I made a mt 2.5.8 avisynth.dll and mt.dll

[Gannjunior]

Hi,

i'm trying to use MT in this script:

SetWorkingDir("d:\avs_plugins\AviSynthPlugins\")
LoadPlugin("MT.dll")
Loadplugin("d:\avs_plugins\warpsharp.dll")

LoadPlugin("SangNom.dll")
LoadPlugin("average.dll")
LoadPlugin("Degrainmedian.dll")
Loadplugin("removegrain.dll")
LoadPlugin("Repair.dll")
LoadPlugin("gradfun2db.dll")

LoadPlugin("DGAVCDecode.dll")
LoadPlugin("DGDecode.dll")

loadplugin("fft3dfilter.dll")
loadplugin("hqdn3d.dll")
LoadPlugin("convolution3d.dll")
LoadPlugin("undot.dll")
LoadPlugin("mvtools2.dll")
LoadPlugin("mt_masktools-26.dll")
LoadPlugin("TTempSmooth.dll")

import("d:\avs_plugins\denoisers\MCTemporalDenoise.v1.4.08.avsi")
import("d:\avs_plugins\denoisers\GradFun2DBmod.v1.5.avsi")
import("LSFmod.v1.9.avsi")
LoadPlugin("GradFun2DB")

LoadPlugin("D:\Program Files (x86)\StaxRip\Applications\AviSynth plugins\DGAVCDecode.dll")

AVCSource("L:\test_5d2\MVI_0135 temp files\MVI_0135.dga")
MCTemporalDenoise(settings="high")

I tried to use SetMTmode() in different way but with no success.

How could I set MT in the correct way to use all my i-920 cores in the script above?

Thanks in advance

ciao!

[osgZach]

All I see is a LoadPlugin() for MT.dll

Where did you attempt to SetMTMode and how did you call it?

[Gannjunior]

before MCTtemporal...

could you help me modify my script adding it in the correct place of the script?

thanks!

[osgZach]

Honestly I'm not that experienced with StMTMode.. Most people know a lot more than me..

Something real basic you could try is this..

SetMTMode(5,0) # put this BEFORE AVCSource - 5 = mode 5 (recommended for source filters). 0 = auto detect threads I believe. You define threads once and only ever change the MODE from then on.

Then after your AVCSource but Before MCTTemporal - SetMTMode(2)

That is one very basic way to try it. I have seen some other examples with a bit more code than that though, so hopefully someone better than me will come along and help.. I am used to using the MT filter on a single line instead of threading the whole script.

[prokhozhijj]

OS: Windows 7 64-bit.
CPU: Intel Core 2 Quad 9505.
RAM: 4Gb
Coder: x264, v.1592 32-bit.
Avisynth: 2.5.8 with dlls from this theme.
Source video resolution: 720x576
MVTools: v.2.5.10 (also tried 2.5.9)

I tried to use script listed below with different numbers of threads and different volume of memory.
With number of threads less than 5 - CPU load is no more than 50%.
With SetMemoryMax=2048Mb - CPU load is no more than 50%.
With number of threads more than 4 and SetMemoryMax=1200Mb - scripts hangs with module name error mvtools2.dll.

Could anybody help me with this problem?

Script looks like this:

SetMemoryMax(1200)
SetMTmode(5,8)

DGDecode_mpeg2source("G:\video\_new\ub_s\s 3\d3\encode\d3_1.d2v", idct=3, info=3)
AssumeTFF()

SetMTmode(2)

crop(10,68,-18,-64)

#######################################
# TempGaussMC
#######################################
Import("D:\arh\media\edit\video\_Filters\TempGaussMC\TempGaussMC_beta2u.avsi")
clip=TempGaussMC_beta2u(EdiMode="nnedi2",qual=3,tr2=2,lossless=-1)


clip=clip.SRestore(frate=25)


return clip

[Didée]

With number of threads less than 5 - CPU load is no more than 50%.

That's normal. You have an HT CPU - 4 physical cores, 8 "logical" cores. When 4 computational intensive threads are running, they will be assigned to the 4 physical cores. Taskmanager will show 50% load because only 4 of the 8 present cores are used. However, the real load in fact is much higher: when the 4 physical cores are already "maxed out", then the additional 4 virtual cores would not give you another 50% of performance. In practice, they could give you some 5%-20% of additional performance (depending on many things). A realistic number is +10%.

With number of threads more than 4 and SetMemoryMax=1200Mb - scripts hangs with module name error mvtools2.dll.

You are in 32bit application land, and your whole processing chain is hitting the "not more than 2GB memory per application" limit.
TGMC needs quite some memory even single-threaded. Now, with some uber-simplification just to make understanding more easy, using SetMTMode(mode,#threads) is quite similar to running TGMC #threads times in parallel. Say that singlethreaded TGMC requires 400MB RAM. Now multiplicate that by 8 threads, and you have a problem. (Yes, I know that SetMTMode in reality uses less ressources.)

Add to that that you're encoding directly to x264, where x264 by itself can easily require anything from 500MB to 1 GB or 1.5GB of RAM, depending on settings. (RC-lookahead plays a major role.)


Alltogether, that's lots of memory that is required .... and since for 32bit applications only 2GB are allowed, it's just too much.

The possibilities are

- stick with SetMTMode(2,4)

- or: make a lossles encoding first, then feed that into x264 (get x264 memory requirement out of the game, so Avisynth has more memory available)

- or: switch to 64bit Avisynth & x264_x64.


In any case, reduce that overly huge SetMemoryMax. It only controls the frame cache of Avisynth, and you really don't need that much. The more frame cache you specify, the less memory is available for the actual script.


My "best guess" is: if you have it running with SetMTMode(2,4), then stick with that. That's about as much as you'll get from TGMC on a CPU with 4 physical cores. You might even consider to specify *less* threads for x264. When the input contains the TGMC turtle, powered by 4 threads already, then it should be safe to reduce x264 to e.g. threads=4. It's default is 12 threads (on a 4-Core with HT), and that won't do much good in this case. 4 threads should be enough to slip into the little free space of Avisynth threads. Anything more will only add unnecessary overhead, for no actual benefit.

[prokhozhijj]

Didée, thank you very much for explanation. I knew nothing about memory allocation within avisynth. So your explanation very useful for me. I'll try to reduce memory size for inner buffer cache.

But let me comment a little some sentences.

You have an HT CPU - 4 physical cores, 8 "logical" cores. When 4 computational intensive threads are running, they will be assigned to the 4 physical cores. Taskmanager will show 50% load because only 4 of the 8 present cores are used. However, the real load in fact is much higher: when the 4 physical cores are already "maxed out", then the additional 4 virtual cores would not give you another 50% of performance. In practice, they could give you some 5%-20% of additional performance (depending on many things). A realistic number is +10%.

When I am running my script with 5 or more threads processing speed equals to around 6 fps. But with 4 or less threads speed is much more modest - it is about 3 fps. So difference is much more than 10%.

[supernater]

In SEt's 2.5.8 MT package he has modified versions of some plugins such as directshowsource and tcpdeliver. Does your version require modified versions of these plugins as well?

[Jeremy Duncan]

I forget, but if I remember correctly the direct show source dll is different from the avisynth dll. Same for tcp deliver dll. I only made the avisynth dll. It's a regular avisynth dll, no special requirements besides using mt dll.

[Mrsash]

OK I reinstalled everything. I remember I had an option other than disabled and libavcodec in ffdshow, but it seems to have disappeared. Am I missing anything? I have just come back after a break to encoding and need some assistance. Basically ffmpeg_mt is missing in the list.
thanks


Forgot to add. Installed latest ffdshow(ffdshow_rev3914_20110629_clsid.exe)

[TheRyuu]

Originally Posted by Mrsash

OK I reinstalled everything. I remember I had an option other than disabled and libavcodec in ffdshow, but it seems to have disappeared. Am I missing anything? I have just come back after a break to encoding and need some assistance. Basically ffmpeg_mt is missing in the list.
thanks


Forgot to add. Installed latest ffdshow(ffdshow_rev3914_20110629_clsid.exe)

This is kind of off topic but everything relevant from ffmpeg-mt has been merged into main.

[Mrsash]

Originally Posted by TheRyuu

This is kind of off topic but everything relevant from ffmpeg-mt has been merged into main.

Sorry I dont understand. Please explain

thanks

[LaTo]

Originally Posted by Mrsash

Sorry I dont understand. Please explain

thanks

ffmpeg is ffmpeg_mt