MT 0.7(+custom avisynth)a filter to run filters multithreaded. Usefull for SMP or HT

[Revgen]

Originally Posted by J-Wo

Hey, decided to give the new MT avisynth.dll for 2.5.7 a try. I'm doing a 2-pass VBR encode with CCE and while my CPU usage started around 95-98%, after 30 minutes it's dropped down to 77-83%. Encoding speed has likewise dropped down from 3.60x to 3.33x. Not sure if that is meaningful at all, just thought I'd post here tho. Here's my script:

setmtmode(2)
loadplugin("D:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")

mpeg2source("E:\MOVIE\VTS_01_1.d2v",idct=6)
Lanczos4Resize(720, 464, 0, 12, 720, 552)
AddBorders(0, 8, 0, 8)
AssumeFPS(23.976, true)
LetterBox(0, 0, 8, 8)
ConvertToYUY2()

CCE is a 1-thread encoder right?

The first pass may not be hindering SetMT from using the cores. Once the 2nd pass starts going, CCE may be bottlenecking Avisynth.

[tsp]

J-Wo: It could be because one of the avisynth threads stalled. TRy downloading process explorer Run it while CCE is running (before and after you experience the slowdown) and double-click on the cce process. Select the thread tab. The cpu utilization between the two avisynth threads should be evenly distributed if one of them is hovering about 0-10% it might be a bug in avisynth.

[J-Wo]

The avisynth threads never get down that low, they seem to be steady around 25% +/- 5%. Again after ~30 mins of encoding, CPU utilization has dropped to 76-80%. If you'd like I could run a comparison with an older version of MT. BTW, is there an easy way to have CCE log how long the encoding process took? I'm not often at my computer during encoding so I often miss the window before it closes...

[tsp]

it would be fine if you could compare it with the 2.5.6MT version and the non-MT 2.5.7 version. The option to enable the log file hides in the options->outputs menu.

[henryho_hk]

I have problem using Writefile() with SetMTMode(). Below is a simplified scenario:

Case 1: The values are nicely placed in c:\1.txt delimited by a space.

Blankclip(pixel_type="yuy2",color=$000000,length=71,width=720,height=240,fps=30000,fps_denominator=1001).killaudio()
trim(0,-1)
space=" "
writefile("c:\1.txt", "Width", "space", "Height", "space", "Framerate", "space", "FramerateNumerator", "space", "FramerateDenominator", "space", "HasAudio", append=false)

Case 2: The space characters become something like "I don't know what "space" is".

SetMTMode(2)
Blankclip(pixel_type="yuy2",color=$000000,length=71,width=720,height=240,fps=30000,fps_denominator=1001).killaudio()
trim(0,-1)
space=" "
writefile("c:\2.txt", "Width", "space", "Height", "space", "Framerate", "space", "FramerateNumerator", "space", "FramerateDenominator", "space", "HasAudio", append=false)

I am using Avisynth.dll in http://www.avisynth.org/tsp/avisynthMT257.2.zip

[foxyshadis]

Try
global space = " "
I guess it's just a scoping issue.

[tsp]

global space =" " would work but it is a bug that happens because each thread has each own var table but they didn't have access to the local var table that are created when the filter chain is created (that contains the variable space in this case) I have fixed this in this version (avisynthMT 2.5.7.3)

[henryho_hk]

tsp, thank you very much.

[halsboss]

A dummy's query, does a socket-939 AMD 64 3500+ Venice Core have hyperthreading ? AMD's website doesn't help me. Another way of putting it, is MT useful with that processor ?

[Boulder]

MT's not useful as your CPU isn't dual-core. AMD doesn't have anything like hyperthreading IIRC.

[Revgen]

Originally Posted by tsp

global space =" " would work but it is a bug that happens because each thread has each own var table but they didn't have access to the local var table that are created when the filter chain is created (that contains the variable space in this case) I have fixed this in this version (avisynthMT 2.5.7.3)

I was having problems with the 2.56 version where the program would quickly exit whenever it would encode a certain frame. For some reason, this version doesn't have any issues. I'll definitely keep it for now.

[Adub]

I believe that I have found a new bug. I was recently trying to encode a movie, Step Up, to x264, using the following script:

SetMTmode(2)
DGdecode_Source("C:/Step Up/decomb ivtc.avs", info=3)
Colormatrix(hints=true)
AssumeTFF.Telecide(guide=1).Decimate()
Autocrop(0)
Spline36resize(1280,702)

Except every time that I go to encode it, eventually x264, versions 620 and 621, spit out an in windows, saying that it needs to close. When I click for more information, the dll in question switches between "decomb.dll" and "Avisynth.dll". Also, I got the error to occur using xvid encraw, so I know that it is not encoder specific. However, I could only get xvid to error out on the Step Up script, not the blankclip script, as I could with x264.

My avisynth.dll is your 2.5.7.2 I suppose. I have yet to try your latest one, but I will check right now.

Edit: okay, 2.5.7.3 errors out as well.
You can find the error using megui if you use the HQ-Slow profile and the following script:

setmtmode(2)

blankclip(pixel_type="YV12",length=100000).AddGrain(100,0,0)
Colormatrix()
AssumeTFF().Telecide(guide=1).Decimate()
#crop
#resize
#denoise

Spline36resize(1280,704)

[Adub]

TSP, can you confirm the bug?

Can anyone confirm the bug?

[tsp]

Merlin7777: 80 % done with second pass x264 HQ-Slow profile and no crash yet with the blankclip script.

[Adub]

What version of decomb.dll are you using?

[webzeb]

Hello,

Is there any solution to use MT/SetMT for resizing (Spline36Resize) under FFDShow ?

SetMTMode seems not working (GetMTMode return '0')...

Thank you,

Fred

[bikes302]

I must say that I am quite thrilled that I finally found this multithread mod to avisynth. I bought a new dual core machine hoping to encode faster, but when I tried it was just as slow. It was taking me 12 hrs per pass, 24 hrs total on my 2.66 Ghz machine. I noticed it was only using one core and I was quite upset. I found out that avisynth was bottlenecking my encode and found this. I added in the MT dll and replaced my avisynth.dll file and my encode now uses 100% both cores and went from 12 hrs per pass to abt 5 hrs per pass and 10 total. I must say I am more than thrilled with this.

Just thought I should mention though that there are a few issues. I am not sure what exactly, but I cant do both passes continuously. Once the one pass finishes in VDub, I have to close and reopen to do the second pass or else I get one of those VDub needs to close, sorry for the inconvience. It also happens occassionally as soon as it starts to encode. If thats the case I just close and reopen and keep trying until it works, eventually it starts to work and once it starts it has no issues. Which puzzles me because sometimes it works and sometimes it doesnt with the same settings, but I dont care since my encode time was cut in half.

[tsp]

Merlin7777: decomb 5.2.2

[Adub]

okay, i am testing on a different computer, same filter versions, only single core. Also, TSP, do you have your threads set to "auto" in your HQ-Slow profile?

[tsp]

merlin7777: yes threads are set to auto