Avisynth 2.6 MT

[LigH]

It will probably depend more on the source plugin, less on the filter plugins. There are video decoders which may not like being called by many concurrent threads, requesting often changing positions from the source video. If they are not programmed "thread-safe" (a.k.a. "reentrant"; it mainly means consequently avoiding global variables, to allow each thread having its own independent set), then it may happen that they partially return content to one thread which another thread requested, like luma and chroma from different frames.

Allegedly, MPEG2Source (DGMPGDec) confused the chrominance of averagedly one frame per 3..5 minutes in this german report (http://forum.gleitz.info/showthread.php?46355&p=437176#post437176) even though it ran in MT mode 3. Hard to prove or disprove this assumption if it doesn't happen on your PC... We are still discussing this case, but the concerned user is not the most rational-thinking. ;)

[Groucho2004]

Hello guys, i need small help with MT, actualy, more like "consultation"

I use this plugin with dfttest, mode 2, but, after chat with fwe people hwo use it back in 2012, i recive not very positive feed backs. Main notice "MT, cause out put of bad frames" i already burned about 10 rips, since yasterday, and didn'tn otice any problem, but it's take about 1,5 - 2 h on burning, since iwhtout Mt - 3.7 - 4.1 fps, but with MT 6.25, or even more.

any how, can any one showme how this "bad frame" look like ? Thank You !
1. There is no need to use dfttest with Avisynth MT, the plugin already features multi-threading
2. I guess the people who told you about the "bad frames" know best what they look like
3. Use a spell checker next time you post

[pila13]

thank you for replyas.

>> LigH
humm, i recive + 2 fps when use MT, before dfttest. I see, but i don't use mt, on a source.

SetMTmode(3,0)
MPEG2Source(".d2v")

only before filters. And i didn't had this problem.

[Genaugmen]

I get errors when trying to extract the file. I have downloaded at least 7-8 times. I've tried extracting with winrar and 7zip. Winar says "avisynth_20130309.7z: Unknown method in avisynth.dll". 7zip says "unsupported compression method for 'avisynth.dll". I can't find a mirror either. Help, please! I should also mention that I have tried all of this with antivirus disabled too, so that's not the issue. I also have the same problem with the previous version. I didn't have an issue with the version prior to that, but it's hosted somewhere else.

[Groucho2004]

I get errors when trying to extract the file. I have downloaded at least 7-8 times. I've tried extracting with winrar and 7zip. Winar says "avisynth_20130309.7z: Unknown method in avisynth.dll". 7zip says "unsupported compression method for 'avisynth.dll". I can't find a mirror either. Help, please! I should also mention that I have tried all of this with antivirus disabled too, so that's not the issue. I also have the same problem with the previous version. I didn't have an issue with the version prior to that, but it's hosted somewhere else.
Works just fine with 7z v9.2 and Winrar v4.

[Genaugmen]

I've been using the same old versions for years. Just tried with the newest version of 7zip and it extracted fine. Thanks!

[LigH]

Version 9.2x introduced the better multi-threaded LZMA2 algorithm.

[cihub]

No matter what scripts I use (mainly with MCTDmod and QTGMC) at some point in time AviSynth tends to stop working. Usually it happens like this: counter of processed frames in x264 (or VirtualDub) stops increasing, but CPU is still used (~30% on my AMD FX-6300), even if I cancel processing (Ctrl+Break with x264 or program closing with VirtualDub).

The scripts I've used mainly look like this:
#plugins
SetMTMode(3)
#source: Mpeg2Source("d2v.d2v")
SetMTMode(2)
#filters: QTGMC(Preset="fast"); SelectEven(); MCTD()
Distributor()
SetMTMode(3)
#post-processing: resize, crop
or
#plugins
SetMTMode(3, 6)
#source: Mpeg2Source("d2v.d2v")
SetMTMode(2)
#filters: QTGMC(Preset="fast", EdiThreads=3); SelectEven(); MCTD()
Distributor()
SetMTMode(3)
#post-processing: resize, crop

Any ideas what's causing crashes?

[Groucho2004]

Any ideas what's causing crashes?
- QTGMC and/or MCTD are probably not getting enough memory. Try playing with "SetMemoryMax()".
- Get rid of "Distributor()". x264 calls that function.
- Use fewer threads.

[Selur]

also there shouldn't be and additional setmtmode calls behind distributor(),...

[cihub]

Groucho2004, Selur thank you. Will try that tomorrow.

BTW, 1) when I do not use Distributor(), at least in AvsP, CPU usage stays at the level of AviSynth 2.5.8, i.e. like there's no MT 2) SetMemoryMax() set to 768--2000 MB had no result at all (well, maybe there was something different like the time it took to crash, but it still crashed).

[Groucho2004]

BTW, 1) when I do not use Distributor(), at least in AvsP, CPU usage stays at the level of AviSynth 2.5.8, i.e. like there's no MT 2)
Don't know about Avsp. If you use x264, omit the Distributor call in the script.

SetMemoryMax() set to 768--2000 MB had no result at all (well, maybe there was something different like the time it took to crash, but it still crashed).
Add the functions one by one (first QTGMC, then MCTD). Also, test the script without the encoder first, use AVSMeter (http://forum.doom9.org/showthread.php?t=165528) to check the speed, CPU and memory usage.

[Jeroi]

Could you add DXVA support also? AInt GFX cards good at video prosessing?

[LigH]

DXVA support in the AviSynth kernel? I doubt that there would be any gain. The video processing features of DXVA will be too specifically related to small parts in video decoding and won't be useful for general purpose.

[junh1024]

You may be able to use DXVA in avs already by using dss or dss2, and an appropriate directshow filter.

[kalehrl]

My script crashes if there is colormatrix entry in it when using Avisynth 2.6 MT.
This is the offending entry:
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
When I remove it, the script opens just fine.
Does anyone know if there is a different version of colormatrix plugin that work with MT?
My existing colormatrix is 2.5.

[kalehrl]

Unfortunately, it is the same with the file you suggested.
This is the script I use:
SetMTMode(3, 4)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\Dreambox\movie\projectx\VTS_02_1[copy].d2v", info=3)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
SetMTMode(2)
AssumeBFF()
QTGMC(Preset="Super Fast")
SelectEven()
Spline36Resize(720,406) # Spline36 (Neutral)
MeGUI won't even load the script unless I remove colormatrix entry.

[Boulder]

Try to put SetMTMode(5) right before ColorMatrix.

[Groucho2004]

Unfortunately, it is the same with the file you suggested.
This is the script I use:
SetMTMode(3, 4)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\Dreambox\movie\projectx\VTS_02_1[copy].d2v", info=3)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
SetMTMode(2)
AssumeBFF()
QTGMC(Preset="Super Fast")
SelectEven()
Spline36Resize(720,406) # Spline36 (Neutral)
MeGUI won't even load the script unless I remove colormatrix entry.
With a SD source and super fast preset for QTGMC, I wonder why you need Avisynth MT at all. What CPU do you have?

[kalehrl]

I've got an old DualCore AMD Athlon II X2 240, 2800 MHz (14 x 200) processor and 4 GB of RAM.
With a default QTGMC setting, I've got only 5 FPS but with super fast around 11.5 FPS.
Without avisynth MT I've got only 3 FPS with a default QTGMC setting.
Even with super fast QTGMC gives better results than YADIF.
Try to put SetMTMode(5) right before ColorMatrix.
Excellent!
It works by just putting SetMTMode(5) before ColorMatrix.
Thank you Boulder.

[Groucho2004]

I've got an old DualCore AMD Athlon II X2 240, 2800 MHz (14 x 200) processor and 4 GB of RAM.
With a default QTGMC setting, I've got only 5 FPS but with super fast around 11.5 FPS.
Without avisynth MT I've got only 3 FPS with a default QTGMC setting.
Even with super fast QTGMC gives better results than YADIF.
OK. As already suggested, try SetMTMode(5) instead of 3. Also, specifying more threads than you have cores makes little sense.
So, your script should look like this:
SetMTMode(5)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\Dreambox\movie\projectx\VTS_02_1[copy].d2v", info=3)
LoadPlugin("D:\Programs\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true)
AssumeBFF()
SetMTMode(2)
QTGMC(Preset="Super Fast")
SelectEven()
Spline36Resize(720,406)

[Boulder]

Excellent!
It works by just putting SetMTMode(5) before ColorMatrix.
Thank you Boulder.Good to know that it helped. I don't know why but there are some filters that do not like mode 3 even though it should be basically a single-thread setting. TDecimate is one those who need SetMTMode(5) to work properly.

[kalehrl]

So, your script should look like this:
Thanks. For simplicity's sake, I'll just stick with SetMTMode(5) at the beginning of the script instead of SetMTMode(3) at the beginning and then again SetMTMode(5) before colormatrix.
Regarding threads, I read that they should be number of cores times 2 for hyperthreading.
I tested the script with SetMTMode(5, 4) and again with SetMTMode(5) and there's very little difference in speed.

[Groucho2004]

Thanks. For simplicity's sake, I'll just stick with SetMTMode(5) at the beginning of the script instead of SetMTMode(3) at the beginning and then again SetMTMode(5) before colormatrix.
Why? Once you set a mode, the processing will remain in that mode until you change it. It's the opposite of simplicity (although won't do any harm).

Regarding threads, I read that they should be number of cores times 2 for hyperthreading.
Your AMD does not support hyperthreading.

[LigH]

@ Boulder:

Despite the fact that SEt claimed all modes from 3 up are now exactly the same in AviSynth MT 2.60 a4 ... obviously still not exactly?

[CarlPig]

Any plan for MT update to 2.6.0 Alpha 5?

[Groucho2004]

Any plan for MT update to 2.6.0 Alpha 5?
That may take a while, SEt hasn't posted here in months.

[CarlPig]

Ok...

[bcn_246]

Haven't had any problems running SEt's 2013-03-09 MT build (avisynth_20130309.7z (https://www.dropbox.com/s/xhqggxamegia420/avisynth_20130309.7z)) with v2.6 Alpha 5 here. Will test it some more.

[LigH]

You misunderstood: SEt's AviSynth MT 2.60 a4 replaces AviSynth 2.60 a5.

[Overdrive80]

Haven't had any problems running SEt's 2013-03-09 MT build (avisynth_20130309.7z (https://www.dropbox.com/s/xhqggxamegia420/avisynth_20130309.7z)) with v2.6 Alpha 5 here. Will test it some more.

If you do that, really working with the alpha version 4.

[bcn_246]

Okay...

Using...
1. AviSynth_130918.exe (CRC32: 973AC633)
2. Which includes the following version of avisynth.dll (CRC32: 7C8B3879)
3. Replaced with SEt's MultiThreaded version of avisynth.dll (CRC32: 4E3DD2EE)

Gave no problems opening...
AVISource("1080pSample.avi")
Crop(0,0, -Width % 8,-Height % 8)
ConvertToYV12()
SetMemoryMax(2048)
SetMTMode(2,8)
DeBlock_QED()
Crop(4,2,-4,-2)
Spline36Resize(1280,720)
GradFun2DBmod(str=0)

[IanB]

@bcn_246,

And your point is? :confused:

SEt's 2013-03-09 MT build is derived from Alpha4, Jan 14th, 2013. If you install Alpha5 and replace it with SEt's Alpha4 avisynth.dll, you are running SEt's Alpha4.


And the SetMemoryMax(2048) is a ridiculous value. As I keep telling people setting the size of the Avisynth frame cache bigger than what the current script needs is a waste.

[LigH]

The core of AviSynth is only this DLL.

[tormento]

As I keep telling people setting the size of the Avisynth frame cache bigger than what the current script needs is a waste.

How should we determine the correct amount?

[Groucho2004]

How should we determine the correct amount?
It's impossible to calculate the correct value because of the complexity of filter combinations like QTGMC.
So, you have to experiment. Start with something like 1500 and reduce it until your script slows down (or crashes). Then go back to the previous value.
I successfully used Avisynth MT/SetMemoryMax(1024) with QTGMC (slow preset) on HD material.

Don't just test it with a few frames, let it run for a few minutes. AVSMeter is a good tool for these experiments.

[ryrynz]

How should we determine the correct amount?

I've had blocky corruption of frames with SetMemoryMax(1000) I've set back to 1500 and everything seems fine now.

[Seedmanc]

I remember some script eating way too much memory and eventually crashing when I used this command, while working nicely without it. Don't know details though.

[LigH]

There may be more updated files between alpha 4 and alpha 5. But the most important file (the "kernel" which represents AviSynth, and defines the version) is only the one DLL.

Other included files are just a few plugins, example scripts, and the documentation. Nothing which really depends on the minor version or would even define it.

[SEt]

Updated to Alpha5.


Despite the fact that SEt claimed all modes from 3 up are now exactly the same in AviSynth MT 2.60 a4 ... obviously still not exactly?
I never claimed such thing... Only that mode 3 and mode 5 for source filters are the same.

[Boulder]

Thanks a lot!

[LigH]

Updated to Alpha5.

:thanks:

I never claimed such thing... Only that mode 3 and mode 5 for source filters are the same.

Well – it is hard to understand, and it stays hard to understand.

There are indeed cases when modes 3..6 do behave differently (e.g. NeatVideo VDF).
__

And there may also be cases when AviSynth MT just can't handle a filter (or vice versa) with more than 1 thread: At least we (in the german board) are currently trying to get DeSpot in an MT sequence running without crashing; the most stable mode in a report of a user appears to be mode 4, but it is always unreliable, sometimes the crash is earlier, sometimes later. There seems to be a discouraging statement about DeSpot in another location.

[Sparktank]

Updated to Alpha5.

:goodpost::thanks:

[PetitDragon]

[QUOTE=SEt;1645637]Updated to Alpha5.

This is just greate!:thanks:

[CarlPig]

A minor updated build of mt_masktools-26.dll for 2.60alpha4's latest interface (current cvs 2013-03-09):
mt_masktools-26-for-2.6alpha4.7z (http://tmod.nmm-hd.org/Misc/mt_masktools-26-for-2.6alpha4.7z)

Patch (on top of Vit (http://forum.doom9.org/showthread.php?p=1568142#post1568142)'s 2.6 fixed one) included.

Please update to 2.60alpha5 MT.

[LigH]

Does AviSynth MT 2.60 alpha 5 introduce a new interface again, so that MaskTools2 have to be adapted again? I didn't hear about any crashes since, yet. So I believe there was only this one change between versions up to alpha 3 and from alpha 4 on?

[Groucho2004]

So I believe there was only this one change between versions up to alpha 3 and from alpha 4 on?
Correct. There are no API changes that would require plugin updates from A4->A5.

[CarlPig]

I dont know, but the answers says that i need a patch.

I got this problem: http://forum.doom9.org/showpost.php?p=1647078&postcount=200

I use 2.60alpha5 MT and the lastest masktools build.

[Groucho2004]

I use ... the lastest masktools build.
This (http://forum.doom9.org/showthread.php?p=1619023) one?

[CarlPig]

Yes..