Avisynth 2.6 MT

[ryrynz]

Yeah, this is very great news. Avisynth development moves at such a slow pace is great to see new revisions.

Everything's working well here with no stability issues. Performance might even be slightly higher judging from the quick test I did. Thanks to you, IanB and everyone else involved.

[SEt]

Oh, I forgot one minor thing in change log:

5) This version accepts brackets-less function calls like official Avisynth.

[SubJunk]

Working well for me too so far :)

[GMJCZP]

I tried to load the VirtualDub filter "MSUSmartSharpen", and although not hung up VirtualDubMod, I just could not carry it, gave me error, as if the configuration that before worked now does not work.

[tormento]

Newer revision works fine and solved some seeking problem too.

Thank you ;)

[hydra3333]

Fresh Win7-64 install, i7-3820 16Gb. Copied 2.6 over the top of tsp's 2.5.7, used VIT functions for QTGMC plus loaded a few other relevant ones for a vanilla QTGMC script with DGindex. Opened script into avs2yuv, and it crashed with no avisynth message. Put 2.5.7 back, backed out to use non-VIT functions in same script and presto, it worked.

In hindsight maybe because I left (only) tsp's "MT" DLLs in the plugins folder and they autoloaded ?

[SEt]

Minor update:

1) Some more VirtualDub plugins compatibility fixes.
2) Issue with Overlay and YV24 (http://forum.doom9.org/showthread.php?p=1567993#post1567993).


hydra3333, MT dlls are not supported. And as base you should start from 2.6 alpha or 2.5.8, not earlier.

[titlis]

Sorry but the latest mt build keep crashing upon loading avspmod.
Other builds work properly.

http://img841.imageshack.us/img841/9458/63379138.png

[SEt]

Can't reproduce.

Latest as 2012.04.03 or 2012.03.31?

[titlis]

Can't reproduce.

Latest as 2012.04.03 or 2012.03.31?

2012.04.03

I'm using AvsPmod 2.2.1

[cweb]

I'm getting a CUDA error and "Failed to create video decoder (100)" running dgdecnv with this latest build you have posted today.. I'm guessing it needs to be recompiled too. Or is there something else which can be done to my script?

[SEt]

I'm even more puzzled: dgdecnv (latest) works without problems for me and changes in build environment were quite minor.
You don't need any recompilation. Try to pinpoint what exactly causes crash: plugin? OS? or it crashes even without plugins with plain AviSource?

[cweb]

I'm even more puzzled: dgdecnv (latest) works without problems for me and changes in build environment were quite minor.
You don't need any recompilation. Try to pinpoint what exactly causes crash: plugin? OS? or it crashes even without plugins with plain AviSource?

So I checked that my script didn't load any other filters. Just a call to dgsource(). The error is returned by dgsource().
CUDA is working with other cuda test programs etc on my pc. I'm running win7 64-bit.

I'm using the latest dgdecnv too.
My source file is an MTS file which I just indexed with dgindexnv, so I can't use AviSource on it. Instead I used AviSource on another avi, and this works without problems. I'm using Virtualdub 1.9.11.
Tried reinstalling my dgdecnv and I get the same result ..

[cweb]

I just checked.. it works with the stock 2.6 release.

[cweb]

I just checked.. it works with the stock 2.6 release.
but not with your avisynth.dll.. at least for me and my 64-bit win7.

[SEt]

Well, I'm using Win7 x64 too (2008R2 actually), so it isn't OS at least.
Let's wait for someone with debugging skills for better crash information - you can use previous version for now, not much difference.

[cweb]

Well, I'm using Win7 x64 too (2008R2 actually), so it isn't OS at least.
Let's wait for someone with debugging skills for better crash information - you can use previous version for now, not much difference.
Just in case I wasn't too clear, as far as I'm aware it's not crashing. It's dgdecnv which gives the message and obviously there's no actual video, but the video itself is replaced with the message 'CUDA error' generated by dgdecnv...

[Atak_Snajpera]

Avisynth 2.6 MT installer (2012.04.03) -> http://www.mediafire.com/?7l0cbjpcd4av1wb
Installer source files -> http://www.mediafire.com/?zu8m4b5zbjmt08e

[kypec]

Avisynth 2.6 MT installer (2012.04.03) -> http://www.mediafire.com/?7l0cbjpcd4av1wb
Installer source files -> http://www.mediafire.com/?zu8m4b5zbjmt08e
Excellent contribution, Atak :thanks:
I was thinking to prepare some installer myself but had no time to get to it.:o

[GMJCZP]

Thanks Atak_Snajpera.

[SEt]

Atak_Snajpera, you don't need to put msvcp60.dll in installer - nothing uses that ancient <censored> except official avisynth.dll.

[Atak_Snajpera]

i didn't know that.

[cweb]

I solved the CUDA problem when using DGDecNV. Here's how.

As a reminder,
1. A dialog box comes up (DGSource) saying 'Failed to create video decoder (100)'.
2. 'CUDA error' is displayed instead of the video.

avisynth.dll is in c:\windows\syswow64

The stable avisynth and the 64 bit avisynth builds worked with dgdecnv without problems for me.
Script is a simple call to dgsource.


It turn out I had an old nvcuvid.dll in the avisynth plugins directory. Deleting this file solved the problem. I'm not aware how this build triggered the problem (the stable build didn't) but I'm glad I found the solution.

[RedDwarf1]

This new version won't work with AvsPmod 2.2.1 or 2.2.0 on my system, whereas the earlier version that I was using worked without problems. I get a CPP runtime library error. This application has requested the Runtime to terminate it in an unusual way.

That is with the plugins folder renamed so no usable plugins and a new copy of AvsPmod, so no scripts or old settings.

2012-03-31 is working okay which I wasn't using previously because I was still on an older version.

[SEt]

cweb, oh, I have an idea now.
Another change I forgot to mention is how plugin DLLs are loaded. Now if they have dependencies – dependencies are searched first in the same folder as loaded plugin DLL. Hopefully it will reduce amount of DLLs you have to place in system32, but as it's different to earlier behavior it can cause unexpected situations like yours.

RedDwarf1, AvsPmod 2.2.1 works ok for me. Need more information to debug things.
"This application has requested the Runtime to terminate it in an unusual way." means there is uncaught C++ exception somewhere.

[Anacletus]

With the last build i got the same error as titlis and RedDwarf1.

I'm on Windows XP SP3 32bit, AvsPmod 2.2.1 keeps crashing with a simple script with just an mpeg2source call.

The same happens wit MeGUI as soon as i try and load the same script. :confused:

[RedDwarf1]

cweb, oh, I have an idea now.
Another change I forgot to mention is how plugin DLLs are loaded. Now if they have dependencies – dependencies are searched first in the same folder as loaded plugin DLL. Hopefully it will reduce amount of DLLs you have to place in system32, but as it's different to earlier behavior it can cause unexpected situations like yours.

RedDwarf1, AvsPmod 2.2.1 works ok for me. Need more information to debug things.
"This application has requested the Runtime to terminate it in an unusual way." means there is uncaught C++ exception somewhere.
What info do you require?

With the last build i got the same error as titlis and RedDwarf1.

I'm on Windows XP SP3 32bit, AvsPmod 2.2.1 keeps crashing with a simple script with just an mpeg2source call.

The same happens wit MeGUI as soon as i try and load the same script. :confused:

No scripts at all when I tried it.

I'm using XP SP3 as well. I can try it on another windows install when I get a chance and reboot.

[Anacletus]

What info do you require?



No scripts at all when I tried it.

I'm using XP SP3 as well. I can try it on another windows install when I get a chance and reboot.
That's true.. avspmod crashs as soon as i click the executable. :sly:

[SEt]

To debug this I need a way to reproduce the error. Or at least location of unhandled exception thrown and stack trace, so someone with debugging skills is needed for second approach.
To pinpoint what is causing your problems start with clean system (you can use VM – it lakes few minutes to set up but very useful for many experiments) and gradually add what you have. I suspect stray DLL in unexpected location that is now loaded instead of expected DLL.

[Boulder]

I suspect stray DLL in unexpected location that is now loaded instead of expected DLL.Would Dependency Walker help with this one?

[SEt]

Unlikely – DLLs can be loaded in runtime.
But you can try list of all process DLLs (with paths of course) in Process Explorer and see if you can spot the difference of previous version and new at the crash point.

[LoRd_MuldeR]

Unlikely – DLLs can be loaded in runtime.

Dependency Walker has a "profiling" feature that can be used to track all the DLL's loaded at runtime too.

Other than that, one can also use VMMap (http://technet.microsoft.com/en-us/sysinternals/dd535533) to show all modules (DLL's) that a given processes has currently loaded...

[Overdrive80]

I will discuss my experience. If I use:

SetMemoryMax(512)
SetMTMode(3,4)
source_filter()
SetMTMode(2)
processing_filters()

Any program freezes, increasing the size of the ram but without result

If I use:

SetMemoryMax(512)
SetMTMode(4,4)
source_filter()
SetMTMode(2)
processing_filters()

I get this error: http://s7.postimage.org/q5llnwpbv/Set_MTMode_4_4.png

Using follow script, run fine.

SetMemoryMax(512)
SetMTMode(5,4)
source_filter()
SetMTMode(2)
processing_filters()

My system is win 7 sp1 x64 on GA-x58-ud5+intel i7 920+ 6gb ram

[Boulder]

I think you are supposed to use SetMTMode(5,x) before a source filter, so the last example is correct (and thus runs without errors).

[TheRyuu]

I think you are supposed to use SetMTMode(5,x) before a source filter, so the last example is correct (and thus runs without errors).

And even then last I checked using mode 5 is highly discouraged.

Mode 3 should be used instead for situations like that IIRC.

[Boulder]

Well, for example DGSource doesn't like SetMTMode(3) on my computer while it works fine with SetMTMode(5). Of course, decoding the source is the smallest thing in my case so it can well run singlethreaded.

[Anacletus]

To debug this I need a way to reproduce the error. Or at least location of unhandled exception thrown and stack trace, so someone with debugging skills is needed for second approach.
To pinpoint what is causing your problems start with clean system (you can use VM – it lakes few minutes to set up but very useful for many experiments) and gradually add what you have. I suspect stray DLL in unexpected location that is now loaded instead of expected DLL.

Hi SEt,

i installed a clean Windows XP Service Pack3 into an Oracle VirtualBox VM, installed official avisynth 2.58 and it was ok. Downloaded your last build of 2.6MT and i got the avsp crash again.

System config:
Oracle VirtualBox 4.1.14 r77440
Windows XP SP3 (with all the security patches at the time of writing)
AvsP.exe 2.0.2.0
Avisynth 2.5.8 + Avisynth 2.6MT (latest version)

I used Dependancy Walker's profiling tool, you can find the profiling log attached, the exception is at line 476 ;)

[SEt]

Found it! Grab the new version.
XP runtime was terminating us for no good reason – I've hacked the compiler a bit to be more XP-friendly. NT6 users are unaffected. No source changes since version 2012.04.03.

[Reel.Deel]

Thanks SEt, your latest version (2012.05.16) works fine. BTW I'm Using Windows XP SP3 (32 bit).

[Anacletus]

Found it! Grab the new version.

:thanks:

[RedDwarf1]

Found it! Grab the new version.
XP runtime was terminating us for no good reason – I've hacked the compiler a bit to be more XP-friendly. NT6 users are unaffected. No source changes since version 2012.04.03.

:thanks: The new one works fine for me as well.

[BugiBugBug]

SEt, I've searched through this threat on information about the MT() function.
I noticed a post you made where you state that the MT.dll is not supported.
Remarkably however, in the beginning of this thread, notably BigDid (http://forum.doom9.org/showthread.php?p=1338268#post1338268) stated he was able to use the latest MT0.7 dll with your AS2.6
Still, MT.dll does not work for me (AV complains "wrong avisynth version. Use avisynth 2.6 or greater or modified avisynth 2.5.6").

Am I correct in assuming that for instance MT("FluxSmoothST(7,7)",threads=2) cannot be used with any of your AviSynth MT builds?

And that the only possibility to use the MT() mode would mean having to use the older 2.5.7 build supplied with the MT.dll?

I'm a great fan of temporal denoisers, and I have perceived the SetMTmodes to limit their effectiveness. If I understand the Wiki correctly the MT() mode is the only true option for effective temporal filters. But this may of course be incorrect; I do not really understand all the differences between SetMTmode - maybe I missed something and the MT() mode is deprecated?

[SEt]

MT() is unsupported by me, though it's not impossible to update it to the current Avisynth 2.6 MT state.
And sure MT() is not your only option besides MTMode: look in the forum – there are other threading attempts like pipelining.

[ryrynz]

Am I correct in assuming that for instance MT("FluxSmoothST(7,7)",threads=2) cannot be used with any of your AviSynth MT builds?


SetMemoryMax(1024)
SetMTMode(3,2)
ffdshow_source()
SetMTMode(2)
FluxsmoothST(7,7)
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

Basically taken from the front page. I use this within ffdshow, I use Fluxsmooth often and had zero issues with SEt's builds.

[SEt]

Never put source filters in mode 2 – you are creating multiple instances of it. Use either mode 3 or 5 for them.

[ryrynz]

Thanks for the tip, I'll edit the previous post.

[Boulder]

I'm a great fan of temporal denoisers, and I have perceived the SetMTmodes to limit their effectiveness.Actually using SetMTMode makes them work exactly like they work in single-threaded mode. A temporal denoiser works over a certain area in a single frame and processes multiple sequential frames for each single frame. Depending on the spatial radius and the overlap value, using MT() could lead to sub-optimal results.

[BugiBugBug]

Actually using SetMTMode makes them work exactly like they work in single-threaded mode. A temporal denoiser works over a certain area in a single frame and processes multiple sequential frames for each single frame. Depending on the spatial radius and the overlap value, using MT() could lead to sub-optimal results.
Thank you for this clarification; I was already wondering how that would work, since I've never had crashes or anything with FluxSmooth or DeGrainMedian using SetMTmode. That is not the reason of my concern.

As you are saying, the filter needs a sequence of sequential/subsequent frames to calculate whether it smooths something. This is also the reason why I understood the threading method of SetMTmode to be sub-optimal for this purpose, because in my endless naivity I assumed using this would mean the following:
thread 1 gets e.g. frame 1,3,5,7 and calculates the temporal pattern of an area on their sequence.
thread 2 gets e.g. frame 2,4,6,9 and calculates the temporal pattern of an area on their sequence.

But if I understand your explanation correctly, two threads are created in which the same frame is thus present (i.e. read) in thread 1 on t=1, and present in thread 2 as well, at t=2.
But the 'noise correction' is applied in thread 1 to frame n, and in thread 2 to frame n+1.


I apologise if I'm bringing things up again that have been explained already (given the fact that MT has been around for some years now), but I have been unable to fully apprehend its use and configuration despite of countless hours of searching and reading. It's these specific questions vs. explanations that keep fogging my understanding.


So this is what I've got now, just for clarification:

SetMTmode(3,0)
LoadPlugin("")
DGDecode_mpeg2source("C:\CT.d2v", info=3)
SetMTmode(2,0)
ColorMatrix(hints=true, threads=0)
trim(125500,126500)
FluxSmoothST(7,7)
crop(10, 10, -8, -14)
Undot()
Spline36Resize(720,576)
Undot()
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

[Boulder]

But if I understand your explanation correctly, two threads are created in which the same frame is thus present (i.e. read) in thread 1 on t=1, and present in thread 2 as well, at t=2.
But the 'noise correction' is applied in thread 1 to frame n, and in thread 2 to frame n+1.Yes. If I have understood the process correctly (=the threads are totally independent of each other) :D. The processing gets a nice boost from the Avisynth cache because the same frame could be needed by many threads and it could already be in the cache when it is requested by a thread. Then again, the cache used to be the nest of many problems in multithreading, but I think that SEt has worked out quite a few of them.

You should make sure that you really need Distributor() in your script. Many applications do not need it since they call it themselves.

[SEt]

That's not how it works. Simplified a bit, it works this way: let's look at certain filter in the script in MTMode – there are indeed 2 threads, but both of them have all source frames, next from one thread frames 0,2,4... are requested and from the other thread frames 1,3,5... If the filter handles non-sequential requests correctly – it'll work exactly as in non-MT mode, but if it doesn't – things will break (note that they will also break in non-MT mode when some filter reorders frames like SelectEven or Trim).

The end
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last
must be used only for certain cases like ffdshow processing, for plain Avisynth usage it must not be used. How to guess? If MTMode doesn't change your CPU utilization no matter how many threads you request, but adding Distributer helps – you need it, otherwise you don't.