MT 0.7(+custom avisynth)a filter to run filters multithreaded. Usefull for SMP or HT

[squid_80]

Originally Posted by Boulder

I've also noticed that having Distributor() in the script causes a lot of problems when loading the script in VDubMod. Most of the time, the application just vanishes when I try to load the script.

Distributor() should only be used when loading the script into applications which talk to avisynth directly e.g. HC and MeGUI. If the program opens the .avs file as an avi file (like virtualdub/mod, x264, xvid_encraw etc.) you don't need (and shouldn't have) the distributor call at the end.

[tsp]

Boulder: The usual way is to figure out which filters that are causing the trouble by commenting them out one at a time but as the crash only occurs sometimes it is less useful. If the crash dialog appears you can see in which dll the crash occurred under the more info link.
Your script crashed every time for me both with and without setmtmode(). I will see if I can figure out why.

[Boulder]

I also emailed hank315 and described the situation, maybe he has some ideas. I tried batch encoding the same script (using 250 frames) 100 times with HC, the first crash occurred around the 80th encode.

I think I should try encoding the script without any imported stuff and having the functions in the script itself and see what that causes.

[Boulder]

Originally Posted by squid_80

Distributor() should only be used when loading the script into applications which talk to avisynth directly e.g. HC and MeGUI. If the program opens the .avs file as an avi file (like virtualdub/mod, x264, xvid_encraw etc.) you don't need (and shouldn't have) the distributor call at the end.

What about MeGUI calling x264, doesn't it get all confused when Distributor() is being used in the script?

I'm going to do some testing as my time allows, starting with a simple script and adding complex functions along the way. I'm particularly interested in seeing if MaskTools v2 causes any instability as Didée's cleaning function includes that one compared to my simple MVDegrain2-function.

What about global idx values, are they needed with the MT-friendly MVTools?

[vcmohan]

My search for Distributor did not yield satisfactory results. I also found no mention in Avisynth main page, external plugins. Is this a plugin or a script function? Can a reference to it be posted. If a plugin then can it be placed in new plugins, functios sticky thread?

[squid_80]

The discussion regarding Distributor started in this thead, at this post.

[IanB]

@vcmohan,

Distributor is an internal part of TSP's Avisynth_MT. It is not intended for normal use, but due to some mis-interaction with other applications it has come to light.

[tsp]

Boulder: It is unnecessary to add distibutor at the end when megui is calling x264. It shouldn't crash if added to the end of the script if the mtmode is between 1 and 4.

The global idx values are still needed to avoid problem if you call the functions more than once in a script

[Morte66]

Hi guys.

I'd like to experiment with a strong denoiser for pre-filtered clips that won't end up in the final video, e.g. to use for motion analysis with mvanalyse but not final denoising with mvdegrain2. I want something MT-friendly that won't hurt overall speed too much.

I've gotten in the habit of fft3dgpu as my fast denoiser, but I don't think it plays ball with MT; and I'm not up to speed on other denoisers. I've tried fft3dfilter, but it slows the overall script too much (and doesn't multithread that well either). Do you guys have any suggestions for a strong denoiser that's an order of magnitude faster than fft3dfilter (bt=5) and works nicely with MT?

My first thought was Convolution3D. Apparently it's 3 frame spatio-temporal, does anybody know if it works with MTMode 2?

[Boulder]

Why not try DegrainMedian, it gets a huge boost at least when wrapped inside an MT call. It's also quite powerful.

[Morte66]

Originally Posted by Boulder

Why not try DegrainMedian, it gets a huge boost at least when wrapped inside an MT call. It's also quite powerful.

I remember trying DegrainMedian about a year ago, and finding that it sort of switched off around moving edges. It's one of the things that got me into motion compensated denoising in the first place. So it might not be ideal for pre-filtering a motion analysis clip. But I'll give it a go.

I figure I ought to try a purely spatial denoiser for the pre-filtering.

[foxyshadis]

Personally, I like VagueDenoiser and some of vcmohan's filters for that. They have a large radius and are really good at highlighting edges. Alternately minblur, or a concoction with mt_edge.

[Morte66]

I tried prefiltering with degrainmedian(mode=0,limitY=8,limitUV=15).HQdn3D(4,3,6,6) and found that:
- It works fine with MT, the speed and CPU utilisation are good.
- IMO the effect was positive but not huge, arguably worth the time.
- It seems to make more difference with smaller blksize in mvanalyse. [The mvtools docs do say that mvanalyse rejects noise better for large blocks.]

I will have a go at the filters foxy suggested next.

[Zep]

ok here is one for the gurus

The below code is from a script Didee wrote and it works fine when not using setmtmode but as soon as I use it even with just a
SetMTMode(5,5) vdub/avisynth crash with an exception error. it works fine with no setmtmode.

it crashes on all the .mt_merge(alt,SAD_***,U=3,V=3)

if I comment them out no crash but of course it then does not do what it is supposed to do

I get 3x speed up when I use SetMTMode(2,5) so I would love to get this to work.

thoughts?



thanks

function Scripted_MVDegrain3(clip c, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", clip "mvbw3", clip "mvfw3",
 \                        int "thSAD", int "plane", int "limit", clip "pelclip", int "idx")
{
thSAD = default(thSAD, 400)
plane = default(plane, 4)
limit = default(limit, 255)
_idx = default(idx, -11)

thSAD = thSAD / 8

alt = c.FFT3DFilter(sigma=10,sigma2=6,sigma3=4,sigma4=2,bw=16,bh=16,ow=8,oh=8,bt=1,degrid=1,ncpu=1)

SAD_fw3 = c.MVMask(mvfw3, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_fw2 = c.MVMask(mvfw2, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_fw1 = c.MVMask(mvfw,  kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw1 = c.MVMask(mvbw,  kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw2 = c.MVMask(mvbw2, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw3 = c.MVMask(mvbw3, kind=1, ml=thSAD, gamma=0.999, Ysc=255)

comp_fw3 = c.MVCompensate(mvfw3, idx=_idx).mt_merge(alt,SAD_fw3,U=3,V=3)
comp_fw2 = c.MVCompensate(mvfw2, idx=_idx).mt_merge(alt,SAD_fw2,U=3,V=3)
comp_fw1 = c.MVCompensate(mvfw,  idx=_idx).mt_merge(alt,SAD_fw1,U=3,V=3)
comp_bw1 = c.MVCompensate(mvbw,  idx=_idx).mt_merge(alt,SAD_bw1,U=3,V=3)
comp_bw2 = c.MVCompensate(mvbw2, idx=_idx).mt_merge(alt,SAD_bw2,U=3,V=3)
comp_bw3 = c.MVCompensate(mvbw3, idx=_idx).mt_merge(alt,SAD_bw3,U=3,V=3)

black    = blankclip(c,color_yuv=$008080)
long     = interleave( comp_fw3,comp_fw2,comp_fw1,  c,   comp_bw1,comp_bw2,comp_bw3 )

long.temporalsoften(3,255,255,24,2)  
SelectEvery(7,3)
}

[Boulder]

I've also had more instability when having Masktools v2 stuff in the function I've used with SetMTMode(2).

[Zep]

Originally Posted by Boulder

I've also had more instability when having Masktools v2 stuff in the function I've used with SetMTMode(2).

trouble is for me anyway it crashes hard every time before even showing the first frame. even with a setmtmode(5) and that is something I never saw before. this is the first time that mode 5 is not safe for a call I have run into so I can not even mode 5 that part out and keep the rest of script at mode 2.

[-=KaMaL=-]

Hello everybody i have big problem with MT when i add in my script anyone script in little script also SetMTMode(2,8), in 1st pass after few mins xvid_encraw.exe is close automatically (xvid_encraw.exe is the processus of the window where i can see the status of my rip like in pourcent ) but in megui in queue for the status of the job i can see error
please help me thanks in advance also i have in processor intel duo quad q6600

[Razorholt]

can we use SetMTMode with Avisynth 2.5.8 now?

Thanks,
- Dan

[Sagekilla]

I don't believe the 2.5.8 build has SetMTmode inside of it right now. Also, the latest version in this thread seems to be 2.5.7. I could be wrong, though.

[Dreassica]

Originally Posted by badshah

I have core2duo, 1GB RAM

I installed avisynth 2.5.7

then from MT_07 package, I have put MT.dll in plugins directory & avisynth.dll from the same pack to windows/system32 folder

I am getting this error, while loading script into Vdub

AVI Import Filter error: (Unknown) (80040154)

If I put original avisynth.dll into system32 folder ... it works fine but then, I am not able to use MT

pls help

regards

I have same problem now as he does, but he didn't mention what he did to fix it. I can't for the life of it fidn the reason no script will load anymore when I use the MT modded avisnth dll.
Version() does same btw.