MT 0.7(+custom avisynth)a filter to run filters multithreaded. Usefull for SMP or HT

[Boulder]

Ok, will do as soon as I've got the time. I'll try MT on each part of the script and see if anything blows up.

[Boulder]

Hmm, how can I use MT with this:

mt_lutxy(orig,NR,yexpr="x y - 128 +",uexpr="y",vexpr="y",Y=3,U=3,V=3)

I always get the "invalid arguments to MT" error. I tried MT("""mt_lutxy(orig,NR,yexpr="x y - 128 +",uexpr="y",vexpr="y",Y=3,U=3,V=3)""",2)

Also this gives an error regardless of what I've tried:
_repair ? repair(comp,clp,1,3) : comp
I've tried repair with single and triple quotes.

I tried the MaskTools v1.5-branch and MaskTools v2-branch versions of the script. They both failed so it might be that MVTools v1.0.3 is actually the culprit. I cannot try Manao's last MVTools version (v0.9.9.1) as it doesn't work with the script.

[tsp]

you can't. MT only accepts 1 input clip. I will need to write a custom scriptparser to handle more than one input also you can't use a named input like this:

a=last.sharpen(0.5)
MT("blur(a,1)")

so mayby comment out the lines that you want to test?

[Kador]

Hello
Why do I get a "MT does not have a named argument"avisynthTS", which I did not get before ?

[Boulder]

Originally Posted by tsp

so mayby comment out the lines that you want to test?

OK, a really quick-and-dirty test showed that MVFlow might be the one that doesn't like MT.

MVTools v1.0.3 can be found at least here : http://home.arcor.de/dhanselmann/_stuff/

[tsp]

Kador: Because version 0.5 requires the included avisynth.dll(or 2.6 when it is released) so I removed that argument.

Boulder: I suspected that MVTools might cause it. The sourcecode is rather complicated so it might take a while until then you could try to use cloudeds motion filter instead (http://forum.doom9.org/showthread.php?t=101859)

[Kador]

Thanx I removed it.

After an update, I'm using the latest filters in ZP+ffdshow :
- masktools2
- MT + avisynth dll (from you link)
- the second LSF from the wiki page
- hqdn3d
I get a crash every time after several seconds of read, of course no crash without those filters ...

if I use the LSF call without MT no crash

any idea ?

[tsp]

how does your script look like? Try commenting/removing the different filters and see if that helps(and post what filter that causes the crash).

[Kador]

My script is very simple

MT("hqdn3d(2)LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=50,overshoot=1)")

[Socio]

Originally Posted by Kador

My script is very simple

MT("hqdn3d(2)LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=50,overshoot=1)")

Try it like this:


MT("hqdn3d(2).LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=50,overshoot=1)")

or this:

MT("hqdn3d(2)
LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=50,overshoot=1)")

[tsp]

Kador: I made it crash with that script now I only needs to figure out why it does it

tsp,

There seems to be an incompatability between the version of avisynth included with MT05 and the version of avisynth required for post 18 versions of masktools (see the limited sharpen thread) - a similar discussion is occuring there also..

Will AviSynth 2.5.6a (28th oct 2005) or AVS 2.5.6 RC2 (7th Oct 2005) both on sourcefourge work with MT05 or only the included version?

Cheers,
Bitey

[Kador]

Originally Posted by tsp

Kador: I made it crash with that script now I only needs to figure out why it does it

Great (that you reproduced, I'm not alone ;-) )

[swaaye]

I'm running a dual core Opteron and am more than a little fascinated by this MT filter. I've read through a bunch of this thread and I'm more than a little unsure just what the limitations or ramifications of using this are.... I'm using a pretty simple script to work on some episodes of a TV show:

MPEG2Source("R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v", idct=7)
tfm(d2v="R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v")
tdecimate(hybrid=1)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10, aspect=-1,threshold=34,samplestartframe=100,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
BicubicResize(640,480,0,0.75)

This, along with multithreaded Divx 6.1.1, puts me around 75%+ CPU, sometimes nearly 100%. But there is room in there for more CPU use, so I thought MT would be worth a try.

Does what I'm using look convertable for use with MT?

[tsp]

Mr.Bitey: So you say that it crash when using my modified avisynth.dll and the latest masktool when you don't use mt or setmtmode? I know that it crach when using mt and masktools together but exactly what goes wrong is not clear yet (other than it looks like a problem with the STL allocator in masktool. I did use a couple of hours yesterday with manao to try debug it).
mt only works with the included avisynth or a fresh build from the latest avisynth 2.6 CVS.

swaaye:
if you want to try I will sugest this script:

Setmtmode(2)
MPEG2Source("R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v", idct=7)
tfm(d2v="R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v")
tdecimate(hybrid=1)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10, aspect=-1,threshold=34,samplestartframe=100,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
BicubicResize(640,480,0,0.75)

with mt 0.5 and the included avisynth.dll.

tsp,

It crashes with a21 of masktools (and I believe any version after a18 (which is ok)) when using setmtmode and MT. I havent tried it without setmtmode or MT..

Glad to hear you and manao are on the case - im sure a workaround or fix wont be long! :-)

Cheers,
Bitey

[Manao]

I'm not that optimistic. I don't own a smp cpu, and i can barely manage to make MT + masktools crash on my computer ( I've succeeded only thrice, and without getting any useful information on what may be going on ).

Tsp is able to systematically reproduce the bug on his computer, so I can give him some custom builds to try, but in any case it'll be tedious, and long, for both of us.

Tsp : I've reviewed this thread entirely, and two nice threads it linked to, which concerned the changes you made on avisynth, and the ugly MakeWritable issue.

Masktools > v2.0a18 might actually suffer from that same issue ( I stumbled on it the before, found a quick and dirty hack without knowing what was really the problem, but now with the nice explanation of IanB and you, I get what's happening, why my hack worked ).

Here is how I get the frames my filters will be needing :

// First, the current frame, processed differently since most filters work in place
PVideoFrame dst = T::is_in_place() ? childs[0]->GetFrame(n, env) : env->NewVideoFrame(vi);

if ( T::is_in_place() )
  env->MakeWritable( &dst );

// request all plane pointers, through GetWritePtr

// then, the other source frames, all read only
std::vector<PVideoFrame> frames;
for ( int i = 0; i < k; i++ )
  frames.push_back(childs[f(i)]->GetFrame(g(i), env));

// request all plane pointers, through GetReadPtr

// do the processing

frames.clear();

return dst;

All masktools filters work alike. f(i) and g(i) say that the ith frame get the g(i)th frame from the f(i)th child.

I don't see anything obviously wrong with the code above.

So, what I'll do is make a "full debug log" build of the masktools, one that in particular checks each plane pointers that are given to the filters. I'll put some DebugPrintf() around the parts that strongly arise suspicion ( especially the T::filter_signature() one ). Whenever you ready, I can send it to you. I expect a huge debug ouput, but hopefully, only the few last hundred lines should be useful.

So feel free to bug me whenever you have the time to do so. The debug version is available here : http://manao4.free.fr/mt_masktools.dll

At all others with smp computers, and willing to help. You can also try that version. Just download the debug version, and DebugView.exe ( first hit on google ), launch it, then virtual dub, then load the faulty script and play it. When it crashes, the DebugView windows will have some information on it. You can send it to me ( manao <at> melix <dot> net, you don't necessarily need to send all, but if you don't send all, select in priority the end of the log ).

[vanessam]

Originally Posted by Manao

So feel free to bug me whenever you have the time to do so. The debug version is available here : http://manao4.free.fr/mt_masktools.dll.

Zoom player crashed with this dll. it closed alone.

[swaaye]

Originally Posted by tsp

swaaye:
if you want to try I will sugest this script:

Setmtmode(2)
MPEG2Source("R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v", idct=7)
tfm(d2v="R:\DVD\TNGS6D6\VTS_02_PGC4\TNGS6D6.d2v")
tdecimate(hybrid=1)
autocrop(mode=0,wmultof=4,hmultof=4,samples=10, aspect=-1,threshold=34,samplestartframe=100,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
BicubicResize(640,480,0,0.75)

with mt 0.5 and the included avisynth.dll.

I tried it out and it did hold the CPU at nearly 100% the whole time. Only gave maybe a few % gain in speed though. I'd say it was pratically equally as fast as without it. Very interesting plugin though and certainly useful in the right situation.

[Manao]

OK, thanks a lot Boulder for the debug log.

Here is another dll : http://manao4.free.fr/mt_masktools2.dll

There no guarantee at all that it shouldn't crash, but at least the debug log he sent me pinpoint without a doubt where the crash was occuring. So if that build doesn't work, I still might be able to find the bug.