Exploiting natural parallelism for multitasking

[pbristow]

Hi again.

I've been dabbling with SetMTmode, and the functions MT and MTi, in my recent work. Basically there are two ways in which the MT-equipped Avisynth can divided up the data to allow parallel processing:
- Time-wise (e.g. MTi() splits the source data into two streams of alternate frames, then re-interleaves them after processing),
- Space-wise (e.g. MT() splits clips either horizontally or vertically, then stitches them together after processing)


In the the job I'm currently doing, neither of these approaches is really ideal - Although I did come up with a trick to exploit MTi quite efficiently (Split the source file in two equal-length halves using Trim(); Interleave these halves into a single (very odd!) clip; use MTi() to re-separate, process and re-interleave the two unrelated streams of frames; Use SelectOdd and SelectEven to recover the original halves, and then use "+" to join them back together. Lots of needless splitting, re-ordering and re-joining going on, but it works!)

However, the data I'm working with has natural parallelism that could be exploited. I'm running pairs or triads of identical or similar functions, on pairs or triads of similar but separate clips, e.g. those created by:

Red = ShowRed(Source)
Green = ShowGreen(Source)
Blue = ShowBlue(Source)

What I *can* do is make a separate script for each of these three data streams, run them in separate instances of Virtuladub, encode them as AVI files and then knit them together with a fourth script... But that's very unwieldy and uses up lots of disk space for the temporary files.

What I'd like is a construct in AviSynth that's a bit like a select or case statement in other languages, but which is used to bracket together and list chunks of code (even if they're only single function calls) that can be safely run in separate threads. e.g.:


Source = AviSource("whatever.avi")

Parallel {
Thread:
Red = ShowRed(Source)
R2 = FancySlowFilter(Red, mode=1).SpatialSoften(1,12,10)
Thread:
Green = ShowGreen(Source)
G2 = FancySlowFilter(Green, mode=2)
Thread:
Blue = ShowBlue(Source)
B2 = FancySlowFilter.MassiveBlur.(Blue, mode=1)
}


Is that feasible? Does anyone else think it would be useful?

[Zarxrax]

What I *can* do is make a separate script for each of these three data streams, run them in separate instances of Virtuladub, encode them as AVI files and then knit them together with a fourth script... But that's very unwieldy and uses up lots of disk space for the temporary files.


I think there is no need to output temporary files there. In the fourth script, just load the other 3 scripts directly.
Now, I'm not certain on this, but if you use Import() to import the other scripts, I believe they will all share the same process, but if you use AviSource() to load them, I think they might use different processes. At least its worth a try.

[pbristow]

Thanks Zarxrax (Did I spell that right? Can't tell 'til I get out of this reply window), I'll give that a try. It could be useful for the simpler cases, although in my current job there's lots of branching out into parallel sections, then recombining for linear sections, then branching out a different way, then recombining... Trying to MT the parallel bits by this technique would be quite unwieldy, with a dozen or so scripts all calling each other!

But it's always good to have another tool in one's toolkit, so thanks for the idea. :)

No one seems to be jumping up down with excitement at my "Parallel" idea... Ho hum.

(GOES OFF TO TRY Z's IDEA OUT)