seggitek
25th July 2011, 17:40
I've been using the MT() and SetMTMode() filters of AviSynth for a while now. What I noticed is that due to poor coding of filters (not thread-safe) scripts often crash. You have to change multi-threading modes in your script and so on. This is pretty cumbersome and something that renders your script quite unreadable.
So what is this all about?
While studying a book called "Patterns for Parallel Programming" I noticed a pattern called "geometric decomposition". It deals with big data structures and splits them up so that processes or threads can work on different parts of the structure.
Why not let the data structure be the video we want to encode? If we have 4 processors, we can split the video stream into 4 parts with the "same" size and let each processor encode one part only.
An AviSynth script could end with this line:
GeoDeComp("mymovie",4)
The filter would have to locate all AviSynth instances running with the same title (mymovie in the example above) and determine the correct frames it has to encode. This would result in pretty much a Trim() command with different parameters in each call.
I tried the approach myself using VirtualDub and a script with Trims. I calculated the frames myself, since the fitler does not exist yet. It worked like a charm. CPU usage around maximum. No crashes. No problems. Each processor being fully utilized.
It is possible to do this manually using only 4 processors. But it becomes more errorprone the more CPUs/cores you utilize. This is why there could be a filter (GeoDeComp above) that calculates the frame numbers automatically.
One could create a batch script to start the multi-process encoding like this:
encoding-app.exe --in my-script.avs --out my-video1.avi
encoding-app.exe --in my-script.avs --out my-video2.avi
encoding-app.exe --in my-script.avs --out my-video3.avi
encoding-app.exe --in my-script.avs --out my-video4.avi
Advantages:
deterministic (CPU usage of 100% should always be possible)
muti-process approach is safer than multi-threading (poor coding of filters can be ignored)
less intrusive in scripting (AviSynth file is easier to read)
Disadvantages:
batch script for encoding necessary
result is split up into single video files that need to be joined afterwards
Is this easy to implement or are there even better solutions rendering the batch file unnecessary? Ideas are welcome.
So what is this all about?
While studying a book called "Patterns for Parallel Programming" I noticed a pattern called "geometric decomposition". It deals with big data structures and splits them up so that processes or threads can work on different parts of the structure.
Why not let the data structure be the video we want to encode? If we have 4 processors, we can split the video stream into 4 parts with the "same" size and let each processor encode one part only.
An AviSynth script could end with this line:
GeoDeComp("mymovie",4)
The filter would have to locate all AviSynth instances running with the same title (mymovie in the example above) and determine the correct frames it has to encode. This would result in pretty much a Trim() command with different parameters in each call.
I tried the approach myself using VirtualDub and a script with Trims. I calculated the frames myself, since the fitler does not exist yet. It worked like a charm. CPU usage around maximum. No crashes. No problems. Each processor being fully utilized.
It is possible to do this manually using only 4 processors. But it becomes more errorprone the more CPUs/cores you utilize. This is why there could be a filter (GeoDeComp above) that calculates the frame numbers automatically.
One could create a batch script to start the multi-process encoding like this:
encoding-app.exe --in my-script.avs --out my-video1.avi
encoding-app.exe --in my-script.avs --out my-video2.avi
encoding-app.exe --in my-script.avs --out my-video3.avi
encoding-app.exe --in my-script.avs --out my-video4.avi
Advantages:
deterministic (CPU usage of 100% should always be possible)
muti-process approach is safer than multi-threading (poor coding of filters can be ignored)
less intrusive in scripting (AviSynth file is easier to read)
Disadvantages:
batch script for encoding necessary
result is split up into single video files that need to be joined afterwards
Is this easy to implement or are there even better solutions rendering the batch file unnecessary? Ideas are welcome.