Cannot get Autolevels() to multithread

[tedkunich]

Autolevels 0.3:

Cannot get multithreading to happen (CPU utilization ~25% on a quad core) with this filter.

No MT:
avisource("film")
autolevels()


SetMTMode:
SetMTMode(5)
avisource("film")
SetMTMode(2)
autolevels()


MT():
avisource("film")
MT("autolevels()")


Deliver all the same FPS (around 20) rendering in VD with the LAGS codec. Avi source is a HDV cam video cropped to 1014x1014. (FWIW, this was found to be the bottleneck in VideoFred's 8mm film restoration script - comment out "autolevels" and I can process at 100 CPU utilization @ 5fps. With autolevels, it drops to 1-2 fps @25%)

The source is straight C++, so I am wondering why this filter will not multithread. Can anyone suggest any options?

Thanks,

Ted

[Gavino]

For each output frame, AutoLevels has to find the luma range used in all frames within the specified radius (default 5). So when multithreading with SetMTMode, most of the work is repeated in each thread and there is little gain.

With MT("autolevels"), I would expect at least some speed improvement, but I would also expect artefacts to appear as it will be applying the autolevels independently to different sections of each frame.

[Didée]

MT() just doesn't make any sense with this filter - when each frame is split in several sections, you surely don't want to have different gains for each frame section. "Spatial multithreading" is not an option.
(However, while the result should be unusable, at least MT() should achieve a speed increase. Shouldn't it?)

"temporal multithreading" ala SetMTmode has another problem: this filter averages the amount of "gain" over consecutive frames
which implies that the threads (as spawned by setmtmode) can not work independently. Due to autolevels' internal temporal dependencies, the whole SetMTmode thingy collapses into a sort of serial execution. (Thread#4 waits for thread#3 waits for thread#2 waits for thread#1 to finish.)


Solution? Well .... it's as easy as re-writing the filter so that it allows external multithreading, or alternatively, does the multithreading directly in the filter! :D

Perhaps it would be possible to use PipeLine() or ThreadRequest to branch-out AutoLevels from the main processing thread, to get at least some acceleration. Might be worth a try.
(But then, better don't ask me ... I'm a newbie with this MT stuff, and am talking about things that I haven't fully understood yet.)

[Gavino]

the threads (as spawned by setmtmode) can not work independently. Due to autolevels' internal temporal dependencies, the whole SetMTmode thingy collapses into a sort of serial execution. (Thread#4 waits for thread#3 waits for thread#2 waits for thread#1 to finish.)
No, I don't think so. The problem is more that the threads do work independently, so each input frame is being analysed independently in each thread and there is no sharing of information between them. There is no temporal dependency between the threads, but there will be a bottleneck as each thread may have to wait for another when reading a frame from the source filter.
Solution? Well .... it's as easy as re-writing the filter so that it allows external multithreading, or alternatively, does the multithreading directly in the filter! :D
Autolevels maintains its own internal cache of the levels found in each frame, but each instance has its own cache. Sharing this between instances (in a thread-safe way, of course) would speed things up when used with SetMTMode.

[Didée]

@Gavino - Okay. If if would understand the matter, then probably I would have said the same. :D

My suggestion for the time being: Try to re-order the script so that AutoLevels either is very early in the chain (the 1st filter, ideally), or that it is very late in the chain (the last filter, ideally). With a little fiddling, chances are good that Fred's script still can deliver same-good results with that modification.

Point#1 is that AutoLevels on its own is a fairly fast filter. At least it seems to me - with just AutoLevels(), I get 110~130 fps on a 720p source. Ain't that fast enough?

Point#2 is that with such a re-ordering, you can rather easily take AutoLevels out of the multithreaded chain (setMTmode 5), while still running the major part of the script multi-threaded (setMTmode 2).

[Gavino]

Try to re-order the script so that AutoLevels either is very early in the chain (the 1st filter, ideally), or that it is very late in the chain (the last filter, ideally).
...
Point#2 is that with such a re-ordering, you can rather easily take AutoLevels out of the multithreaded chain (setMTmode 5), while still running the major part of the script multi-threaded (setMTmode 2).
I don't follow this.
If I understand correctly, putting SetMTMode(5) before the last filter in the chain will cause the entire chain to be run single-threaded, as all other threads will be blocked until that filter's GetFrame returns.

Further, if a filter works with SetMTMode(2), I don't see how overall performance can be improved by using SetMTMode(5), no matter where it is in the chain.

[Didée]

You're correct. Having the stumblestone at the end of the script is not a good idea. But having it at the beginning, that's much better than having it somewhere in the middle.


Script#1: no AutoLevels at all

setmtmode(5,8)

o = mpeg2source("test-720p.d2v")

setmtmode(2)

sup = o.msuper()
bv2 = sup.manalyse(isb=true,delta=2,search=4)
bv1 = sup.manalyse(isb=true,delta=1,search=4)
fv1 = sup.manalyse(isb=false,delta=1,search=4)
fv2 = sup.manalyse(isb=false,delta=2,search=4)

stage1 = o.mdegrain2(sup,bv1,fv1,bv2,fv2)

sup2 = stage1.msuper()
bv11 = sup2.manalyse(isb=true,delta=1,search=4)
fv11 = sup2.manalyse(isb=false,delta=1,search=4)

stage2 = stage1.mdegrain1(sup2,bv11,fv11)

return(stage2)

~27 fps , 95~98% CPU


Script#2: Autolevels() somewhere in the middle. Keep using SetMTmode(2) for the whole construct.
setmtmode(5,8)

o = mpeg2source("test-720p.d2v")

setmtmode(2)

sup = o.msuper()
bv2 = sup.manalyse(isb=true,delta=2,search=4)
bv1 = sup.manalyse(isb=true,delta=1,search=4)
fv1 = sup.manalyse(isb=false,delta=1,search=4)
fv2 = sup.manalyse(isb=false,delta=2,search=4)

stage1 = o.mdegrain2(sup,bv1,fv1,bv2,fv2)
stage1 = stage1.AutoLevels(8)

sup2 = stage1.msuper()
bv11 = sup2.manalyse(isb=true,delta=1,search=4)
fv11 = sup2.manalyse(isb=false,delta=1,search=4)

stage2 = stage1.mdegrain1(sup2,bv11,fv11)

return(stage2)

~9.2 fps , 20~25% CPU. That's not good.


Script#3. AutoLevels somewhere in the middle. Use SetMTmode(2) for everything BUT AutoLevels, which uses SetMTmode(5).

setmtmode(5,8)

o = mpeg2source("test-720p.d2v")

setmtmode(2)

sup = o.msuper()
bv2 = sup.manalyse(isb=true,delta=2,search=4)
bv1 = sup.manalyse(isb=true,delta=1,search=4)
fv1 = sup.manalyse(isb=false,delta=1,search=4)
fv2 = sup.manalyse(isb=false,delta=2,search=4)

stage1 = o.mdegrain2(sup,bv1,fv1,bv2,fv2)

SetMTmode(5)
stage1 = stage1.AutoLevels(8)

SetMTmode(2)
sup2 = stage1.msuper()
bv11 = sup2.manalyse(isb=true,delta=1,search=4)
fv11 = sup2.manalyse(isb=false,delta=1,search=4)

stage2 = stage1.mdegrain1(sup2,bv11,fv11)

return(stage2)

~15.3 fps , ~41% CPU. That's better, but not yet good.


Script#4: Autolevels with SetMTmode(5) at the very beginning. All the rest with SetMTmode(2).

setmtmode(5,8)

o = mpeg2source("test-720p.d2v")
o = o.AutoLevels(8)

setmtmode(2)

sup = o.msuper()
bv2 = sup.manalyse(isb=true,delta=2,search=4)
bv1 = sup.manalyse(isb=true,delta=1,search=4)
fv1 = sup.manalyse(isb=false,delta=1,search=4)
fv2 = sup.manalyse(isb=false,delta=2,search=4)

stage1 = o.mdegrain2(sup,bv1,fv1,bv2,fv2)

sup2 = stage1.msuper()
bv11 = sup2.manalyse(isb=true,delta=1,search=4)
fv11 = sup2.manalyse(isb=false,delta=1,search=4)

stage2 = stage1.mdegrain1(sup2,bv11,fv11)

return(stage2)

~26.8 fps , 90~98% CPU.

Deal.

***
Edit - oops! Forgot one scenario!

"SetMTmode(2) for everything, with Autolevels at the beginning"

~23~26.8 fps , 70~98% CPU.

(Don't ask me why those fluctuations. Did run it 3 times, and 3 times the same.)

[tedkunich]

You're correct. Having the stumblestone at the end of the script is not a good idea. But having it at the beginning, that's much better than having it somewhere in the
middle.

snip....


Thanks Didee, you are the Man!!

In regards to Fred's restoration script, I'll probably just break it into two scripts - all the denoising and sharpening really need MT - I see about 5-6fps on my Phenom 940 with my 1440x1080 HD source when I do not use the autolevels function. It is structured in the script as a final processing step, so it is easy to break it out.

Thanks,
Ted

[tedkunich]

Thanks Didee, you are the Man!!

In regards to Fred's restoration script, I'll probably just break it into two scripts - all the denoising and sharpening really need MT - I see about 5-6fps on my Phenom 940 with my 1440x1080 HD source when I do not use the autolevels function. It is structured in the script as a final processing step, so it is easy to break it out.

Thanks,
Ted

I played with Fred's script and found that if I called the distributor() function prior to the autolevels() call, I am able to sustain 5-6fps (source 1440x1080, final dest 720x480) With the Autolevels() call in place, whereas before I was only seeing 1-2fps.

Now that I am happy with the performance of the post process, I need to devote my attention to my film capture rig and optimizing that setup. :scared:

T

[um3k]

ThreadRequest() works with AutoLevels(). Might be a viable alternative to SetMTMode for scripts containing the filter.