MVTools

[Fizick]

Originally Posted by mikeytown2

Is there any way to improve MVAnalyse?

Yes, I am sure. But I do not know how exactly

More seriously: I consider how to use blocks of different sizes.
It is knewn method (macroblocks, etc). For big features it is better to use big blocks (stable and robust motion estimation), but for small features we need in small blocks.
To do no broke and rewrite MVTools (which coded with constant block size only approach), i try to add function MVRefine for two-stage estimation.

First stage: coarse_vectors=MVAnalyse(blksize=big)
Second stage: fine_vectors=MVRefine(vectors, blksize=small, thSAD=200)
Old vectors are used as a predictors.
This way we will preserve good vectors (update their SAD only),
and replace (re-estimate) bad vectors. Old vectors are used as a predictors. Probably refining is at finest level only.

I almost finished the first beta, hope can sent it to beta-testers soon

Of course, I am not sure, that results will be excited.
But it certainly will be slower.

[Ranguvar]

Of course it'll be slower xD It's something of an AviSynth law...

but seriously, thanks for your great work You, josey_wells, Manao, everyone

[josey_wells]

Originally Posted by Fizick

I tried latest version on old Ahtlon 64, so I use classic MVAnalyse-MVDegrain2 script.
The speed is about 8 fps (blksize=16, overlap4).
But I have 10 fps with v1.9.5.7 for same script.
BTW, I recompile v1.9.6.7 with my development env (VCToolkit2003), but speed is still 8 fps.
Disabling mt=false do nothing.

I had disabled the BitBlt isse routines and doing memcpy only. The isse routines have been reinstated.

I finally got the rest of what I feel was missing to the multithreaded implementation of MVTools. Here is the new version.

http://rapidshare.com/files/13913936...9.6.8.rar.html

This version does away with the MT flag to control multithreading and adds the following two parameters for MVAnalyseMulti, MVDegrainMulti, MVDegrain1, MVDegrain2, MVDegrain3:

1. PreFetch - This will prefetch x frames before starting the threads for processing(default is 1)
2. Threads - This is the maximum number of threads to run concurrently(Default is 64) i.e all threads at once.

For example with the following line
MVMulti=last.MVAnalyseMulti(refframes=2, PreFetch=2, Threads=4)

There can be at most 2*refframes*prefetch=8 threads at once, and at most 4 are running at a time.

For example with the following line
MVDegrainMulti(MVMulti, PreFetch=2, Threads=4)

There can be at most 3*prefetch=6 threads at once, and at most 4 are running at a time.

Some notes:
1. When any thread completes and more data is to be processed a new thread is immediately started
2. Any prefetch greater than 1 will cause results to be cahced and the result return if still within cache. this will be seen as incrementing by prefetch in virtualdub since the cached result is returned immediately

As an example of a script which sets the parameters on a quad core.

#motion denoise
function Denoise(clip c, int "iLevel", int "iTHSAD", int "iBlksize", int "iOverlap", int "iPel", bool "bChroma", \
		      bool "bTrueMotion", int "iSearch", int "iSharp", int "iDct", int "iPrefetchAnalyse", \
		      int "iPrefetchDegrain", int "iThreads", int "iQueuedThreads")
{
    iLevel          =default(iLevel, 2)                    #default level for degrain 
    iBlksize        =default(iBlksize, 8)                  #default block size
    iOverlap        =default(iOverlap, iBlksize/2)         #default 1/2 block size
    iTHSAD          =default(iTHSAD, 400)                  #default
    iPel            =default(iPel, 4)                      #quarter pixel
    bChroma         =default(bChroma, true)	           #use chroma
    bTrueMotion     =default(bTrueMotion, true)            #use true motion
    iSearch         =default(iSearch, 2)                   #diamond search
    iSharp          =default(iSharp, 2)                    #6 tap sharp filter
    iDct            =default(iDct, 1)                      #use DCT for SAD 
    iThreads        =default(iThreads, 4)                  #max number of threads
    iQueuedThreads  =default(iQueuedThreads, iThreads)     #number of queuedthreads
    iPrefetchAnalyse=default(iPrefetchAnalyse, LCM(2*iLevel, iQueuedThreads)/(2*iLevel)) 
    iPrefetchDegrain=default(iPrefetchDegrain, LCM(3, iQueuedThreads)/3)                

    MVMulti=c.MVAnalyseMulti(refframes=iLevel, pel=iPel, blksize=iBlksize, overlap=iOverlap, chroma=bChroma, \
			     truemotion=bTrueMotion, search=iSearch, sharp=iSharp, dct=iDct, Threads=iThreads, \
			     prefetch=iPrefetchAnalyse, idx=1)
    return c.MVDegrainMulti(MVMulti, thSAD=iTHSAD, Threads=iThreads, prefetch=iPrefetchDegrain, idx=1)  
}

# Computes the Greatest Common Divisor of two integers
# (Euclid's Algorithm)
function GCD(int a, int b)
{
    return (b == 0) ? a : GCD(b, a % b)
}

# Computes the Least Common Multiple of two integers
function LCM(int a, int b)
{
    return (a*b) / GCD(a, b)
}

This will prefetch the minimum amount of frames so that the total threads avaialbe is an integral multiple of the number of threads allowed to queued.

Hello I have found a bug.
Download this steam http://rapidshare.com/files/13884330...muxed.zip.html
Load it (MPEG2 file) in avisynth and add those lines to the avisynth script:
source=last
MVForBack = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=2, overlapv=2, sharp=1, idx = 1,blksize=16,search=3,chroma=true,global=true)
source.mvshow(MVForBack)

Look at the frame just before the scenechange, motiondetection is totaly broken.

The line for show should be
source.MVShow(MVForBack.MVMultiExtract(0)) #show bac1

With this build I acheived about a 10-20% improvement and up to 80-90% improvement depending on script.

You may have to set SetMemoryMax down to a value such as 256 is memory problems arise since more prefetching cost more memory and under Win32 memory is limited to about 2GB.

Have fun!

[mikeytown2]

@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.

[josey_wells]

Originally Posted by mikeytown2

@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.

I have found the problem and it has to do with multiple MVAnalyseMulti or MVDegrain in the script.

I will post fix tomorrow.

[morsa]

Could this be of any use?

http://research.microsoft.com/~larryz/ZitnickICCV05.pdf


A question: What would happen if one were analizing a wavelet decomposed image, then interpolating it and recomposing it again?
Could it give any improvements quality wise?

[josey_wells]

Originally Posted by mikeytown2

@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.

Here is the fix. I skipped V1.9.6.9 since I consdider this a major release.

http://rapidshare.com/files/13925780...9.7.0.rar.html

I now consider all optimizations to MVAnalyseMulti and MVDegrain's completed.

If there is enough interest then I can look at multithreading some of the other functions but need a break first. I will also post complete comparisons of this code vs V1.9.5.7 soon disregarding all white space changes.

Have fun!

[Adub]

If I may, but why is the "Threads" option default to 64? Why not have it default to the number of cores found or something along those lines?

Edit: And also, thank you very much for your hardwork!! By all means, take a break, you deserve it!

Edit2:

Okay, bug report from kokotier:

<kokotier> hello
<kokotier> i saw you have asnwered
<kokotier> to the mvtools thread
<Merlin7777> yes?
<kokotier> canyou tell to josey_wells than the last buid is buggy
<kokotier> http://forum.doom9.org/showthread.ph...74#post1172974
<Merlin7777> Wait, did you read his reply?
<kokotier> yes but is doesnt change anything
<kokotier> motiondetection is broken
<kokotier> you can test yourself
<kokotier> dl my sample
<kokotier> and on the latest buid i have green frames with it :-
<Merlin7777> It's having trouble connecting. But didn't you post it online? Anyways, I will post and tell him that you are still having problems. Same script?
<kokotier> i cant post on doom9 sorry
<kokotier> i hve to wait more days
<Merlin7777> Yeah, I know. But are you using the same script? Or are you also encountering the bug with the suggestion he posted?
<kokotier> source=last
<kokotier> MVForBack = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=2, overlapv=2, sharp=1, idx = 1,blksize=16,search=3,chroma=true,global=true)
<kokotier> source.MVShow(MVForBack.MVMultiExtract(0))
<kokotier> with the suggestion i have the same bug
<kokotier> it seems that is overlapv=2 that is buggy

Edit3:

its weird
<kokotier> with refframe=2
<kokotier> 2 frame before the scene chnge is bad
<kokotier> with refframe=3
<kokotier> 3 frame before the scene change is bad
<kokotier> and so on..

Edit 4: (man, I am really going to town on this thing.)

Kokotier, (who is in the waiting period after registering, by the way) walked me through what he is doing, and I have to agree, it is a bit odd. with Refframe=1, the frame before the scenechange is filled with motion vectors, even though there is no motion. But since it is a scenechange, it might be that way on purpose.

However, that's not the issue. As Kokotier pointed out, if you change the refframe to 2, then two frames before the scenechange is filled with motion vectors, then a "good frame", aka, no motion vectors, THEN the scenechanges. It is some odd behavior I must say. But I don't necessarily know if it is wrong. So I will let the experts decide.

[morsa]

I have found that if I use the histogram equalization filter from VCmohan at the MVanalyze stage, results get improved a lot.
Does it sound logical to use histogram equalization internally Fizick?

[thetoof]

Originally Posted by josey_wells

If there is enough interest then I can look at multithreading some of the other functions but need a break first.

Thanks a lot! Could you also consider updating the docs? (well, you or anyone feeling like it )
mvflowfps would be the next function to multithread imo...

[Sagekilla]

@morsa: How exactly did you use the equalization, and how much did the results improve?

[Delerue]

Originally Posted by thetoof

Thanks a lot! Could you also consider updating the docs? (well, you or anyone feeling like it )
mvflowfps would be the next function to multithread imo...

I think the same.

EDITED: BTW, I downloaded the last version, and use it with this old script, but I got crashes all the time:

SetMtmode(1,5)
source=ffdshow_source()
SetMTMode(2)
LoadPlugin("C:\arquivos de programas\avisynth\plugins\mvtools.dll")
backward_vec = source.MVAnalyse(blksize=16, overlap=0,  isb = true,  pel=2, search=2, idx=1)
forward_vec = source.MVAnalyse(blksize=16, overlap=0, isb = false, pel=2, search=2, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=2*FramerateNumerator(source), \
   den=FramerateDenominator(source), mask=1, idx=1)
distributor()

I tried to remove the MT lines, but it doesn't help. Also tried other CPU versions of MVTools (although I have a Core2Duo), and it doesn't help either.

[egrimisu]

for using the next code in multithread :
source = last
backward_vec3 = source.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)

should be called as :
source = last
backward_vec3 = source.MVAnalysemulti(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalysemulti(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalysemulti(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalysemulti(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalysemulti(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalysemulti(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)

[Dreassica]

Using that line I'm gettting a "MVAnalysemulti has no named argument "isb"" error msg.

[Sagekilla]

Because that's the incorrect usage of MVAnalyseMulti. It has no "delta" or "isb" syntax anymore, since a single instance of MVAnalyseMulti is supposed to be used as so:

# Ref frames is your delta, but it looks backwards AND forwards at the same time. No need for isb=true/false
vectors = source.MVAnalyseMulti(refframes = 3)

I don't remember if vector extraction has been implemented, but it was supposed to be done as so if you want old school MVDegrain:

vectors = source.MVAnalyseMulti(refframes=1)
bvec1 = vectors.MVMultiExtract(0)
fvec1 = vectors.MVMultiExtract(1)
source.MVDegrain1(bvec1,fvec1)

[egrimisu]

sorry, i'm a hard head so what will be the code in the new version for the code that writen up?

[mikeytown2]

old

source = last
backward_vec3 = source.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)

NEW

source = last
vectors = source.MVAnalyseMulti(refframes=3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

refframes=1 is MVDegrain1()
refframes=2 is MVDegrain2()
refframes=3 is MVDegrain3()
refframes=4 is MVDegrain4()
ect...

[egrimisu]

There's no need for adding delta = 3 anymore?

[Sagekilla]

Nope. Delta is now refframes, but refframes encompasses both delta and isb. There's no need to set as being backwards or forwards vector since it stores both in the same clip.

[bairradino]

@ Mikeytown2

Your code

source = last
vectors = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)

gives the following error:
there is no function named "MVAnaliseMulti".