about MVtools...

[morsa]

@Manao:
Could any of this be usefull for MVinterpolate??

http://www.utdallas.edu/~aria/mcl/motion.html

[Manao]

This interdependence necessitates a costly iterative approach to the computation of motion. [...] This is not a trivial task, since a straight forward insertion of BMA motion vectors in the mesh model leads to unpredictable and erratic results. Yes, it could be useful, but no, I won't implement it, because results will be too slow and too complicated to use.

[morsa]

Ohh, sorry didn't know it or understood it. :(

[Fizick]

Manao, thanks for your hard work!
1. As i understand, you still have no plan to include overlapped blocks motion estimation and compensation?
(for deblocking)

2. So, may i consider your vectors format as stable?
I want try use vectors from MVTools for global motion estimation for Depan as alternative method.
Or MVtools have some internal global motion estimation?

[Fizick]

Manao, i still will post here about development :)

I developed first beta (alpha) version of MVtools to Depan interface, as MVDepan function of MVTools.
The global motion extraction algorithm is still not very clever and robust in this version, but it works as idea.

I used v.0.9.8 as base.
I also add MVDepan function description to updated doc.
http://bag.hotmail.ru/mvtools/mvtoolsdepan.zip
I add 2 files (MVdepan.cpp and MVdepan.h) and add some lines to interface.cpp.

May we use this method of MVTools-Depan interface?
Or it is inconvenient for development and using?

I also try use v 0.9.8.1, but got compiler (VC ++ Toolkit 2003) error:
...MVCore.h(9) : fatal error C1083: Cannot open include file: 'boost/shared_ptr.hpp': No such file or directory

[Manao]

Sorry about the "boost/shared_ptr", i forgot to delete it before releasing the filter ( i was experimenting a little )

I've release MVTools v0.9.8.2 (http://manao4.free.fr/MVTools-v0.9.8.2.zip) containing MVDepan. If you make modifications, make them from that package ( portability.h changed a little, it won't suffer from the ugly bugs that were in 0.9.8 ( it's a miracle that it was working ... ), and boost won't bother you again )

It will be the last release of the MVTools for some time.

[Fizick]

Thanks, it is compiled and works now!



BTW, do you know about DIRAC project?
http://dirac.sourceforge.net/
It has some motion vectors code.

[scharfis_brain]

I want to place a complete different question here.
I wanted to write some AVS-stuff that is able to handle the mismatches of mvtools.

my intention is, to detect the deviations of full compensated frames vs. their uncompensated originals.

so, one is able to replace extreme mismatches in a compensated frame with its original areas.

the current thing I have written in AVS is only able to do this with one compensated clip.

but if I want to use two or three compensated clips
(depanned, fwd-compensated, bwd-compensated)
I really don't know how to handle this with AVS.

Is there a possibility to get such thing written as plugin for AVS?

possible syntax:

corrector(clip original, clip compensated1, clip "compensated2", clip "compensated3", clip "compensated4", int threshold)

where the first compensated clip has the most priority above the following ones.

if all clips are matching the original, the clips are getting blended together.
if only one is not matching, all clips exept the non-matching one are mixed together.
if only one clip is correct, only this correct one will be returned.
if no clip matches, the original is given to the output.

(of course, not framewise, all is meant to be pixelwise.)

I hope, this wasn't too confusing

[Manao]

I've got some free time, so here it is :

http://manao4.free.fr/MVTools-v0.9.8.3.zip

Filter's name is Corrector. It can take any number of clips as input. Threshold's name is simply 'th'.

[scharfis_brain]

That's great!
I didn't expected such fast reaction!

but unfortunately I won't be able to test this until next weekend :(

but maybe another idea:
only return the clip with the lowest deviation.
(or give this clip the most weight in the blend)

only if all clips reach the thresold (too high deviation), the original gets returned.

I intend to use this corrector in mvbob() from mugfunky.
this should speed up things while improving quality.

but as I said: wait for next weekend..

[Mug Funky]

maybe it's just my whacked-out computer, but mvcompensate doesn't seem to be compensating anymore.

try:
pel = default(pel,1)
blocksize = default(blocksize,8)

vfw = c.mvanalyse(pel=pel, isb=false, blksize=blocksize)
vbw = c.mvanalyse(pel=pel, isb=true, blksize=blocksize)

interleave(c.mvcompensate(vfw),c,c.mvcompensate(vbw))


it's just returning the current frame, so i get 3 dupes instead of 1 frame and 2 compensations.

[edit]

what on Earth's going on here? if i precede this with "reduceby2" all's well. but if i leave it out it does a NOP.

full script:

function MCdespot (clip c, int "limit", int "pel", int "blocksize")
{
limit = default(limit,6)
pel = default(pel,1)
blocksize = default(blocksize,8)

vfw = c.mvanalyse(pel=pel, isb=false, blksize=blocksize)
vbw = c.mvanalyse(pel=pel, isb=true, blksize=blocksize)

interleave(c.mvcompensate(vfw),c,c.mvcompensate(vbw))
#yv12lutxy(last,last.clense(),yexpr="x y - abs")
#degrainmedian(mode=1,limitY=limit,limitUV=limit).selectevery(3,1)
#selectevery(3,1)

}



#ASYNTHER AVISource
AVISource("C:\Ack\test2.avi")

tdeint(1,1,tryweave=false)
#reduceby2()
converttoyv12()
reduceby2()
#MCdespot()

mcdespot()


test clip is huffy. you can find it here:
http://forum.doom9.org/showthread.php?s=&threadid=86163

it's cheops' capture, not mine :)


[edit 2]

seems to be a size limit. 720x576 didn't work, nor 640x480, or 512x384. but 384x288 and 384x384 worked. i wonder why it never did this before?

[Manao]

Just a tip : MVCompensate doesn't compensate on what he thinks is a scene change....

[scharfis_brain]

Hi Manao.
I forgot to mention one really important thing for the corrector. Sorry about that!

It should be like this:


Corrector_double( \
clip original_analysis, clip compensated_1_analysis, [...], clip compensated_n_analysis, \
clip original_output, clip compensated_1_output, [...], clip compensated_n_output, \
int threshold, int mode)


its analysis behaviour should be exactly like the current corrector (with both modes! many thanks for that!).

the analysis_clips are building a mask (or whatever) to let the output of Corrector_double() being built using those masks from the analysis-clip and the output clips.

only with this implemtation of corrector, mvbob will benefit from it, cause I do the analysis on vertical blurred clips. and the correction inherently is done on some other crip clips.

(If I would to the analysis and correction on the crisp clips, all line-jaggyness will sneak through the corrector :( )

[Mug Funky]

Originally posted by Manao
Just a tip : MVCompensate doesn't compensate on what he thinks is a scene change....

has the syntax changed? thSCD1 is throwing an error for me.

i think i'll dig up and older version of MVtools for now.

[Manao]

It works on my computer. Strange. What error message do you get ?

[Mug Funky]

ah, never mind. i was putting it in the wrong place.

[tsp]

I found a memory leak that I think originates from MVTools when used with Invoke.

It's present in my filters ml3dex an medianblurt wtih motion compensation enabled. See this thread (http://forum.doom9.org/showthread.php?s=&threadid=84636) to get the filter including source.
At the moment it only works with MVTools 0.9.8.1 until I figure out how to use a named argumentlist with Invoke. The following script will cause a memory leak:

ml3dex()

while won't:

c=last
vfw = c.mvanalyse(pel=1, isb=false)
vbw = c.mvanalyse(pel=1, isb=true)
interleave(c.mvcompensate(vfw, thSCD1=300),c,c.mvcompensate(vbw, thSCD1=1000)).ml3dex(false).selectevery(3,1)


the code in ml3dex that invokes MVTools look like this:
[/code]
PVideoFrame srca[3];
if(MC){
PClip vectors;
AVSValue MVAnalyseArgs[12]={child,blksize,pel,0,2,1,false,lambda,1,true,true,12};
AVSValue MVCompensateArgs[6]={child,vectors,true,0,thSCD1,thSCD2};

vectors = env->Invoke("MVAnalyse",AVSValue(MVAnalyseArgs,12)).AsClip();
MVCompensateArgs[1]=vectors;
PClip compen=env->Invoke("MVCompensate",AVSValue(MVCompensateArgs,6)).AsClip();
srca[0]=compen->GetFrame(n,env);
srca[1]=src;
MVAnalyseArgs[6]=true;

vectors = env->Invoke("MVAnalyse",AVSValue(MVAnalyseArgs,12)).AsClip();
MVCompensateArgs[1]=vectors;
srca[2]=env->Invoke("MVCompensate",AVSValue(MVCompensateArgs,6)).AsClip()->GetFrame(n,env);
if(markscenechange)
if(srca[0]->GetReadPtr(PLANAR_Y)==srca[1]->GetReadPtr(PLANAR_Y)||srca[2]->GetReadPtr(PLANAR_Y)==srca[1]->GetReadPtr(PLANAR_Y))
scenechange=true;
else
scenechange=false;
}
[/code]

and I can't see that it should cause a memory leak.

[Manao]

Where exactly do you use that code ? I'm afraid you're using it in the GetFrame method of your filter, which is highly uneficcient, because there's a lot of memory assignment when initializing MVAnalyse.

Now, for the 'memory leak' explanation : since MVTools 0.9.7, there's a static object, MVCore::mvCore, which handle the memory allocation & stockage for every frame that is interpolated. The releasing of this memory is done when the object is destroyed. It basically means that with the way you're using the MVTools, mvCore is stocking every frames in memory, hence the memory leak.

Of course, the issue will disappear once you use MVanalyse in the constructor of your filter.

On a sidenote, a much more clean way to use it, imho, would be to provide with your filter an avsi script wrapping your filter an invoking MVAnalyse & MVCompensate. You'd then give to your filter only results of MVanalyse and MVCompensate, making it easier to manage if i (once again) change the dyntax of the mvtools. And it will let the 'power users' use their own custom settings for MVanalyse if they want to.

[tsp]

Thanks for the explanation. The code was placed in GetFrame after I moved it to the constructor the leak disappeared as you predicted and the speed increased 30%. The problem with placing the code in an avsi file is that at least with medianblurt the temporal radius is variable and until there is a loop construction in avisynth it's not possible to make such a script. Also the power user can always disable the motion compensation and do it them self and for the beginner it's easier to use.
It is a problem what I'm quite dependent on you not changing the syntax to much. It will not be such a big problem after I figured out how to use Invoke with named args. So if you don't change the parameter names to often it will be fine :) .

Another way to cause a leak with MVTools is this script:


scriptclip(s,"ml3dex1(s,10)")

function nop(clip c,int sc){
vfw = c.mvanalyse(pel=1, isb=false, blksize=8)
vbw = c.mvanalyse(pel=1, isb=true, blksize=8)
interleave(c.mvcompensate(vfw, thSCD1=sc),c,c.mvcompensate(vbw, thSCD1=sc)).selectevery(3,1)
}

The strange thing is that the destructor is called after every frame so it shouldn't cause a leak. Replacing sc with a constant doesn't change this :confused: .

Still MVTools is a really powerful tool for all these temporal filters even if you can't use it with scriptclip.

[Manao]

My MVCore::mvCore object is staying alive during the whole script execution, so no deallocation. If you want to avoid that constant memory allocation, use the idx parameter of MVAnalyse, and set it to a positive value : in that case, frames will be allocated only once, and when needing a new memory space, the oldest frame stored will be reused.

Now, why do you want to use script clip for ? What is the parameters you want to make vary ?

And finally, you'd get a better speed if you were using the interface i provide with the MVTools in order your filter to use them directly : you need to include a lot of files from the MVTools yet ( I didn't get time to make a proper interface yet, I'll do it today ), but deriving your class from GenericMotionFilter would allow you to get the result of the compensation without having to extract it with MVCompensate. And you'd also get the validity of the motion estimation for a particular block ( through the block's sad value )

[Fizick]

My small comments:
1.
an avsi script wrapping your filter an invoking MVAnalyse & MVCompensate. You'd then give to your filter only results of MVanalyse and MVCompensate, making it easier to manage if i (once again) change the syntax of the mvtools. And it will let the 'power users' use their own custom settings for MVanalyse if they want to.

I think it is the best method, if you do not use vectors,SAD, etc.

2.
using the interface i provide with the MVTools in order your filter to use them directly : you need to include a lot of files from the MVTools yet

If we all will make our filtes so, the MVTools soon will become the single remaining Avisynth filter :D

[Manao]

No, i'm making an interface right, so you don't have to include the filter inside the mvtools. I never done interfaces yet, so it may takes some times.

[tsp]

[qutoe]
My MVCore::mvCore object is staying alive during the whole script execution, so no deallocation. If you want to avoid that constant memory allocation, use the idx parameter of MVAnalyse, and set it to a positive value : in that case, frames will be allocated only once, and when needing a new memory space, the oldest frame stored will be reused.
[/quote]
Using idx didn't work. It only slowed the process down. I tried this script:

global s=AVISource("c:\test.avi")converttoyv12()
scriptclip(s,"nop(s,10)")

function nop(clip c,int sc){
vfw = c.mvanalyse(pel=1, isb=false, blksize=8,idx=12)
vbw = c.mvanalyse(pel=1, isb=true, blksize=8,idx=12)
interleave(c.mvcompensate(vfw, thSCD1=1000),c,c.mvcompensate(vbw, thSCD1=1000)).selectevery(3,1)
}


and virtualdubmod crashes after about 1800 frames when the allocated memory reaches 2 GB. Seems as if there is a problem reusing the old frames.


Now, why do you want to use script clip for ? What is the parameters you want to make vary ?


At the moment I don't need it. Just pointing out that if someone is using MVTools or a filter what calls MVTools inside scriptclip/conditionalfilter or frameevaluate the problem will arise.


And finally, you'd get a better speed if you were using the interface i provide with the MVTools in order your filter to use them directly : you need to include a lot of files from the MVTools yet ( I didn't get time to make a proper interface yet, I'll do it today ), but deriving your class from GenericMotionFilter would allow you to get the result of the compensation without having to extract it with MVCompensate. And you'd also get the validity of the motion estimation for a particular block ( through the block's sad value )

That would be a good idea. It will be the best solution. I will look forward to what speed improvement calling MVtools directly will bring. Maybe in avisynth 3.0 MVTools will be in the core.

The speed difference now between calling MVTools using Invoke and wrapping the filter in an avsi file is about 0.3%(in favor of Invoke).