Req: MVTools help

[FredThompson]

I'm using this script to clean up a very noisy non-CGI animation source. It works wonderfully, but slowly, on almost every part. However, there is a very short portion in which some rapid motion is screwing everything up. The motion compensation (Fizick's sample code from the DeSpot page) has a wonder effect of helping with noise reduction. In this particular scene, the robot's head is spinning around his neck very quickly. If the motion compensation is removed, there is no corruption but there is a lot more noise and a huge amount of mosquito junk. Because of this, I'd like to keep the motion compensation but tune it to better handle this rapid motion. I've been reading the MVTools documentation and my head is swimming. Would someone who has experience with MVTools help me out with this?

I realize the quick fix is to tweak for these few particular frames but what if there are other high-motion areas with the same corruption? If at all possible, I'd rather have a script which works "properly" on a wider range of source than have to inspect the whole source.

ConvertToYUY2()
GrapeSmoother(Smoothing=35)
PeachSmoother(NoiseReduction=50,Spatial=120)
i = ConvertToYV12()
vf = MVAnalyse(i, isb=false, lambda=2000)
f = MVCompensate(i, vf)
vb = MVAnalyse(i, isb=true, lambda=2000)
b = MVCompensate(i, vb)
Interleave(f, i, b)
Undot()
DeSpot(sign=-1,p1=40,mthres=25,p2=20,pwidth=50,pheight=30,mwidth=12,mheight=10,interlaced=false,blur=1)
DeSpot(sign=1,p1=30,mthres=15,p2=10,pwidth=10,pheight=10,mwidth=12,mheight=10,interlaced=false,blur=1)
DeHalo_Alpha()
LRemoveDust_YV12(10,2)
Soothe(LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4),last,24)
FastLineDarken(strength=6, luma_cap=191, threshold=4, thinning=0)
SelectEvery(3,1)
trim(79129,79853)

[Didée]

First, let me suggest something against
It works wonderfully, but slowly,
You really don't want ALL those filters to run on the long interleaved clip ...
Arrange it like

ConvertToYUY2()
GrapeSmoother(Smoothing=35)
PeachSmoother(NoiseReduction=50,Spatial=120)
i = ConvertToYV12()
i = i.Undot()
vf = MVAnalyse(i, isb=false, lambda=2000)
f = MVCompensate(i, vf)
vb = MVAnalyse(i, isb=true, lambda=2000)
b = MVCompensate(i, vb)
Interleave(f, i, b)
DeSpot(sign=-1,p1=40,mthres=25,p2=20,pwidth=50,pheight=30,mwidth=12,mheight=10,interlaced=false,blur=1)
DeSpot(sign=1,p1=30,mthres=15,p2=10,pwidth=10,pheight=10,mwidth=12,mheight=10,interlaced=false,blur=1)
LRemoveDust_YV12(10,2)
SelectEvery(3,1)
DeHalo_Alpha()
Soothe(LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4),last,24)
FastLineDarken(strength=6, luma_cap=191, threshold=4, thinning=0)
trim(79129,79853) and things will run much faster.


Then, regarding the question ... accidentially, it has been answered already (http://forum.doom9.org/showpost.php?p=762597&postcount=10). ;)

[FredThompson]

I was pretty sure putting the SelectEvery(3,1) after all the filters gave a smoother cleaning of the solid colors in the source and far less mosquito noise. Then again, it was late. I'm retesting to see if that was actually the case.

edit: Without a doubt, the motion compensation helps DeHalo_alpha. The combination almost completely removes all mosquito noise and is very obvious in high contrast, small detail areas.

Thanks, I'll follow the link.

edit: Your post make 2 statements: use simple denoising or use simple denoising. Huh?

Shouldn't the solution to this be related to total change between adjacent frames? If the frames are very similar, the motion compensation works, if they are very different, don't do the compensation. Question is, how to script for that?

The docs are not so clear. At the top of the page, there is a mention of motion detection vs. motion compensation. OK, fair enough. I can't tell from the parameter explanations how to account for this. For example, the explanation of the lamba parameter of MVAnalyse talks about the coherence of a vector. Humor me, I haven't had any geometry in 20 years. I vaguely remember something about the term coherent but don't remember what it means. For some reason, I want to think it has to do with the right-hand rule for torque but that's probably wrong. I do understand the isb parameter.

In this case, I think the spinning effect is so fast and regular that frame n looks a lot like frame n+2 and n-2 while nothing like frame n+1 or n-1.

I'm also a little confused because it appears the corruption fits the block size, as I understand this filter.

Could it be the DeSpot calls are the ones getting confused, not MVTools or are these large blocks a result of both interacting in confusion?

[FredThompson]

Alright, I've just tested the location of the SelectEvery(3,1) command with the DeSpot calls removed. There is a very distinct quality difference wrt cleanliness if the SelectEvery(3,1) is before or after the LRemoveDust_YV12 call. There doesn't seem to be any benefit to having the Soothe and FastLineDarken calls inside. This sample's a PNG so it should be more obvious. The differences, when looking at individual frames, are small but there is a HUGE difference shimmering during playback of the encoded result.

It seems the problem I'm chasing has something to do with the interaction between DeSpot and MVTools. If MVTools is removed from the script, there is no corruption.

If DeSpot's sign parameter is -1, for white spots only, the result is added black blocks . If the value is 1, for white spots only, the result is black parts of outlines are missing. The combination of both is truely ugly.

[AVIL]

My two pence:

I check that the mocomped block don't have a SAD greater than a threshold (look at mt_binarize). If that happens i use the original block instead the mocomped one.

--------------------------------------------------------------
i = source.mvanalyse(isb = false, lambda = 2000, blksize = 8, delta = 1, pel = 2,search=3,searchparam=3,chroma=true,idx=1,truemotion=true)
j = source.mvanalyse(isb = true, lambda = 2000, blksize = 8, delta = 1, pel = 2,search=3,searchparam=3,chroma=true,idx=1,truemotion=true)

b1=a1.refina(i)
c1=a1.refina(j)

interleave (b1,source, c1)
....
filters
....
selectevery(3,1)

function refina(clip a, clip j){

u = a.mvcompensate(j, mode = 0, thscd1 = 300, thscd2 = 170)
ie=mt_LUTxy(a,u,yexpr="x y - abs" ,u=1,v=1).blockaverage().pointresize(width(a)/2,height(a)/2).blockaverage().pointresize(width(a)/4,height(a)/4).blockaverage().mt_binarize(threshold=9,upper=true,u=1,v=1).PointResize(width(a),height(a))
mt_merge(a,u,ie,u=2,v=2)

}

-----------------------------------------------------------------
the expression with pointresize/blockaverage do an average of the SAD of the pixels in the block (8x8). So all the block is tested instead individual pixels.

Good luck

[FredThompson]

What is "source" which is used to define i and j in your code?

I'm using MVTools 9.9.1 and there doesn't appear to be a parameter "truemotion" for MVAnalyse. Which version are you using?

[AVIL]

Sorry for the mistake but the code must be :

--------------------------------------------------------------
i = source.mvanalyse(isb = false, lambda = 2000, blksize = 8, delta = 1, pel = 2,search=3,searchparam=3,chroma=true,idx=1,truemotion=true)
j = source.mvanalyse(isb = true, lambda = 2000, blksize = 8, delta = 1, pel = 2,search=3,searchparam=3,chroma=true,idx=1,truemotion=true)

b1=source.refina(i)
c1=source.refina(j)

interleave (b1,source, c1)
....
filters
....
selectevery(3,1)

function refina(clip a, clip j){

u = a.mvcompensate(j, mode = 0, thscd1 = 300, thscd2 = 170)
ie=mt_LUTxy(a,u,yexpr="x y - abs" ,u=1,v=1).blockaverage().pointresize(width(a)/2,height(a)/2).blockaverage().pointresize(width(a)/4,height(a)/4).blockaverage().mt_binarize(threshold=9,upper=true,u=1,v=1).PointResize(width(a),height(a))
mt_merge(a,u,ie,u=2,v=2)

}

-----------------------------------------------------------------

Then, source is the clip to be filtered. In your script it is the clip i.


I'm using the latest version 1.0.2 but this isn't important. If you adapt mvanalyse to your version the result will be the same.

[FredThompson]

OK, I grabbed MVTools 1.0.3 and your suggested script pukes. There's no mt_LUTxy.

1.0.2 isn't around any more. There's a note about some bugs that were squashed with 1.0.3.


Did you get this code to run? Any ideas?

[foxyshadis]

mt_lutxy is from masktools 2. You can convert it to masktools1 by renaming it yv12lutxy and mt_binarize->binarize. (Might as well just get 2 though.)

[Didée]

I check that the mocomped block don't have a SAD greater than a threshold (look at mt_binarize). If that happens i use the original block instead the mocomped one.
Insufficient. SAD is not a good enough measure. You're guaranteed to still get loads of blocking - in nebula, steam, in dark scenes ...
And if you set the SAD threshold low enough to safely avoid blocking, you'll replace >90% of the compensated frames with the original frame ...

BTW, instead of ie=mt_LUTxy(a,u,yexpr="x y - abs" ,u=1,v=1).blockaverage()
\ .pointresize(width(a)/2,height(a)/2).blockaverage()
\ .pointresize(width(a)/4,height(a)/4).blockaverage()
\ .mt_binarize(threshold=9,upper=true,u=1,v=1)
\ .PointResize(width(a),height(a)) what about ie=mt_LUTxy(a,u,yexpr="x y - abs" ,u=1,v=1)
\ .DCTFilterD(14).mt_exand().mt_inpand()
\ .mt_binarize(threshold=9,upper=true,u=1,v=1)
Dunno if it's faster ... the ex+inpand is needed because of that "stripe" bug in DCTFilter. At least it looks more simple ;)

[FredThompson]

Well, his code does work for this high-contrast section. I haven't tested it in darker areas, though.

The combination of MVTools and DeSpot does a great job of removing single-frame defects in the background of old animation. The slight jiggle movement of the background is effectively removed which must help DeSpot identify actual spots instead of missing them due to slight motion. There's got to be a way to keep it from going haywire.

I wish there was an easy way to demonstrate the huge difference in quality that happens on this dirty animation transfer. Still images can't come close to showing how dramatic the results are.

[FredThompson]

Insufficient. SAD is not a good enough measure. You're guaranteed to still get loads of blocking - in nebula, steam, in dark scenes ...
And if you set the SAD threshold low enough to safely avoid blocking, you'll replace >90% of the compensated frames with the original frame ...Can you recommend a better method?