MFlowFps artifacts in zooming regular patterns?

[shae]

When using MFlowFps on video with a regular repeating pattern (like, a grid) that zooms, sometimes it produces artifacts. It looks as if some block vectors latch to the wrong part of the pattern.

In some cases increasing MAnalyse's blksize helps (blksize=64 for 1920x1080), but not always.

Is there a better way to fix it? Maybe force it somehow to treat the whole frame as one cohesive zooming unit, and so have the block vectors move together?

I don't have a specific example to post, but hopefully the description is clear enough.

[Ceppo]

Try QTGMC method:

repair0.BilinearResize( w/2, h/2 ).RemoveGrain( 12,CMrg, planar=true ).BilinearResize( w, h )

Basically you bilinear to half size and then removegrain then you upscale it back to the original resolution and you use that as search clip.

[johnmeyer]

All motion estimation algorithms break down, even the professional stuff that is being used to destroy/improve (depending on your point of view) all the old TV shows shot on film decades ago and which are now being remastered.

You will always have problems with objects entering/leaving the scene in the foreground; strong vertical shapes (a picket fence is going to look awful); opposing motion (someone walking across the frame, with their legs going back and forth), etc.

Checkerboard patterns are going to be almost impossible. You'll have to try really large or really small blksize and try different overlaps. You may also get better performance if you estimate in steps using MRecalculate. Here is the code I've posted many times which became a benchmark of sorts for comparing improvements in motion estimated results using other tools and techniques:

prefiltered = RemoveGrain(source,22)
super = MSuper(source,hpad=16, vpad=16, levels=1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad=16, vpad=16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize=16,overlap=4,search=3,dct=0)
forward = MAnalyse(superfilt, isb = false, blksize=16,overlap=4,search=3,dct=0)
forward_re = MRecalculate(super, forward, blksize=8, overlap=2, thSAD=100)
backward_re = MRecalculate(super, backward, blksize=8, overlap=2, thSAD=100)
output=MFlowFps(source,super, backward_re, forward_re, num=48000, den=1001,ml=200,mask=2)

Here are a few other threads you might want to skim:

SVP-like frame interpolation? (https://forum.doom9.org/showthread.php?t=174410)

FrameRateConverter (Official) (http://forum.doom9.org/showthread.php?t=174793)

[edit] You will, of course, have to change the num and den values to fit your desired fps.

[shae]

Thanks Ceppo and john for the suggestions and pointers.

The above two snippets didn't help in my specific case, which is composed of two problematic parts: one zooms in slowly and ends with a partial fade, the other zooms out much quicker.

I also tried InterFrame (https://forum.doom9.org/showthread.php?p=1486831) (based on SVP), which did one part well, and the other probably worse than I started with.

For now, I settled for FrameRateConverter (MysteryX's), which did the same part also well, and the other part not perfect but on-par or slightly better than where I started.

[Frank62]

All motion estimation algorithms break down, even the professional stuff that is being used
...

That's true. And because it is, I still prefer MBlockFps. It also fails in the mentioned cases, but the result is much less ugly to my eyes. Worth a try.

[Ceppo]

If you provide the mentioned clip I might look into it.

[shae]

@Ceppo

Thanks! But no need to expend more energy on this, the result will suffice.

[DTL]

To decrease probability of gross search error you can simply decrease number of search levels in MAnalyse. It will lost some very long vectors but will stop to point to false long vectors. If the motion speed in the footage is not too fast - many levels are not good and can produce very large errors. If the motion shift in the 2 neibour frames is less than about half of the repeating of pattern size - limiting number of levels should help.