MVTools

[Fizick]

Almost similar for big best SAD - we do not know if match is wrong (local minimum instead of global) or this object was deformed.
May be we can store second (or more) best matches for every vector.
Other option may be somehow use SADs of coarser hierahical levels.
I do not have the working algo.
But i tried implement so call meander vector scan (left to right - right to left, end so on). Seems, it give more satble results for various left-right movements.

Released version 2.5.2

MSuper: separable width and height reducing with assembler iSSE optimization
MAnalyse: limit levels value

MAnalyse: Added meander parameter for blocks scan in rows from left to right and from right to left
Small optimization of YUY2 conversion

MSuper: More full size of sublevels
MAnalyse: Increased number of sublevels (for more long vectors)
MAnalyse: Replaced parameter name level by levels, changed meaning of its positive and negative values
MAnalyse: Disabled lambda for most coarse level
MAnalyse: Full search with large radius at smallest level

Note: These 2.5 changes have experimental (beta) status (now public beta), and may be removed in next versions (for example, it is hard to implement meander scan with variable block size).
Variable block size has also problem with overlap.
Insted of meander it is possible to get two vectors set for different scan directions, but it will be twice slower.

[MfA]

Originally Posted by TSchniede

This is an issue I have been thinking about myself. Right now we simply don't know if a low SAD means that the match is good or that there isn't much information anyway.
We would want some sort of metric that tells us how reliable our guess on the movement is.
I had hoped this could be an alternative approach for the movement analysis. but so far I haven't found a way to get the concept into a working algorithm. I'm not even sure it is possible to do it in a way that the produced vectors and computation time is better or at least equivalent.

Why doesn't the smoothness constraint take care of this?

[Didée]

For vector search stage, I agree it's a difficult case ... a labyrinth filled with cans of worms. It's hard to predict which theoretical means really would improve, and which are only pipe dreams.

However, for vector usage stage (e.g. in MDegrain), and in reply to the "make it and show proof" - retort in #1259 :

I already have given proof, long time ago.
It's in TempGaussMC, in the section commented with "Needed to later make MVDegrain work as expected ...the "SAD concept" doesn't fit to the realities -- so, let's tailor the realities to fit the darn concept ...".
Since SAD rating can't be modulated at the script level, it's done the reverse way: instead of rating SAD according to contrast, the contrast is altered to fit the uniformly-everywhere method.

Point is: without that effort, TGMC would not produce much usable. With it, it does. Period.

This' I consider a sufficient [practical] proof. (...for a relation so obvious that giving proof is almost superfluid.)


And BTW, this is pretty similar to the concept I once had proposed for "static-detection" in deinterlacing/bobbing. It's been decades that all deinterlacers only used primitive hard-thresholding for static detection: if a pixel differs by more than "x", it's not static.
At some point I was annoyed enough by that insufficient concept, and build the adaptive thresholding system found in MCBob. It worked better, which was expected, and in the meantime is known as TMM() plugin function.

And it's really the same idea: areas with strong signal need a different treatment than those with weak signal.

Nobody would expect the very poor people to pay the same taxes as the very rich. So, why should pixels?

[Boulder]

I get an access violation with Avisynth 2.5.8 and the latest MVTools2 version with this script (VDub sometimes crashes, pointing to mvtools2.dll):

MPEG2Source("bamse.d2v")
Crop(16,4,-16,-4,true)
interleave(last,MCTemporalDenoise(chroma=true,flat=true))
AddBorders(0,0,16,8)

If I comment out the cropping and AddBorders, it works fine.

MCTemporalDenoise is the latest one (v1.3.0.0) from this thread: http://forum.doom9.org/showthread.php?t=139766

[Fizick]

Boulder,
I do not support MCTemporalDenoise. I need in simple script to reproduce crash, as well as short sample video.
Anyway, how about older MVTools versions?

[Boulder]

I'll get you a sample in the next few days, and also try to reproduce it with a simple script.

Version 2.5.2 does crash, v2.4.7 does not. I couldn't find the two versions between those on your site.

[Fizick]

I now added also v2.5.0 and 2.5.1 to my site.

[Fizick]

Some problem with SetMTMode(2) of MVTools v2.5.3 was reported by MAG79.
It seems, that temporal predictor mode is not compatible with multi-threading.

I released v2.5.4 - temporal problem is NOT resolved, but I set default temporal=false according to doc (it was wrongly =true in v2.5.3)

[Boulder]

Here's a sample file: http://www.mediafire.com/?sharekey=9...db6fb9a8902bda

The highest MVTools2 version which works is v2.4.7. The 2.5 branch doesn't work with the script above and the sample clip. I'll post a link to this discussion in the MCTemporalDenoise thread, maybe LaTo has some ideas. The MCTemporalDenoise function uses MVTools2 function quite a lot so the problem could lie anywhere.

[LaTo]

The crash occurs when hpad=0 & vpad=0 & chroma=true in MSuper... (tested only with v2.5.4)

I don't know why but:

2.5.0 (01.08.2009 by Fizick)

* MSuper: More full size of sublevels

Maybe it is the problem...

Search all "hpad=0,vpad=0" in the script and delete them if you want to use v2.5.x with MCTD but it is sub-optimal (speed-wise)

[LaTo]

I have checked a bit the source: all modifications in MVSuper.h (2.4.7->2.5.0) is about vpad/hpad and chroma

[Fizick]

Boulder, LaTo:
try v2.5.5

Anyway, padding=0 is not a very good. And I wil try improve padding handling in next version(s)

[Boulder]

Works fine, thanks

[LaTo]

Great!

[thewebchat]

I see that we are now at 2.5.8 of MVTools. Is the 2.5 branch still considered experimental, or is it generally safe to use?

[Fizick]

v2.5.8.1 it is rather safe

"temporal" is the experimental feature, not compatible with SetMTMode (by one user report)

[thewebchat]

Actually, I do not remember reading anything about the "temporal" option in this thread, and the description of it on avisynth.org.ru is fairly limited. Could you explain what the goal of the "temporal" option is? Regardless, I will upgrade to 2.5.8.1 and test it later, since you have confirmed that it is stable.

[Redsandro]

I was wondering if anyone is interested in bringing rolling shutter compensation to MVTools, or if something similar has already been attempted?

Example, so bad it cannot be repaired but it illustrates the problem perfectly:
http://www.youtube.com/watch?v=0qC0_nIUq9s

Excellent article illustrating the problem with CMOS sensors and the rolling shutter effect:
http://dvxuser.com/jason/CMOS-CCD/

Right now afaik the only available solution is using Rolling Shutter compensation option in Gunnar Thalin's DeShaker plugin for VirtualDub.
However the compensation is linear in nature (skew). It is very awesome, saves a lot of footage, but only half-descent for footage more complicated (speed changes, moving people etc) than just steady static pans.

Chia-Kai Liang's article on rolling shutter compensation:
http://mpac.ee.ntu.edu.tw/Exhibition...ng-shutter.php
Curved compensation can be better than linear compensation for nonsteady pans (speed changes within frame), but this implementation is commercial, hardware only and thus not available. Just food for thoughts. This curvy compensation is imo theoretically superior to DeShaker's compensation for incalculating horizontal speed changes within the frame, but not vertical speed changes.

TheFoundy working on a commercial motion compensated rolling shutter node for Nuke 6.
http://www.thefoundry.co.uk/blog.aspx#rollingshutter
They compensate motion from different moving objects. I think they themselves were inspired by watching failing divx movies or even MVTools itself.

In my imagination, a combination of the compensation from the DeShaker model, temporally motion compensated for objects moving independent from the camera and maybe spatially compensated for camera movement speed changes within one frame, is superior to all, and doable with the way MVTools detects motion and MFlow can realisticly (un)distort images. Not literally, but it seems like a lot of the work has already been done.

[johnmeyer]

As a first approximation, if there was a way to apply a ramp function to the MFlowInter function's time parameter, and then apply this increasingly ramped correction top to bottom (or bottom to top) to each scan line, it might provide a first approximation to a solution.

[Terka]

deshaker has some technique for correcting it.