MVUtensils - The future of MVTools

[Myrsloik]

MVUtensils (https://github.com/myrsloik/mvutensils) is a large scale cleanup and refactoring of mvtools to make it cleaner, use sane algorithms to avoid memory bloat and add optimizations for moderns CPUs. IT HAS ALMOST 100% SPEED GAINS FOR 16 BIT MATERIAL ON MODERN CPUS. I believe this to now be the fastest mvtools fork ever with a good margin.

I also fixes the longstanding issue of edges not being processed and some arguments being completely incomprehensible and without sane scaling.

Just do pip install vapoursynth-mvutensils and try it for yourselves. Note that the defaults have been slightly changed and some arguments modified relative the original mvtools.

Test script:
Blocksize: 16x16
Overlap: 8x8
BestSource("bbb_sunflower_1080p_30fps_normal.mp4") => Super => Analyse => Degrain3

MVUtensils 16bit:
Output 8001 frames in 62.40 seconds (128.22 fps) 95% FASTER

MVTools 16bit:
Output 8001 frames in 122.00 seconds (65.58 fps)

MVUtensils 8bit:
Output 8001 frames in 45.16 seconds (177.18 fps) 37% FASTER

MVTools 8bit:
Output 8001 frames in 62.07 seconds (128.91 fps)

CPU 9800X3D

[Selur]

Nice!
Thanks!

Cu Selur

[Myrsloik]

I forgot to mention that it has float support as well so mvsf is probably obsolete too.

[Selur]

Nice, but reading https://github.com/myrsloik/mvutensils#porting-from-mvtools this will take me ages to adjust all my scripts (https://github.com/Selur/VapoursynthScriptsInHybrid/).
Anyway, that's a 'me problem' and I'm happy to get a faster 'mvtools' :)
Thanks!

Cu Selur

[Myrsloik]

I can see v2 wasn't good enough and I apologize. Now v3 is out and it's ~5% faster on the original benchmark script.

[ChaosKing]

Nice, but reading https://github.com/myrsloik/mvutensils#porting-from-mvtools this will take me ages to adjust all my scripts (https://github.com/Selur/VapoursynthScriptsInHybrid/).
Anyway, that's a 'me problem' and I'm happy to get a faster 'mvtools' :)
Thanks!

Cu Selur

Just tell an ai to do it for you :devil:

[Blue_MiSfit]

Hooray! It's been a long time since I've done any AVS / VS scripting, but this is great to see!

Also yes, coding agents are extremely good at refactoring. I wouldn't think twice about it.

[Selur]

Added a MotionVectors-class as a Wrapper to my misc.py (https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/misc.py), this way I can call MV.Super&Co and without having to worry what mvtools variation is used.

Cu Selur

[Myrsloik]

I've released v4. It fixes all bugs that have been discovered so far.

It's also slightly faster on sse2/avx2 cpus.

[Selur]

Nice!

[sdml]

Recommended way to avoid processing through scene changes?

[Myrsloik]

Recommended way to avoid processing through scene changes?

It already doesn't. Adjust thsad if it does. Blocks are already rejected if a match is too bad.

Same as in all mvtools.

[sdml]

DepanStabilise kinda buggy... the lower dxmax, dymax the more it feels... From crooked frames up to "access violation" crushes, on some specific scene changes. Crush always point to some avx2 (or mmx for plain dll) stuff.

[Myrsloik]

DepanStabilise kinda buggy... the lower dxmax, dymax the more it feels... From crooked frames up to "access violation" crushes, on some specific scene changes. Crush always point to some avx2 (or mmx for plain dll) stuff.

Full script and input clip format?

The crash dump would also be very helpful.

[DTL]

To make better performance with SIMD computing we need to use SIMD at higher levels of the program execution.

Example with possibly best in SAD-wise quality esa (really expanding) search in mvtools: https://github.com/myrsloik/mvutensils/blob/336c4d69c3de3bd01d90b37ae3d73c2dfddd9f02/src/MotionBlockPyramid.cpp#L598

With SIMD at single block single position SAD computing only it is a conditionless sequence of CheckMV() calls with a sequence of different ref coordinates. Also the order of the 2D scan creates some anisotropy if pnew != 0 - the first good MV (at the beginning of for loop scan top left corner) can be locked and not replaced with better MV in the next scan order position. With AVX2 and AVX512 we get new SIMD instructions to compute SADs of several positions with single instruction. This also decreases data reload from cache to register file. In better case (with not very big radius but typical usable r=2) we can load once src and ref blocks and compute SADs of all required search positions with instructions like mpsadbw. And analyse output 2D array of SAD data (also exist minpos SSE instruction to find lowest value of a SIMD vector).

With a full 2D array of SADs for a given search radius computed by better SIMD instructions inside the register file we can make many versions of esa search like:
1. complete isotropy best SAD single pick from total search area
2. expanding multi-step search isotropic with best SAD selection at each radius
3. anisotropic expanding search equal to current version

Though hardware instructions for multi-position SAD computing looks only compatible with pel=1 image data storage mode so pel=2 and pel=4 may be emulated by processing x2 and x4 upsized frames (single frame buffer image after MSuper mode). This also will reduce the number of memory read streams for each ref block position for pel 2 and 4 and expect to help caching.

Though my tests between 1 and 2 and 3 show very small quality differences.

Also to make multi-generation MVs refining scripts we need dual-input MAnalyse. It is a small addition of a second src or ref clip input and if provided - use src of ref frames from separate clip. Also a full list of a possibly useful new features for better mvtools from old ages you can see at https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/new_features_list.ods .

[Myrsloik]

Nobody's missing BlockFPS, right?

[Selur]

I removed it everywhere, where I used it for frame interpolation (wasn't worth keeping it),... so "right" from me :)

Cu Selur

[MeteorRain]

https://github.com/myrsloik/mvutensils/blob/0b1380be6ae1d9864467311a3752d1a1158179f5/src/Mask.cpp#L94

Range seems incorrect when VS_USE_LATEST_API‎ or VS_USE_API_42 is not defined. Code would write the opposite value into _Range.

[Myrsloik]

https://github.com/myrsloik/mvutensils/blob/0b1380be6ae1d9864467311a3752d1a1158179f5/src/Mask.cpp#L94

Range seems incorrect when VS_USE_LATEST_API‎ or VS_USE_API_42 is not defined. Code would write the opposite value into _Range.

Right, will fix.

[MysteryX]

One of the issues I had porting code to VapourSynth was the more limited block sizes; 12 and 24 I believe were missing. Are these available here?

[Myrsloik]

One of the issues I had porting code to VapourSynth was the more limited block sizes; 12 and 24 I believe were missing. Are these available here?

No, they add a lot of complexity and with the optimizations I've done you can simply use a one step smaller size with the same performance as other mvtools. If it's motion vector coherence you care about I think large block => recalculate with smaller block will be better as well.

[MeteorRain]

In https://github.com/dubhatervapoursynth/vapoursynth-mvtools/commit/5bcf3df1384923714c70aba6d20137a6e9c7dd11
There was a change of

nVx >> 1

to

nVx / xRatioUV

that changed the results for negative inputs. Previously it rounds towards negative, now it truncates towards zero.

Previously, when nVx = -3, nVx >> 1 => -2. Now, nVx / xRatioUV => -1. This was a breaking change against the original mvtools.

[Myrsloik]

It seems like a reasonable change. Otherwise subsampled chroma would move different amounts depending on left/right or up/down.

So hardly breaking apart from bitexact output.

[MeteorRain]

https://github.com/HomeOfAviSynthPlusEvolution/neo-mv/pull/1
MVU and vapoursynth-mvtools are also impacted.

[Myrsloik]

So I tested the fields and tff arguments. Turns out they're completely useless and produce worse results. This even after fixing the fact that the field shift can be applied multiple times which is worse than doing nothing at all.

Objections to removing them entirely? If not the arguments will be gone in the next release.

[ChaosKing]

Does it affect the new new qtgmc re-write?

[Myrsloik]

Does it affect the new new qtgmc re-write?

Since it does nothing useful probably not. You can grab the most recent mvu artifact and test yourself if it does work.