working to get the most out of SVPFlow (getting rid of smudginess)

[a1s2d3f4]

(This is posted already on the SVP-team forum, but it's not very active there.)

I have used MVTools for 7 years converting PAL or 24fps videos to 60fps.

I use AVISynth with VirtualDub
My MVTools part of the script would go something like this:

#currently I have PAL 25fps video as my last operation
num_frames = 12
super = MSuper(pel=2)
backward_vec = MAnalyse(super, overlap = 4, isb = true, search = 3)
forward_vec = MAnalyse(super, overlap = 4, isb = false, search = 3)
v100=MFlowFps(super, backward_vec, forward_vec, num = num_frames * FramerateNumerator(last), den = FramerateDenominator(last))
v200 = ConvertToY8(v100).AssumeFPS(300).ChangeFPS(60)



Now, with SVPFlow, I have:
num_frames = 12

super = SVSuper("{pel: 2, gpu: 1}")
vectors = SVAnalyse(super, "{overlap: 4}")
forward_vec = SVConvert(vectors, false)
backward_vec = SVConvert(vectors, true)
super_mv = MSuper(pel=2, hpad=0, vpad=0) #padding should be zero here!
v100 = MFlowFps(super_mv, backward_vec, forward_vec, num = num_frames * FramerateNumerator(last), den = FramerateDenominator(last))
v200 = ConvertToY8(v100).AssumeFPS(300).ChangeFPS(60)


I can already see the improvement:
mvtools:
https://drive.google.com/file/d/1mX4Q_chei6Oudi1a2DM0fzb_mRDaAJHt/view?usp=sharing
SVPFlow:
https://drive.google.com/file/d/1qdk2qJAnTkWiS__ULk913UmXEaoCv9l8/view?usp=sharing

What I am wondering about is can I do something else to further improve the "smudginess"?
I am including the original frames from the 25fps footage between which the interpolation occurs.
I wish there was a way to get the interpolation algorithm to recognize the non-moving background (piano keys) and the moving object - hand.
https://drive.google.com/file/d/12_w6AjaFI9W2J9ZyVWnjwCk2hC5iTM3y/view?usp=sharing
https://drive.google.com/file/d/1nT3xnSgWT9aUbGWw4HlSYQFum9ShxTLo/view?usp=sharing

Speed is not an issue as I export to a separate file and can let the computer sit overnight if necessary.

[hello_hello]

I wish there was a way to get the interpolation algorithm to recognize the non-moving background (piano keys) and the moving object - hand.

That sounds like the holy grail of frame interpolation. ;)

This isn't perfect, but based on my limited experience, it's generally better.
FrameRateConverter (https://github.com/mysteryx93/FrameRateConverter/releases)

https://forum.doom9.org/showthread.php?t=174793

PS You probably should use it with these versions if you're not using them already:
https://github.com/pinterf/masktools/releases/
https://github.com/pinterf/mvtools/releases

[johnmeyer]

One of the keys to getting optimal results from motion estimation frame rate changes is to use your own block sizes. In both the code snippets you posted, you failed to do this.

If you look at some of the threads feeding into that FrameRateConverter thread you'll see some MVTools2 code I posted which became one of the benchmarks used to compare the improvements the FrameRateConverter author was trying to achieve. He worked really hard at making improvements and did some very clever things, but I am not convinced that his results are always better than what you can get with MVTools2 or SVPFlow, which is the same code re-worked and slimmed down (features removed) in order to run in real time using a GPU.

I strongly suggest you read through this thread, which discusses many of the issues surrounding frame rate conversion:

SlowMo: Simple slow motion, using interpolation (https://forum.doom9.org/showthread.php?t=174089)

Slow motion is the exact same thing as changing the frame rate. The only difference is that you add a playback speed change when doing slow motion.

That thread led to this one:

SVP-like frame interpolation? (https://forum.doom9.org/showthread.php?t=174410)

That thread then led to MysteryX developing FrameRateConverter. You can read about that here:

FrameRateConverter (Official) (https://forum.doom9.org/showthread.php?t=174793)

[a1s2d3f4]

Cool. Lots of reading. Hopefully, I can find some time over the weekend to read and try applying.

@johnmeyer Yes, I agree that if I figured out a way to use these "block sizes" correctly (I am a bit slow, and even though I have read through MVTools website, I haven't figured out what to do with all the parameters) things would improve.

Thanks for the replies.