Log in

View Full Version : Any ML based frame rate interpolation for Vapoursynth?

Selur
19th March 2021, 07:52
Just wondering is someone keeping an eye on 'newer' ML based motion interpolation stuff, like:

DAIN https://github.com/baowenbo/DAIN
CAIN https://github.com/myungsub/CAIN
RIFE https://github.com/hzwer/arXiv2020-RIFE
FLAVR https://github.com/tarun005/FLAVR

I'm only aware that:

DAIN can be used through: https://github.com/BurguerJohn/Dain-App
CAIN, RIFE and FLAVR can be used through: https://github.com/n00mkrad/flowframes
(both have patreon pages)
and there are ncnn implemenation for:
https://github.com/nihui/cain-ncnn-vulkan
https://github.com/nihui/rife-ncnn-vulkan
sadly for none of those there are Vapoursynth filters, or is there and I'm simply not aware of it.

So I'm wondering:
a. Are there any ML based frame interpolation filters for Vapoursynth I'm not aware of?
b. Is someone keeping an eye on the development of those filters?

Cu Selur
Selur
2nd June 2021, 18:58
Lots of thanks to HolyWu for implementing a RIFE Vapoursynth version (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan) !
Thanks!

Cu Selur
poisondeathray
2nd June 2021, 19:55
Lots of thanks to HolyWu for implementing a RIFE Vapoursynth version (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan) !
Thanks!

Cu Selur


Yay!

Thanks
Selur
4th December 2021, 12:21
Just as a small follow up: Nowadays there are two RIFE ports for Vapoursynth:

https://github.com/HolyWu/vs-rife
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan

both work fine here, but sadly there are no ports of DAIN, CAIN and FLAVR so far.

Cu Selur
aegisofrime
7th December 2021, 06:29
Just wondering, for the people who have tried RIFE, how's the speed vs what we currently have like svpflow, interframe etc?
Selur
7th December 2021, 20:35
RIFE is way slower and SVP and mvtools based interpolation and in no way usable for real time playback.
Selur
14th December 2021, 20:32
Btw. did anyone do some testing with the new vs-rife which is now based on Practical-RIFE and would share some of his impressions?
(a. seems faster, b. allows higher (whole number) multipliers than 2)
Selur
2nd October 2022, 16:55
https://github.com/styler00dollar/VapourSynth-RIFE-ncnn-Vulkan is a fork from HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan which adds a bunch of new RIFE models.

Nihui always just exports the fastest model (ensemble False, Fast True), while the other ones can result in a better image. Ensembling combines the flow of two different interpolations within the model and if fastmode is false it calls contextnet. You can see it here.
see: https://github.com/nihui/rife-ncnn-vulkan/issues/50
might also be interessting.

Cu Selur