FrameRateConverter (Official)

[poisondeathray]

Yes, 2 versions . The Pytorch/CUDA version is significantly faster than Vulkan if you have a Nvidia GPU. But Vulkan is accessible to everyone - so that version should definitely be included for general use

The other issue is model availability - I'm finding 2.3 and 2.4 generally better than the 3.x versions . But we can probably ask HolyWu to make other models available for the Vapoursynth Pytorch version

There was some discussion on why original Vulkan version was slower, but apparently it just is
https://github.com/nihui/rife-ncnn-vulkan/issues/22

[MysteryX]

I can't get vapoursynth version to run, never seen this error before
I thought I was storing Core from the initializer and using it elsewhere; perhaps from wrong thread or something. I looked through the code. There was 1 instance where I was storing Core in a field, and it wasn't used at all. So it's not that.

I don't know then...

[ChaosKing]

vs-rife was just updated with 1.8, 2.3 and 2.4 models. https://github.com/HolyWu/vs-rife

[Selur]

Nice! btw. is there some info about what the differences are between the models? (I mean is model xy better suited for content wz?)

--
I also think it would be nice to choose whether
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan
or
https://github.com/HolyWu/vs-rife
would be used.

Cu Selur

[Selur]

As a side note, using Beta11:
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="rife", blkSize=8, blkSizeV=8, rife=1, rifeModel=1, rifeTta=False, rifeGpu=0) # new fps: 50
causes blending while
clip = core.rife.RIFE(clip)
does not with this (https://drive.google.com/file/d/1xchtRU6uZyLm9xJWh4HgNxDrSqC70zhl/view?usp=sharing) file (Frame 359).
Neither does 'FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)'.

Cu Selur

[Selur]

Any news on RIFE incorporation into FrameRateConverter?

[poisondeathray]

As a side note, using Beta11:
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="rife", blkSize=8, blkSizeV=8, rife=1, rifeModel=1, rifeTta=False, rifeGpu=0) # new fps: 50
causes blending while
clip = core.rife.RIFE(clip)
does not with this (https://drive.google.com/file/d/1xchtRU6uZyLm9xJWh4HgNxDrSqC70zhl/view?usp=sharing) file (Frame 359).
Neither does 'FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)'.

Cu Selur


You can disable some or all of the frc features . Adding output="flow"gives you interpolation only.

You can set individual masking features you don't want to "0" , such as maskThr, maskOcc, skipThr, blendOver, skipOver



There is an issue with stp ; stp=0 to disable strip detection produces error, but values >0 produce no error msg


File "PATH\FrameRateConverter.py", line 285, in FrameRateConverter
EMstp = havs.ChangeFPS(EMstp, newNum, newDen)
UnboundLocalError: local variable 'EMstp' referenced before assignment

[Dogway]

MysteryX, what's the point of having different mask settings for different clip sizes as shown here (https://forum.doom9.org/showthread.php?p=1945840#post1945840)? Does it have to do with the MAnalyse vectors?
I ask because it would make sense to have the same mask for the same clip whatever its size is.

This was my test code:

msk="C.BicubicResize(Round(C.Width()/blk/4.0)*4, Round(C.Height()/blk/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(thr,u=-128,v=-128)
\ .Blur(.6)
\ .BicubicResize(1280, 720)"

a=bicubicresize(640,360,-0.5,0.25) # 12 blksize
b=bicubicresize(854,480,-0.5,0.25) # 12 blksize
c=bicubicresize(1280,720,-0.5,0.25) # 16 blksize
d=bicubicresize(1920,1080,-0.5,0.25) # 24 blksize

a=Eval(ReplaceStr(msk,"thr,","120-10").ReplaceStr("blk","11").ReplaceStr("C.","a."))
b=Eval(ReplaceStr(msk,"thr,","120-10").ReplaceStr("blk","12").ReplaceStr("C.","b."))
c=Eval(ReplaceStr(msk,"thr,","120" ).ReplaceStr("blk","16").ReplaceStr("C.","c."))
d=Eval(ReplaceStr(msk,"thr,","120+10").ReplaceStr("blk","24").ReplaceStr("C.","d."))

StackVertical( \
StackHorizontal(a,b), \
StackHorizontal(c,d))

[pintcat]

Tried a very simple script on beta 11 and got this error:
Avisynth open failure:
Script error: Invalid argument to function "StipeMaskPass"
(FrameRateConverter.avsi, line 376)
(FrameRateConverter.avsi, line 219)

Script was something like
directshowsource(test.mp4)
framerateconverter()
Nothing fancy and it used to work with FrameRateConverter v1.3.

[manolito]

The correct function name should be "StripeMaskPass"...

[pintcat]

Yeah, that's a typo on my side. I did not copy/paste the error message. Yet, I have no idea what's wrong here. Am I the only one getting this error?

[Floatingshed]

Just downloaded version 1.3 zip, but the avsi file states that it is version 1.21...

[manolito]

This is just a small oversight by MysteryX. The script is really version 1.3.

[Floatingshed]

Great, thanks for letting me know.

[MysteryX]

MysteryX, what's the point of having different mask settings for different clip sizes as shown here (https://forum.doom9.org/showthread.php?p=1945840#post1945840)? Does it have to do with the MAnalyse vectors?
Not sure I understand your question; but at the link you pointed to, I take blkSize into the equation so that the resulting frame is resolution-independent; meaning a "mask zone" will be about let's say 1/20th of the width on a 480p clip and 1/20th of the width on a 4K clip. Will look similar when looking at that clip full-screen without regards to input resolution.

[MysteryX]

I got tired of Windows and switched to Linux.

For when I get back to this; does RIFE work under Linux? It will be a bit more of an adventure to get all the plugins working in Linux.

[manolito]

I got tired of Windows and switched to Linux.


Alright, I guess you will continue support for the upcoming Linus version in the Linux section at Doom9. Just one question:

Will you publish a final stable Windows version 2.0, or will you just abandon it and refer Windows users to the old stable version 1.3 ?

Whatever, thank you very much for your work on FrameRateConverter.

Cheers
manolito

[StainlessS]

or will you just abandon it and ...
Of course he would not just abandon it, MX is a very respectable person
with very strong principles and would not ever do that,
a final stable ver$ 2.0 is the least you can expect from the noble gentleman. :)

[MysteryX]

I also have Windows running within Linux with near-native performance with QEMU+KVM with GPU passthrough; enough to play games with native Windows performance.

Been busy with lots of other stuff. Including porting some of my software to Linux by switching UI from WPF to Avalonia (https://avaloniaui.net/), and porting MvvmDialogs (https://github.com/mysteryx93/mvvm-dialogs) to support Avalonia and other UI platforms. Oh, and converting the Media Player UI to work on Avalonia/Linux (https://github.com/mysteryx93/MediaPlayerUI.NET) was a pain too (and not completed).

There's FrameRateConverter to finish, and xClean to finish.

Not counting my usual work in the astral realms.

[StainlessS]

There's FrameRateConverter to finish, and xClean to finish.
"Complete", better than 'finish".
And Good Luck in the "Astral Realms", [EDIT: whatever the hell they are {I'm guessin' close friend of Mystic Meg}],
also see U at some point in the Linux Realm :)
(Thank you MX)

[kedautinh12]

I think xclean avisynth version still don't finish

[MysteryX]

Oh and I spent a full week to get GPU passthrough to work for my Windows Virtual Machine... this is what I had to go through... (https://github.com/mysteryx93/GPU-Passthrough-with-Optimus-Manager-Guide)

yes xClean Avisynth is what needs to be finished. Also a comparison video I want to produce. Got a long list of things to do...

[real.finder]

Any news on RIFE incorporation into FrameRateConverter?

don't know if it possible or not but I think most mvtools problems come from bad motion vectors that MAnalyze do, so I wonder if MAnalyze can be updated to use some ideas from RIFE (neural network) to make perfect solution not only for FrameRateConverter but for any thing use MAnalyze like Motion Compensated and Temporal Denoising (like SMDegrain)

[johnmeyer]

don't know if it possible or not but I think most mvtools problems come from bad motion vectors that MAnalyze do, so I wonder if MAnalyze can be updated to use some ideas from RIFE (neural network) to make perfect solution not only for FrameRateConverter but for any thing use MAnalyze like Motion Compensated and Temporal Denoising (like SMDegrain)Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

[real.finder]

Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

yes I know it will not be perfect improved

with "perfect solution" I mean as "RIFE incorporation into FrameRateConverter" plus improve mvtools in general

[zambelli]

Any plans for 10-bit YUV support?

[takla]

Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

Interpolating 60 fps from 24 fps is just moronic.

[Selur]

Any updates on Vapoursynth&RIFE support? (adding support for RIFE-v4 model and the new skip&skip_threshold might be a good idea)

[Selur]

Any news?

[MysteryX]

Updates? Released a few software for Windows/Linux/MacOS

https://sourceforge.net/projects/converter432hz/
https://sourceforge.net/projects/player432hz/
https://sourceforge.net/projects/yangdownloader/
https://sourceforge.net/projects/powerliminals-player/
plus xClean script

Haven't got back to FrameRateConverter yet.

[Selur]

Haven't got back to FrameRateConverter yet.
:( fingers crossed this will change soon, really like the idea of FrameRateConverter using RIFE properly :)

[kedautinh12]

Yeah, RIFE support avs and vs now. Maybe need new ver of FrameRateConverter using RIFE

[MysteryX]

if RIFE supports AVS, it should be easy to do.

It's porting it to VapourSynth that is more complicated; particularly that the MvTools2 masks are completely different under VapourSynth!

I'm now on Linux and not currently setup for Avisynth development, but if someone wants to do it...

Just edit the frame blending clip like this: run RIFE once or twice before frame blending and voilà!

Could have a parameter "rife" set to -1 for automatic, 0 to disable, 1 or more to manually specify how many passes of doubling frames. We still need frame-blending to reach the exact desired framerate.

[kedautinh12]

:( fingers crossed this will change soon, really like the idea of FrameRateConverter using RIFE properly :)

Dogway add RIFE to FrameRateConverter now
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverterMIX.avsi

[MysteryX]

How is RIFE working out?

[Dogway]

I did some tests and found that DCT=1, DCTRe=4 and CalcDiff=true (so 'Slowest' preset with DCTRe=4) made improvements for blending and some distortions.
Only tested on one shot but compared to no-RIFE is simply at another level. RIFEAnime stays at 'Slow' preset. Maybe you can check if implementation is correct.

[johnmeyer]

DCT=1 sometimes (not always) makes improvements in motion estimation artifacts, but at a 5x speed penalty! It is definitely worth testing on each video before proceeding.

[Dogway]

I think RIFE is also very slow so maybe it can be thought as a 'Placebo' preset, despite not being placebo at all.
I will try to run it through some popular clips like the one with the fence or the famous one @zorr used for zopti.

[takla]

Script error: the named argument "str" to StripeMask had the wrong type
FrameRateConverter.avsi, line 219 (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi#L219)

And
with Dogways FrameRateConverterMIX.avsi (https://raw.githubusercontent.com/Dogway/Avisynth-Scripts/master/MIX%20mods/FrameRateConverterMIX.avsi) I get

Evaluate: operands of "%" must be integers
ExTools.avsi, line 7509 (https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi#L7509)