FrameRateConverter (Official)

[MysteryX]

After discussing and developing FrameRateConverter in this thread, it's time for FrameRateConverter to have its own thread.

Latest version: 1.3
https://github.com/mysteryx93/FrameR...erter/releases

Documentation


Based on feedback, quality is good enough on a wide range of content. Performance and stability however still could be improved. DCT=1 occasionally freezes with MT, and MvTools2 in general gives poor performance with MT.

FrameRateConverter has special handling for stripes that consistently failed. Thus, stripes look good with this. SVP/Interframe has blurrier results than it should due to a bug (after comparing with MvTools with exact same settings), and FrameRateConverter has superior artifact masking, resulting in overall much better results.

FrameRateConverter also works well with anime, but DCT=0 gives crap so you'll want to use Preset="slow" for DCT=4, or manually set DCT=1.

[Groucho2004]

Just a note - This is not a bug, I pointed out the cause a while ago.

[MysteryX]

Have I updated the headers since, or it slipped by?

According to timestamp, headers were uploaded 3 months ago which is before your post. So I still need to update headers.

Assembly optimization also needs to be added (Pinterf volunteered to do it when he'll have time)

[MysteryX]

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

None
RgTools.RemoveGrain(22)
RgTools.RemoveGrain(21)
modPlus.Median(uu=true,vv=true)
modPlus.Minvar(lx=120, ty=100)





Median is a LOT slower. Minvar gives a bit more of a plastic effect. So far RemoveGrain still wins, and RemoveGrain(21) shows notable benefits over RemoveGrain(22)

[MysteryX]

Here are some encoding samples

HYUNA - How's This
Preset=Normal
Preset=Slower

Girl's Day - Female President
Preset=Normal
Preset=Slower

I have the issue that x265 60fps content can't play properly in MPC-HC nor decode in real-time in a Avisynth script, so I'm encoding in x264 instead.

[MysteryX]

Performance-wise, Preset=Normal (DCT=0) is working relatively well with MT. I'm encoding a 1080p video at 5fps right now with 8 threads, at 79% CPU usage (1.87ghz of 2.38ghz). It might however freeze after a while or become unstable on the long run. On its own it doesn't make very good use of CPU but combined with x264 encoding, it fills up the CPU pretty well.

Presets Slow, Slower and Slowest (DCT=1 and 4) don't work well with MT at all. Performance is low and it will regularly freeze. It works best by running several ST instances in parallel.

[MysteryX]

To convert 1080p YouTube videos to 60fps, I recommend this script

Encode with x264 Q=23 Preset="slower". Single-threaded, but you can run 2 or 4 instances at once for best performance.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Preset="slower", Prefilter=RemoveGrain(21))

or for faster processing

Encode with x264 Q=23 Preset="slow". Here you can use MT.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Prefilter=RemoveGrain(21))
Prefetch(8)

If downloading videos from YouTube, always use the MP4 version if possible, not the VP9 video. Why?

Here's a comparison between MP4 and MP9 after encoding to x264. The MP4 video is just slightly larger but VP9 is 40% more efficient thus MP4 should be lower quality.

After encoding however... MP4 / VP9


VP9 doesn't re-encode well at all back to x264 because these 2 formats don't keep the same kind of data. x264 works with pixels, VP9 works with vectors. Converting back to x264 causes massive loss of details and a plastic effect. x265 doesn't decode well at 60fps.

If there are recommendations to improve this process, you can post your suggestions (very light denoising perhaps?)

[SpoCk0nd0pe]

Originally Posted by MysteryX

- Added Preset Slowest that calculates diff between DCT=1 and DCT=0

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?

What are your favourite settings for encoding modern high quality blue ray content?

Thank you very much for your continued work on this!

[MysteryX]

Originally Posted by SpoCk0nd0pe

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?

That's what preset Slower does.

Originally Posted by SpoCk0nd0pe

What are your favourite settings for encoding modern high quality blue ray content?

I use preset Slower for HD content; and for SD content. For 1080p, I have to count a minute of processing per second, or an hour per minute, mostly due to poor MT support.

The reason I go for Slower is that you're going to lose quality by re-encoding. You need sufficiently high quality output to make up for it.


I'm also wondering. Is DCT=1 / DCT=4 something with which the GPU would perform well?

[burfadel]

Originally Posted by MysteryX

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

How about if the prefilter is only applied to the recalculation steps, and not the main analysis? Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.

[MysteryX]

Originally Posted by burfadel

How about if the prefilter is only applied to the recalculation steps, and not the main analysis?

I trust John tested extensively and have reason to do it that way. You can play with that but I doubt you'll see improvements.

Originally Posted by burfadel

Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.

Good catch! Main code block also had this missing. This however has no impact unless BlkSizeV is different than BlkSize.

Originally Posted by burfadel

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.

I tried changing searchparam from 2 to 1. Sometimes it's better, sometimes it's worse. No definite gain.

[MysteryX]

Something that *could* be done is doing a diff between slightly different settings since small settings changes can result in considerably different output, which is good for diff. We just need same block size for it to work well. For example, Search could be different for both versions. This could allow removing more artifacts without using DCT, but I doubt the result will be as good.

[MysteryX]

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

[Selur]

Any plans for a Vapoursynth version of this filter?

[MysteryX]

Originally Posted by Selur

Any plans for a Vapoursynth version of this filter?

I have no experience with VapourSynth whatsoever

[Selur]

Using a simple script:

SetMemoryMax(768)
SetMTMode(5,16) # change MT mode
LoadCPlugin("G:\Hybrid\AVISYN~1\ffms2.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\FrameRateConverter.dll") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
LoadPlugin("G:\Hybrid\AVISYN~1\mvtools2.dll") # from: https://github.com/pinterf/mvtools/releases/tag/2.7.21.22
LoadPlugin("G:\Hybrid\AVISYN~1\masktools2.dll")  # from: https://github.com/pinterf/masktools/releases/tag/2.2.10
LoadPlugin("G:\Hybrid\AVISYN~1\GRunT.dll") # from https://forum.doom9.org/showthread.php?t=139337
Import("G:\Hybrid\avisynthPlugins\FrameRateConverter.avsi") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
# loading source: F:\TestClips&Co\test.avi
#  input luminance scale tv
FFVideoSource("F:\TESTCL~1\test.avi",cachefile="H:\Temp\avi_0197468a3716844ade54f3f30f60eeda_4827_1_0.ffindex",fpsnum=25)
# current resolution: 640x352
SetMTMode(2) # change MT mode
FrameRateConverter(NewNum=50,NewDen=1)
distributor()
return last

I get:

MAnalyse: Block's size must be 4x4, 8x4, 8x8, 16x2, 16x8, 16x16, 32x16, 32x32

Is this a bug or am I missing something?
(first thought this was due to my resolution of 640x352, but even when using a 1920x1080 source I get the same error,..)

-------------------

@ Preset - The speed/quality preset [slower|slow|normal|fast]. (default=normal)

vs.

Pset = Preset == "slowest" ? P_SLOWEST : Preset == "slower" ? P_SLOWER : Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1

-> you might want to add 'slowest' to the description text,...


Cu Selur

[MysteryX]

Additional block sizes were added in Pinterf's recent version of MvTools2, you'll need that otherwise you need to manually set BlkSize to one of those values

[Selur]

Ah, okay, I thought I was using that version, but you are right I redownloaded it again and now it's working.
Thanks!

[MysteryX]

I was looking for a way to fix duplicate frames and the solution was to replace them with frame interpolation, and I found FillDropsI from johnmeyer -- except that it uses basic interpolation without artifact masking. So I thought.. how about modifying it to use FrameRateConverter for interpolation?

Any comments on this?

I want to add "keep", 1 to keep 1st frame and interpolate the 2nd, and 2 to keep 2nd frame and interpolate the first, because in the video I'm testing the 2nd frame is better. I tried simply replacing the function YDifferenceFromPrevious with YDifferenceToNext but that didn't work. Any idea how to get that to work? Also is there a way to interpolate without separating fields? I might give better interpolation on whole frames, but I wasn't successful at doing that.

Once this script is polished I'll release it in the same script file.

function InterpolateDoubles(clip c, float "thr", string "preset")
{
    thr = default(thr, .1)
    preset = default(preset, "normal")

    even       = c.SeparateFields().SelectEven()
    even_flow  = FrameRateConverter(even, Preset=preset, FrameDouble=true).SelectOdd()

    odd        = c.SeparateFields().SelectOdd()
    odd_flow   = FrameRateConverter(odd,  Preset=preset, FrameDouble=true).SelectOdd()

    even_fixed = ConditionalFilter(even, even_flow, even, "YDifferenceFromPrevious()", "lessthan", string(thr))
    odd_fixed  = ConditionalFilter(odd,  odd_flow,  odd,  "YDifferenceFromPrevious()", "lessthan", string(thr))

    Interleave(even_fixed, odd_fixed)
    return Weave()
}

Running this script before mClean kind of works but it freezes (no MT)

[SpoCk0nd0pe]

Originally Posted by MysteryX

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

I am currently running a test with two parallel workers on 1080p content (Game of Thrones).
My Avisynth templates reads:
<input>
<deinterlace>
<crop>
<resize>
<denoise>
FrameRateConverter(23976, 500, "slower", 16)

My x264 settings are essentially "slower" (I added some higher quality settings) with constant quality 20.

My I7 920@3.8 GHz is doing two workers at 0.64 FPS each. My ram (6gb) is pretty much filled. I think the performance is reasonable. When Vega 56 cards with board partner coolers arrive, I will switch to a Ryzen 7 1700 with 16gb ram, that should help performance a lot.

Is there a setting for 3d material? That is the content profiting the most from 48 fps imho.