FrameRateConverter (Official)

[MysteryX]

After discussing and developing FrameRateConverter in this thread (https://forum.doom9.org/showthread.php?t=174410), it's time for FrameRateConverter to have its own thread.

Latest version: 1.3
https://github.com/mysteryx93/FrameRateConverter/releases

Documentation (https://github.com/mysteryx93/FrameRateConverter)


Based on feedback, quality is good enough on a wide range of content. Performance and stability however still could be improved. DCT=1 occasionally freezes with MT, and MvTools2 in general gives poor performance with MT.

FrameRateConverter has special handling for stripes that consistently failed. Thus, stripes look good with this. SVP/Interframe has blurrier results than it should due to a bug (after comparing with MvTools with exact same settings), and FrameRateConverter has superior artifact masking, resulting in overall much better results.

FrameRateConverter also works well with anime, but DCT=0 gives crap so you'll want to use Preset="slow" for DCT=4, or manually set DCT=1.

[Groucho2004]

Just a note - This (https://github.com/mysteryx93/FrameRateConverter/commit/0b613d4d7a8bb7055d9c57e6dbcdb11128de6d5a) is not a bug, I pointed out the cause (https://forum.doom9.org/showthread.php?p=1810240#post1810240) a while ago.

[MysteryX]

Have I updated the headers since, or it slipped by?

According to timestamp, headers were uploaded 3 months ago which is before your post. So I still need to update headers.

Assembly optimization also needs to be added (Pinterf volunteered to do it when he'll have time)

[MysteryX]

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

None
RgTools.RemoveGrain(22)
RgTools.RemoveGrain(21)
modPlus.Median(uu=true,vv=true)
modPlus.Minvar(lx=120, ty=100)

https://s27.postimg.org/m7d77oshb/Prefilter7076_None.png (http://postimg.org/image/m7d77oshb/) https://s27.postimg.org/jmfpmmnz3/Prefilter7076_Remove_Grain21.png (http://postimg.org/image/jmfpmmnz3/) https://s27.postimg.org/cyb0xkjlb/Prefilter7076_Remove_Grain.png (http://postimg.org/image/cyb0xkjlb/) https://s27.postimg.org/vbbm84u27/Prefilter7076_Median.png (http://postimg.org/image/vbbm84u27/) https://s27.postimg.org/9q6jkixbj/Prefilter7076_Minvar.png (http://postimg.org/image/9q6jkixbj/)

https://s27.postimg.org/ayuf65p9r/Prefilter7249_None.png (http://postimg.org/image/ayuf65p9r/) https://s27.postimg.org/f1tj7p49r/Prefilter7249_Remove_Grain21.png (http://postimg.org/image/f1tj7p49r/) https://s27.postimg.org/o4eiq08j3/Prefilter7249_Remove_Grain.png (http://postimg.org/image/o4eiq08j3/) https://s27.postimg.org/4jve9hijz/Prefilter7249_Median.png (http://postimg.org/image/4jve9hijz/) https://s27.postimg.org/3t2o3pg6n/Prefilter7249_Minvar.png (http://postimg.org/image/3t2o3pg6n/)

Median is a LOT slower. Minvar gives a bit more of a plastic effect. So far RemoveGrain still wins, and RemoveGrain(21) shows notable benefits over RemoveGrain(22)

[MysteryX]

Here are some encoding samples

HYUNA - How's This (https://www.youtube.com/watch?v=y882AFjrSOM)
Preset=Normal (https://mega.nz/#!ON4RxT4T!3rrSkJFCvOq2nP1R9mDNIctG3VeMQCoV5AF4dQB0LO4)
Preset=Slower (https://mega.nz/#!LABDhT7K!kPscph_VgYbPsFDe6gtcQ-Sizu2g3PLuaR99ypLQsTs)

Girl's Day - Female President (https://www.youtube.com/watch?v=v0f9ifrDSp8)
Preset=Normal (https://mega.nz/#!WZIDCbqQ!xfJB1j_SP4aRr3rMm2x3A6f2eh1K0u_yv6LP8P46EmU)
Preset=Slower (https://mega.nz/#!nERUwLIB!ebjGFO7Et4j0x8OzkazDe_H5wwDmgWCnb5UHmikHLLU)

I have the issue that x265 60fps content can't play properly in MPC-HC nor decode in real-time in a Avisynth script, so I'm encoding in x264 instead.

[MysteryX]

Performance-wise, Preset=Normal (DCT=0) is working relatively well with MT. I'm encoding a 1080p video at 5fps right now with 8 threads, at 79% CPU usage (1.87ghz of 2.38ghz). It might however freeze after a while or become unstable on the long run. On its own it doesn't make very good use of CPU but combined with x264 encoding, it fills up the CPU pretty well.

Presets Slow, Slower and Slowest (DCT=1 and 4) don't work well with MT at all. Performance is low and it will regularly freeze. It works best by running several ST instances in parallel.

[MysteryX]

To convert 1080p YouTube videos to 60fps, I recommend this script

Encode with x264 Q=23 Preset="slower". Single-threaded, but you can run 2 or 4 instances at once for best performance.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Preset="slower", Prefilter=RemoveGrain(21))


or for faster processing

Encode with x264 Q=23 Preset="slow". Here you can use MT.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Prefilter=RemoveGrain(21))
Prefetch(8)


If downloading videos from YouTube, always use the MP4 version if possible, not the VP9 video. Why?

Here's a comparison between MP4 and MP9 after encoding to x264. The MP4 video is just slightly larger but VP9 is 40% more efficient thus MP4 should be lower quality.

After encoding however... MP4 / VP9
https://s28.postimg.org/x3qhybce1/Encode264.png (http://postimg.org/image/x3qhybce1/) https://s28.postimg.org/6jxww6buh/Encode_VP9.png (http://postimg.org/image/6jxww6buh/)

VP9 doesn't re-encode well at all back to x264 because these 2 formats don't keep the same kind of data. x264 works with pixels, VP9 works with vectors. Converting back to x264 causes massive loss of details and a plastic effect. x265 doesn't decode well at 60fps.

If there are recommendations to improve this process, you can post your suggestions (very light denoising perhaps?)

[SpoCk0nd0pe]

- Added Preset Slowest that calculates diff between DCT=1 and DCT=0

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?

What are your favourite settings for encoding modern high quality blue ray content?

Thank you very much for your continued work on this!

[MysteryX]

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?
That's what preset Slower does.

What are your favourite settings for encoding modern high quality blue ray content?
I use preset Slower for HD content; and for SD content. For 1080p, I have to count a minute of processing per second, or an hour per minute, mostly due to poor MT support.

The reason I go for Slower is that you're going to lose quality by re-encoding. You need sufficiently high quality output to make up for it.


I'm also wondering. Is DCT=1 / DCT=4 something with which the GPU would perform well?

[burfadel]

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

How about if the prefilter is only applied to the recalculation steps, and not the main analysis? Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.

[MysteryX]

How about if the prefilter is only applied to the recalculation steps, and not the main analysis?
I trust John tested extensively and have reason to do it that way. You can play with that but I doubt you'll see improvements.

Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.
Good catch! Main code block also had this missing. This however has no impact unless BlkSizeV is different than BlkSize.

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.
I tried changing searchparam from 2 to 1. Sometimes it's better, sometimes it's worse. No definite gain.

[MysteryX]

Something that *could* be done is doing a diff between slightly different settings since small settings changes can result in considerably different output, which is good for diff. We just need same block size for it to work well. For example, Search could be different for both versions. This could allow removing more artifacts without using DCT, but I doubt the result will be as good.

[MysteryX]

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

[Selur]

Any plans for a Vapoursynth version of this filter?

[MysteryX]

Any plans for a Vapoursynth version of this filter?

I have no experience with VapourSynth whatsoever

[Selur]

Using a simple script:
SetMemoryMax(768)
SetMTMode(5,16) # change MT mode
LoadCPlugin("G:\Hybrid\AVISYN~1\ffms2.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\FrameRateConverter.dll") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
LoadPlugin("G:\Hybrid\AVISYN~1\mvtools2.dll") # from: https://github.com/pinterf/mvtools/releases/tag/2.7.21.22
LoadPlugin("G:\Hybrid\AVISYN~1\masktools2.dll") # from: https://github.com/pinterf/masktools/releases/tag/2.2.10
LoadPlugin("G:\Hybrid\AVISYN~1\GRunT.dll") # from https://forum.doom9.org/showthread.php?t=139337
Import("G:\Hybrid\avisynthPlugins\FrameRateConverter.avsi") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
# loading source: F:\TestClips&Co\test.avi
# input luminance scale tv
FFVideoSource("F:\TESTCL~1\test.avi",cachefile="H:\Temp\avi_0197468a3716844ade54f3f30f60eeda_4827_1_0.ffindex",fpsnum=25)
# current resolution: 640x352
SetMTMode(2) # change MT mode
FrameRateConverter(NewNum=50,NewDen=1)
distributor()
return last
I get:
MAnalyse: Block's size must be 4x4, 8x4, 8x8, 16x2, 16x8, 16x16, 32x16, 32x32
Is this a bug or am I missing something?
(first thought this was due to my resolution of 640x352, but even when using a 1920x1080 source I get the same error,..)

-------------------

@ Preset - The speed/quality preset [slower|slow|normal|fast]. (default=normal)
vs.
Pset = Preset == "slowest" ? P_SLOWEST : Preset == "slower" ? P_SLOWER : Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
-> you might want to add 'slowest' to the description text,...


Cu Selur

[MysteryX]

Additional block sizes were added in Pinterf's recent version of MvTools2, you'll need that otherwise you need to manually set BlkSize to one of those values

[Selur]

Ah, okay, I thought I was using that version, but you are right I redownloaded it again and now it's working.
Thanks!

[MysteryX]

I was looking for a way to fix duplicate frames and the solution was to replace them with frame interpolation, and I found FillDropsI from johnmeyer -- except that it uses basic interpolation without artifact masking. So I thought.. how about modifying it to use FrameRateConverter for interpolation?

Any comments on this?

I want to add "keep", 1 to keep 1st frame and interpolate the 2nd, and 2 to keep 2nd frame and interpolate the first, because in the video I'm testing the 2nd frame is better. I tried simply replacing the function YDifferenceFromPrevious with YDifferenceToNext but that didn't work. Any idea how to get that to work? Also is there a way to interpolate without separating fields? I might give better interpolation on whole frames, but I wasn't successful at doing that.

Once this script is polished I'll release it in the same script file.


function InterpolateDoubles(clip c, float "thr", string "preset")
{
thr = default(thr, .1)
preset = default(preset, "normal")

even = c.SeparateFields().SelectEven()
even_flow = FrameRateConverter(even, Preset=preset, FrameDouble=true).SelectOdd()

odd = c.SeparateFields().SelectOdd()
odd_flow = FrameRateConverter(odd, Preset=preset, FrameDouble=true).SelectOdd()

even_fixed = ConditionalFilter(even, even_flow, even, "YDifferenceFromPrevious()", "lessthan", string(thr))
odd_fixed = ConditionalFilter(odd, odd_flow, odd, "YDifferenceFromPrevious()", "lessthan", string(thr))

Interleave(even_fixed, odd_fixed)
return Weave()
}


Running this script before mClean kind of works but it freezes (no MT)

[SpoCk0nd0pe]

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

I am currently running a test with two parallel workers on 1080p content (Game of Thrones).
My Avisynth templates reads:
<input>
<deinterlace>
<crop>
<resize>
<denoise>
FrameRateConverter(23976, 500, "slower", 16)

My x264 settings are essentially "slower" (I added some higher quality settings) with constant quality 20.

My I7 920@3.8 GHz is doing two workers at 0.64 FPS each. My ram (6gb) is pretty much filled. I think the performance is reasonable. When Vega 56 cards with board partner coolers arrive, I will switch to a Ryzen 7 1700 with 16gb ram, that should help performance a lot.

Is there a setting for 3d material? That is the content profiting the most from 48 fps imho.

[burfadel]

I was looking for a way to fix duplicate frames and the solution was to replace them with frame interpolation, and I found FillDropsI from johnmeyer -- except that it uses basic interpolation without artifact masking. So I thought.. how about modifying it to use FrameRateConverter for interpolation?

Any comments on this?

I want to add "keep", 1 to keep 1st frame and interpolate the 2nd, and 2 to keep 2nd frame and interpolate the first, because in the video I'm testing the 2nd frame is better. I tried simply replacing the function YDifferenceFromPrevious with YDifferenceToNext but that didn't work. Any idea how to get that to work? Also is there a way to interpolate without separating fields? I might give better interpolation on whole frames, but I wasn't successful at doing that.

Once this script is polished I'll release it in the same script file.


function InterpolateDoubles(clip c, float "thr", string "preset")
{
thr = default(thr, .1)
preset = default(preset, "normal")

even = c.SeparateFields().SelectEven()
even_flow = FrameRateConverter(even, Preset=preset, FrameDouble=true).SelectOdd()

odd = c.SeparateFields().SelectOdd()
odd_flow = FrameRateConverter(odd, Preset=preset, FrameDouble=true).SelectOdd()

even_fixed = ConditionalFilter(even, even_flow, even, "YDifferenceFromPrevious()", "lessthan", string(thr))
odd_fixed = ConditionalFilter(odd, odd_flow, odd, "YDifferenceFromPrevious()", "lessthan", string(thr))

Interleave(even_fixed, odd_fixed)
return Weave()
}


Running this script before mClean kind of works but it freezes (no MT)

There's been various scripts like that. There's blendupes:
https://forum.doom9.org/showpost.php?p=1764263&postcount=18

Morphdupes_MI
https://forum.doom9.org/showpost.php?p=1764867&postcount=20
https://forum.doom9.org/showpost.php?p=1764867&postcount=21
(split script)

Filldrops3
https://forum.doom9.org/showpost.php?p=1764192&postcount=1

Filldrops3 could very easily and effectively be incorporated :). It would also have little impact on performance seeing as the mvanalyse has already been done.

[burfadel]

Did you find the solution? Using MFlowInter is a simple solution.

[MysteryX]

I haven't looked into it further. If I want to use MFlowInter, then I can't simply call FrameRateConverter and benefit from its features.

I'll also looking into increasing SkipOver from 120 to something like 210. Highly animated scenes get skipped and it breaks the smoothness.

Also using DCT=4 for MRecalculate is a good idea.

[burfadel]

Animated sources are difficult, ideally you need to incorporate using MCompensate, or MFlow and then use correction masks. MFlow does a great job most of the time, and makes up for it by adding it's own special effects at scene changes. Then you need the scene detection function... The good thing about using any of that is it can be used for normal scenes as well. Use normal clip for MAnalyse, use that for compensation, then use a super of that for MRecalculate. Or a variation on that principle.

[SpoCk0nd0pe]

From the latest changelog in mvTools2:

Fix: [DCT 8x8@8bit] safe multithreading for integer DCT (8x8 block size, 8 bit video): assembly had a single working buffer.

Does this fix the MT problems?

If I got it correctly, your block size recommendations are for 4:3. What blocksize would you chose for 1080p?

[MysteryX]

From the latest changelog in mvTools2:

Fix: [DCT 8x8@8bit] safe multithreading for integer DCT (8x8 block size, 8 bit video): assembly had a single working buffer.

Does this fix the MT problems?
With BlkSize=8, CPU still goes only up to 29% with Prefetch(8)

If I got it correctly, your block size recommendations are for 4:3. What blocksize would you chose for 1080p?
I treat 4:3 or widescreen videos the same.

As per documentation, default block sizes are, based on height
0-359: 8
360-749: 12
750-1199: 16
1200-1699: 24
1600-2160: 32

[MysteryX]

Version 1.1 is released! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.1)

What's new:
- Added InterpolateDoubles function to replace double frames with interpolated frames using FrameRateConverter
- Added DctRe parameter to specify DCT for MRecalculate
- DCT now also applied for MRecalculate by default
- Preset Normal now uses DCT=4 for MRecalculate. Old Normal is now Fast and old Fast is now Faster.
- Presets Slow and Slower also now use DCT=4 for MRecalculate
- SkipOver default treshold increased from 120 to 210
- BlendOver default treshold increased from 60 to 65
- SkipOver now specified for thSCD2 of MMask. Previously, scene changes would cause 2 adjacent frames to be skipped. Specifying thSCD2 causes only 1 frame to be marked for scene change.
- Updated Avisynth headers in DLL
- StripeMask: renamed trh parameter to thr

I also just did some tests with Stripes="Skip" and it's not practical for real videos, it often gives very weird results. I might remove it in next version.

I'm also open to feedback to improve InterpolateDoubles.

[MysteryX]

There is one issue with using DCT=4 for MRecalculate: it causes masks to be MUCH weaker, and thus pretty much disables artifact masking!

Would MFlowBlur be useful in that script? Would it be possible to use MFlowBlur instead of frame blending for artifact masking?

[burfadel]

It's possible, I was thinking that myself for mClean. For video temporal quality is much more important than single frame comparisons, particularly those where people have to zoom in or look hard to determine the difference. If there is fast motion it should be perceptively better. With motion analysis derived results, like an arm moving fast you get that opaque overlay look which is particularly noticeable in animation. MFlowBlur may make this area 'heavier' which wouldn't be good as it will make the arm look unnaturally fat momentarily. If you can detect that there is motion opaqueness caused by MFlowFPS, you can mask this and return the good detail from the previous or next frame. This will likely make the arm appear thinner. If you then apply MFlowBlur, you should get a smooth, and perceptively be motion artifact free assuming the source was free of this.

I'd say it would be difficult to implement! The temporal quality should be much higher though, and spatial comparisons should be more favourable as well.

[MysteryX]

How do you use MFlowBlur anyway? It doesn't have a FPS parameter like MFlowFPS, so how do I generate a matching clip?

[burfadel]

I believe it recreates existing frames based on the motion vector data of previous and next frames provided by MAnalyse. I was just thinking that if you ran MFlowBlur or Mcompensate before MflowFPS, and work out the clip stuff (super,superfilt) based on this, it could produce good results? I think it all comes down to experimenting with the functions.

[MysteryX]

I asked Chainik about it
> Does MFlowBlur generate artifacts?

yes, since it uses the very same (i.e. wrong) motion vectors


It's useless then.

I also tried it. Takes the source clip and kind of blurs/distorts it. It's not generating any in-between frames. Useless.

[MysteryX]

OK after testing the mask of MRecalculate with DCT=4, overall, MAnalyze gives a much weaker mask with DCT=4 than with DCT=0. DCT=4 and DCT=1 give the same mask. If you do MAnalyze with DCT=0 and MRecalculate with DCT=4, you get the same mask as if MAnalyze was done with DCT=4.

Thus, with the latest release, presets Slow/Slower that were already using DCT=4 will behave similarly but with slightly better quality. Preset Normal will have much lower artifact masking because of MRecalculate. For artifact masking, perhaps some adjustment could be made to take this mask strength discrepancy into consideration.

This however can cause issues for scene change detection if masks are too weak.

In terms of mask strength discrepancy, here's what I'm talking about

MAnalyze DCT 0, 1, 2, 3, 4
https://s26.postimg.org/in1q4s35x/MAnalyze0.png (http://postimg.org/image/in1q4s35x/) https://s26.postimg.org/3sms1gpad/MAnalyze1.png (http://postimg.org/image/3sms1gpad/) https://s26.postimg.org/in1q4s35x/MAnalyze0.png (https://postimg.org/image/in1q4s35x/) https://s26.postimg.org/n09wy27lx/MAnalyze3.png (http://postimg.org/image/n09wy27lx/) https://s26.postimg.org/jt5q3krgl/MAnalyze4.png (http://postimg.org/image/jt5q3krgl/)

MAnalyze+MRecalculate DCT 0, 1, 2, 3, 4
https://s26.postimg.org/3sd4qltl1/MRecalculate0.png (http://postimg.org/image/3sd4qltl1/) https://s26.postimg.org/p3kc5q7et/MRecalculate1.png (http://postimg.org/image/p3kc5q7et/) https://s26.postimg.org/5oe5qd3id/MRecalculate2.png (http://postimg.org/image/5oe5qd3id/) https://s26.postimg.org/qmkbog3d1/MRecalculate3.png (http://postimg.org/image/qmkbog3d1/) https://s26.postimg.org/6aytrafb9/MRecalculate4.png (http://postimg.org/image/6aytrafb9/)

[hello_hello]

MysteryX,
I was experimenting with frame interpolation on some animation today and noticed FrameRateConverter (both versions 1.0 and 1.1) tends to lose the plot during fade-outs. I haven't fiddled with settings too much yet so I don't know if there's an adjustment that'd fix it, although it seems to be the sort of thing that probably shouldn't happen by default.

There's three MKVs in the zip file I've linked to if you'd care to look. "12fps.mkv" is simply a section of the original DVD cropped and resized and decimated to 12fps. It's for reference.
"FrameRateConverter.mkv" is the result of adding FrameRateConverter(24,1) to the script after resizing, and "Interframe.mkv" is the result of adding Interframe(cores=1, NewNum=24, NewDen=1).

FrameRateConverter seems to do a better job than Interframe aside from the fade-outs. If you look at the grey area near the middle of the picture a bit after frame 120 and step through the frames one at a time as it fades out, you'll see the shadow over the grey area appears to move up and down. A similar thing happens a little later as the girl fades out and the shot of the earth behind her distorts a little in the direction it's moving. It appears FrameRateConverter might be mistaking the fading-out for movement.

Cheers.

12fps-24fps.zip (https://files.videohelp.com/u/210984/12fps-24fps.zip) (3.3MB)

Edit: For the record I tried manolito's earlier version of the script here (https://forum.doom9.org/showthread.php?p=1805050#post1805050) and the fade-out problem was far worse. I must have saved StainlessS's DoubleRate (https://forum.doom9.org/showthread.php?p=1785080#post1785080) script at some stage so I tried it and the effect was reduced, although still present.

Thanks.

[MysteryX]

I've now adjusted the masks to look similar with different DCT values. Still need to do some testing.

hello_hello, for anime, use Preset="slow". With DCT=4, the fade out is good. There's a bit of a warping effect at 239-242. When the lady fades out, the mask is too weak to skip that, but on fade in, I get a Skip value of 46. Set BlendOver=40. I think it's a good idea to set BlendOver to a lower value for anime.

Perhaps I could add a preset "Anime" that changes BlendOver/SkipOver values, but that would require more testing. I'll release this version with re-adjustments to the masks and we'll see from there what settings need to be tweaked for anime.

Also, if you double the frame rate, use FrameDouble=true instead of NewNum to preserve source frames. I'm editing the script to set FrameDouble to true if NewNum is double the frame rate.

[MysteryX]

Version 1.2 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2)

What's new:
- Mask strength now adjusted to produce similar artifacts masking with different DCT values
- Removed Stripes parameter as Skip isn't practical
- Added Stp parameter, whether to detect and blend stripes
- Changed MaskTrh default from 100 to 120
- Changed BlendOver default from 65 to 70
- If NewNum is twice the clip's frame rate, set FrameDouble to true by default to preserve source frames

You can test preset="anime" to see which settings work best. This new preset hasn't yet been tested and is likely to be tweaked.

[manolito]

Edit: For the record I tried manolito's earlier version of the script here (https://forum.doom9.org/showthread.php?p=1805050#post1805050) and the fade-out problem was far worse.

The only difference with your command line between FRC 1.0 and my simplified script is the block size. FRC defaults to 12 for this frame size while my script uses 16 (a block size of 12 is not supported by older versions of MVTools2).

I did a few tests, and for me the clip looks best with a block size of 32 (happens often with anime). DCT=1 did not make much difference, though. Using blksize=32 fixes the fluttering grey shadow completely, but when the planet enters the window, the first couple of frames have distorted movement. I think that this is far less visible than the grey shadow, it only can be detected by stepping through the frames.


Cheers
manolito

[burfadel]

Version 1.2 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2)

What's new:
- Mask strength now adjusted to produce similar artifacts masking with different DCT values
- Removed Stripes parameter as Skip isn't practical
- Added Stp parameter, whether to detect and blend stripes
- Changed MaskTrh default from 100 to 120
- Changed BlendOver default from 65 to 70
- If NewNum is twice the clip's frame rate, set FrameDouble to true by default to preserve source frames

You can test preset="anime" to see which settings work best. This new preset hasn't yet been tested and is likely to be tweaked.

You didn't make a binary of this version, you only provided the source code :).

[hello_hello]

Thanks for the info MysteryX and manolito.

I haven't had a chance to play around more yet. Probably not until tomorrow, but I was wondering.....

I didn't give any thought to it at the time, but would frame interpolation work better at higher resolutions? I resized first, thinking it'd be faster:

TDecimate(mode=2, rate=12, Input="D:\ep1.txt")
crop(14, 4, -14, -4)
Spline36Resize(704,396)
FrameRateConverter(24,1)

I'll use FrameDouble=true next time, but would it make much difference if I put FrameRateConverter before the cropping and resizing? It's a PAL DVD so there's 720x576 resolution to begin with, even if there's not 720x576 worth of picture detail as such.

Cheers.

[MysteryX]

I did a few tests, and for me the clip looks best with a block size of 32 (happens often with anime).
Right, forgot about that. Preset "anime" should use higher block sizes by default, although that still needs to be tested and tweaked.

You didn't make a binary of this version, you only provided the source code :).
Oups forgot to upload! Fixed :)

would it make much difference if I put FrameRateConverter before the cropping and resizing?
For 288p to 768p, it's much better if interpolation is done last. For 720p to 1080p, I'm not sure whether it's best to interpolate before or after, I still need to test that. One thing for sure: interpolation is expensive. If you interpolate after upscaling, it will be slower. Test both ways, opening each version in a different instance of VirtualDub, and compare the frames.

I've just re-encoded a very difficult and animated video with this version. Quality is considerably improved. Output file size is also considerably different: 209mb instead of 205mb.

[hello_hello]

I was actually downscaling ie 720x576 to 704x396. Chances are it's not enough to matter but I thought I'd ask.
I still haven't had a chance to do more testing but I'll report back when I do.

Thanks.

[MysteryX]

Normally, interpolate at the end.

If the details are too low and interpolation is causing artifacts, then you can also try interpolating first. Let me know which works best.

You may want to use AvisynthShader's SSIM downscaler. I haven't tested it much myself but it's giving great results. (I'll use it to downscale 4K YouTube content to 1080p)

[SpoCk0nd0pe]

I tried your new script version and got a divide by 0 error in line 132 :/

1080p content at slower, stock settings

[MysteryX]

I tried your new script version and got a divide by 0 error in line 132 :/

1080p content at slower, stock settings
Line 132 is

B = C.ConvertFps(NewNum, NewDen)


Did you specify valid NewNum and NewDen? What's your script?

[MysteryX]

I saw a movie clip with a calm background and high-artifact motions in the middle. The frame should get blended but because the background is calm the SkipOver value is too low.

The only way I can think to fix that would be to analyze the vectors data to determine what percentage of vectors are below a certain movement threshold. If it's a low-movement background, blending the whole frame doesn't degrade quality much. I could say if the amount of slow-moving blocks is above a certain percentage of the frame, reduce BlendOver by half. This should help in those scenes.

I have no idea what vectors data looks like though! Anyone knows how this works?

According to doc (http://www.avisynth.nl/users/fizick/mvtools/mvtools2.html)


outfile: name of file to write motion vectors data. This data may be used by some external program or may be by next MVTools versions for second pass coding, etc. Produced binary file has a header (MVAnalysisData structure, see MVInterface.h source code), and the data sequence:
frame number, vector data (Vx, Vy, SAD) of every block, next valid frame number, this frame vector data, and so on.


I was thinking to simply use YDifferenceFromPrevious to detect how much movement is in the frame but that would cause low-contrast action scenes to be blended.

As for Anime preset, perhaps we could simply double default BlkSize; this would have to be tested.

[hello_hello]

The only difference with your command line between FRC 1.0 and my simplified script is the block size. FRC defaults to 12 for this frame size while my script uses 16 (a block size of 12 is not supported by older versions of MVTools2).

I may have done your version of the script a disservice. I didn't realise you'd updated it since I saved it. The version I was using is dated 18-May-2017. Is this the latest version of your version? https://forum.doom9.org/showthread.php?p=1805050#post1805050
I'll try again fairly soon with the newer version while I'm testing the newer FrameRateConverter and the settings suggested by MysteryX, but if that's the latest, you might want to go through it and replace all instances of trh with thr. I recall MysteryX doing the same thing. I think there's only one line where trh is used as an argument and causes an error, but it'd make sense to change MaskTrh to MaskThr and SkipTrh to SkipThr anyway.

Edit The 22-June-17 version of your version of the script is better in respect to the fade-out issue than the 18-May-2017 version I tried originally, so it's probably no worse than the official FrameRateConverter script in that respect.

Cheers.

[hello_hello]

MysteryX,
The help file seems to imply BlkSize=6 is supported, but FrameRateConverter(FrameDouble=true, BlkSize=6) results in this (version 1.2):

MVTools: no BLITCHROMA function for block size 1x1
(FrameRateConverter.avsi, line 153)

[hello_hello]

As for Anime preset, perhaps we could simply double default BlkSize; this would have to be tested.

I tried it but I prefer the default block size (I think I even preferred the default to reducing it to 8 (I assume at 704x396 the default BlkSize is 12?).

BlkSize=24 seems to fix the fade-out issue, or at least improve it greatly, but it increases distortion in areas where one object is moving across another stationary object. I think I'd prefer some minor wobble during fade-outs.

You may want to use AvisynthShader's SSIM downscaler. I haven't tested it much myself but it's giving great results. (I'll use it to downscale 4K YouTube content to 1080p)

I gave it a try but kept getting an error relating to a missing function. It was yesterday and I didn't make a note of it because I thought I'd work on one problem at a time, but I think it was "no function get_bitdepth" or something similar.

I tried switching MeGUI to using it's portable version of AvIsynth+ (I have the standard AVIsynth installed) but that didn't help. Apparently MeGUI is using r2508. Or maybe it's an XP thing?

Anyway, that'll be a problem for another day. I'll keep playing around with FrameRateConverter settings for the moment.

Thanks.

[MysteryX]

hello_hello, when testing settings, set debug=true to see what it's doing

"no function get_bitdepth" that means you didn't load AvisynthShader.dll

[hello_hello]

There's shader.dll in the zip file but no AvisynthShader.dll, is that something different again? I definitely had shader.dll in the Avisynth plugins folder. Maybe it's a dependency problem. My video card is pretty ancient so I'm not to optimistic as to how well it'd work anyway.

I tried version 1.50 just to see what'd happen and it resulted in "no function named ConvertToShader" and when I went back to 1.6.2 it was "no function named shader_GetBitdepth".

Best as a can tell, and keeping in mind I have no idea what I'm talking about, the Dependency Walker clues seem to indicate it's an XP issue.
https://msdn.microsoft.com/en-us/library/windows/desktop/bb219676%28v=vs.85%29.aspx

Not to worry, I'm not fussed about that for the moment. I'm still playing around with FrameRateConverter.

PS This: FrameRateConverter(FrameDouble=true, BlkSize=6, debug=true)
still results in this error:
MVTools: no BLITCHROMA function for block size 1x1
(FrameRateConverter.avsi, line 153)

(mvtools2 2.7.22)