FrameRateConverter (Official)

[MysteryX]

After discussing and developing FrameRateConverter in this thread (https://forum.doom9.org/showthread.php?t=174410), it's time for FrameRateConverter to have its own thread.

Latest version: 1.3
https://github.com/mysteryx93/FrameRateConverter/releases

Documentation (https://github.com/mysteryx93/FrameRateConverter)


Based on feedback, quality is good enough on a wide range of content. Performance and stability however still could be improved. DCT=1 occasionally freezes with MT, and MvTools2 in general gives poor performance with MT.

FrameRateConverter has special handling for stripes that consistently failed. Thus, stripes look good with this. SVP/Interframe has blurrier results than it should due to a bug (after comparing with MvTools with exact same settings), and FrameRateConverter has superior artifact masking, resulting in overall much better results.

FrameRateConverter also works well with anime, but DCT=0 gives crap so you'll want to use Preset="slow" for DCT=4, or manually set DCT=1.

[Groucho2004]

Just a note - This (https://github.com/mysteryx93/FrameRateConverter/commit/0b613d4d7a8bb7055d9c57e6dbcdb11128de6d5a) is not a bug, I pointed out the cause (https://forum.doom9.org/showthread.php?p=1810240#post1810240) a while ago.

[MysteryX]

Have I updated the headers since, or it slipped by?

According to timestamp, headers were uploaded 3 months ago which is before your post. So I still need to update headers.

Assembly optimization also needs to be added (Pinterf volunteered to do it when he'll have time)

[MysteryX]

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

None
RgTools.RemoveGrain(22)
RgTools.RemoveGrain(21)
modPlus.Median(uu=true,vv=true)
modPlus.Minvar(lx=120, ty=100)

https://s27.postimg.org/m7d77oshb/Prefilter7076_None.png (http://postimg.org/image/m7d77oshb/) https://s27.postimg.org/jmfpmmnz3/Prefilter7076_Remove_Grain21.png (http://postimg.org/image/jmfpmmnz3/) https://s27.postimg.org/cyb0xkjlb/Prefilter7076_Remove_Grain.png (http://postimg.org/image/cyb0xkjlb/) https://s27.postimg.org/vbbm84u27/Prefilter7076_Median.png (http://postimg.org/image/vbbm84u27/) https://s27.postimg.org/9q6jkixbj/Prefilter7076_Minvar.png (http://postimg.org/image/9q6jkixbj/)

https://s27.postimg.org/ayuf65p9r/Prefilter7249_None.png (http://postimg.org/image/ayuf65p9r/) https://s27.postimg.org/f1tj7p49r/Prefilter7249_Remove_Grain21.png (http://postimg.org/image/f1tj7p49r/) https://s27.postimg.org/o4eiq08j3/Prefilter7249_Remove_Grain.png (http://postimg.org/image/o4eiq08j3/) https://s27.postimg.org/4jve9hijz/Prefilter7249_Median.png (http://postimg.org/image/4jve9hijz/) https://s27.postimg.org/3t2o3pg6n/Prefilter7249_Minvar.png (http://postimg.org/image/3t2o3pg6n/)

Median is a LOT slower. Minvar gives a bit more of a plastic effect. So far RemoveGrain still wins, and RemoveGrain(21) shows notable benefits over RemoveGrain(22)

[MysteryX]

Here are some encoding samples

HYUNA - How's This (https://www.youtube.com/watch?v=y882AFjrSOM)
Preset=Normal (https://mega.nz/#!ON4RxT4T!3rrSkJFCvOq2nP1R9mDNIctG3VeMQCoV5AF4dQB0LO4)
Preset=Slower (https://mega.nz/#!LABDhT7K!kPscph_VgYbPsFDe6gtcQ-Sizu2g3PLuaR99ypLQsTs)

Girl's Day - Female President (https://www.youtube.com/watch?v=v0f9ifrDSp8)
Preset=Normal (https://mega.nz/#!WZIDCbqQ!xfJB1j_SP4aRr3rMm2x3A6f2eh1K0u_yv6LP8P46EmU)
Preset=Slower (https://mega.nz/#!nERUwLIB!ebjGFO7Et4j0x8OzkazDe_H5wwDmgWCnb5UHmikHLLU)

I have the issue that x265 60fps content can't play properly in MPC-HC nor decode in real-time in a Avisynth script, so I'm encoding in x264 instead.

[MysteryX]

Performance-wise, Preset=Normal (DCT=0) is working relatively well with MT. I'm encoding a 1080p video at 5fps right now with 8 threads, at 79% CPU usage (1.87ghz of 2.38ghz). It might however freeze after a while or become unstable on the long run. On its own it doesn't make very good use of CPU but combined with x264 encoding, it fills up the CPU pretty well.

Presets Slow, Slower and Slowest (DCT=1 and 4) don't work well with MT at all. Performance is low and it will regularly freeze. It works best by running several ST instances in parallel.

[MysteryX]

To convert 1080p YouTube videos to 60fps, I recommend this script

Encode with x264 Q=23 Preset="slower". Single-threaded, but you can run 2 or 4 instances at once for best performance.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Preset="slower", Prefilter=RemoveGrain(21))


or for faster processing

Encode with x264 Q=23 Preset="slow". Here you can use MT.

file="Source.mp4"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
FrameRateConverter(NewNum=60, NewDen=1, Prefilter=RemoveGrain(21))
Prefetch(8)


If downloading videos from YouTube, always use the MP4 version if possible, not the VP9 video. Why?

Here's a comparison between MP4 and MP9 after encoding to x264. The MP4 video is just slightly larger but VP9 is 40% more efficient thus MP4 should be lower quality.

After encoding however... MP4 / VP9
https://s28.postimg.org/x3qhybce1/Encode264.png (http://postimg.org/image/x3qhybce1/) https://s28.postimg.org/6jxww6buh/Encode_VP9.png (http://postimg.org/image/6jxww6buh/)

VP9 doesn't re-encode well at all back to x264 because these 2 formats don't keep the same kind of data. x264 works with pixels, VP9 works with vectors. Converting back to x264 causes massive loss of details and a plastic effect. x265 doesn't decode well at 60fps.

If there are recommendations to improve this process, you can post your suggestions (very light denoising perhaps?)

[SpoCk0nd0pe]

- Added Preset Slowest that calculates diff between DCT=1 and DCT=0

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?

What are your favourite settings for encoding modern high quality blue ray content?

Thank you very much for your continued work on this!

[MysteryX]

What about a preset that calculates diff between DCT=4 and DCT=0? Or is that what Slower does now?
That's what preset Slower does.

What are your favourite settings for encoding modern high quality blue ray content?
I use preset Slower for HD content; and for SD content. For 1080p, I have to count a minute of processing per second, or an hour per minute, mostly due to poor MT support.

The reason I go for Slower is that you're going to lose quality by re-encoding. You need sufficiently high quality output to make up for it.


I'm also wondering. Is DCT=1 / DCT=4 something with which the GPU would perform well?

[burfadel]

I'm taking another look at the Prefilter for HD content (when not using a denoiser)

How about if the prefilter is only applied to the recalculation steps, and not the main analysis? Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.

[MysteryX]

How about if the prefilter is only applied to the recalculation steps, and not the main analysis?
I trust John tested extensively and have reason to do it that way. You can play with that but I doubt you'll see improvements.

Also in the script I noticed in the calcdiff section that overlapV is specified for the recalculated fwd2, bak2, as well as Manalyse bak2, but not Manalyse fwd2.
Good catch! Main code block also had this missing. This however has no impact unless BlkSizeV is different than BlkSize.

Also by tweaking the search parameters slightly it seemed to improve fast motion quality where it passes over a line.
I tried changing searchparam from 2 to 1. Sometimes it's better, sometimes it's worse. No definite gain.

[MysteryX]

Something that *could* be done is doing a diff between slightly different settings since small settings changes can result in considerably different output, which is good for diff. We just need same block size for it to work well. For example, Search could be different for both versions. This could allow removing more artifacts without using DCT, but I doubt the result will be as good.

[MysteryX]

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

[Selur]

Any plans for a Vapoursynth version of this filter?

[MysteryX]

Any plans for a Vapoursynth version of this filter?

I have no experience with VapourSynth whatsoever

[Selur]

Using a simple script:
SetMemoryMax(768)
SetMTMode(5,16) # change MT mode
LoadCPlugin("G:\Hybrid\AVISYN~1\ffms2.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\FrameRateConverter.dll") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
LoadPlugin("G:\Hybrid\AVISYN~1\mvtools2.dll") # from: https://github.com/pinterf/mvtools/releases/tag/2.7.21.22
LoadPlugin("G:\Hybrid\AVISYN~1\masktools2.dll") # from: https://github.com/pinterf/masktools/releases/tag/2.2.10
LoadPlugin("G:\Hybrid\AVISYN~1\GRunT.dll") # from https://forum.doom9.org/showthread.php?t=139337
Import("G:\Hybrid\avisynthPlugins\FrameRateConverter.avsi") # from https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.0
# loading source: F:\TestClips&Co\test.avi
# input luminance scale tv
FFVideoSource("F:\TESTCL~1\test.avi",cachefile="H:\Temp\avi_0197468a3716844ade54f3f30f60eeda_4827_1_0.ffindex",fpsnum=25)
# current resolution: 640x352
SetMTMode(2) # change MT mode
FrameRateConverter(NewNum=50,NewDen=1)
distributor()
return last
I get:
MAnalyse: Block's size must be 4x4, 8x4, 8x8, 16x2, 16x8, 16x16, 32x16, 32x32
Is this a bug or am I missing something?
(first thought this was due to my resolution of 640x352, but even when using a 1920x1080 source I get the same error,..)

-------------------

@ Preset - The speed/quality preset [slower|slow|normal|fast]. (default=normal)
vs.
Pset = Preset == "slowest" ? P_SLOWEST : Preset == "slower" ? P_SLOWER : Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
-> you might want to add 'slowest' to the description text,...


Cu Selur

[MysteryX]

Additional block sizes were added in Pinterf's recent version of MvTools2, you'll need that otherwise you need to manually set BlkSize to one of those values

[Selur]

Ah, okay, I thought I was using that version, but you are right I redownloaded it again and now it's working.
Thanks!

[MysteryX]

I was looking for a way to fix duplicate frames and the solution was to replace them with frame interpolation, and I found FillDropsI from johnmeyer -- except that it uses basic interpolation without artifact masking. So I thought.. how about modifying it to use FrameRateConverter for interpolation?

Any comments on this?

I want to add "keep", 1 to keep 1st frame and interpolate the 2nd, and 2 to keep 2nd frame and interpolate the first, because in the video I'm testing the 2nd frame is better. I tried simply replacing the function YDifferenceFromPrevious with YDifferenceToNext but that didn't work. Any idea how to get that to work? Also is there a way to interpolate without separating fields? I might give better interpolation on whole frames, but I wasn't successful at doing that.

Once this script is polished I'll release it in the same script file.


function InterpolateDoubles(clip c, float "thr", string "preset")
{
thr = default(thr, .1)
preset = default(preset, "normal")

even = c.SeparateFields().SelectEven()
even_flow = FrameRateConverter(even, Preset=preset, FrameDouble=true).SelectOdd()

odd = c.SeparateFields().SelectOdd()
odd_flow = FrameRateConverter(odd, Preset=preset, FrameDouble=true).SelectOdd()

even_fixed = ConditionalFilter(even, even_flow, even, "YDifferenceFromPrevious()", "lessthan", string(thr))
odd_fixed = ConditionalFilter(odd, odd_flow, odd, "YDifferenceFromPrevious()", "lessthan", string(thr))

Interleave(even_fixed, odd_fixed)
return Weave()
}


Running this script before mClean kind of works but it freezes (no MT)

[SpoCk0nd0pe]

As an update to some of the performance issues I was reporting where CPU would work at full frequency for a while and then slow down, it may be due to my fan being dirty and the CPU heating up. After cleaning it, I'm running 4 instances with preset="slower" on 1080p content and I get stable 80-100% CPU usage @ 2.38ghz with ~2.3fps encoding rate.

I am currently running a test with two parallel workers on 1080p content (Game of Thrones).
My Avisynth templates reads:
<input>
<deinterlace>
<crop>
<resize>
<denoise>
FrameRateConverter(23976, 500, "slower", 16)

My x264 settings are essentially "slower" (I added some higher quality settings) with constant quality 20.

My I7 920@3.8 GHz is doing two workers at 0.64 FPS each. My ram (6gb) is pretty much filled. I think the performance is reasonable. When Vega 56 cards with board partner coolers arrive, I will switch to a Ryzen 7 1700 with 16gb ram, that should help performance a lot.

Is there a setting for 3d material? That is the content profiting the most from 48 fps imho.

[burfadel]

I was looking for a way to fix duplicate frames and the solution was to replace them with frame interpolation, and I found FillDropsI from johnmeyer -- except that it uses basic interpolation without artifact masking. So I thought.. how about modifying it to use FrameRateConverter for interpolation?

Any comments on this?

I want to add "keep", 1 to keep 1st frame and interpolate the 2nd, and 2 to keep 2nd frame and interpolate the first, because in the video I'm testing the 2nd frame is better. I tried simply replacing the function YDifferenceFromPrevious with YDifferenceToNext but that didn't work. Any idea how to get that to work? Also is there a way to interpolate without separating fields? I might give better interpolation on whole frames, but I wasn't successful at doing that.

Once this script is polished I'll release it in the same script file.


function InterpolateDoubles(clip c, float "thr", string "preset")
{
thr = default(thr, .1)
preset = default(preset, "normal")

even = c.SeparateFields().SelectEven()
even_flow = FrameRateConverter(even, Preset=preset, FrameDouble=true).SelectOdd()

odd = c.SeparateFields().SelectOdd()
odd_flow = FrameRateConverter(odd, Preset=preset, FrameDouble=true).SelectOdd()

even_fixed = ConditionalFilter(even, even_flow, even, "YDifferenceFromPrevious()", "lessthan", string(thr))
odd_fixed = ConditionalFilter(odd, odd_flow, odd, "YDifferenceFromPrevious()", "lessthan", string(thr))

Interleave(even_fixed, odd_fixed)
return Weave()
}


Running this script before mClean kind of works but it freezes (no MT)

There's been various scripts like that. There's blendupes:
https://forum.doom9.org/showpost.php?p=1764263&postcount=18

Morphdupes_MI
https://forum.doom9.org/showpost.php?p=1764867&postcount=20
https://forum.doom9.org/showpost.php?p=1764867&postcount=21
(split script)

Filldrops3
https://forum.doom9.org/showpost.php?p=1764192&postcount=1

Filldrops3 could very easily and effectively be incorporated :). It would also have little impact on performance seeing as the mvanalyse has already been done.

[burfadel]

Did you find the solution? Using MFlowInter is a simple solution.

[MysteryX]

I haven't looked into it further. If I want to use MFlowInter, then I can't simply call FrameRateConverter and benefit from its features.

I'll also looking into increasing SkipOver from 120 to something like 210. Highly animated scenes get skipped and it breaks the smoothness.

Also using DCT=4 for MRecalculate is a good idea.

[burfadel]

Animated sources are difficult, ideally you need to incorporate using MCompensate, or MFlow and then use correction masks. MFlow does a great job most of the time, and makes up for it by adding it's own special effects at scene changes. Then you need the scene detection function... The good thing about using any of that is it can be used for normal scenes as well. Use normal clip for MAnalyse, use that for compensation, then use a super of that for MRecalculate. Or a variation on that principle.

[SpoCk0nd0pe]

From the latest changelog in mvTools2:

Fix: [DCT 8x8@8bit] safe multithreading for integer DCT (8x8 block size, 8 bit video): assembly had a single working buffer.

Does this fix the MT problems?

If I got it correctly, your block size recommendations are for 4:3. What blocksize would you chose for 1080p?

[MysteryX]

From the latest changelog in mvTools2:

Fix: [DCT 8x8@8bit] safe multithreading for integer DCT (8x8 block size, 8 bit video): assembly had a single working buffer.

Does this fix the MT problems?
With BlkSize=8, CPU still goes only up to 29% with Prefetch(8)

If I got it correctly, your block size recommendations are for 4:3. What blocksize would you chose for 1080p?
I treat 4:3 or widescreen videos the same.

As per documentation, default block sizes are, based on height
0-359: 8
360-749: 12
750-1199: 16
1200-1699: 24
1600-2160: 32

[MysteryX]

Version 1.1 is released! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.1)

What's new:
- Added InterpolateDoubles function to replace double frames with interpolated frames using FrameRateConverter
- Added DctRe parameter to specify DCT for MRecalculate
- DCT now also applied for MRecalculate by default
- Preset Normal now uses DCT=4 for MRecalculate. Old Normal is now Fast and old Fast is now Faster.
- Presets Slow and Slower also now use DCT=4 for MRecalculate
- SkipOver default treshold increased from 120 to 210
- BlendOver default treshold increased from 60 to 65
- SkipOver now specified for thSCD2 of MMask. Previously, scene changes would cause 2 adjacent frames to be skipped. Specifying thSCD2 causes only 1 frame to be marked for scene change.
- Updated Avisynth headers in DLL
- StripeMask: renamed trh parameter to thr

I also just did some tests with Stripes="Skip" and it's not practical for real videos, it often gives very weird results. I might remove it in next version.

I'm also open to feedback to improve InterpolateDoubles.

[MysteryX]

There is one issue with using DCT=4 for MRecalculate: it causes masks to be MUCH weaker, and thus pretty much disables artifact masking!

Would MFlowBlur be useful in that script? Would it be possible to use MFlowBlur instead of frame blending for artifact masking?

[burfadel]

It's possible, I was thinking that myself for mClean. For video temporal quality is much more important than single frame comparisons, particularly those where people have to zoom in or look hard to determine the difference. If there is fast motion it should be perceptively better. With motion analysis derived results, like an arm moving fast you get that opaque overlay look which is particularly noticeable in animation. MFlowBlur may make this area 'heavier' which wouldn't be good as it will make the arm look unnaturally fat momentarily. If you can detect that there is motion opaqueness caused by MFlowFPS, you can mask this and return the good detail from the previous or next frame. This will likely make the arm appear thinner. If you then apply MFlowBlur, you should get a smooth, and perceptively be motion artifact free assuming the source was free of this.

I'd say it would be difficult to implement! The temporal quality should be much higher though, and spatial comparisons should be more favourable as well.

[MysteryX]

How do you use MFlowBlur anyway? It doesn't have a FPS parameter like MFlowFPS, so how do I generate a matching clip?

[burfadel]

I believe it recreates existing frames based on the motion vector data of previous and next frames provided by MAnalyse. I was just thinking that if you ran MFlowBlur or Mcompensate before MflowFPS, and work out the clip stuff (super,superfilt) based on this, it could produce good results? I think it all comes down to experimenting with the functions.

[MysteryX]

I asked Chainik about it
> Does MFlowBlur generate artifacts?

yes, since it uses the very same (i.e. wrong) motion vectors


It's useless then.

I also tried it. Takes the source clip and kind of blurs/distorts it. It's not generating any in-between frames. Useless.

[MysteryX]

OK after testing the mask of MRecalculate with DCT=4, overall, MAnalyze gives a much weaker mask with DCT=4 than with DCT=0. DCT=4 and DCT=1 give the same mask. If you do MAnalyze with DCT=0 and MRecalculate with DCT=4, you get the same mask as if MAnalyze was done with DCT=4.

Thus, with the latest release, presets Slow/Slower that were already using DCT=4 will behave similarly but with slightly better quality. Preset Normal will have much lower artifact masking because of MRecalculate. For artifact masking, perhaps some adjustment could be made to take this mask strength discrepancy into consideration.

This however can cause issues for scene change detection if masks are too weak.

In terms of mask strength discrepancy, here's what I'm talking about

MAnalyze DCT 0, 1, 2, 3, 4
https://s26.postimg.org/in1q4s35x/MAnalyze0.png (http://postimg.org/image/in1q4s35x/) https://s26.postimg.org/3sms1gpad/MAnalyze1.png (http://postimg.org/image/3sms1gpad/) https://s26.postimg.org/in1q4s35x/MAnalyze0.png (https://postimg.org/image/in1q4s35x/) https://s26.postimg.org/n09wy27lx/MAnalyze3.png (http://postimg.org/image/n09wy27lx/) https://s26.postimg.org/jt5q3krgl/MAnalyze4.png (http://postimg.org/image/jt5q3krgl/)

MAnalyze+MRecalculate DCT 0, 1, 2, 3, 4
https://s26.postimg.org/3sd4qltl1/MRecalculate0.png (http://postimg.org/image/3sd4qltl1/) https://s26.postimg.org/p3kc5q7et/MRecalculate1.png (http://postimg.org/image/p3kc5q7et/) https://s26.postimg.org/5oe5qd3id/MRecalculate2.png (http://postimg.org/image/5oe5qd3id/) https://s26.postimg.org/qmkbog3d1/MRecalculate3.png (http://postimg.org/image/qmkbog3d1/) https://s26.postimg.org/6aytrafb9/MRecalculate4.png (http://postimg.org/image/6aytrafb9/)

[hello_hello]

MysteryX,
I was experimenting with frame interpolation on some animation today and noticed FrameRateConverter (both versions 1.0 and 1.1) tends to lose the plot during fade-outs. I haven't fiddled with settings too much yet so I don't know if there's an adjustment that'd fix it, although it seems to be the sort of thing that probably shouldn't happen by default.

There's three MKVs in the zip file I've linked to if you'd care to look. "12fps.mkv" is simply a section of the original DVD cropped and resized and decimated to 12fps. It's for reference.
"FrameRateConverter.mkv" is the result of adding FrameRateConverter(24,1) to the script after resizing, and "Interframe.mkv" is the result of adding Interframe(cores=1, NewNum=24, NewDen=1).

FrameRateConverter seems to do a better job than Interframe aside from the fade-outs. If you look at the grey area near the middle of the picture a bit after frame 120 and step through the frames one at a time as it fades out, you'll see the shadow over the grey area appears to move up and down. A similar thing happens a little later as the girl fades out and the shot of the earth behind her distorts a little in the direction it's moving. It appears FrameRateConverter might be mistaking the fading-out for movement.

Cheers.

12fps-24fps.zip (https://files.videohelp.com/u/210984/12fps-24fps.zip) (3.3MB)

Edit: For the record I tried manolito's earlier version of the script here (https://forum.doom9.org/showthread.php?p=1805050#post1805050) and the fade-out problem was far worse. I must have saved StainlessS's DoubleRate (https://forum.doom9.org/showthread.php?p=1785080#post1785080) script at some stage so I tried it and the effect was reduced, although still present.

Thanks.

[MysteryX]

I've now adjusted the masks to look similar with different DCT values. Still need to do some testing.

hello_hello, for anime, use Preset="slow". With DCT=4, the fade out is good. There's a bit of a warping effect at 239-242. When the lady fades out, the mask is too weak to skip that, but on fade in, I get a Skip value of 46. Set BlendOver=40. I think it's a good idea to set BlendOver to a lower value for anime.

Perhaps I could add a preset "Anime" that changes BlendOver/SkipOver values, but that would require more testing. I'll release this version with re-adjustments to the masks and we'll see from there what settings need to be tweaked for anime.

Also, if you double the frame rate, use FrameDouble=true instead of NewNum to preserve source frames. I'm editing the script to set FrameDouble to true if NewNum is double the frame rate.

[MysteryX]

Version 1.2 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2)

What's new:
- Mask strength now adjusted to produce similar artifacts masking with different DCT values
- Removed Stripes parameter as Skip isn't practical
- Added Stp parameter, whether to detect and blend stripes
- Changed MaskTrh default from 100 to 120
- Changed BlendOver default from 65 to 70
- If NewNum is twice the clip's frame rate, set FrameDouble to true by default to preserve source frames

You can test preset="anime" to see which settings work best. This new preset hasn't yet been tested and is likely to be tweaked.

[manolito]

Edit: For the record I tried manolito's earlier version of the script here (https://forum.doom9.org/showthread.php?p=1805050#post1805050) and the fade-out problem was far worse.

The only difference with your command line between FRC 1.0 and my simplified script is the block size. FRC defaults to 12 for this frame size while my script uses 16 (a block size of 12 is not supported by older versions of MVTools2).

I did a few tests, and for me the clip looks best with a block size of 32 (happens often with anime). DCT=1 did not make much difference, though. Using blksize=32 fixes the fluttering grey shadow completely, but when the planet enters the window, the first couple of frames have distorted movement. I think that this is far less visible than the grey shadow, it only can be detected by stepping through the frames.


Cheers
manolito

[burfadel]

Version 1.2 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2)

What's new:
- Mask strength now adjusted to produce similar artifacts masking with different DCT values
- Removed Stripes parameter as Skip isn't practical
- Added Stp parameter, whether to detect and blend stripes
- Changed MaskTrh default from 100 to 120
- Changed BlendOver default from 65 to 70
- If NewNum is twice the clip's frame rate, set FrameDouble to true by default to preserve source frames

You can test preset="anime" to see which settings work best. This new preset hasn't yet been tested and is likely to be tweaked.

You didn't make a binary of this version, you only provided the source code :).

[hello_hello]

Thanks for the info MysteryX and manolito.

I haven't had a chance to play around more yet. Probably not until tomorrow, but I was wondering.....

I didn't give any thought to it at the time, but would frame interpolation work better at higher resolutions? I resized first, thinking it'd be faster:

TDecimate(mode=2, rate=12, Input="D:\ep1.txt")
crop(14, 4, -14, -4)
Spline36Resize(704,396)
FrameRateConverter(24,1)

I'll use FrameDouble=true next time, but would it make much difference if I put FrameRateConverter before the cropping and resizing? It's a PAL DVD so there's 720x576 resolution to begin with, even if there's not 720x576 worth of picture detail as such.

Cheers.

[MysteryX]

I did a few tests, and for me the clip looks best with a block size of 32 (happens often with anime).
Right, forgot about that. Preset "anime" should use higher block sizes by default, although that still needs to be tested and tweaked.

You didn't make a binary of this version, you only provided the source code :).
Oups forgot to upload! Fixed :)

would it make much difference if I put FrameRateConverter before the cropping and resizing?
For 288p to 768p, it's much better if interpolation is done last. For 720p to 1080p, I'm not sure whether it's best to interpolate before or after, I still need to test that. One thing for sure: interpolation is expensive. If you interpolate after upscaling, it will be slower. Test both ways, opening each version in a different instance of VirtualDub, and compare the frames.

I've just re-encoded a very difficult and animated video with this version. Quality is considerably improved. Output file size is also considerably different: 209mb instead of 205mb.

[hello_hello]

I was actually downscaling ie 720x576 to 704x396. Chances are it's not enough to matter but I thought I'd ask.
I still haven't had a chance to do more testing but I'll report back when I do.

Thanks.

[MysteryX]

Normally, interpolate at the end.

If the details are too low and interpolation is causing artifacts, then you can also try interpolating first. Let me know which works best.

You may want to use AvisynthShader's SSIM downscaler. I haven't tested it much myself but it's giving great results. (I'll use it to downscale 4K YouTube content to 1080p)

[SpoCk0nd0pe]

I tried your new script version and got a divide by 0 error in line 132 :/

1080p content at slower, stock settings

[MysteryX]

I tried your new script version and got a divide by 0 error in line 132 :/

1080p content at slower, stock settings
Line 132 is

B = C.ConvertFps(NewNum, NewDen)


Did you specify valid NewNum and NewDen? What's your script?

[MysteryX]

I saw a movie clip with a calm background and high-artifact motions in the middle. The frame should get blended but because the background is calm the SkipOver value is too low.

The only way I can think to fix that would be to analyze the vectors data to determine what percentage of vectors are below a certain movement threshold. If it's a low-movement background, blending the whole frame doesn't degrade quality much. I could say if the amount of slow-moving blocks is above a certain percentage of the frame, reduce BlendOver by half. This should help in those scenes.

I have no idea what vectors data looks like though! Anyone knows how this works?

According to doc (http://www.avisynth.nl/users/fizick/mvtools/mvtools2.html)


outfile: name of file to write motion vectors data. This data may be used by some external program or may be by next MVTools versions for second pass coding, etc. Produced binary file has a header (MVAnalysisData structure, see MVInterface.h source code), and the data sequence:
frame number, vector data (Vx, Vy, SAD) of every block, next valid frame number, this frame vector data, and so on.


I was thinking to simply use YDifferenceFromPrevious to detect how much movement is in the frame but that would cause low-contrast action scenes to be blended.

As for Anime preset, perhaps we could simply double default BlkSize; this would have to be tested.

[hello_hello]

The only difference with your command line between FRC 1.0 and my simplified script is the block size. FRC defaults to 12 for this frame size while my script uses 16 (a block size of 12 is not supported by older versions of MVTools2).

I may have done your version of the script a disservice. I didn't realise you'd updated it since I saved it. The version I was using is dated 18-May-2017. Is this the latest version of your version? https://forum.doom9.org/showthread.php?p=1805050#post1805050
I'll try again fairly soon with the newer version while I'm testing the newer FrameRateConverter and the settings suggested by MysteryX, but if that's the latest, you might want to go through it and replace all instances of trh with thr. I recall MysteryX doing the same thing. I think there's only one line where trh is used as an argument and causes an error, but it'd make sense to change MaskTrh to MaskThr and SkipTrh to SkipThr anyway.

Edit The 22-June-17 version of your version of the script is better in respect to the fade-out issue than the 18-May-2017 version I tried originally, so it's probably no worse than the official FrameRateConverter script in that respect.

Cheers.

[hello_hello]

MysteryX,
The help file seems to imply BlkSize=6 is supported, but FrameRateConverter(FrameDouble=true, BlkSize=6) results in this (version 1.2):

MVTools: no BLITCHROMA function for block size 1x1
(FrameRateConverter.avsi, line 153)

[hello_hello]

As for Anime preset, perhaps we could simply double default BlkSize; this would have to be tested.

I tried it but I prefer the default block size (I think I even preferred the default to reducing it to 8 (I assume at 704x396 the default BlkSize is 12?).

BlkSize=24 seems to fix the fade-out issue, or at least improve it greatly, but it increases distortion in areas where one object is moving across another stationary object. I think I'd prefer some minor wobble during fade-outs.

You may want to use AvisynthShader's SSIM downscaler. I haven't tested it much myself but it's giving great results. (I'll use it to downscale 4K YouTube content to 1080p)

I gave it a try but kept getting an error relating to a missing function. It was yesterday and I didn't make a note of it because I thought I'd work on one problem at a time, but I think it was "no function get_bitdepth" or something similar.

I tried switching MeGUI to using it's portable version of AvIsynth+ (I have the standard AVIsynth installed) but that didn't help. Apparently MeGUI is using r2508. Or maybe it's an XP thing?

Anyway, that'll be a problem for another day. I'll keep playing around with FrameRateConverter settings for the moment.

Thanks.

[MysteryX]

hello_hello, when testing settings, set debug=true to see what it's doing

"no function get_bitdepth" that means you didn't load AvisynthShader.dll

[hello_hello]

There's shader.dll in the zip file but no AvisynthShader.dll, is that something different again? I definitely had shader.dll in the Avisynth plugins folder. Maybe it's a dependency problem. My video card is pretty ancient so I'm not to optimistic as to how well it'd work anyway.

I tried version 1.50 just to see what'd happen and it resulted in "no function named ConvertToShader" and when I went back to 1.6.2 it was "no function named shader_GetBitdepth".

Best as a can tell, and keeping in mind I have no idea what I'm talking about, the Dependency Walker clues seem to indicate it's an XP issue.
https://msdn.microsoft.com/en-us/library/windows/desktop/bb219676%28v=vs.85%29.aspx

Not to worry, I'm not fussed about that for the moment. I'm still playing around with FrameRateConverter.

PS This: FrameRateConverter(FrameDouble=true, BlkSize=6, debug=true)
still results in this error:
MVTools: no BLITCHROMA function for block size 1x1
(FrameRateConverter.avsi, line 153)

(mvtools2 2.7.22)

[hello_hello]

This doesn't relate to my animation problem as such, but while playing around with various settings I had a couple of thoughts....

MaskThr=0 disables artefact masking according to the help file, but the script is very insistent on SkipThr being lower than MaskThr, yet the range for each is stated as being 0-255. It turns out setting SkipThr to -1 bypasses the error (hopefully that's all it does), but I wondered if it might be an idea to limit settings when it's required? For example if you set MaskThr=100, the script would ensure SkipThr doesn't exceed 99, even if you specify something larger.

IMHO it would be awesomely awesome if the debug info included the current frame number, for those times when you have two instances of a script open with slightly different settings (I do that quite a bit using multiple instances of MPC-HC). And also because the debug info and Info() both want to display in the upper left hand corner....

Cheers.

[hello_hello]

MysteryX,
I've played around some more, but so far I don't think I've come up with anything that does a better job than your new animation tuning, so I think I'll just go with that for the moment.

Thanks.

PS. You've probably had a good look at the inner workings of Interframe. If so, do you know what the difference between the default tuning and the animation tuning might be? I ask, because the default tuning does a reasonable job on my animation sample (not as good as FrameRateConverter though, aside from the fade-out problem), whereas the animation tuning is just horrible. Stuff wobbles around all over the place near any object that's moving. I can't imagine when it'd be an improvement over the default tuning, or when you'd want to use it.

Thanks again.

[MysteryX]

Yes, it's shader.dll. Somehow it's not loading. Are you using Windows XP? AvisynthShader requires Windows Vista or above.

BlkSize=6 is valid; but MRecalculate with BlkSize=3 isn't. That's the problem. You can only use BlkSize with Preset="faster" where MRecalculate is disabled.

Why would you set SkipTrh to 0? If you want to disable scene change detection, set SkipOver=0

To get or set current frame number, in MPC-HC or VirtualDub, press CTRL+G. I often find it easier to compare in VirtualDub.

From my test the new animation settings work pretty well, only exception being the woman "warping out" after the planet moves to the right. If I end up implementing a "dynamic blending threshold", it might help with cases like that with still backgrounds and high-artifacts motions.

As for Interframe's Anime tuning, AFAIK, it mostly tunes the settings way down. I don't know the details.

[hello_hello]

Yes, I'm using XP.

I wasn't wanting to set SkpThr to zero as such, but I wanted to try disabling MaskThr by setting it to zero and if you do that, this line kicks in, effectively preventing you from setting MaskThr=0.

Assert(SkipThr < MaskThr, "FrameRateConverter: SkipThr must be lower (stronger) than MaskThr")

Plus it seemed to me it might be better for the script to automatically reduce SkpThr if required according to the MaskThr value, but either way, if you set MaskThr=0, the specified SkipThr range of 0 to 255 will always result in an error.

There's a similar relationship between SkipOver & BlendOver (from memory) I thought could maybe be handled in a similar manner rather than result in an error. Plus the error states SkipOver must be greater than BlendOver, but it doesn't tell you what BlendOver is. ;)

Ctrl+G isn't quite the same thing as having the frame number constantly displayed. The latter just makes it a little easier to compare two videos when stepping through frames. I created a little function called FrameNum() a long time ago and it's easy enough to stick it at the end of the script, but I thought adding the frame number to debug mode might be a nice touch.

The main reason I prefer to use MPC-HC for previewing/comparing scripts is it has a fullscreen mode, and from memory there's no way to prevent VirtualDub from locking scripts when it opens them. If you disable MPC-HC's internal AVS source filter it doesn't lock them. I'm quite used to a script displaying in MPC-HC while I edited it with Notepad and save the edited version, then use CTRL+E to re-load it.
There's probably advantages to VirtualDub I'm unaware of as I've not used it much.

I think I know what you mean about the woman "warping out", or something similar. For frame 241 the slowest preset doesn't distort the planet at all, but the guy "warbles in", whereas the slow preset seems to blend the planet movement, but the guy "appears" nice and cleanly. Some sort of dynamic handling might be nice for that sort of thing, but I don't know how hard it'd be to implement. Mind you it's only a single frame so it passes quite quickly and isn't the end of the world.

I guess this frame interpolation stuff isn't easy. :)

Preset="slow", BlendOver=40, SkipOver=140
https://image.ibb.co/bu3Roa/slow.jpg

Preset="slowest", BlendOver=40, SkipOver=140
https://image.ibb.co/dEi3ZF/slowest.jpg

[MysteryX]

For "anime", use preset=slow (DCT=4) because DCT=0 generally gives crap. Slowest only inserts some of that crap back in. If you want to see the output without artifact masking, use Output="flow"

[hello_hello]

If you want to see the output without artifact masking, use Output="flow"

No problem.

Although I still think it'd be an idea to change this in the script, given it's impossible to set MaskThr to 0 without it resulting in an error (or unless you also set SkipThr to -1)

@ MaskThr - The threshold where a block is considered bad, between 0 and 255. Smaller = stronger. 0 to disable artifact masking. (Default = 120)

Thanks.

[pinterf]

BlkSize=6 is valid; but MRecalculate with BlkSize=3 isn't. That's the problem.
It can be valid if the source is 4:4:4 (YV24). BlkSize=3 for YV12 format means that chroma block size is 1 (3/2) which is not supported.

[urukazuto]

Hello I'm trying to encode 720p anime by using megui with this setting:
x264 = CRF 23, Very Slow Preset at 8 bit.
Avisyth :

SetMemoryMax(512)
SetMTMode(3,4)
PluginPath = "D:\Program Files (x86)\MeGUI_2715\tools\avisynth_plugin\"
LoadPlugin(PluginPath+"FrameRateConverter.dll")
LoadPlugin(PluginPath+"mvtools2.dll")
LoadPlugin(PluginPath+"masktools2.dll")
LoadPlugin(PluginPath+"GRunT.dll")
Import(PluginPath+"FrameRateConverter.avsi")
<input>
SetMTMode(2)
<deinterlace>
<crop>
<denoise>
<resize>
FrameRateConverter(NewNum=60, NewDen=1, Preset="Anime")


And When I trying to encode, the speed encode is only 1 fps... is this normal?

is there a way to increase speed without changing the preset / setting on framerateconverter?

Edit:
I'm kind of new to this things, i will try to tweak setting a bit.

My Specs:
Intel Core i5-4690 3.5Ghz
MSI Geforce GTX 960 2048MB DDR5 - Gaming 100ME - Limited Edition
8GB DDR3 RAM
Windows 10 Pro 64 Bit (Build 14393)

and yes the Avisynth i'm using is x86 one... version 2.6.0 from Official Builds : https://sourceforge.net/projects/avisynth2/
and after that using the MT one from here : https://forum.doom9.org/showthread.php?t=148782

For filters i dunno which version I'm using....

[burfadel]

Whether it is 'normal' depends on the other filters used that you haven't listed, in addition to the versions of the filters and Avisynth, and in particular, your system specs. I see you are using x86 Avisynth, that isn't 'ideal' for performance. Also is there any reason specifying SetMemoryMax? Is your system short on memory?

[MysteryX]

I also use x86, AFAIK it doesn't make much of a difference.

However, for Anime you're using DCT=4, which isn't working well with MT. Not only are you getting slow performance, but it will crash as well at some point. Currently all presets from Normal to Slowest use some DCT=4 or DCT=1 (it remains to be tested whether MRecalculate DCT=4 has the same issues as MAnalyze... probably)

So you're limited to single-threaded mode. You can however run several instances at once. On a 8-CPU system, I get best performance by running 4 instances in parallel. However, cutting the video in segments and running these segments in parallel isn't something MeGUI supports :)

[hello_hello]

So you're limited to single-threaded mode. You can however run several instances at once. On a 8-CPU system, I get best performance by running 4 instances in parallel. However, cutting the video in segments and running these segments in parallel isn't something MeGUI supports :)

Not automatically, but it's fairly easy.

I use the AVS Cutter under the tools menu to add cuts to divide the script into sections, then make the appropriate number of copies of the script and edit each to remove all but one Trim() range.

You can add each copy of the script to the queue and run them simultaneously but don't forget to check "stitchable" in the x264 encoder configuration (under the Misc tab) so you won't have any problem appending the encoded segments, which of course you'll have to do yourself with something like MKVToolnixGUI.

[bblogoss]

Is there a way to only skip scene changes or sudden lightning change? I want the smoothest output possible like the output="flow" and only skip the frames that the debug info indicates SKIP.
I have been tweaking the SkipOver argument with no success.
In brief , I want frame interpolation only except for scene changes or when it becomes absolutely awful like a sudden flash lightning between 2 frames.

Thanks for this wonderful script by the way. Results even better than Premiere Pro optical flow which tend to add some blur to the interpolate frame.

[MysteryX]

yeah even SVP gives better results than Premiere Optical Flow.

Set BlendOver=0 to disable frame blending.

[bblogoss]

I have also tried with blendover=0 without succes too

Here is a zoom sample with BlendOver=0 (pretty much the same as default settings)

https://image.ibb.co/dzGcJa/blendover0.jpg

And here it is with output="flow"

https://image.ibb.co/jmieWv/Output_Flow.jpg

[MysteryX]

Oh, you want to disable artifact masking?

Set MaskTrh=0; but I believe someone reported a validation bug with that. Alternatively, setting MaskTrh=255 will also ensure it doesn't trigger.

[bblogoss]

I managed to get something near what I want with

MaskThr=255, SkipThr=0, BlendOver=0, SkipOver=220

Thanks for the help.

[Fabulist]

Can someone explain how to use this tool? Is it a live converter like SVP?

[rsd2015]

Can someone explain how to use this tool? Is it a live converter like SVP?

I defer to the gurus on this board, everything you need to know is in this thread. FRC is meant for encoding video, not playback like SVP. Load the plugin, import the script and specify the frame rate variables you need, e.g. FrameRateConverter (NewNum=60, New Den=1) for 60 FPS.

[Fabulist]

I defer to the gurus on this board, everything you need to know is in this thread. FRC is meant for encoding video, not playback like SVP. Load the plugin, import the script and specify the frame rate variables you need, e.g. FrameRateConverter (NewNum=60, New Den=1) for 60 FPS. I see, thanks, I was hoping it is a playback tool like SVP.

[Magik Mark]

MysteryX,

Can you recommend a good setting for:

1080p HD videos 23.976fps to 29,970fps no or minimum soap opera effect, action movies / drama?

I use staxrip, how do I integrate awarsharp2?

[MGarret]

MysteryX

I would really like to try your filter but I keep running into errors. I've installed everything required, latest MaskTools2, MvTools2 (pinterf) but latest error I'm getting now is:

Avisynth open failure:

mt_inpand: wrong colorspace, only YV12 and I420 allowed
(FrameRateConverter.avsi, line 165)

I checked with Info() and the colorspace is YV12. How can I fix this?
I assume I'm missing some additional updated plugins?

[burfadel]

Latest Avisynth+ r2508 as well?

[manolito]

Does "FrameRateConverter.dll" no longer work under plain vanilla AviSynth 2.6 ?

On my Win7-64 computer FrameRateConverter crashes with the latest DLL version introduced with v.1.1. The changelog had this: - Updated Avisynth headers in DLL
Replacing the DLL with the version from FRC 1.0 (from the first post of this thread) fixed it.


Cheers
manolito

[hello_hello]

manolito,
version 1.2 is playing nice for me and Avisynth 2.6 on XP.
Not that it helps you, but I assume it means it's not an Avisynth related problem.

[Groucho2004]

Does "FrameRateConverter.dll" no longer work under plain vanilla AviSynth 2.6 ?
1.1 and 1.2 work fine for me on AVS 2.6 and XP. Run the script with AVSMeter and see if that produces errors.

[manolito]

Thanks guys, AVSMeter revealed the (stupid and unnecessary) problem:

I mixed the FRC script v1.0 with the DLL v1.2, and this caused the crash. In addition to updating the AviSynth headers MysteryX also decided to change the shortcut for "Threshold" from "trh" to "thr" and hardcoded it into the DLL. The DLL version needs to match the script version, if it doesn't it will crash.

For me this is a prime example of sloppy programming. Nothing in the docs says that the DLL is version specific and requires a certain version of the script. In the future please document such changes and maybe include the version number in the DLL name... :devil:


Cheers
manolito

[burfadel]

The trh to thr was because of a spelling mistake 'treshold' instead of 'threshold'. It's typical of any program that you can't mix files, it doesn't really need to be clarified. Putting versions in file names isn't ideal, many people will then end up having multiple versions of the same file.

[hello_hello]

For me this is a prime example of sloppy programming. Nothing in the docs says that the DLL is version specific and requires a certain version of the script. In the future please document such changes and maybe include the version number in the DLL name...

I guess you missed my post a while back. :)
https://forum.doom9.org/showthread.php?p=1817518#post1817518

I'm a little on the fence about giving dlls version numbers in the name. I usually add them myself before putting them in the Avisynth plugns folder anyway, but if a script calls a plugin specifically such as deblock1.2_deblock(), you can't replace deblock1.2.dll with deblock1.3,dll without breaking the script, although admittedly that's only been an issue for me once that I can think of off the top of my head.

It'd be nice to be able to discover the dll version using the Explorer right click/properties menu though.

[manolito]

Sorry if my post came across being a little harsh, but I still think that the problem I encountered should not have happened.

A plugin DLL has one or more functions, there is an interface how the plugin functions have to be called, and this interface needs to be documented. I think we all agree on this.

The script command which caused the crash was this one:
EMstp = C.StripeMask(blksize=BlkSize, blksizev=BlkSizeV, str=min(SkipThr*2+20, 255), strf=min(SkipThr+10, 255), thr=23)

The variable "thr" in the StripeMask function of the DLL had a different name "trh" in the older DLL version.

But this variable is not exposed in the FrameRateConverter.avsi script which calls this function. Users have no way to read the documentation and edit the variable name other than editing the script itself (at line 212).


My specific situation was that I did not care for the script changes between FRC v1.0 and v1.2, I wanted to stick with my (modified) v1.0 script. Seeing that MysteryX had updated the DLL with newer AviSynth headers I figured that it would be a good idea to update the DLL to the latest version, but obviously this was not a good idea at all... :eek:


Cheers
manolito

[MysteryX]

All changes, including the parameter changes, are very well documented in the change log.

Why would anyone expect 2 files of different versions to work together, and why would anyone do that? Replace any files of your drivers by a version from older versions and you'll get a blue screen of death. Nothing unexpected about that.

[Ben_Nicholls]

Looks promising, honestly kind of surprised... I always felt SVP was quite a way ahead of MVTools.

It might just be that to use any kind of MVTools convetion in real time you have to set use awful quality settings, SVP does seem to be better optimised.

I would point out that (in my opinion) SVP only comes into its own when the artefact masking is set to maximum (and shader complexity is set to 23). I know InterFrame's default settings don't really bring out the best in SVP in many cases. Also v4 is showing big quality improvements over v3 (while the program's GUI isn't free any more the core SVPflow DLLs are still free to use in scripts).

Will do some side by side comparisons shortly.

[Hotte]

I was recently testing FRC in oder to replace Interframe/SVP for editing. I am converting 2K or 4K from 25/29.97 to 50p.

I am looking for the best motion compensation and avoid any object- or frame-blending. The only way to achieve this with FRC was to set output="flow" AND blksize=8. With any larger blocksize, FRC would start to blend moving objects in the frame. Quality="fast" was good enough to replace interframe in most cases. Very good work really!

However, I am wondering why I am getting blending free results only with the smallest blocksize and why output="auto" with various other parameter settings leads to blended rather than "morphed" objects in almost any cases.

Any ideas ?

[manolito]

If you want to avoid any artifact masking by blending, skipping or whatever you might want to try the core jm_fps script from here:
https://forum.doom9.org/showthread.php?p=1800439#post1800439

FRC is based upon this script with additional artifact masking (which is often beneficial, but sometimes it is not...)


Cheers
manolito

[mark0077]

Hi guys, I have been using SVP for many years. I now see this thread, looks great. So whats everyone using it with, avisynth or avisynth+. Which gives better performance? I'll give it a go using whatever is recommended parameters to see if my system can handle it vs SVP (Using Interframes Film / Medium settings).

[MysteryX]

AVS+ is better but either way it should work just fine with default settings. There are various presets. You get better performance with "faster" preset and better quality with "slower" preset.

[Hotte]

Thanks manolito, this is a good basic script with all the functions I need. Quality is good. The only thing I missed was scene change detection. Do I have to go back to FrameRateConverter to get it ?

Also in your script, blksize determines inframe object movement behaviour. Small blksizes like 8 or 12 will morph, higher blksizes will more and more blend the movement. But in any case small movement will be handled differently than big movement. I understood this is a common concept. It´s simple to handle with only one parameter to change for different situations. Not bad, really.

[MysteryX]

Thanks manolito, this is a good basic script with all the functions I need. Quality is good. The only thing I missed was scene change detection. Do I have to go back to FrameRateConverter to get it ?

You can handle scene changes using thSCD2. Setting it to 255 disables scene change detection if I'm not mixing things up.

Flow = MFlowFps(C, super, bak, fwd, num=NewNum, den=NewDen, blend=false, ml=200, mask=2, thSCD2=255)

[mark0077]

So is anyone using this in realtime? Trying to get it running in real time here, not sure if its a runner. I'm trying the fast preset first (Using core i7 6700K at 4.5GHZ, GTX 980). I'm running this script via ffdshow which is how I also use SVP.

For ffdshow settings itself, I have it set to "Add ffdshow video source" disabled. Buffer back/ahead set to 0/15. Again this is what works well for SVP.

Heres my script so far but its definitely not smooth :) I assume it needs tweaking to work in ffdshow and maybe the buffer settings in ffdshow arn't optimal. I havn't seen anyone give a full example of use with ffdshow.

Setmemorymax(1024)
SetMTMode(5,16)
LoadPlugin("C:\Program Files (x86)\SVP\FRC\FrameRateConverter.dll")
LoadPlugin("C:\Program Files (x86)\SVP\FRC\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\SVP\FRC\masktools2.dll")
#LoadPlugin("C:\Program Files (x86)\SVP\FRC\GRunT.dll")
Import("C:\Program Files (x86)\SVP\FRC\FrameRateConverter.avsi")
Input = ffdshow_source()
SetMTMode(2)
FrameRateConverter(Input, Preset="fast")
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

[hello_hello]

mark0077,
To the best of my knowledge the ffdshow Avisynth section ignores some AviSynth functions, and I'm fairly sure changing the frame rate is one of them. It makes sense, because if you change the frame rate and it in turn changes the duration, the audio won't stay in sync.
Plus the renderer or other filters down the line probably need to know the frame rate and changing it might cause issues there.

Try putting something simple like ChangeFPS(20,1) in ffdshow's Avisynth section.

SVP is probably a different story, but I can't say I've used it for more than 10 seconds. Doesn't it install stuff for controlling it outside of ffdshow?

Do you per chance use Potplayer? Many of ffdshow's filters have been borrowed and added to Potplayer, and it should let you change the frame rate. In fact I think it comes with scripts for doing just that. Have a look in Preferences/Video/Avisynth.

[mark0077]

Thanks hello hello. Actually I use SVP DLLs without any of the additional tray icons or software and I think madVR always says the frame rate is the original rate. Not the converted rate but all still says in sync. I am hoping to replicate the same pattern with FRC. Has anyone used it via ffdshow in real time?

[hello_hello]

My bad... I appear to have remembered wrong. ChangeFPS() works in the ffdshow Avisynth filter, but AssumeFPS() doesn't, so FrameRateConverter should work, assuming it's not too slow.

[iG0R]

MysteryX
Hi. Could you give some advises how to speed up the conversion cause 4-6fps is too boring on 4790K@5GHz and 1080ti? Is it possible somehow to use also GPU for this process?
Thank you for such quality filter.

[burfadel]

MysteryX
Hi. Could you give some advises how to speed up the conversion cause 4-6fps is too boring on 4790K@5GHz and 1080ti? Is it possible somehow to use also GPU for this process?
Thank you for such quality filter.

I can answer that one :). In terms of using the GPU for processing, the answer is unfortunately no. For speed, it all depends on how you currently have things set up. If you use the latest (currently rev 2544) 64-bit Avisynth+ with the latest Masktools, MVTools, as well as other filters in your script, it will help with speed quite a bit!

[iG0R]

I can answer that one :). In terms of using the GPU for processing, the answer is unfortunately no. For speed, it all depends on how you currently have things set up. If you use the latest (currently rev 2544) 64-bit Avisynth+ with the latest Masktools, MVTools, as well as other filters in your script, it will help with speed quite a bit!
Thank you for the answer.
Yes, I'm using the latest Avisynth+ (AvisynthPlus-r2544-MT.7z), MaskTools (masktools2-v2.2.10.7z), MVTools (mvtools-v2.5.11.22.zip) and FRC (FrameRateConverter-v1.2.zip) but the speed is still awful :( - with Preset="slowest", BlendOver=40, SkipOver=140 fps is around 1-1.2 and with Preset="Anime" fps is about 4-6.

[MysteryX]

What CPU usage are you getting? Are you using MT mode, and if so, how many threads are you using?

[iG0R]

What CPU usage are you getting? Are you using MT mode, and if so, how many threads are you using?
Screenshot:
https://snag.gy/QwAs7F.jpg
The internal AviSynth Portable r2508 of MeGUI doesn't support MT though it says that it does (Error message for your reference: Script error: There is no function named 'SetMTMode') so I tried to use AviSynth MT from this post https://forum.doom9.org/showthread.php?p=1817809#post1817809, but after some time of processing, encoding is crashed so I decided not to use it and tried AviSynth r2544MT, I don't know why MeGUI doesn't support it and switches to internal portable r2508.
Here is my script:

SetMemoryMax(6000)
PluginPath = "C:\Program Files (x86)\MeGUI\tools\avisynth_plugin\"
LoadPlugin(PluginPath+"FrameRateConverter.dll")
LoadPlugin(PluginPath+"mvtools2.dll")
LoadPlugin(PluginPath+"masktools2.dll")
LoadPlugin(PluginPath+"GRunT.dll")
Import(PluginPath+"FrameRateConverter.avsi")
<input>
<deinterlace>
<crop>
<denoise>
<resize>
FrameRateConverter(NewNum=60, NewDen=1, Preset="slow", BlendOver=40, SkipOver=140)

PS. Sorry for such stupid question but what is the "spoiler" tag for this forum? I tried to insert screenshot as an image but didn't find the spoiler tag and without this tag my screenshot is too big to publish it in the message.

[Groucho2004]

Screenshot:
https://snag.gy/QwAs7F.jpg
The internal AviSynth Portable r2508 of MeGUI doesn't support MT though it says that it does (Error message for your reference: Script error: There is no function named 'SetMTMode') so I tried to use AviSynth MT from this post https://forum.doom9.org/showthread.php?p=1817809#post1817809, but after some time of processing, encoding is crashed so I decided not to use it and tried AviSynth r2544MT, I don't know why MeGUI doesn't support it and switches to internal portable r2508.
According to your screen shot you're attempting to use Avisynth+ 64 Bit with Megui. Apparently, Megui does not support 64 bit Avisynth and therefore switches to the "portable" 32 bit version.

The MT error message is quite clear, Avisynth+ does not have a "SetMTMode" function. Multi-threading in AVS+ is implemented differently, see here (http://avisynth.nl/index.php/AviSynth%2B#MT_Notes).

[iG0R]

According to your screen shot you're attempting to use Avisynth+ 64 Bit with Megui. Apparently, Megui does not support 64 bit Avisynth and therefore switches to the "portable" 32 bit version.

The MT error message is quite clear, Avisynth+ does not have a "SetMTMode" function. Multi-threading in AVS+ is implemented differently, see here (http://avisynth.nl/index.php/AviSynth%2B#MT_Notes).

Could you advice another tool like MeGUI that supports 64bit AviSynth+? Hybrid doesn't support 64bit AviSynth+ too :(
Thank you, now I understand what was wrong with AviSynth+.

[Groucho2004]

Could you advice another tool like MeGUI that supports 64bit AviSynth+?Staxrip. I think there's also a 64 bit version of Megui.

[StainlessS]

Think Current ver$ of Staxrip support ONLY 64 bit [Win7+). (JFYI)

EDIT:
http://staxrip.readthedocs.io/intro.html#about

Requirements

Windows 7 x64 or Windows 10 x64, StaxRip is x64 only
.NET 4.7
Intel Skylake or newer for HEVC/H.265 hardware encoding
NVIDIA Maxwell gen2 card or newer for HEVC/H.265 hardware encoding
AMD Polaris card or newer for HEVC/H.265 hardware encoding
AviSynth+ x64, the installer is bundled with StaxRip or alternativly VapourSynth which requires Python



EDIT: On searching StaxRip Docs (builtin StaxRip search) found no mention of MeGUI [at least for current version, x64].
http://staxrip.readthedocs.io/search.html?q=megui&check_keywords=yes&area=default

[Groucho2004]

Think Current ver$ of Staxrip support ONLY 64 bit [Win7+). (JFYI)
Yes, I'm aware.


On searching StaxRip Docs (builtin StaxRip search) found no mention of MeGUI [at least for current version, x64].Now you've got me stumped. Why would you search for Megui in the staxrip docs?

[StainlessS]

Sorry, was just saying that StaxRip dont come bundled with MeGUI. [ I just woke up, gotta get a coffee :) ]

[Yanak]

If i may :

FrameraterConverter plugin is part of Staxrip plugins package already and have a filter dedicated implemented in the program :
https://s14.postimg.org/ifl24ij5d/51885420171127132346cr.png

The parameters for the plugin are left blank by default "FrameRateConverter()", need to right click on the AVS filters box list and select "Edit Code" of the contextual menu to add you own parameters after adding it like in the picture above, but if you know the parameters you want by default you can add them to the default filter or create some more filters with different settings.

Then you can add any of those created filters in one click to your video projects, if you need some help with this just ask in the Staxrip dedicated thread i'll try explain more in detail how to add the filters with your own parameters there.

[iG0R]

Groucho2004, StainlessS, Yanak
Thank you! That is what I was looking for!

Also I tried MeGUI 64bit with 64bit plugins without MT and was impressed:
32bit
Preset "Anime" - 4.12-6.56fps
Preset "Slowest", BlendOver=40, SkipOver=140) - 0.98-1.21fps
64bit
Preset "Anime" - 7.38fps-9.98fps
Preset "Slowest", BlendOver=40, SkipOver=140) - 1.85-1.99fps

PS. During installation of 64bit MeGUI with other 64bit plugins and 64bit AviSynth+ I have encountered some issues so i would like to warn those who will face the same problems. It was needed to copy some DLLs into %Windir%\System32 folder which is dedicated for 64bit DLLs and I used for this TotalCommander which is 32bit app so all DLLs ran into %Windir%\SysWOW64 which is dedicated for 32bit DLLs. To copy 64bit DLLs into correct folder (%Windir%\System32) you need to use WindowsExplorer or %Windir%\Sysnative folder in TotalCommander(32bit).

[Groucho2004]

I used for this TotalCommander which is 32bit app so all DLLs ran into %Windir%\SysWOW64 which is dedicated for 32bit DLLs.You're not the first one falling into this trap. :)

[dioxin]

Help me please. I'm using the x32 version of FrameRateConverter. In the settings I select the "slowest" preset and get an error: mt_merge:"luma" is unsupported in 422 and 444. If select the "Normal" preset, then everything works.

[MysteryX]

Version 1.2.1 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2.1)

Slight improvement to quality based on comments I got from ShyK.

Two additions to the MSuper parts of the script:
sharp=1
The catrom method is the best in general. The "sharper" Wiener interpolation offers nothing over it except more artifacts. It's not really sharper (catrom never has perceivably "less detail"), and its artifacts can only make everything worse.

rfilter=4
it makes the search slightly better, and the result is less artifacts. The performance impact is negligible.


I did some testing and it is indeed consistently better, both on HD videos and especially on low-quality videos. The parade torture sample clip does see good improvement based on this change.



Help me please. I'm using the x32 version of FrameRateConverter. In the settings I select the "slowest" preset and get an error: mt_merge:"luma" is unsupported in 422 and 444. If select the "Normal" preset, then everything works.
Make sure you have Pinterf's latest version of MaskTools2

[hello_hello]

Version 1.2.1 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.2.1)

Thanks!

[Lirk]

Can InterpolateDoubles() restore 2 or 3 frames besides 1 by default? Is there "blend frames" option instead interpolation to avoid some artifacts? Does ConvertFPS() support only integer values of framerate and it can't be changed to 29,97 fps?

[StainlessS]

... and it can't be changed to 29,97 fps?

ConvertFPS(clip clip, float new_rate [, int zone, int vbi]) # 29,97
ConvertFPS(clip clip, int numerator [, int denominator, int zone, int vbi]) # 30000, 1001

ConvertFPS(clip clip1, clip clip2 [, int zone, int vbi]) # same framerate as clip2
ConvertFPS(clip clip1, string preset [, int zone, int vbi]) # "ntsc_video"

http://avisynth.nl/index.php/FPS

[Lirk]

MysteryX, can you add to InterpolateDoubles() availability for detection 1-3 frames successively instead only 1 doubled frame? With highest threshold value script can define few frames as dropped, but in this case script affects non-dropped frames.

[byteshare]

Can someone help me figure out my syntax?
I'm using RipBot and getting the error that I have an invalid argument in the FrameRateConverter. My script is:
FrameRateConverter(NewNum="25", NewDen="1")
I also tried:
FrameRateConverter(preset="Faster")
I was told that it didn't know what "Faster" was

[Atak_Snajpera]

video=FrameRateConverter(video,NewNum="25", NewDen="1")

[Selur]

@byteshare: remove the quotes,..
FrameRateConverter(NewNum=25,NewDen=1)
+ last time I used it, preset only supported: slowest, slower, slow, normal and fast,... even if 'faster' is now supported it is probably all lower-case.
Yup,..
Pset = Preset == "slowest" ? P_SLOWEST : Preset == "slower" ? P_SLOWER : Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : Preset == "faster" ? P_FASTER : Preset == "anime" ? P_ANIME : -1
source: https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi line 92

Cu Selur

[byteshare]

Thank you Atak_Snajpera and Selur
video=FrameRateConverter(video, preset="faster")
worked :)

[Seedmanc]

FrameRateConverter( preset="slower",blksize=32 ) gives http://puu.sh/BouYf/f4a79c59d1.png even though I'm using masktools from the same author.

[poisondeathray]

FrameRateConverter( preset="slower",blksize=32 ) gives <snip> even though I'm using masktools from the same author.

Post your full script

What colorspace does your source filter return ? Check with info() after the source filter.

[Seedmanc]

Script is as follows
dss2("D:\kf2.mp4")
converttoyv12
FrameRateConverter( preset="slowest",blksize=16 )
I said "I'm using masktools from the same author" but it was a mistake, I meant pinterf's masktools. Before then I also used the default masktools but it gave the same results.

[Taurus]

maybe:
converttoyv12()
:D

[manolito]

This will make the color conversion a little faster (at least in classic AVS, no idea about AVS+), but it should not cause a crash.

[Seedmanc]

The video is already in YV12, it was only put there to demonstrate that.

[poisondeathray]

what avs version? check your plugins again or load them explicitly - maybe you have multiple versions

[StainlessS]

Run AvsMeter against your installation, and see what version, and what plugs you got.
Today, G2K4 got a nice shiney new version to play with.

[Groucho2004]

Run AvsMeter against your installation, and see what version, and what plugs you got.More precisely - Run "avsmeter avsinfo -log" and post the log file ("avsinfo_x86.log" or "avsinfo_x64.log").

[Seedmanc]

Avisynth 2.6, the one with SetMTmode.
Do masktools and mvtools have their own version() function? Such large plugin packs with lots of dependencies should have that so you don't have to check it manually in plugin directory every time to see which one got loaded.

When I just call MMask(last) (mmask is among the offending functions in that line that FRC is complaining about) I get a strange "Access violation /syswow64/ntdll.dll".

Here's log, compressed. Why does everything has to be so difficult here in 2018.

[StainlessS]

Seedmanc. dont think you log is likely to overflow Usage Forum post limit of 16KB (Dev forum=20KB).

Awaiting mods to approve. :)

[Groucho2004]

Here's log, compressed. Why does everything has to be so difficult here in 2018.The size of the log is not the issue, the time it takes for a moderator to approve your attachment is. You'd be better off posting it on pastebin or similar.

[Seedmanc]

I tried it on a different OS with a fresh installation of AVS+, still getting errors. https://pastebin.com/5Kz7Wsft log
Depending on the masktools version the errors are different:
MT a48 : mt_inpand unsupported colorspace, only planar YUV, etc, line 164
MT pinterf : mt_merge clips should have identical bit depths, line 189
Was this even tested?

[Groucho2004]

I tried it on a different OS with a fresh installation of AVS+, still getting errors. https://pastebin.com/5Kz7Wsft log
Depending on the masktools version the errors are different:
MT a48 : mt_inpand unsupported colorspace, only planar YUV, etc, line 164
MT pinterf : mt_merge clips should have identical bit depths, line 189
Was this even tested?
Delete "mt_masktools-25.dll" and try again.

[Seedmanc]

That helped, thank you very much.

But it looks like this plugin just blends frames together most of the time. Is there a configuration example I can use to remove "safety locks" so it never resorts to blending and instead shows its best attempt at interpolation even if it's not passing some good metrics?

[StainlessS]

You wann give those that use this script, some idea what you are trying to do, low framerate up to super duper framerate, perhaps ?

[Lirk]

0 value doesn't disable BlendOver (and maybe SkipOver) parameters. 0 value just sets threshold to 0, but not disable this parameters.

[Seedmanc]

If merely doubling to 48-60fps counts as super for this script then my point still stands.

I'd rather see all the artifacts the script produces instead of the ghosting doubleframes, it would at least allow to compare to the artifacting other scripts do, to see which gets closer to the original (assuming I can take 60 fps, halve the framerate and try to reconstruct dropped frames). The way it is now the only measurement I can do is about its speed.

Here are the examples, one from the Sintel clip http://www.framecompare.com/image-compare/screenshotcomparison/0J9MNNNU the other from idolmaster ingame footage http://www.framecompare.com/image-compare/screenshotcomparison/WD7PNNNX for more 2D-ish kind of material. In both cases I had to use the "flow" output of FRC to get at least something similar to motion interpolation instead of pure doubling like ConvertFPS on auto. But even then the simple mvtools with recalculation did better. I tried setting various threshold params of FRC to 0 as the readme says but it didn't help.

[zorr]

...it would at least allow to compare to the artifacting other scripts do, to see which gets closer to the original (assuming I can take 60 fps, halve the framerate and try to reconstruct dropped frames). The way it is now the only measurement I can do is about its speed.

...

But even then the simple mvtools with recalculation did better. I tried setting various threshold params of FRC to 0 as the readme says but it didn't help.

You could try AvisynthOptimizer (https://forum.doom9.org/showthread.php?t=175723) for this task, it can search optimal settings for you (semi)automatically. If you're interested I can provide you a frame-doubling MVTools script that AvisynthOptimizer can work with. Should be easy to make one for FramerateConverter as well. Warning: running this optimization takes a long time (hours, maybe days if you get really serious about it) and is very addictive. ;)

[Seedmanc]

zorr, as a matter of fact yes I was going to try your tool for a while already, been watching your topic closely, waiting till it 'stabilizes' more or less about parameters and software involved. Maybe it's about time I give it a try.
I've been extensively testing various framerate conversion plugins recently,trying to figure out the best settings, but for now I've ran out of options (seems the pinterf's 64x64 does the trick better than the rest, though I had my hopes high for SVP). Particularly I'm interested in figuring out the usable truemotion parameters for blocksizes 32 and above, as they seem to be incompatible.

[zorr]

zorr, as a matter of fact yes I was going to try your tool for a while already, been watching your topic closely, waiting till it 'stabilizes' more or less about parameters and software involved. Maybe it's about time I give it a try.

While the optimizer is still officially beta I don't expect any major changes anymore. Please do try it and if you have any problems I will try to help. Your feedback on any issues you may have will be valuable.

I've been extensively testing various framerate conversion plugins recently,trying to figure out the best settings, but for now I've ran out of options (seems the pinterf's 64x64 does the trick better than the rest, though I had my hopes high for SVP). Particularly I'm interested in figuring out the usable truemotion parameters for blocksizes 32 and above, as they seem to be incompatible.

All right, here's a frame-doubling script using Pinterf's MVTools2.

TEST_FRAMES = 10
MIDDLE_FRAME = 25

# original framerate
FPS_NUM = 18
FPS_DEN = 1

# source clip
AVISource("d:\optimizer\bridge\bridge_cropped.avi")
AssumeFPS(FPS_NUM, FPS_DEN)

#return last

# needed for some parameter combinations
ConvertToYV24()

orig = last

# preprocessing - blur and/or degrain
#searchClip = orig
#searchClip = last.Blur(1.58).Blur(1.58)
searchClip = RemoveGrain(orig, 22) # optimize RemoveGrain(orig, _n_) | -1..24 | rmgrain


super_pel = 4 # optimize super_pel = _n_ | 2,4 | super_pel
super_sharp = 2 # optimize super_sharp = _n_ | 0..2 | super_sharp
super_rfilter = 2 # optimize super_rfilter = _n_ | 0..4 | super_rfilter
super_search = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, searchClip)
super_render = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, orig, levels=1)

blockSize = 32 # optimize blockSize = _n_ | 4,6,8,12,16,24,32,48,64 ; min:divide 0 > 8 2 ? ; filter:overlap overlapv max 2 * x <= | blockSize
searchAlgo = 5 # optimize searchAlgo = _n_ | 0..7 D | searchAlgo
searchRange = 2 # optimize searchRange = _n_ | 1..30 | searchRange
searchRangeFinest = 2 # optimize searchRangeFinest = _n_ | 1..60 | searchRangeFinest
lambda = 1000*(blockSize*blockSize)/(8*8) # optimize lambda = _n_ | 0..20000 | lambda
lsad=1200 # optimize lsad=_n_ | 8..20000 | LSAD
pnew=0 # optimize pnew=_n_ | 0..256 | pnew
plevel=1 # optimize plevel=_n_ | 0..2 | plevel
overlap=0 # optimize overlap=_n_ | 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32 ; max:blockSize 2 / ; filter:x divide 0 > 4 2 ? % 0 == | overlap
overlapv=0 # optimize overlapv=_n_ | 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32 ; max:blockSize 2 / | overlapv
divide=0 # optimize divide=_n_ | 0..2 ; max:blockSize 8 >= 2 0 ? overlap 4 % 0 == 2 0 ? min | divide
globalMotion = true # optimize globalMotion = _n_ | false,true | globalMotion
badSAD = 10000 # optimize badSAD = _n_ | 4..10000 | badSAD
badRange = 24 # optimize badRange = _n_ | 4..50 | badRange
meander = true # optimize meander = _n_ | false,true | meander
temporal = false # optimize temporal = _n_ | false,true | temporal
trymany = false # optimize trymany = _n_ | false,true | trymany
dct = 0 # optimize dct = _n_ | 0,2,3,4,5,6,7,8,9,10 | dct

delta = 1
useChroma = true
bv = MAnalyse(super_search, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv,
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct)
fv = MAnalyse(super_search, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv,
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany, dct=dct)



threshold = 10000
maskScale = 70 # optimize maskScale = _n_ | 1..300 | maskScale
mask_fps = 3 # optimize mask_fps = _n_ | 0..2 | mask_fps
inter = orig.MFlowFPS(super_render, bv, fv, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold, thSCD2=100)

# return this to look at the clip with doubled framerate
#return inter

fps_only = inter.SelectOdd()

# second pass
super_search2 = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, fps_only)
super_render2 = MSuper(pel=super_pel, sharp=super_sharp, rfilter=super_rfilter, fps_only, levels=1)
bv2 = MAnalyse(super_search2, isb = true, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv,
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany)
fv2 = MAnalyse(super_search2, isb = false, blksize=blockSize, search=searchAlgo, searchparam=searchRange, pelsearch=searchRangeFinest,
\ chroma=useChroma, delta=delta, lambda=lambda, lsad=lsad, pnew=pnew, plevel=plevel, global=globalMotion, overlap=overlap, overlapv=overlapv,
\ divide=divide, badSAD=badSAD, badrange=badRange, meander=meander, temporal=temporal, trymany=trymany)
inter2 = fps_only.MFlowFPS(super_render2, bv2, fv2, num=FPS_NUM*2, den=FPS_DEN, mask=mask_fps, ml=maskScale, thSCD1=threshold, thSCD2=100)
fps_only2 = inter2.SelectOdd()


delimiter = "; "

inter_yv12 = fps_only2.ConvertToYV12()
orig_yv12 = orig.ConvertToYV12()

# for comparison original must be forwarded one frame
orig_yv12 = trim(orig_yv12,1,0)

inter_yv12 = inter_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
orig_yv12 = orig_yv12.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = inter_yv12

global total = 0.0
global ssim_total = 0.0
global avstimer = 0.0
frame_count = FrameCount()
FrameEvaluate(last, """
global ssim = SSIM_FRAME(orig_yv12, inter_yv12)
global ssim_total = ssim_total + (ssim == 1.0 ? 0.0 : ssim)
""", args="orig_yv12, inter_yv12, delta, frame_count")

# NOTE: AvsTimer call should be before the WriteFile call
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (ssim, time)
resultFile = "D:\optimizer\perFrameResults.txt" # output out1="ssim: MAX(float)" out2="time: MIN(time) ms" file="D:\optimizer\perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "ssim", "delimiter", "avstimer")
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "ssim_total", append=true)

return last

You will need to change the source (obviously) and the folder of the result file. Change MIDDLE_FRAME to find a good spot for the test. TEST_FRAMES sets the number of frames tested, larger than 10 will make the script more robust but will make the optimization process slower. Set the frame rate of your clip using FPS_NUM and FPS_DEN.

If your video has a large resolution it might be a good idea to crop it to a smaller size, leaving just the interesting motion in the center. Make a cropped video and load it directly into this script to make the optimization faster.

My approach at frame rate doubling is this: I double the rate of the original video, remove original frames, then double the rate again and compare against the original frames. That has been working well but I haven't done comparisons to other ways (like dropping every other frame and then doubling the rate). When you write out the optimized scripts take out the comment in front of "return inter" to see the frame-doubled version (ie without the second pass which recreates the original frames).

This script has a lot of parameters and the ranges are mostly not restricted in any way (some parameters like lambda and lsad don't have an upper limit so I just used a very large number there). I did take out dct=1 because with some parameter combinations it takes *extremely* long time to calculate (like 8 minutes when normal time is 1,5 seconds). That's a shame because in my testing dct=1 is usually the best quality-wise.

Instead of using truemotion true/false this script sets the parameters controlled by it (lambda, lsad, pnew, plevel, global) individually.

If you already know for example which blockSize is good then just adjust the search ranges accordingly.

This script doesn't have MRecalculate, I recommed that you first try to find optimal values without it and once you have them add MRecalculate and optimize only its parameters (I can give an example script of that as well). Actually you could stack as many recalculates as you want always using smaller and smaller blocksize... I think it could be a way to make a really robust script, starting with blocksize 64 and halving it with every recalculate. I haven't tried that yet so it's just a theory. :)

Note that not all the parametes of MVTools2 are there, I haven't gotten around to trying everything yet...

The script includes preprocessing the search clip with RemoveGrain, in my tests I have found it can help. Obviously we are searching an optimal value for that function as well.

Oh and one more thing, I came up with a way to limit the ugly artifacts you often get with MFlowFPS when good motion vectors are not found. Basically you reconstruct the frame created with MFlowFPS using MCompensate and the original frames. You just need to find good parameters for that, which you can do with the optimizer. I will give an example of that later. :)

[parbruek]

Confirming the same issue Sharc saw previously:
https://forum.doom9.org/showthread.php?p=1808322&highlight=access+violation#post1808322
(https://forum.doom9.org/showthread.php?p=1808322&highlight=access+violation#post1808322)
FrameRateConverter runs into Access Violation whenever I set the speed to anything slower than "normal". The error seems to be occuring in line 143:

bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizev=BlkSizeV, overlap = BlkSize>4?(BlkSize/4+1)/2*2:0, overlapv = BlkSizeV>4?(BlkSizeV/4+1)/2*2:0, search=3, dct=Dct)

Just tested with blu-ray and dvd compliant material with the same result, so it probably isn't triggered by resolution or bit depth...

I've tested and found the same issue with current versions of Avisynth 32 bit and Avisynth+. Should I fiddle with BlkSize, or does someone else know of a fix?

[parbruek]

Just found the issue - My apologies! I had a bad version of FFTW3.dll. And I'm not even sure where it was from. I grabbed the version from here (http://www.avisynth.nl/users/warpenterprises/) and dropped it in syswow64. Everything runs properly now.

[attackworld]

Sorry for lack of technical terms but...

Tested this with many many different settings, and the best interpolation I can get is just a blend of the two different frames. Only very small movement will the frame look like its a new frame altogether, but with bigger movements you can see a faded version of the two original frames in the "interpolated" frame.

At least with SVP it attempts to create an image that is somewhere between the two original frames.

[Dogway]

I'm unable to run this at all at a reasonable speed. I'm in AVS+ x64, and using fftw338-x64-AVX2. It falls into sub 1fps. Either with or without mp_pipeline using slower preset. What fftw do you guys use? Is there a special MT mode I have to use for FrameRateConverter.dll?

[ChaosKing]

Hmm I get 14fps on a 1080p source. Just with FrameRateConverter() alone. CPU is only @ 25%

Have you installed the latest plugins? You can use AVSRepo to download/install framerateconverter https://forum.doom9.org/showthread.php?t=175822

[Dogway]

I spent a week finding latest and fastest builds of everything manually, since I have some preference.
As I said the only ones I wasn't sure at all were fftw and also dfttest (the latest v1.9.4.3 gives me issues).
As far as I know the bulk of this script is MVTools, masktools, fftw (for slower modes) and Resizers for which I might change the code to use the faster multithreaded version by jpsdr.
Is that speed with slower or slowest preset?
I noticed that in the repo the latest FFTW3 version listed is 3.3.5, any reason to not use any higher?

[ChaosKing]

I used the default settings... just FrameRateConverter()

I need direct links + 3.3.5 is as fast as the newer builds, maybe max 1 fps slower.

[Dogway]

Default preset is normal so it uses DCT=0 (no use of fftw library).
Direct links are in the top of your AVSRepo thread, (https://forum.doom9.org/showpost.php?p=1844512&postcount=1), the Wolfberry builds.
AVX2 instruction should be faster, but I haven't seen speed tests on that though.

edit: I updated the fftw library with a build from Wolfberry and now my dfttest works, I'm going to run another test with FRC.

[ChaosKing]

I made my own speed tests. As far as I see I can not directly link to files in the gdrive share folder.

[MysteryX]

I don't know anything about 3D material.

I have this nagging feeling that there has to be something better than plain frame blending as a backup on artifact areas and stripes... ? Using original frames on those areas wasn't giving a good results -- maybe a variation of frame blending that is always 80/20% at most, so no 50/50 blended frames?

hum... if that could be done, it would provide a trade-off between mode skip where the artifact zones jump in chunks, and mode blend where artifact zones can be a blurry mess

[Dreamject]

O.o why are mvtools/FrameRateConverter so difficult and does not have any preset like 'import it to your player and just use'.
It's more easy to make presets that use core file.

https://s17.directupload.net/images/190526/scnxd3hw.png

Sending to ffdshow looks like

[HKEY_CLASSES_ROOT\.avs]
@="avsfile"
"PerceivedType"="text"

[HKEY_CLASSES_ROOT\avsfile]
@="AviSynth+ Script"

[HKEY_CLASSES_ROOT\avsfile\shell]


[HKEY_CLASSES_ROOT\avsfile\shell\Послать в FFDSHOW X64]

[HKEY_CLASSES_ROOT\avsfile\shell\Послать в FFDSHOW X64\command]
@="REG ADD \"HKCU\\Software\\GNU\\ffdshow64_raw\\default\" /v \"avisynthScriptMULTI_SZ\" /t REG_MULTI_SZ /d import(\\\"\"%1\"\\\")\\0 /f"

[HKEY_CLASSES_ROOT\avsfile\shell\Послать в FFDSHOW X86]

[HKEY_CLASSES_ROOT\avsfile\shell\Послать в FFDSHOW X86\command]
@="REG ADD \"HKCU\\Software\\GNU\\ffdshow_raw\\default\" /v \"avisynthScriptMULTI_SZ\" /t REG_MULTI_SZ /d import(\\\"\"%1\"\\\")\\0 /f\""



PS: I found it has presets, but I dont know how to use them

[MysteryX]

Are you trying to use it for real-time playback? This won't render at real-time speed. For that, have SVP project.

[StainlessS]

PS: I found it has presets, but I dont know how to use them

Apart from noobs like yourself, I dont think anybody [meaning regular Avs users] still uses ffdshow (or PotPlayerSource [I was not even aware of it till last few weeks], or whatever its called),
perhaps I'm wrong. Maybe best place find out about use of ffdshow, or Potplayer, is in forums specifically supporting those items.

EDIT:
This won't render at real-time speed.
Absolutely.

EDIT: I guess that you could start your own thread specifically for ffdhow/potplayer use with Avs in a particlar player, at least then you might get responses all in one place.
Maybe best forum would be Software Players forum, near bottom of Main D9 forum page.

EDIT: Most I ever do in a player is change aspect ratio or switch on subtitles, pretty much nothing else.

[Dreamject]

~_~ I'm not very stupid (and not very smart), just AviSynth is closed for understanding. You need use knowledges and logics from ~1990. It could be funny if you scare you grandson 'we had to use eval(""" """) widthout highlight instead of {}, we set filename in scripts by hand and them did not works without it', but using it in present is not funny. I still can not get why avisynth can not do processing in realtime. SVP does it, but svpflow1 based on mvtools2 and as I know svpflow2 (frame rendering plugin) based on another plugin.

>Most I ever do in a player is change aspect ratio or switch on subtitles, pretty much nothing else.
As I know you used interframe
https://forum.doom9.org/showthread.php?t=160226&page=48

But its dead and uses his own and very weird logic which has nothing to do with reality.
https://s17.directupload.net/images/190528/y77q4e57.png
I just translated svp logic from js to avisynth and it works mostly like svp. Only ofter that I gen knowledge that natively avisynth can not render in realtime O.o

[Groucho2004]

AviSynth is closed for understanding. You need use knowledges and logics from ~1990.I still use knowledge and logic from the late 17th century such as Newton's calculus and the laws of gravity. Should we forget about these too?

If Avisynth is so closed to you, try Vapoursynth, maybe that's more to your liking.

By the way - here's (https://www.youtube.com/watch?v=fPV05eZgEpc) a funny video about Millennials.

[StainlessS]

As I know you used interframe

Actually, that was MvTools2[in interframe thread]. (I have on occasion used interframe, but only ever with defaults, apart from output framerate).

My point was that best place to seek advise was from people that are likely to have used it (avisynth) with your player,
I'm certainly not gonna go figure out how it works and what is required.

Also, if anything should insert filename into an avisynth script, then it is you (or your player, where source script would be some kind of template [probably created by you or your lackey]).

Seriously, player forum best place. If regular AVS user wanted use in player, would likely not even bother to figure out how presets work, would be no effort at all to set up script exactly how he/she/it wanted it.

[Dreamject]

~_~ it's only one popular project on avisynth (as I know) - SVP, cause it usable, like click and run, but it's not very perfect. If avisynth(vs) will be more usable for people, better postprocessing filter will be.

[StainlessS]

@G2K4,
Your vs suggestion [EDIT: not to me] was a stroke of genius, thanx muchly :)

[MysteryX]

Dreamject, most people using Avisynth use it for high-quality video encoding, heavy-processing often running at 1 to 5 fps.

If you want to do high-quality video encoding at 60fps, use FrameRateConverter.

If you want real-time playback, use SVP which uses GPU acceleration but doesn't provide the same level of quality.

Groucho2004, GREAT video!! Shared it on Facebook.

[MysteryX]

Got tons of more important stuff to do but figured out this would be quick to implement.

FrameRateConverter was doing a great job at identifying artifact areas, but the backup plan sucked (plain frame blending) so the quality of artifact mask wasn't resulting in that much better output -- replacing artifacts with double-shadows. This version eliminates double-shadows and thus greatly enhance the overall result by having clearer pictures in the artifact areas without breaking the motion.

Version 1.3 is ready! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v1.3)

What's new:
- Greatly enhanced quality in artifact areas! Ghost effect eliminated.
- Added ConvertFpsLimit to blend with reduced blending ratios. Ratio=0 is frame copy and Ratio=100 is full blending. Ratio=50 reduces blending by half.
- Now perform soft blending with a default ratio of 40, meaning frames blending at 50/50 will blend as 20/80 (50 * .4 = 20)
- You can adjust frame blending ratio with BlendRatio

I haven't done much testing to find the best blend ratio but 40 is working good. Post your findings.

Another version could done so that ConvertFpsLimit falls back to the standard ConvertFps or ChangeFps when ratio is 0 or 100, and the soft vs hard blend option can be removed as setting this BlendRatio completely replaces it: hard blend is BlendRatio=0.

[MysteryX]

I'm just realizing that this program blows every commercial solution completely out of the water for frame interpolation.

[videoh]

Be careful with your hyperbole!

[Dreamject]

>I'm just realizing that this program blows every commercial solution completely out of the water for frame interpolation.
How to try it in any player?

[MysteryX]

>I'm just realizing that this program blows every commercial solution completely out of the water for frame interpolation.
How to try it in any player?

You can open AVS script in MPC-HC but only for preview. As I said twice already, you cannot use this for real-time playback, so I won't repeat it a 4th time.

[Dreamject]

You said it blows commercial projects, I thought now it's work in real time. BTW svp is not fully commercial, mostly it's free

[MysteryX]

Commercial is mostly for video editing softwares such as Optical Flow in Adobe Premiere.

There are other consumer real-time solutions being implemented in graphic cards and TVs, as well as SVP, but real-time consumer interpolation can never equal professional video editing tools.

For consumer (real-time) playback, SVP does the best job. For professional video editing, FrameRateConverter does the best job. Period.

Although both are based on the same old MvTools2 code that nobody really understands. The most critical part is in the DCT modes which I don't think even Pinterf understands that were written 50 years ago. If someone could understand that part and implement new and more modern DCT modes, we could see further quality and/or performance improvement. I'm sure there's been progress in that kind of algorithm over the years.

[manolito]

Any chance that somebody could compile a separate ConvertFpsLimit plugin for classic AVS 2.60 which does not require SSE2? The latest FrameRateConverter.dll does not work for my machines.

[MysteryX]

Should be pretty easy to do for someone with an older compiler; I only have Visual Studio 2019 installed on my machine, and no XP support. Just copy the project and cut out a few files.

The source code contains a non-SIMD version of GetFrame that is commented so it can be put back easily.

Oh, forgot to mention, the latest version requires VC++ 2019!

btw I'm surprised there are still lots of people using Avisynth with old CPUs. Modern CPUs are designed mostly FOR video encoding; it's pretty much the only use nowadays for such computing power.

[wonkey_monkey]

Manolito's dual monitor video processing setup (http://magazine.loyola.edu/issue/fall09/images/stories/mainframe/old_comp.jpg).

[ChaosKing]

Manolito's dual monitor video processing setup (http://magazine.loyola.edu/issue/fall09/images/stories/mainframe/old_comp.jpg).With touch support! :O

[StainlessS]

MX, can you update first page of thread with current version, still points at v1.0 from 2017.
Thanks.

EDIT: You seem to be using an old Avisynth.h(+) header from 2017 (presume that other headers are likewise dated).

EDIT:
Also, Capi.h has been updated this year,
Alignment.h and config.h were updated 17 days ago.

And C header, avisynth_c.h also updated this year.

Suggest that you update ALL avs+ headers.

Headers on github:- https://github.com/pinterf/AviSynthPlus/tree/MT/avs_core/include

(update at least for next version, dont suppose any great need to update to new ver$ immediately - unless P disagrees).

[MysteryX]

Thanks StainlessS, first post updated. Will update headers next time.

[StainlessS]

Any chance that somebody could compile a separate ConvertFpsLimit plugin for classic AVS 2.60 which does not require SSE2? The latest FrameRateConverter.dll does not work for my machines.

Having a go, can't promise anything.

Attempting compile on VS2008, requires some headers from VS2010 (C99, stdint.h cstint), currently extracting VS2010 headers, dont know if any further problems will ensue).

Still dont know yet if any .NET crap is needed, if so then will probably abandon attempt.

Wish me luck as you wave me goodbye, ...

EDIT: Nice monitor setup Mani, I miss not havin' wot ewe got.

[MysteryX]

nothing .NET-related in there at all

[StainlessS]

MX,
I cant really continue.
I got no idea whatever bout how that template stuff works, nor the type auto and other thingy stuff that seems to require minimum VS 2013 (eg constexpr).
I might be able to expand the template to full source, but dont really wanna do that.
So, pretty much about to abandon attempt at Manolito Non SSE2 plug for ConvertFpsLimit().

Havin' said that, I think that I may have discovered why is not working for Mani, when I guess that you probably expected it to work ok.

From Merge.cpp

#include "merge.h"
//#include "merge_avx2.h"
//#include <emmintrin.h> // SSE2
//#include <smmintrin.h> // SSE4.1


#include <xmmintrin.h> // SSE, Not originally included
#include <mmintrin.h> // MMX Not originally included

#include "avs/alignment.h"
#include <stdint.h>


In red missing, could that be the problem ? (maybe just need recompile with the missing headers added [EDIT: or do the ones in BLUE include the lower SSE and MMX headers themselves])

EDIT: I guess I really could do as Mani requested, only supporting v2.60 standard and not avs+ additional colorspaces.

[manolito]

I guess I really could do as Mani requested, only supporting v2.60 standard and not avs+ additional colorspaces.

This would be all I need... :)
The reason I would like to have this is that I find this blend concept very intriguing. Maybe this could be used for some other fps conversion tasks where the standard ConvertFPS looks terrible and ChangeFPS is just too jerky.

Nice monitor setup Mani, I miss not havin' wot ewe got.
Yeah, I love it,too. And this was at a time when I still had hair... :D


Cheers
manolito

[MysteryX]

Merge.cpp is a direct copy from AVS. If you go in ConvertFpsLimit::GetFrame and comment out SIMD optimization, and uncomment the code right above, then you won't need Merge.cpp at all. It won't be as fast but for testing will work just fine.

[StainlessS]

MX,
I've hacked it down to bare bones avs v2.60 std colorspace, no idea yet if it works at all (I no longer have either mmx or iSSE only machines, min P4).

You got any idea what will happen if just using below, compiles ok with the other mmx thing commented out. [merge.cpp]

#include "merge.h"

#include <xmmintrin.h> // SSE, Not originally included
//#include <mmintrin.h> // MMX Not originally included


I really wanna get some sleep soon.

[StainlessS]

MX, compiler option "/arch:SSE" or "Not Set" better for PIII ? [VS2008]

Ill try post something before beddy-byes.

[MysteryX]

I'm not the one to ask about SIMD optimizations and processor directives, but I'll change something if you point me to something to tweak.

The simple fact is, there's a commented routine in there using no SIMD optimizations at all and not using Merge.cpp -- so can just use that, exclude Merge.cpp from project, set project to no Arch, and it will compile without requiring any specific CPU.

[manolito]

I dug out the old johnmeyer torture clip (the one with the antlers) and tested what the new ConvertFPSLimit routine can do. And the results are quite impressive IMO. Download here:
https://www.sendspace.com/file/4rh967

First I just did a blend conversion with a limit of 40, and I believe that it can be very useful in a lot of cases. Just don't step through the frames, you need to watch it in motion.

Then I did some conversions using the old pre-pinterf plugins which restrict the allowed block sizes and do not support deep color and high bitdepth. My simplified mx_fps modded version is also included.

The last thing I did was downloading the latest pinterf plugins and using FRC with default settings and with slowest settings.


All conversions used the default limit value of 40, I think this is a very good compromise. And at least with this clip my simplified mx_fps build with limited options does it just as well as the original FRC version. And IMO the "Slowest" parameter is not worth the speed sacrifice.


Cheers
manolito

[StainlessS]

Thanks MX, already done dll's C + MMX/iSSE.
No idea at all if it works or not, wanna get some sleep. [only guarantee is that it compiles]

http://www.mediafire.com/file/mjxsdunsmdd93st/ConvertFpsLimit_v0.01_20190603.7z/file

Will be a nice surprise for me [when I wake] if it dont crash immediately :)

Goodnight.

EDIT: I think the iSSE only happens if 50.0/50.0 blend, otherwise MMX [obviously not in the C only version].

Included VS2008 hacked source.

[MysteryX]

Will be a nice surprise for me [when I wake] if it dont crash immediately :)
Would be a nice surprise if it turns into a BSOD that corrupts his BIOS flash memory and grills his CPU.

Then he'd have to buy a new one. (upgrading, preferably)

[manolito]

Will be a nice surprise for me [when I wake] if it dont crash immediately :)


Tested the MMX/iSSE version on my ancient machine, works wonderfully, no problems whatsoever. Many many thanks... :D

Cheers
manolito

[manolito]

Would be a nice surprise if it turns into a BSOD that corrupts his BIOS flash memory and grills his CPU.

Then he'd have to buy a new one. (upgrading, preferably)

I know that you hate me for my philosophy to support sustainability, no problem... :mad:

My tests with your latest ConvertFPSLimit addition only showed that all these "modernized" plugins and new fancy deep color and high bitdepth capabilities did absolutely nothing to the core conversion quality. I will continue to use older hardware (and repair it instead of buying new gadgets every year), and I do not feel like I am giving up anything.

[wonkey_monkey]

and I do not feel like I am giving up anything.

I hope you never need any of my recent plugins then ;)

[manolito]

Hi David,

well, I did not need any of your previous plugins so far, and I think I won't need any in the future... :devil:

Just to make things clear, this is my basic profile:

Birth Date: 1953
First Computer: 1986
Turbo Pascal, MASM+TASM, always hated GUI programming
Viewing device: CRT, 100Hz no-flicker
Media streaming box: Xtreamer Sidewinder, no HEVC
Smart Phone: None
Car: None
Motorcycle: Yamaha XJ 900 from 1991

I generally do not care for HD video content, my TV does not support it, and I will not give up this TV set for any LCD because so far nothing can match its natural colors. HDR, Deep Color, High Bitdepth are things I will certainly not explore in my lifetime. And my general goal is sustainability, I refuse to throw away things which work just because they are not in fashion any more.

So you can ridicule me as much as you want, I can live with this.

[wonkey_monkey]

Oh, no, I don't ridicule you at all - any teasing is meant in good spirits. If you were to complain about a lack of compatibility for very old systems from new plugins, then perhaps I would, but you don't as far as I know (complaining about a lack of <AVX-512 support would be valid; <SSE, not so much!). And sustainability is a very worthy goal. I hate replacing things unless it's truly necessary and I'm only begrudgingly considering getting a new laptop now because my current one is literally falling to pieces. And even then I may just get the same model from Ebay and swap the parts around.

[StainlessS]

no problems whatsoever.

Dont know why, all I did was remove SSE2/AVX stuff, and change to use iSSE header (which presumably also includes the mmx header).
Did not notice anything that could have been responsible for prev probs.
[I presume that there aint any bugs in the VS2008 SSE2/SSE4.1 intrinsic headers.]
[EDIT: Or rather, in the VS2019 (or whatever MysteryX used) headers.]

EDIT: Smart Phone: None
I only just recently upped from non-smart SamSung SGH-E250 (my real phone, I had 3 of them over bout 10/12 years), to Motorola RAZRi (circa 2012, I think), I always fancied
one of those with the Kevlar case back (I wear it in my shirt pocket, just incase I get shot in the heart, might save my bacon). Got about 5 or 6 other
Android phones (but all locked to some network, only use them as Wifi devices). Also [recently] got a Blackberry Samoa, nice little £20.00 unlocked backup phone but browser is crap
as screws up on any HTTPS site. I still miss an old Nokia that I had for a time (non-smart, got nicked), 1661 I think, had 21 day standby, dont get that often on your smarts.
Currently looking at UmiDigi with VoLTE 4G and dual SIM, as next buy [android], or might even get a Lumia 950 XL, now that nobody else wants them (I like being awkward).

[Groucho2004]

Just to make things clear, this is my basic profile:You forgot your bra size. :D

Motorcycle: Yamaha XJ 900 from 1991Very nice. I had a XJ650 back in the 80's, loved it.

[Dreamject]

>For consumer (real-time) playback, SVP does the best job. For professional video editing, FrameRateConverter does the best job.
What is the problem for realtime processing?

SVP in first releases used mvtools https://www.videohelp.com/software/SmoothVideo-Project/old-versions#downloadold . There is a lot of directions for mvtools, but they just not works in realtime. For my (user) side avisynth is like pixel shader. If it works correct, I do not want to know how it works, its syntax, a code is not interesting for me until it begins works bad.

From quality side, SVP reduces quality of video a lot (it has no noises and frames, it has a lot of artifacts). If you replace all frames to svp frames (or blended frames) it will be bad. By default SVP wants to drop half of original frames in 24fps, dev says in 25fps it wants to leave only 5 original frames. For quality its very bad solution (but maybe good for smoothness). It has settings for autodetection of scenes, but they just do not works in auto mode and you have to create profile manually

[real.finder]

manolito, You are older than of my parents

I agree with your CRT vs LCD, but all SONY TVs in our Home dead now, there are no replacement Except LCD and those new technologies like OLED, HD and HDR are good, but I don't mind life with SD things

we have pentium 4 1.7 ghz pc (made in 2001-2002) which I upgrade many things in it previous years and it still work but no one use it

also even if I have Smart Phones I hate use them

[manolito]

You forgot your bra size. :D


Sorry, forgot about this, but my boobs are not really worth mentioning... :eek:

[MysteryX]

What is the problem for realtime processing?
You have to cut corner to process at very high speed. It's always a trade-off between quality and performance.

SVP in first releases used mvtools
It's still based on mvtools, except they made their own version with GPU acceleration. Which is why it can render in real-time.

For my (user) side avisynth is like pixel shader. If it works correct, I do not want to know how it works, its syntax, a code is not interesting for me until it begins works bad.
AFAIK VapourSynth is in a more "it just works" state -- although I have yet to port this plugin to it. Even if it works more smoothly with multi-threading and full CPU usage, you can't process heavy script in real-time. In fact, VapourSynth doesn't even have audio support so forget about live playback.

Also, I tried taking the auto-generated script from SVP and running it manually as an AVS script instead of passing through ffdshow. Curiously enough, it rendered better through ffdshow than when opening the clip directly! Problems start occurring when skipping a frame -- when playing the script directly it wants to render every single frame so it never catches up.

Sorry, forgot about this, but my boobs are not really worth mentioning... :eek:
Are you shy?

[Dreamject]

>It's still based on mvtools, except they made their own version with GPU acceleration. Which is why it can render in real-time.
NO. You can use GPU acceleration, it decrease CPU load, thats it. My laptop has i7 3232QM with hd4000 (nvidia burned) and its not enought for 60fpsSVP 1080p with GPU + deband shader. My cpu can use highest svp profiles. Your words are like 'you even cant play video without gpu decoding"

[hfrforever]

Hey, I am interested in making this work, but I have almost no experience in scripting (some highschool java and matlab). I have been looking for the past week to see if i can find a decent tutorial on how to make it work properly. I would appreciate any direction you could provide, or point me to some instructions... I am interested in converting some of my videos to 60 fps, but I do not know how to use most of the software involved... what are the prerequisites, and if someone could post a working script, I am quite good at modifying scripts. any help would be appreciated.

[MysteryX]

if someone could post a working script, I am quite good at modifying scripts. any help would be appreciated.

Both of you, start by learning the basics here
https://www.spirton.com/convert-videos-to-60fps/

then can easily replace InterFrame with FrameRateConverter

[hfrforever]

I followed the instructions, and I think i have it set up properly
this is my best attempt at making the script work... can someone critique my work please?

also, I am unsure of the file format i am supposed to use, and if i need to convert the mp4, to mkv or avi?


Cores=4
SetMemoryMax(6000)
SetMTMode(4,Cores)
PluginPath = "C:\Program Files (x86)\MeGUI\tools\avisynth_plugin"
LoadPlugin(PluginPath+"FrameRateConverter-x64.dll")
Import(PluginPath+"FrameRateConverter.avsi")
<input>.ConvertToYV12()
SetMTMode(2)
<deinterlace>
<crop>
<denoise>
<resize>
InterFrame(Cores=Cores)

[hfrforever]

PluginPath = "C:\Program Files (x86)\MeGUI\tools\avisynth_plugin"
LoadPlugin(PluginPath+"FrameRateConverter.dll")
Import(PluginPath+"FrameRateConverter.avsi")
FrameRateConverter()

# I get an error with this code that says "error parsing avs file, script error: invalid arguments to function 'FrameRateConverter'

[StainlessS]

hfrforever,

Can you clarify, are you only using Avisynth provided by MeGUI ? [do you have a version of avisynth installed, if so which one].

If using avisynth installed by you, then put plugins in the Avisynth autoload directory, not the MeGUI avisynth_plugin directory, and they will be autoloaded automatically without LoadPlugin thing.
Likewise, *.avsi files are autoloaded from Avisynth plugins directory.
(MeGUI only uses it's own default version of Avisynth, if you do not have your own version of it installed).

Apart from that, you need to have a source clip, your script dont have one, so nothing to process.

EDIT: Also suggest that you forget about multithreading and such until you get the basics sorted out, and let SetMemoryMax default, at least for now.

EDIT:
invalit arguments to function 'FrameRateConverter'
It needs a clip arg.

Suggest try something like

Avisource("D:\MyVideoClip.avi") # This implicitly assigns clip to a special variable called Last (when not explicitly assigned to anything else)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last


EDIT: With your own version of Avisynth installed, you can also use eg VirtualDub2 to view/edit script and view results, and load your avs file into MeGUI for final output to eg MP4.
VDub2 AVS script editing via Tools menu. Also good idea to have a good text/script editor.

EDIT: Spotted problem in original script

PluginPath = "C:\Program Files (x86)\MeGUI\tools\avisynth_plugin[B]\" # <<<<<<===== in red, Need directory node separator (BACKSLASH ie '\') at end of path, else BAD PATH.
LoadPlugin(PluginPath+"FrameRateConverter.dll")
Import(PluginPath+"FrameRateConverter.avsi")
FrameRateConverter()

[hfrforever]

Thank you very much for your thorough answer!
Ok, so to answer your first question, I have avisynth installed separately, and I have the FrameRateConverter plugin in both. I tried your code and got an error saying that it could not open the file. I think it might be because I am trying to use an mp4, but I do not know how to convert it to an avi. also, what is the code for making the FrameRateConverter produce a 60fps result?

Avisource("C:\Users\padge\Documents\test video.mp4") # This implicitly assigns clip to a special variable called Last (when not explicitly assigned to anything else)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last

[StainlessS]

Avisource() only opens AVI files [although it (a little surprisingly) will open WAV audio files].
Can use DirectShowSource () for most files, BUT, everybody will advise against this as DirectShow is not frame accurate.
Suggest either FFMS [FFVideoSource] or LSMash [LSMASHVideoSource for ISO files like mp4, and LWLibavVideoSource for non ISO],
unless AVI or MPG [use DGIndex, MPeg2SourcE("d:\somepath\mympg.d2v"), d2v files created by DGIndex].
Weirdly, FFMS/FFMS2 and LSmash only have threads on Development forum (go figure - devs probably think nobody uses them).

Here script TEMPLATE for Avisynthesizer that I posted a few weeks back:- http://forum.doom9.org/showthread.php?p=1874068#post1874068

I've removed the template insertion point for this post for use without Avisynthesizer.

#ASYNTHER Simple_AVI_LSmash_FFMS_Reader ### EDIT: This line is used only in Avisynthesizer: where template name is Simple_AVI_LSmash_FFMS_Reader.avst

# Requires RT_Stats


###########################
Function GetSeq(String vFn,String "aFn") {
Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}
Function IsRiffFileName(String s) {s=RT_GetFileExtension(s) Return(s==".avi"||s==".wav")}
myName="GetSeq: " aFn = Default(aFn,"") aFn = (aFn=="") ? vFn : aFn
Assert(vFn!="",RT_String("%sVFn Cannot be ''",myName)) vFn = vFn.RT_GetFullPathName IsRiffV = vFn.IsRiffFileName IsIsoV = vFn.IsISOFileName
Assert(aFn!="",RT_String("%saFn Cannot be ''",myName)) aFn = aFn.RT_GetFullPathName IsRiffA = aFn.IsRiffFileName IsIsoA = aFn.IsISOFileName
c=0 a=0
Try { # 1st Try specialized AVI
c = (IsRiffV) ? vFn.AviSource : NOP
(c.IsClip) ? RT_DebugF("AviSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
a = (c.IsClip && c.HasAudio && aFn==vFn) ? c : (IsRiffA) ? aFn.WavSource : NOP
(a.IsClip && a.HasAudio) ? RT_DebugF("%s opened Audio\n '%s'",(c.IsClip && c.HasAudio && aFn==vFn)?"AviSource":"WavSource",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try specialized ISO LSMash
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (IsIsoV) ? LSMASHVideoSource(vFn) : c
(IsIsoV) ? RT_DebugF("LSMASHVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (IsIsoA) ? LSMASHAudioSource(aFn) : a
(IsIsoA) ? RT_DebugF("LSMASHAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try FFMS2
Try {
IsOpenV=(c.IsClip && c.HasVideo)
(!IsOpenV) ? FFIndex(vFn) : NOP
c=(!IsOpenV) ? FFVideoSource(vFn) : c
(!IsOpenV) ? RT_DebugF("FFVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a=(!IsOpenA) ? FFAudioSource(aFn) : a
(!IsOpenA) ? RT_DebugF("FFAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try LSmash non ISO Video
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (!IsOpenV && !IsIsoV) ? LWLibavVideoSource(vFn) : c
(!IsOpenV && !IsIsoV) ? RT_DebugF("LWLibavVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (!IsOpenA && !IsIsoA) ? LWLibavAudioSource(aFn) : a
(!IsOpenA && !IsIsoA) ? RT_DebugF("LWLibavAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

Assert(c.IsClip, RT_String("%s failed open on\n '%s'",myName,vFn))
(!c.HasAudio && a.IsClip && a.HasAudio) ? AudioDubEx(c,a) : c
(!HasAudio) ? RT_DebugF("Audio failed Open on\n '%s",aFn,name=myName) : NOP
return Last
}

#[GetSeq("___FILE___")] ### COMMENTED OUT, NOT FOR USE in Avisynthesizer


GetSeq("C:\Users\padge\Documents\test video.mp4")
Return last


The above
requires RT_Stats but should make things easier for you. [also needs FFMS and LSMash, see Dev Forum]

RT_Stats here:- http://forum.doom9.org/showthread.php?t=165479&highlight=RT_Stats

ABove script will use either AVISource, or the other mentioned source filters, but not MPeg2Source().

EDIT:
I have the FrameRateConverter plugin in both
The one in MeGUI directory will never be used.

EDIT:
You may have to inquire in devs forum about which is better version of FFMS/FFMS2(FFmpegSource/FFVideoSource) as there seem to be dozens of versions (and apparently names too) knocking about. (I dont often use it).

[hfrforever]

I am getting yet another error, ffindex cant open C:\Users\padge\Documents\test video.mp4 and says line 1 is the problem

ffms2("C:\Users\padge\Documents\test video.mp4") # This implicitly assigns clip to a special variable called Last (when not explicitly assigned to anything else)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last

EDIT I am using the link below for the ffmpeg

http://avisynth.nl/index.php/FFmpegSource

[StainlessS]

Alas, FFMS/FFMS2 is the hip (modern/fashionable) name for it, but is not the name you use in scripts (go figure),

Give me a few moments to put something together.

[StainlessS]

Which version AVS are you using, x86 or x64 ?

[hfrforever]

according to the wiki, FFMS2 should work...

http://avisynth.nl/index.php/FFmpegSource

[StainlessS]

Which version avs are you using ? (x86, x64)

[hfrforever]

I also tried the LSMASHVIDEO source, but got similar errors. I also checked the wiki, and FFMS2 should be correct syntax

[hfrforever]

I am using the x86

[hfrforever]

Not really sure why I am using this version...

[Groucho2004]

I am using the x86
Download this tool (https://forum.doom9.org/showthread.php?t=176079), run it, save the info and post that.

[hfrforever]

The installed Avisynth version is a 2.6 Alpha or RC:
C:\WINDOWS\SysWOW64\avisynth.dll

Update to Avisynth 2.6 Release or Avisynth+

[hfrforever]

I am inclined to save the plugins i have, and then uninstall avisynth, and try installing it again.

[StainlessS]

You want latest one, the one you have is quite old:- https://github.com/pinterf/AviSynthPlus/releases

[hfrforever]

[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz / Haswell (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, FMA3, AVX, AVX2
4 physical cores / 8 logical cores


[Avisynth info]
VersionString: AviSynth 2.60, build:Mar 31 2015 [16:38:54]
VersionNumber: 2.60
File / Product version: 2.6.0.6 / 2.6.0.6
Interface Version: 5
Multi-threading support: No
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2015-03-31, 06:40:58 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files (x86)\AviSynth\plugins


[CPP 2.5 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth\plugins\LSMASHSource.dll [2017-02-25]

[CPP 2.6 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth\plugins\DirectShowSource.dll [2.6.0.3]
C:\Program Files (x86)\AviSynth\plugins\ffms2.dll [2016-12-29]
C:\Program Files (x86)\AviSynth\plugins\FrameRateConverter.dll [2019-06-05]
C:\Program Files (x86)\AviSynth\plugins\TCPDeliver.dll [2.6.0.7]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth\plugins\colors_rgb.avsi [2015-03-30]
C:\Program Files (x86)\AviSynth\plugins\FFMS2.avsi [2015-05-22]
C:\Program Files (x86)\AviSynth\plugins\FrameRateConverter.avsi [2019-06-05]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth\plugins\avcodec-57.dll [57.81.100.0]
C:\Program Files (x86)\AviSynth\plugins\avformat-57.dll [57.66.102.0]
C:\Program Files (x86)\AviSynth\plugins\avresample-3.dll [3.2.0.0]
C:\Program Files (x86)\AviSynth\plugins\avutil-55.dll [55.47.100.0]
C:\Program Files (x86)\AviSynth\plugins\swscale-4.dll [4.3.101.0]

[Uncategorized files]
C:\Program Files (x86)\AviSynth\plugins\ChangeLog.txt [2019-06-05]
C:\Program Files (x86)\AviSynth\plugins\COPYING [2016-10-18]
C:\Program Files (x86)\AviSynth\plugins\ffms2-avisynth.md [2016-10-13]
C:\Program Files (x86)\AviSynth\plugins\ffms2.lib [2016-12-29]
C:\Program Files (x86)\AviSynth\plugins\ffmsindex.exe [2016-12-29]
C:\Program Files (x86)\AviSynth\plugins\LICENSE [2019-06-05]
C:\Program Files (x86)\AviSynth\plugins\README.md [2019-06-05]



[Plugin errors/warnings]
________________________________________________________________________________________________________________________________________

LoadPlugin: unable to load "C:\Program Files (x86)\AviSynth\plugins\LSMASHSource.dll", Module not found. Install missing library?

Dependencies that could not be loaded:
avutil-55.dll
avcodec-57.dll
avformat-57.dll
swscale-4.dll
avresample-3.dll
VCRUNTIME140.dll

Note: Visual Studio 2015/2017 Runtime doesn't seem to be installed
________________________________________________________________________________________________________________________________________

LoadPlugin: unable to load "C:\Program Files (x86)\AviSynth\plugins\FrameRateConverter.dll", Module not found. Install missing library?

Dependencies that could not be loaded:
VCRUNTIME140.dll

Note: Visual Studio 2015/2017 Runtime doesn't seem to be installed
________________________________________________________________________________________________________________________________________

[Groucho2004]

Visual Studio 2015/2017 Runtime doesn't seem to be installed
Download the All-In-One MS runtime installer you can find here (https://github.com/abbodi1406/vcredist/releases). It updates all runtimes and uninstalls outdated ones if required.

[hfrforever]

alright I updated to the one you linked and ran the runtime installer. Here is the info from the tool. I tried it again, and get this error https://drive.google.com/open?id=1ueaNHLyIQQfeDlmks-stbO6BYzbyIRk-


[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz / Haswell (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, FMA3, AVX, AVX2
4 physical cores / 8 logical cores


[Avisynth info]
VersionString: AviSynth+ 0.1 (r1576, x86)
VersionNumber: 2.60
File / Product version: 2.6.0.5 / 2.6.0.5
Interface Version: 4
Multi-threading support: No
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2014-01-03, 02:14:26 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins+


[CPP 2.5 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\LSMASHSource.dll [2017-02-25]

[CPP 2.6 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins+\DirectShowSource.dll [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins+\ImageSeq.dll [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins+\Shibatch.dll [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins+\TimeStretch.dll [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins+\VDubFilter.dll [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins\DirectShowSource.dll [2.6.0.3]
C:\Program Files (x86)\AviSynth+\plugins\ffms2.dll [2016-12-29]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.dll [2019-06-05]
C:\Program Files (x86)\AviSynth+\plugins\TCPDeliver.dll [2.6.0.7]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.avsi [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins\FFMS2.avsi [2015-05-22]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.avsi [2019-06-05]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\avcodec-57.dll [57.81.100.0]
C:\Program Files (x86)\AviSynth+\plugins\avformat-57.dll [57.66.102.0]
C:\Program Files (x86)\AviSynth+\plugins\avresample-3.dll [3.2.0.0]
C:\Program Files (x86)\AviSynth+\plugins\avutil-55.dll [55.47.100.0]
C:\Program Files (x86)\AviSynth+\plugins\swscale-4.dll [4.3.101.0]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2014-01-03]
C:\Program Files (x86)\AviSynth+\plugins\ChangeLog.txt [2019-06-05]
C:\Program Files (x86)\AviSynth+\plugins\COPYING [2016-10-18]
C:\Program Files (x86)\AviSynth+\plugins\ffms2-avisynth.md [2016-10-13]
C:\Program Files (x86)\AviSynth+\plugins\ffms2.lib [2016-12-29]
C:\Program Files (x86)\AviSynth+\plugins\ffmsindex.exe [2016-12-29]
C:\Program Files (x86)\AviSynth+\plugins\LICENSE [2019-06-05]
C:\Program Files (x86)\AviSynth+\plugins\README.md [2019-06-05]

EDIT:

Here is the script i ran that gave me that error.

ffms2 ("C:\Users\padge\Documents\test video.mp4") # This implicitly assigns clip to a special variable called Last (when not explicitly assigned to anything else)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last

[hfrforever]

i am in bad storms rn, and the power just went out. I may be offline until tomorrow. I want to save my computer battery. I will try to check in a few hours.

[Groucho2004]

You need MVTools2 (https://github.com/pinterf/mvtools/releases). While you're at it, also download some essential plugs you'll probably need sooner or later:
MaskTools2 (https://github.com/pinterf/masktools/releases)
RGTools (https://github.com/pinterf/RgTools/releases)

[Groucho2004]

VersionString: AviSynth+ 0.1 (r1576, x86)D'oh. Use the link for AVS+ that StainlessS gave you: https://github.com/pinterf/AviSynthPlus/releases

[StainlessS]

You dont need any of these in the plugins directory

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2014-01-03] # Descriptive only
C:\Program Files (x86)\AviSynth+\plugins\ChangeLog.txt [2019-06-05] # Changelog for ffms2
C:\Program Files (x86)\AviSynth+\plugins\COPYING [2016-10-18] # stuff for ffms2
C:\Program Files (x86)\AviSynth+\plugins\ffms2-avisynth.md [2016-10-13] # documentation for ffms2
C:\Program Files (x86)\AviSynth+\plugins\ffms2.lib [2016-12-29] # Some compiler stuff for ffms2 (developer thingy)
C:\Program Files (x86)\AviSynth+\plugins\ffmsindex.exe [2016-12-29] # for indexing video file outside of avisynth (command line)
C:\Program Files (x86)\AviSynth+\plugins\LICENSE [2019-06-05] # documentation for ffms2
C:\Program Files (x86)\AviSynth+\plugins\README.md [2019-06-05] # documentation for ffms2


Dont just dump everything you find into the plugins directory.

When you have ffms2 and LSmash working then try the previously given multiple source loader (given in post #198)

For anyone else wanting to try the multi-source loader script function, here is a 7zip compressed file (~31MB: AVI_LSMASH_FFMS2_SOURCE.7z):-
http://www.mediafire.com/file/2kuq0flzz145ct8/AVI_LSMASH_FFMS2_SOURCE.7z.7z/file
Above 7zip file using Ultra Compression, so may need latest version 7Zip if yours is old (saved an extra 10MB).

Contains both x86 and x64
FFMS2 (XP compatible [C Filter NOT CPP, but autoloads OK in AVS+ without LoadCPlugin call])
LSmash(NOT XP Compatible, CPP filter)
Multi-Source Script function [ GetSeq() ], tries AVI, both LSmash filters and FFMS. (and dubs audio to video)
Also contains RT_Stats latest, v2.00 Beta 12(used by the multi-source loader script).

EDIT:Also, with DebugView (Google or here:- https://docs.microsoft.com/en-us/sysinternals/downloads/debugview)

Can show debugging info from the script function (GetSeq), below succeeds,

00000015 1.08073246 [1384] FFMS2 avs plugin: Initializing...
00000016 1.19320846 [1384] GetSeq: LSMASHVideoSource opened Video
00000017 1.19328237 [1384] GetSeq: 'D:\TEMPLATE\heima_720p_500.mp4'
00000018 1.27220118 [1384] GetSeq: LSMASHAudioSource opened Audio
00000019 1.27224672 [1384] GetSeq: 'D:\TEMPLATE\heima_720p_500.mp4'


EDIT: FFMS2 can fail on some MP4(ISO) files, LSmashVideoSource() is specifically for ISO files.

Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}

[hfrforever]

Alright, I have ____ min of internet left before my battery backup dies, but even after installing that plugin, It still gives me the same error as above https://drive.google.com/open?id=1ueaNHLyIQQfeDlmks-stbO6BYzbyIRk-

edit, here is the script i am using. If i have time, i will try the other one as well.

LSmashVideoSource ("C:\Users\padge\Documents\test video.mp4") # This implicitly assigns clip to a special variable called Last (when not explicitly assigned to anything else)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last

[hfrforever]

Here is the error I get with your script. https://drive.google.com/open?id=11Q_S8SyggCGYjQgGUak-z7dfuvt1JCoG

[Groucho2004]

Alright, I have ____ min of internet left before my battery backup dies, but even after installing that plugin, It still gives me the same error as above https://drive.google.com/open?id=1ueaNHLyIQQfeDlmks-stbO6BYzbyIRk-Run the info tool again and post the log.

[hfrforever]

here you are.

[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz / Haswell (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, FMA3, AVX, AVX2
4 physical cores / 8 logical cores


[Avisynth info]
VersionString: AviSynth+ 0.1 (r2772, MT, i386)
VersionNumber: 2.60
File / Product version: 0.1.0.0 / 0.1.0.0
Interface Version: 5
Multi-threading support: Yes
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2018-12-20, 21:06:26 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins+


[CPP 2.5 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\LSMASHSource.dll [2017-02-25]

[CPP 2.6 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins+\ConvertStacked.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\DirectShowSource.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\ImageSeq.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\Shibatch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\TimeStretch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\VDubFilter.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins\DirectShowSource.dll [2.6.0.3]
C:\Program Files (x86)\AviSynth+\plugins\ffms2.dll [2016-12-29]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.dll [2019-06-05]
C:\Program Files (x86)\AviSynth+\plugins\TCPDeliver.dll [2.6.0.7]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.avsi [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins\FFMS2.avsi [2015-05-22]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.avsi [2019-06-05]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\avcodec-57.dll [57.81.100.0]
C:\Program Files (x86)\AviSynth+\plugins\avformat-57.dll [57.66.102.0]
C:\Program Files (x86)\AviSynth+\plugins\avresample-3.dll [3.2.0.0]
C:\Program Files (x86)\AviSynth+\plugins\avutil-55.dll [55.47.100.0]
C:\Program Files (x86)\AviSynth+\plugins\swscale-4.dll [4.3.101.0]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins\ffmsindex.exe [2016-12-29]

[Groucho2004]

So, where did you put the mvtools2.dll?

[StainlessS]

(You can post images on PostImage.org without an account, then copy Thumbnail for forum or HotLink for forum, paste in post).
https://i.postimg.cc/bSqxbXqb/error2.png (https://postimg.cc/bSqxbXqb)

RT_String is one of the hundred or so functions in RT_Stats

#ASYNTHER Simple_AVI_LSmash_FFMS_Reader ### EDIT: This line is used only in Avisynthesizer: where template name is Simple_AVI_LSmash_FFMS_Reader.avst

# Requires RT_Stats


###########################
Function GetSeq(String vFn,String "aFn") {
Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}
Function IsRiffFileName(String s) {s=RT_GetFileExtension(s) Return(s==".avi"||s==".wav")}
myName="GetSeq: " aFn = Default(aFn,"") aFn = (aFn=="") ? vFn : aFn
Assert(vFn!="",RT_String("%sVFn Cannot be ''",myName)) vFn = vFn.RT_GetFullPathName IsRiffV = vFn.IsRiffFileName IsIsoV = vFn.IsISOFileName
Assert(aFn!="",RT_String("%saFn Cannot be ''",myName)) aFn = aFn.RT_GetFullPathName IsRiffA = aFn.IsRiffFileName IsIsoA = aFn.IsISOFileName
c=0 a=0
Try { # 1st Try specialized AVI
c = (IsRiffV) ? vFn.AviSource : NOP
(c.IsClip) ? RT_DebugF("AviSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
a = (c.IsClip && c.HasAudio && aFn==vFn) ? c : (IsRiffA) ? aFn.WavSource : NOP
(a.IsClip && a.HasAudio) ? RT_DebugF("%s opened Audio\n '%s'",(c.IsClip && c.HasAudio && aFn==vFn)?"AviSource":"WavSource",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try specialized ISO LSMash
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (IsIsoV) ? LSMASHVideoSource(vFn) : c
(IsIsoV) ? RT_DebugF("LSMASHVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (IsIsoA) ? LSMASHAudioSource(aFn) : a
(IsIsoA) ? RT_DebugF("LSMASHAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try FFMS2
Try {
IsOpenV=(c.IsClip && c.HasVideo)
(!IsOpenV) ? FFIndex(vFn) : NOP
c=(!IsOpenV) ? FFVideoSource(vFn) : c
(!IsOpenV) ? RT_DebugF("FFVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a=(!IsOpenA) ? FFAudioSource(aFn) : a
(!IsOpenA) ? RT_DebugF("FFAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try LSmash non ISO Video
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (!IsOpenV && !IsIsoV) ? LWLibavVideoSource(vFn) : c
(!IsOpenV && !IsIsoV) ? RT_DebugF("LWLibavVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (!IsOpenA && !IsIsoA) ? LWLibavAudioSource(aFn) : a
(!IsOpenA && !IsIsoA) ? RT_DebugF("LWLibavAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

Assert(c.IsClip, RT_String("%s failed open on\n '%s'",myName,vFn))
(!c.HasAudio && a.IsClip && a.HasAudio) ? AudioDubEx(c,a) : c
(!HasAudio) ? RT_DebugF("Audio failed Open on\n '%s",aFn,name=myName) : NOP
return Last
}

#[GetSeq("___FILE___")] ### COMMENTED OUT, NOT FOR USE in Avisynthesizer


GetSeq("C:\Users\padge\Documents\test video.mp4")
Return last

The above
requires RT_Stats but should make things easier for you. [also needs FFMS and LSMash, see Dev Forum]

RT_Stats here:- http://forum.doom9.org/showthread.ph...light=RT_Stats

ABove script will use either AVISource, or the other mentioned source filters, but not MPeg2Source().



And for posted package including dll's and script
Contains both x86 and x64
FFMS2 (XP compatible [C Filter NOT CPP, but autoloads OK in AVS+ without LoadCPlugin call])
LSmash(NOT XP Compatible, CPP filter)
Multi-Source Script function [ GetSeq() ], tries AVI, both LSmash filters and FFMS. (and dubs audio to video)
Also contains RT_Stats latest, v2.00 Beta 12(used by the multi-source loader script).


Here's the link (Direct) again, RT_Stats v2.00Beta12:- http://www.mediafire.com/file/xv4foo7gdzxhic7/RT_Stats_25%2626_x86_x64_dll_v2.00Beta12_20181125.zip

EDIT: Dont despair just yet, it will get easier, bit of a steep hill at the moment (but should get a bit less tricky when the multi-source script thingy set up ok).

EDIT: And get mvtools2.dll into your plugs directory like Groucho said.

[Groucho2004]

...but even after installing that plugin, It still gives me the same error as above https://drive.google.com/open?id=1ueaNHLyIQQfeDlmks-stbO6BYzbyIRk-Instead of posting images you can run the script with AVSMeter (https://forum.doom9.org/showthread.php?t=174797) and grab the error message(s) in plain text format.

[hfrforever]

Okay, so the script loads correctly into megui, but when I hit encode, It gives me the same error in the debug window. (I hope the images work...)
https://i.postimg.cc/Qx1QMTBb/error4.png (https://postimages.org/)intelligent deathclaws (https://falloutfacts.com/goris-the-intelligent-deathclaw)

https://i.postimg.cc/TPcPpRGh/error3.png (https://postimages.org/)gas station nearby with diesel (https://gasstation-nearme.com/diesel)

EDIT: I am currently using notepad for a text editor, and I believe I saw someone suggest that I should have something else?

Edit: I meant to say hit queue

[StainlessS]

So have you put RT_stats (x86 version) dll into your plugins directory ? [that would be Avisynth plugins directory, not MeGUI version of avisynth directory]
Have you installed mvtools2.dll there, also ?

Again post result of Groucho avsmeter avsinfo or from the avs info tool (so we [and you] can see that both do indeed exist where they should).

EDIT: RT_Stats version 2.00 Beta 12 requires VS 2008 runtimes, older version (v1.43) no runtime requirements. (Did you install Groucho linked All-In-One Runtimes)

Also you want the avs 2.60 x86 version, not the v2.58 or 2.60 x64 versions.

EDIT guessin' that you installed x64 version which is for x64 version avs, irrespective of bittage of your machine, you want v2.60 x86 for 32 bit avs.
32 bit avs requires all x86 dll's, avs x64 req all x64 dll's (and runtimes), same as any other windows executable.

[hfrforever]

[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz / Haswell (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, FMA3, AVX, AVX2
4 physical cores / 8 logical cores


[Avisynth info]
VersionString: AviSynth+ 0.1 (r2772, MT, i386)
VersionNumber: 2.60
File / Product version: 0.1.0.0 / 0.1.0.0
Interface Version: 5
Multi-threading support: Yes
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2018-12-20, 21:06:26 (UTC)
PluginDir2_5 (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Program Files (x86)\AviSynth+\plugins+


[CPP 2.5 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\LSMASHSource.dll [2017-02-25]

[CPP 2.6 Plugins (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins+\ConvertStacked.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\DePan.dll [2.13.1.4]
C:\Program Files (x86)\AviSynth+\plugins+\DePanEstimate.dll [2.10.0.3]
C:\Program Files (x86)\AviSynth+\plugins+\DirectShowSource.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\ImageSeq.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\mvtools2.dll [2.7.41.0]
C:\Program Files (x86)\AviSynth+\plugins+\Shibatch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\TimeStretch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins+\VDubFilter.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins\DirectShowSource.dll [2.6.0.3]
C:\Program Files (x86)\AviSynth+\plugins\ffms2.dll [2016-12-29]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.dll [2019-06-05]
C:\Program Files (x86)\AviSynth+\plugins\RT_Stats_x86.dll [2.0.12.0]
C:\Program Files (x86)\AviSynth+\plugins\TCPDeliver.dll [2.6.0.7]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.avsi [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins\FFMS2.avsi [2015-05-22]
C:\Program Files (x86)\AviSynth+\plugins\FrameRateConverter.avsi [2019-06-05]

[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\avcodec-57.dll [57.81.100.0]
C:\Program Files (x86)\AviSynth+\plugins\avformat-57.dll [57.66.102.0]
C:\Program Files (x86)\AviSynth+\plugins\avresample-3.dll [3.2.0.0]
C:\Program Files (x86)\AviSynth+\plugins\avutil-55.dll [55.47.100.0]
C:\Program Files (x86)\AviSynth+\plugins\swscale-4.dll [4.3.101.0]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins\ffmsindex.exe [2016-12-29]

[hfrforever]

should I restart my computer when I add a plugin?

[ChaosKing]

should I restart my computer when I add a plugin?

no need to

[StainlessS]

Load your script into VirtualDub2 or any player (except VLC, and ideally not MPC-HC), what happens ? [right click, open with]

RT_Stats dll is clearly in your plugs, so should work, no alarm raised in avsmeter about missing runtimes.

EDIT: Also, get rid of this lot. [out of plugins] (unless they are needed for somethiing, anybody know ???, I suppose get it working then remove and put back dll's if then problems)
EDIT: Think the uncat dll's are from LSMash Works AvsUtil stuff, which you dont need, remove from plugs, but keep handy just in case of probs.


[Uncategorized DLLs (32 Bit)]
C:\Program Files (x86)\AviSynth+\plugins\avcodec-57.dll [57.81.100.0]
C:\Program Files (x86)\AviSynth+\plugins\avformat-57.dll [57.66.102.0]
C:\Program Files (x86)\AviSynth+\plugins\avresample-3.dll [3.2.0.0]
C:\Program Files (x86)\AviSynth+\plugins\avutil-55.dll [55.47.100.0]
C:\Program Files (x86)\AviSynth+\plugins\swscale-4.dll [4.3.101.0]

[Uncategorized files]
C:\Program Files (x86)\AviSynth+\plugins+\colors_rgb.txt [2016-07-05]
C:\Program Files (x86)\AviSynth+\plugins\ffmsindex.exe [2016-12-29]


I dont see anything else wrong in setup or plugs, perhaps the eagle eyed G2K4 will find something.

[hfrforever]

i loaded it into windows media player, and it played the video. It did not give me any kind of error. I will remove the unwanted plugs.

[hfrforever]

Okay, so i removed the uncat dlls, and the avs info tool said that it needed them for dependencies, so i put them back in.

[StainlessS]

here for verification

Colorbars.KillAudio.RT_Stats()

https://i.postimg.cc/VryG3zCn/ver-00.jpg (https://postimg.cc/VryG3zCn)

EDIT: Just curious, did it say what you need them for ? [Did it say what you need, maybe it was saying need VS 2008 runtimes for RT_Stats, try the RT_Stats verification thing above.]

[hfrforever]

I ran it, and It gave me that picture with no errors in the log.

[hfrforever]

Is my rt_stats in the right plug folder? There are 4 plug folders, and i am not sure what each one means.

[StainlessS]

Right, load your megui script into some player or Vdub2, and check it works OK (good idea to ALWAYS DO that before attempting MeGUI).

[StainlessS]

C:\Program Files (x86)\AviSynth+\plugins
C:\Program Files (x86)\AviSynth+\plugins+


Those are your plugins folders, the ones in MeGUI directories are NOt USED when you have separate AVS installed. [if that is the other 2 you talk of]

EDIT: Maybe you have used a dual setup instller which installs both x86 and x64 versions of avsynth, if so, then try
eg AVSMeter64 avsinfo

will show you directies and stuff for your x64 version of avs, which requires ALL 64 bit dll's and runtimes.

[hfrforever]

i was talking about those 2, and the x64 ones.

[hfrforever]

there are some x64 plugins in there...


[OS/Hardware info]
Operating system: Windows 10 (x64) (Build 18362)

CPU: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz / Haswell (Core i7)
MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, FMA3, AVX, AVX2
4 physical cores / 8 logical cores


[Avisynth info]
VersionString: AviSynth+ 0.1 (r2772, MT, x86_64)
VersionNumber: 2.60
File / Product version: 0.1.0.0 / 0.1.0.0
Interface Version: 5
Multi-threading support: Yes
Avisynth.dll location: C:\WINDOWS\SYSTEM32\avisynth.dll
Avisynth.dll time stamp: 2018-12-20, 20:55:16 (UTC)
PluginDir2_5 (HKLM, x64): C:\Program Files (x86)\AviSynth+\plugins64
PluginDir+ (HKLM, x64): C:\Program Files (x86)\AviSynth+\plugins64+


[CPP 2.6 Plugins (64 Bit)]
C:\Program Files (x86)\AviSynth+\plugins64+\ConvertStacked.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64+\DePan.dll [2.13.1.4]
C:\Program Files (x86)\AviSynth+\plugins64+\DePanEstimate.dll [2.10.0.3]
C:\Program Files (x86)\AviSynth+\plugins64+\DirectShowSource.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64+\ImageSeq.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64+\mvtools2.dll [2.7.41.0]
C:\Program Files (x86)\AviSynth+\plugins64+\Shibatch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64+\TimeStretch.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64+\VDubFilter.dll [2018-12-20]
C:\Program Files (x86)\AviSynth+\plugins64\DePan.dll [2.13.1.4]
C:\Program Files (x86)\AviSynth+\plugins64\DePanEstimate.dll [2.10.0.3]
C:\Program Files (x86)\AviSynth+\plugins64\mvtools2.dll [2.7.41.0]

[Scripts (AVSI)]
C:\Program Files (x86)\AviSynth+\plugins64+\colors_rgb.avsi [2016-07-05]

[hfrforever]

I tried your script in windows media player, and it played normally, but without the interpolation.

[StainlessS]

See edit in my prev post.
Those are for your 64 bit version of avs, you must have used a dual setup installer.
The 64 bit avs requies ALL 64 bit dll's and runtimes.
Can check with eg AvsMeter64 avsinfo

I tried your script in windows media player, and it played normally, but without the interpolation.

Yep, that just checked that RT_Stats was no longer a problem.
By the Way, 32 bit MeGUI requires 32 bit AVS, 64 bit MeGUI requires 64 bit AVS (and all associated dll's etc).

The GetSeq thing just replaces you source filter eg AviSourcE(Filename) or ffms(Filename), just substitute remainder of your script after tha Getseq call (using your filename in GetSeq)

[hfrforever]

ahhh, ok so do i need to disable the 64 bit or uninstall?

[StainlessS]

Just leave it alone. You may later want to upgrade to x64 or use sometimes, and use 32 bit other/most times.

Again, see above edit.


EDIT:
You may later want to upgrade to x64
But be warned, going x64 can be tricky :)

[StainlessS]

So in full something like this


Function GetSeq(String vFn,String "aFn") {
Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}
Function IsRiffFileName(String s) {s=RT_GetFileExtension(s) Return(s==".avi"||s==".wav")}
myName="GetSeq: " aFn = Default(aFn,"") aFn = (aFn=="") ? vFn : aFn
Assert(vFn!="",RT_String("%sVFn Cannot be ''",myName)) vFn = vFn.RT_GetFullPathName IsRiffV = vFn.IsRiffFileName IsIsoV = vFn.IsISOFileName
Assert(aFn!="",RT_String("%saFn Cannot be ''",myName)) aFn = aFn.RT_GetFullPathName IsRiffA = aFn.IsRiffFileName IsIsoA = aFn.IsISOFileName
c=0 a=0
Try { # 1st Try specialized AVI
c = (IsRiffV) ? vFn.AviSource : NOP
(c.IsClip) ? RT_DebugF("AviSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
a = (c.IsClip && c.HasAudio && aFn==vFn) ? c : (IsRiffA) ? aFn.WavSource : NOP
(a.IsClip && a.HasAudio) ? RT_DebugF("%s opened Audio\n '%s'",(c.IsClip && c.HasAudio && aFn==vFn)?"AviSource":"WavSource",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try specialized ISO LSMash
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (IsIsoV) ? LSMASHVideoSource(vFn) : c
(IsIsoV) ? RT_DebugF("LSMASHVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (IsIsoA) ? LSMASHAudioSource(aFn) : a
(IsIsoA) ? RT_DebugF("LSMASHAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try FFMS2
Try {
IsOpenV=(c.IsClip && c.HasVideo)
(!IsOpenV) ? FFIndex(vFn) : NOP
c=(!IsOpenV) ? FFVideoSource(vFn) : c
(!IsOpenV) ? RT_DebugF("FFVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a=(!IsOpenA) ? FFAudioSource(aFn) : a
(!IsOpenA) ? RT_DebugF("FFAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try LSmash non ISO Video
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (!IsOpenV && !IsIsoV) ? LWLibavVideoSource(vFn) : c
(!IsOpenV && !IsIsoV) ? RT_DebugF("LWLibavVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (!IsOpenA && !IsIsoA) ? LWLibavAudioSource(aFn) : a
(!IsOpenA && !IsIsoA) ? RT_DebugF("LWLibavAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

Assert(c.IsClip, RT_String("%s failed open on\n '%s'",myName,vFn))
(!c.HasAudio && a.IsClip && a.HasAudio) ? AudioDubEx(c,a) : c
(!HasAudio) ? RT_DebugF("Audio failed Open on\n '%s",aFn,name=myName) : NOP
return Last
}

FN="C:\Users\padge\Documents\test video.mp4"

GetSeq(FN)

FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last


OR can save first part of above script as eg "GetSeq.avsi" and put in Plugins, it will autoload.
(the part before FN="... etc")

And use eg just this


FN="C:\Users\padge\Documents\test video.mp4"

GetSeq(FN)
FrameRateConverter() # defaults FrameRate * 2, both takes and produces clip Last
return Last

[hfrforever]

I tried this code and it said i have a syntax error on line 60 column 7

EDIT: in case you were wondering what the error was...

there was an extra quotes mark.

FN=""C:\Users\padge\Documents\test video.mp4"

[hfrforever]

Okay so i found the error on line 60, but now i have this error.

https://i.postimg.cc/g2BDnpMG/error6.png (https://postimages.org/)

[StainlessS]

Yep, sorry I already fixed it. Is doubled Double Quote at beginning of FN, remove one.

FN=""C:\Users\padge\Documents\test video.mp4"

EDIT: Put these two dll's in your SysWOW64 folder [req by FFT3DFilter and DftTest, and MvTools when arg DCT!=0]]

http://www.mediafire.com/file/hv1y096h58idkuz/FFT.7z/file

(Groucho likes to put them somewhere else, but I prefer above [at least for now]).

[hfrforever]

Is it safe for me to use 64bit versions of the plugs, or should I only use the 32?

[StainlessS]

Only in 64 bit plugins, and only with x64 avs (one or other cannot use both at same time, windows thing).
See prev edit.

[hfrforever]

i did that and now get this error (sigh)

https://i.postimg.cc/CxqyNwRx/error7.png (https://postimages.org/)

[hfrforever]

is that masktools2?

[StainlessS]

Yes, thats Masktools2.

Didnt we have a wonderful day, the day we went to Bangor, A beautiful day, we had lunch on the way, and all for under a pound you know,
and on the way back, I cuddled with Jack and opened a bottle of cider, singin' a few of our favourite songs as the wheels went round ...
https://www.cosgan.de/images/smilie/musik/e050.gif https://www.cosgan.de/images/smilie/musik/a075.gif https://www.cosgan.de/images/smilie/musik/c025.gif

EDIT: Fiddler's Dram - "Day Trip to Bangor (Didn't We Have a Lovely Time ... https://www.youtube.com/watch?v=wfwQi8L9h8M

[hfrforever]

it was mask tools 2. It still says that there is no function named 'RT_String'.

EDIT: I am officially to the ripping hair out stage.

thank you very much for your infinite patience

[StainlessS]

Comment out the FrameRateConvert line (precede with '#' character)
Try in VDub (NOT x64 version, not x64 version media player, MUST use ALL 32bit or ALL 64 bit), if complaining about RT_string, its because its the very first used external
function from dll, I'm guessin' because you are using x64 AVS via x64 player, and ALL plugins are 32 bit, so ALL will fail, its just that RT_string is first in line pending fail.
EDIT: Probably Vdub2 is your favourite bet, as has both x86 and x64 versions.

If works OK, then uncomment.

[hfrforever]

i tried it as is, and it seemed to work in vdub (at a really low framerate)

[hfrforever]

it also works commented out.

[hfrforever]

how can I make it work in megui then? I seem to be using 32bit megui (is in the x86 program files directory, and what I downloaded seems to be 32 bit.

[StainlessS]

Have you tried in MeGUI since now got it working ? Should work OK unless you have installed 64 bit MeGUI, (which is not quite stable as yet).

EDIT: And yes, motion compensated processing can be slow. (A few years back I spent 3 days rendering a movie [LOTR-ROTK]).

[hfrforever]

yes, same error as before.


https://i.postimg.cc/6QrKfB8N/error8.png (https://postimages.org/)

[StainlessS]

Check [expand] first line in log, Versions.

[hfrforever]

https://i.postimg.cc/j2SrT9Rw/error9.png (https://postimages.org/)

[StainlessS]

And the other stuff, eg redistributables (ie runtimes).
https://i.postimg.cc/2yVZVgtR/Me-GUI-VERS.jpg (https://postimages.org/)

EDIT: At bottom of page looks like MeGUI bug (somebody post it in MeGUI forum), says path to dll is Windows\system32\Avisnth.dll (on x64 system, should be showing SysWOW64),
the one its showing is the 64 bit dll (in 32 bit MeGUI). [On my system both dll's present in system32 & SysWOW64]

[hfrforever]

https://i.postimg.cc/bYVvJ04c/error10.png (https://postimg.cc/qgnrDKkj)

[StainlessS]

You seem to be missing the RT_Stats VS2008 redistributables (runtimes).
You will likely also need VS2012.
Not sure about VS2013, maybe VS2013 -> 2019 all some, cant remember.

See Here:- https://support.microsoft.com/en-gb/help/2977003/the-latest-supported-visual-c-downloads

You need 2008, 2012, 2013.

and from MS above link, looks like you best install vs2019.
Note Visual C++ 2015, 2017 and 2019 all share the same redistributable files.

For example, installing the Visual C++ 2019 redistributable will affect programs built with Visual C++ 2015 and 2017 also. However, installing the Visual C++ 2015 redistributable will not replace the newer versions of the files installed by the Visual C++ 2017 and 2019 redistributables.

This is different from all previous Visual C++ versions, as they each had their own distinct runtime files, not shared with other versions.


ABOVE, BOTH x86 and x64 REQUIRED [you will need all, sooner or later]

[hfrforever]

Where would I get them from?

[StainlessS]

The given link above.

[Groucho2004]

Where would I get them from?Didn't you already install the All-In-One MS runtime thingy? Also, if runtimes were missing, the Info Tool would complain.

Edit - Maybe you're using the "other" Avisynth in megui?

[StainlessS]

In my MeGUI image, shows VS2005 and VS2015, missing, but they are present in Programs And Features thing.
Maybe should check in Programs And Features, perhaps an additional bug in MeGUI [I had I think noticed that before, but ignored it].

EDIT: Or maybe MeGUI only shows runtimes that it relies upon(or its tools), dont know.

EDIT: I think hfrforever version MeGUI is current Stable (2896).

[StainlessS]

From Your Image

Avisynth Status: Ignored, as Portable build is FORCED.

We Got it. https://www.cosgan.de/images/smilie/froehlich/c010.gif

No idea how to unforce it, probably in Settings.

EDIT: Easy Peasy, first setting "Always Use Included Avisynth", Untick.

See just above Avisynth Portable, in the last image you posted (and also Avisynth Portable entries).

EDIT: I think that MeGUI now clutters up its directories with loads of dll's and stuff, real messy, perhaps its just showing the VS runtimes included within MeGUI Directories.

[hfrforever]

maybe, how would i know if I am??

[Groucho2004]

I am officially to the ripping hair out stage.Avisynth is quite complex and you (please don't take offense) are a complete noob and have not read any documentation. The spoon-feeding we're doing can only get you so far and most importantly, you're not learning anything, you just follow commands you don't understand (yet).

So, you may have a look at the tutorials on avisynth.nl:
http://avisynth.nl/index.php/Main_Page
("New to AviSynth – Start Here")

[hfrforever]

your probably right. I will come back if i cant figure it out in a few days.

[Groucho2004]

your probably right. I will come back if i cant figure it out in a few days.Well, we've been at this for almost 100 posts so let's at least get you going on this particular venture. :D

Did you find the megui switch off portable thing?

[StainlessS]

To be fair, he has encountered quite a few hurdles, and the Forced MEGUI thing would have tripped me up a bit, was not even aware of that setting.

Anyways, there is now it seems light at the end of the drain pipe, nothing but fun from here on in. https://www.cosgan.de/images/smilie/froehlich/a065.gif

EDIT:
Maybe a MODERATOR could separate off from post #192:- https://forum.doom9.org/showthread.php?p=1876263#post1876263
into a new thread with hfrforever as OP, and let MysteryX have his thread back.

Suggested title pending rename by hfrforever, "A Painless Intro to Avisynth.".

[hfrforever]

it works. I am soooo dumb. i was hitting the wrong *** button. this whole time...... argh. argh.... there are two queue buttons (now i know)

https://i.postimg.cc/K8MbFjKt/success2.png (https://postimages.org/)

thank you both very much for your help and patience working with me.

[StainlessS]

I personally use (almost) nothing but AutoEncode. [EDIT: And most use CRF (Const Qual) rather than two pass for x264, I usually use CRF 21.5, many use lower (higher qual) setting]

EDIT: Also, not sure but think some version of AAC would perform better with x264 MP4, than mp3. [also has more bang for you're buck]

EDIT: I have no idea which AAC flavour is best bet, I personally use Nero AAC, which I think is no longer available for download,
and most others use something else, I just stick with what I've always used. [AAC, half file size for same quality : compared to mp3]

[ChaosKing]

EDIT: I have no idea which AAC flavour is best bet, I personally use Nero AAC, which I think is no longer available for download,
and most others use something else, I just stick with what I've always used. [AAC, half file size for same quality : compared to mp3]

The best is currently the AAC encoder from Apple. Install /extract Itunes and download (or via megui) qaac wrapper, done.

[Dreamject]

Opus betteryoutube uses it

[videophile]

Hi, I am new to this forum and this is my first post. English is not my mother tongue so I apologize in advance for any mistake.

I have developed a 8 mm / Super 8 transfer bench, which is working quite well. I do some post-processing after capture, which includes denoising, color correction, image stabilization and frame rate interpolation from 18 to 36 fps.

I use an Avisynth script and I am struggling with frame rate interpolation.

I have first used this script, which, I think, comes from John Meyer:
super= processed.MSuper()
backward_vec= MAnalyse(super, blksize=block_size, blksizev= block_size_v, overlap=block_overlap, isb=true, truemotion=false, search=3)
forward_vec= MAnalyse(super,blksize=block_size, blksizev= block_size_v, overlap=block_overlap, isb= false, truemotion=false, search=3)
interpolated= processed.MFlowFps(super, backward_vec, forward_vec, num=36, den= 1, ml=200)

In this script, "processed" in the input clip, and "interpolated" is the output clip.

This script works quite well, except on fast motion where it produces artifacts.

Then I tried FrameRateConverter.

Here is the FRC script I used:
interpolated = processed.FrameRateConverter(Preset="slow", FrameDouble=true)

On very complex scenes with a lot of motion, FRC reverts back to frame blending, and the visual result is more pleasant than the JM script. However, I feel that the algorithm used by FRC to detect problematic areas and fill them with blended frames is way overzealous. As a result, FRC is sometimes better, but too often worst, which is not what I would expect.

Here is an example clip (FRC on the left, JM on the right): https://cp.sync.com/dl/7fa3fbd80/ncpe3rku-nv7j5qrq-vwccx56c-8apevkms

In this screenshot from the clip, we see that FRC has blended the dress, whereas JM has done a good job in spite of the repetitive pattern (which is always difficult for motion detection):
https://i.imgur.com/fFVMywJ.jpg

Here, FRC has blended the whole frame: we see the ghost hair of the lady (red circle), whereas JM has done a good interpolation, except on the eagle (blue circle):
https://i.imgur.com/fDaTgFR.jpg

This example shows how FRC often fails: although there was a moderate motion of the eagle's head, FRC has decided to blend (see the eye and the surrounding area on the left image), whereas JM has interpolated it properly:
https://i.imgur.com/ia94Q2b.jpg

This last example shows another situation where FRC has unnecessarily blended the whole frame:
https://i.imgur.com/v2qLz67.jpg

I have tried to play with the FRC parameters, but no luck. The parameters are quite numerous and the documentation is not very detailed, so I am not sure what to do.

I feel that FRC is not doing what it is expected to do.

Any advice?

[johnmeyer]

I wanted FRC to work. I was rooting for it to work. It is a much-needed piece of technology. The guy who wrote it put in a ton of time and effort, and obviously built something that can and does work, in certain circumstances.

However, I had the same experience as you, and finally gave up. I think it still needs a lot more work on both the masking and detection, the two key technologies that underlie its basic concept.

[videophile]

Are you THE John Meyer ?

If yes, then thank you for your great script which I am using since 2013! Is there somewhere an official/updated version?

Regarding FrameRateConverter, that's bad news :(

I think I will stick to my current technique: I create two clips at 36 fps, one with your method, the other one with frame blending, and then I manually select the one that works best on each scene, using my NLE Software (Edius).

[johnmeyer]

Are you THE John Meyer?I am one of thousands, but yes, I am the guy who spent half a day trying hundreds of combinations of MVTools2 settings, trying to understand what each one did, and figure out what impact they had on the typical motion estimation artifacts that all motion estimation tools seem to create. These problems are present even in the expensive commercial programs, such as Twixtor, After Effects, Motionperfect, etc.

I managed to find some sort of "sweet spot" that has held up pretty well for many people. The most important setting was block size. What I have found is that for frame rate conversion you want to use big block size, which makes perfect sense, since this keeps small details from producing "morphs." By contrast, when doing motion-estimated de-noising, you almost always want to use smaller block sizes.

The mask idea is a good one, and when I am doing really critical work, I'll create both a motion estimated version as well as a frame-blended version, but then manually go through each frame and either cut between them for that one frame, or create a manual mask, which is the equivalent of what this tool tries to do. I only do this for paid jobs, of course, and it is very, very tedious.

[kolak]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.

[johnmeyer]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.Thanks for listing those two. I need to look at the results they can produce. I'm always looking for better tools, even if they cost money.

[videophile]

The mask idea is a good one, and when I am doing really critical work, I'll create both a motion estimated version as well as a frame-blended version, but then manually go through each frame and either cut between them for that one frame, or create a manual mask, which is the equivalent of what this tool tries to do. I only do this for paid jobs, of course, and it is very, very tedious.

Yes, this is exactly what I am doing currently. I wished I could rely on a more automated - i.e. less time consuming - method.

[doxel]

I've found two john meyer scripts on this forum with only two lines that are different and I'm wondering which version people are using:

super = MSuper(source, hpad = 16, vpad = 16, levels = 1)
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1, sharp = 1, rfilter = 4)
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16, sharp = 1, rfilter = 4)

What does 'sharp' and 'rfilter' do? Should I use them?

[manolito]

Have a look at this post:
https://forum.doom9.org/showthread.php?p=1841952#post1841952

IMO these two params do make sense, but you need to test it for yourself...

[kolak]

Thanks for listing those two. I need to look at the results they can produce. I'm always looking for better tools, even if they cost money.


They cost serious money. Xfile is 10k$. Tachyon similar.

[poisondeathray]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.



You mentioned this a year or two ago . Is that marketing speak or actual testing ?

I don't know about Tachyon, but for Alchemist Ph.C / Ph.C high effort, it's terribly over rated in terms of the motion compensation conversions

I got a chance to test Alchemist at a trade show, then later in more detail where I know somebody at facility that has it . The artifacts were quite similar as to other solutions, and it would fail in exactly the same conditions and situations. I ran it though typical clips where most ME approaches fail in automatic mode, not some cherry picked marketing demo clips. It was slightly better in terms of the artifacts were maybe slightly smaller (there weren't massive fails, just slightly smaller edge morphing fails)

I think the product got taken over by another company ( Grass Valley now? ), so maybe it improved 100x . But when it was still Snell, it was similar to other solutions in pure automatic mode.

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.

[johnmeyer]

I don't know about Tachyon, but for Alchemist Ph.C / Ph.C high effort, it's terribly over rated in terms of the motion compensation conversions

I got a chance to test Alchemist at a trade show, then later in more detail where I know somebody at facility that has it . The artifacts were quite similar as to other solutions, and it would fail in exactly the same conditions and situations. I ran it though typical clips where most ME approaches fail in automatic mode, not some cherry picked marketing demo clips. It was slightly better in terms of the artifacts were maybe slightly smaller (there weren't massive fails, just slightly smaller edge morphing fails)

I think the product got taken over by another company ( Grass Valley now? ), so maybe it improved 100x . But when it was still Snell, it was similar to other solutions in pure automatic mode.

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.That's really interesting information and will save me some time trying to do my own tests. I too have found that the artifact generation seems to be an issue with the concept itself, and not with the specific implementation: if I take video of a picket fence while driving by (one of the best torture tests), really bad things are going to happen when I use any of these products.

The problems are also highly dependent on the original frame rate: if you apply ME to 10 fps material and try to take it to 60 fps, all heck will break loose. But, if you want to take 120 fps to 720 fps, it will look perfect, no matter what program you use.

I have always thought that somewhere in that last sentence lies the solution to the problem of getting better results, perhaps by doing multiple estimations sequentially, rather than all at once.

As (admittedly flimsy) support for that idea, when doing broadband audio noise reduction (e.g., single-ended hiss removal), you get much better results by reducing the noise only a little bit, and then doing a second (and third, fourth, fifth, etc.) pass, each one doing just a little reduction. You get far fewer artifacts this way.

However, I don't know if it is possible to only do a "little ME" in each pass.

I'm just trying to come up with ideas ...

[poisondeathray]

The problems are also highly dependent on the original frame rate: if you apply ME to 10 fps material and try to take it to 60 fps, all heck will break loose. But, if you want to take 120 fps to 720 fps, it will look perfect, no matter what program you use.

I have always thought that somewhere in that last sentence lies the solution to the problem of getting better results, perhaps by doing multiple estimations sequentially, rather than all at once.

As (admittedly flimsy) support for that idea, when doing broadband audio noise reduction (e.g., single-ended hiss removal), you get much better results by reducing the noise only a little bit, and then doing a second (and third, fourth, fifth, etc.) pass, each one doing just a little reduction. You get far fewer artifacts this way.

However, I don't know if it is possible to only do a "little ME" in each pass.

I'm just trying to come up with ideas ...



I don't think that's applicable here, but go ahead and test it out

The reason why higher FPS sources work better - in general - is you are starting with more frequent true motion samples. So in general, the distance that a given object moves between frame A and frame B is smaller for the higher FPS source (of course you have exceptions like whip pans, explosions etc..., but in general terms) . The larger the motion (and distance travelled by objects), the more difficult to interpolate.

Not only that, but complex motion paths, and things in nature are not linear. Someone walking in straight line does not mean their muscles bend and flex linearly. Where that FPS snapshot frame take of where an arm bend might miss the peak of the bend. You miss the trajectory and motion curves

And that only addresses some of the problems with ME. The other huge problem is occlusions and object boundaries . Objects passing over another. Objects deforming (yet the same object). This involves user intervention in the higher end solutions . If that could be done accurately, automatically, you have a winner. Some of the github projects look amazingly accurate for object/mask generation , the ones for superresolution too

[Matt Kirby]

Hi guys,

I have a problem with "framerateconverter" and "anime".
I have ugly blends or ghosts in my frames when I use it to slowmotion my anime video. My original video has 25 fps. I pump it up with interpolated frames to 37.5 fps. Then I slow down it to 25 fps. The video is factor 1.5 longer than the original. (I need it in this speed, don't ask why :D)

My script:
LoadPlugin("...\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("...\Mach a S Test.mkv")
FrameRateConverter(NewNum=37500, NewDen=1000, Preset="Anime")
AssumeFPS(25)


The ugly result is : Look at "Attachment"

Here is the original testfile:

https://www.dropbox.com/sh/pl29vs4t20wanco/AAD3FPYO_TwyoR0kdinij6Q8a?dl=0


Is it possible to convert the video to 37.5 fps without these effects?

[manono]

Frame interpolation often doesn't work well with animations. You might be better off with frame duplication.

ChangeFPS(37.5)
AssumeFPS(25)

[Matt Kirby]

OK, it looks much better. Thank you!

[poisondeathray]

The Nvidia demos looks great and promising, but until it's a usable product under more a few cherry picked test scenarios, it's still unproven vapourware in my mind.


Mini review - Current status of available Optical Flow AI research methods (that are semi-usable right now and don't require a PhD in programming to use)

1) Super SloMo - This one was the biggest "letdown" for me :(

Remember the "Super SloMo" Nvidia AI demo awhile back ?
https://www.youtube.com/watch?v=MjViy6kyiqs
https://people.cs.umass.edu/~hzjiang/projects/superslomo/

It's going to be released for NGX for RTX cards eventually, the author said it could be implemented in pytorch
https://developer.nvidia.com/rtx/ngx

So there is a working pytorch implementation with pretrained model using the Adobe240fps dataset
https://github.com/avinashpaliwal/Super-SloMo

The results? Mediocre. Slightly better/slightly worse in areas than mvtools2. Basically the same problems as other optical flow methods - object/edge artifacts , picket fence problems etc.. It might be an issue with this implementation, or perhaps the dataset and training aren't as good as the other unavailable set. I tried several tests, including samples cut from Nvidia's own demo video and the results weren't as good as the demo. Their demo video had clean edges, almost no warping artifacts. But the trained model used was different, and there might be pytorch implementation differences. Hopefully the official NGX release will improve with different distributed trained models

There is a tensorflow implementation of Super-Slomo using the same Adobe240fps dataset, but the provided tensorflow model has motion problems. Produced results are definitely worse than gif bike demo and the pytorch implementation
https://github.com/rmalav15/Super-SloMo

2) CyclicGen, voxelflow based
I got CyclicGen to run, but results are poor. Lots of edge morphing artifacts. Tried both small and large models
https://github.com/alex04072000/CyclicGen

3) sepconv using "lf" model seems to do ok, similar quality to super slo mo; ie. Slightly better/slightly worse in some areas than mvtools2 based optical flow.
https://github.com/sniklaus/pytorch-sepconv


Of course, most of my tests were on "problematic" samples where OF typically "fails" or has problems, so it's probably not representative of general use

[johnmeyer]

I just tuned past a re-run of M*A*S*H on MeTV and they have used optical flow to make it look like it was shot on video rather than film. I assume they had access to whatever "best of breed" is available for professional use. It too doesn't look very good.

[StainlessS]

Anybody done any tests to see if hi bitdepth performs better in mvtools than lo bitdeth.
For lo BD and eg pel =4, so max vector length about 32, & so if movement
greater then cannot be
Used.

God I hate auto correct, Mobile.

Edit: Presumably vectors can be much longer in HBD.
Edit: Of greater importance where HD clip.

[Adam65]

Hi,
How to install FrameRateConverter on Windows with MPC-HC/BE ?

[Sharc]

Hi,
How to install FrameRateConverter on Windows with MPC-HC/BE ?
It would put too much load on the CPU. Playback would just stutter.

[Treaties Of Warp]

At the "best" settings (Preset = "slowest",Dct=1,DctRe=1), it's WAAAAY better than Interframe at not producing noticeable artifacts. But it's also WAAAAAY slower. :)

It does seem to have one problem... it doesn't do well with scrolling end credits, especially white text on a black background. "Boxes" or sections of credit text seem to "slide" around a little as they scroll. Has anyone encountered that and have any settings that might fix it?

[Maxiuca]

Best motion adaptive engines are in XFile (ex Alchemist) and in Tachyon. Both GPU based. Apparently Nvidia neural based engine is also good.

That really depends on your source material. Tachyon is very fast, the quality is ok, but it does artifacts a lot.

I haven't seen what the software Alchemist can produce but results from the hardware Alchemist are just so-so.

If you have the time and the processing power, FrameRateConverter can give much better results.

BTW. You don't have to buy a license anymore, Tachyon along with other tools made by Cinnafilm are now available online and you just pay per minute of processed material: pixelstrings.com

The commercial tools are also useful when you need to downconvert the frame-rate, which the FrameRateConverter can't do directly. On the other hand, downconversion can be done indirectly with FRC and the results are really good.


I just tuned past a re-run of M*A*S*H on MeTV and they have used optical flow to make it look like it was shot on video rather than film. I assume they had access to whatever "best of breed" is available for professional use. It too doesn't look very good.

Don't assume they have any idea what they are doing or have access to the "best of breed". Most people (technicians) in Hollywood don't even know anymore what interlace or 3:2 pulldown is.
For example, if you look closely at a stock shot at 11:48 mark ("Surigacal Appliances" storefront) in Seinfeld episode 4x22 ("The Handicap Spot"), you'll see interlace, yet it's 23.976p. And it's not the only this one shot (that one I've remembered for some reason), the HD remaster is ridden with such issues and clearly shows that the company that prepared the 2006 HD remaster had no idea what they were doing. Fast-forward 14 years and nowadays most companies and their employees know even less about old formats and video in general.

[hahadoom]

Hi,
How to install FrameRateConverter on Windows with MPC-HC/BE ?

use AviSynth Filter + AviSynth+
https://github.com/CrendKing/avisynth_filter

Uncheck input formats 10bit 16bit for AviSynth Filter.

1080p avs:

#8 core CPU
global Threads=8
AvsFilterSource()
FrameRateConverter(last,BlkSize = 16,Preset ="Slow",Output ="auto",Stp =false,Dct = 0,FrameDouble=true)
AvsFilterDisconnect()
Prefetch(Threads)


amd 1700x smooth，but image error may occur for changing the playback progress.

[Selur]

Seeing that there are a few files over at https://github.com/mysteryx93/FrameRateConverter/tree/master/Src hinting at Vapoursynth I was wondering is there a way to use FrameRateConverter natively in Vapoursynth?

Cu Selur

[binba]

Thank you for developing this script, and for all the optimizations. It's a challenging field, but FRC absolutely seems to be one of the best tools out there, commercial or not.

As Maxiuca mentioned, I ran my same test clips on Tachyon via Cinnafilm Pixelstrings. ~$2/min. is IMO a fair and handy licensing model, and I could definitely afford running some tests. And FrameRateConverter was markedly better.

They cost serious money. Xfile is 10k$. Tachyon similar.
Only $10K for Xfile? My Googling said it was $125K. 🤑

[binba]

Some questions, and my apologies for reading only the Git readme and 40 out of the 300 posts in the thread (including the last of hfrforever’s adventure �� )… in case this had been answered earlier.


1. 10-bit support: No support for bit depths higher than 8bpc? What's missing for it to be possible?


2. Performance: I have a 24-core Threadripper, and use AviSynth+ with all-64bit plugins whenever possible. FRC was barely using 3% of 2 cores. I didn't expect stellar multicore/multithreaded performance, but is that the expected performance, with how optimization is at this point? On average, FRC ran x16 slower than real-time.

I was able to run multiple VirtualDub instances which was nice, I wonder if could process, say, 12 instances in parallel. :devil: But it also makes me wonder, can the .avsi be modified so at first there's a scene-detection-only pass, and then many instances are spun to motion-interpolate scenes in parallel? Then stitch them back into the final output file. This could help get around some of the limitations of MT.


3. 30p to 24p
The commercial tools are also useful when you need to downconvert the frame-rate, which the FrameRateConverter can't do directly. On the other hand, downconversion can be done indirectly with FRC and the results are really good.

Um I just ran a frame rate reduction directly... fed it a 29.97p video (via LSMASHVIdeoSource) and ran FrameRateConverter(Preset="Slow", FrameDouble=false, NewNum=24000, NewDen=1001) and got a 23.976p file. AviSynth/VDub/FRC definitely never complained and I didn't see any obvious artifacts. Was I doing it wrong?... Or skipped every 5th frame without noticing or something of that sort?

[manolito]

3. 30p to 24p

Um I just ran a frame rate reduction directly... fed it a 29.97p video (via LSMASHVIdeoSource) and ran FrameRateConverter(Preset="Slow", FrameDouble=false, NewNum=24000, NewDen=1001) and got a 23.976p file. AviSynth/VDub/FRC definitely never complained and I didn't see any obvious artifacts. Was I doing it wrong?... Or skipped every 5th frame without noticing or something of that sort?

This depends on the fps reduction ratio and on the FRC settings. The underlying fps conversion routine by johnmeyer certainly has no problems with downconversions. It is the mask creation which can cause errors.

I found 2 possible causes. The first one can happen when the blend routine calls ConvertFPSLimit outside of the conversion ratio restrictions, and the other one lies in creating the artifact correction mask. In some cases the blocksize has to be corrected to make this routine work.


Attached is a fixed FrameRateConverter.avsi which took care of downconversion errors for me...


//EDIT//
For better speed you can try the parameter "DCT=0". Your preset "Slow" uses DCT=4 which in my experience does not always improve quality. The fftw3.dll filter is the major speed block for FRC, and if you do not use it by specifying "DCT=0" your conversion speed will improve considerably.

//EDIT2//
Not impressed with the attachment approval speed...
Here is another download link:
https://files.videohelp.com/u/172211/FrameRateConverter_fixed.zip

//EDIT3//
Sorry, I mixed up some calling parameters between different FRC versions. The current version of the fixed AVSI is now tested to work with the latest FRC v. 1.3. Please redownload...

[binba]

Thanks, manolito! I'll see when I find myself back in this research, since the main release happened to work quite well for my fps reduction. (I try to avoid using the term "downconversion" since it's mostly used to mean resolution, and some times bitrate, conversions; but not frame rate.)

I do hope MysteryX takes a look at it and possibly incorporates it into the production release, or else it will get buried in post #306 of thread 174793.. Makes you wonder how much Doom9 wisdom & effort gets lost because it's not always possible to read thousands of posts..

[manolito]

...since the main release happened to work quite well for my fps reduction.

I do hope MysteryX takes a look at it and possibly incorporates it into the production release...

Try to reduce the framerate from 60 fps to 23.976... :devil:

I believe that MysteryX did not really care for fps reduction (at least he does not mention it in the header of the script). Most users were also more interested in doubling the rate or doing slow motion conversions. Anyways, MysteryX has not been active in this forum for more than 6 months, so I do not know if he will take a look anytime soon.

My fixes just use the "try ... catch" construct of the AVS script language. Probably a little clumsy, but it works.

[kolak]

Thank you for developing this script, and for all the optimizations. It's a challenging field, but FRC absolutely seems to be one of the best tools out there, commercial or not.

As Maxiuca mentioned, I ran my same test clips on Tachyon via Cinnafilm Pixelstrings. ~$2/min. is IMO a fair and handy licensing model, and I could definitely afford running some tests. And FrameRateConverter was markedly better.


Only $10K for Xfile? My Googling said it was $125K. ��

125K$ is an old price for hardware box which was about always made to order as stocking them was simply too expensive.
XFile gives better quality and needs just fairly good GPU.

[kolak]

I haven't seen what the software Alchemist can produce but results from the hardware Alchemist are just so-so.

If you have the time and the processing power, FrameRateConverter can give much better results.



Problem with old hardware box was fact that it had to work in real time. It was a compromise as they could not make it faster (to keep it been realtime), so they had to reduce quality.

It's not the case for Xfile anymore, so quality is better. It's similar to Tachyon or open source tools, but also quite fast and can work over many GPUs.

I did not use FrameRateConverter itself, but original InterFrame. I think Xfile was bit better (if we take many different source types). You also don't have to play with it as it's quite consistent and has not many settings.

[MysteryX]

Can someone test: how does RIFE compare to FrameRateConverter?
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan

[poisondeathray]

Can someone test: how does RIFE compare to FrameRateConverter?
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan



Better in some ways , worse in others

In general, RIFE is better with occlusions, object boundaries, y-axis rotational vectors than mvtools2 based tools. I use RIFE to solve problems that FRC or mvtools2 has and composite the results

There are some video comparisions and discussion in the vapoursynth sub forum, with "picket fence" backgrounds . Object boundaries are much cleaner, basically usable with RIFE

RIFE is slower, but ok and usable with a decent GPU. Less tweakable , no settings to adjust (you can use 1 version of a model at a time), and it can fail on some "simple" things. You need to combine /composite results to get the "best" of everything

I tested the non vulkan, non vpy, cuda version of RIFE which is supposedly faster if you have a compatible card. I tried the RIFE vpy version quickly , but I couldn't get the .dll to load, I'll revisit it later when I have time

DAIN is similar to RIFE in terms of results, but 20-30x slower. But there are cases where DAIN produced better results

[kedautinh12]

I think need mix ver of RIFE and FRC or mvtools2 :D :D :D

[MysteryX]

Are you saying that RIFE could replace frame blending for artifact masking?

FRC is currently using 3 clips. One fast to calculate, one much slower -- merging both based on a mask. And then frame blending. RIFE could then replace 1 or 2 of those?

First, would have to get RIFE to work in Avisynth.

[ChaosKing]

First, would have to get RIFE to work in Avisynth.

You have always the http://avisynth.nl/index.php/VapourSource option.

[poisondeathray]

Are you saying that RIFE could replace frame blending for artifact masking?

FRC is currently using 3 clips. One fast to calculate, one much slower -- merging both based on a mask. And then frame blending. RIFE could then replace 1 or 2 of those?

First, would have to get RIFE to work in Avisynth.

I'm saying in some cases it can. Very clean object edges. That's the main strength. Much better than mvtools2 as a starting point in some cases.

But in other cases - not so much, other BG parts of same scene can have problems too where mvtools2 with mrecalculate did not.

You need to mix/match composite versions with mvtools2 / other optical flow to get the best results. I've been using RIFE to fix mvtools2 (and other traditional optical flow) "fails" such as picket fence, or object rotation, where mvtools2 is basically unusable

[MysteryX]

in the case of picket fence -- it's basically RIFE vs Frame Blending. Not a tough competition to win.

On a side note; "the highs don't help as much as the lows hurt". Better quality is great; but it's the artefacts and consistency that matter even more.

[poisondeathray]

in the case of picket fence -- it's basically RIFE vs Frame Blending. Not a tough competition to win.


If you force interpolation, the foreground object boundaries are clean. That's a seriously impressive result. Mvtools2 just can't do it. zorr looked at avs optimizer and running iterations and settings, but it looks like it just can't be done with mvtools2

Rotoscoping is the most time consuming and tedious post production activity. RIFE cuts hours/days/weeks of work off



On a side note; "the highs don't help as much as the lows hurt". Better quality is great; but it's the artefacts and consistency that matter even more.

And that's why you need to combine/composite them to get ideal results

Different people have different usage cases. eg. For playback, svpflow scenarios, artifacts might be acceptable.

I'm looking for clean interpolation. Not blending. Post work if necessary

[Milardo]

I tried to use the RIFE vapoursynth filter for video playback, but it doesn't work.

[Selur]

I tried to use the RIFE vapoursynth filter for video playback, but it doesn't work.
I would be surprised if any ML based method was usable for real time playback,... (at least on any hardware I know)

[zorr]

If you force interpolation, the foreground object boundaries are clean. That's a seriously impressive result. Mvtools2 just can't do it. zorr looked at avs optimizer and running iterations and settings, but it looks like it just can't be done with mvtools2

TBD. I'm still running tests and I have only reported the first batch which didn't even use all the available options of MVTools2 (starting from this post (https://forum.doom9.org/showthread.php?p=1942410#post1942410) for those who are interested). No holy grail findings yet but the results are improving. Now testing MRecalculate. There are a couple of tricks that can be used to improve the results even further (and I have like 10 more ideas I haven't tested).

That being said, my bet is that MVTools will not beat RIFE when it comes to object boundaries.

[poisondeathray]

TBD. I'm still running tests and I have only reported the first batch which didn't even use all the available options of MVTools2 (starting from this post (https://forum.doom9.org/showthread.php?p=1942410#post1942410) for those who are interested). No holy grail findings yet but the results are improving. Now testing MRecalculate. There are a couple of tricks that can be used to improve the results even further (and I have like 10 more ideas I haven't tested).

That being said, my bet is that MVTools will not beat RIFE when it comes to object boundaries.



Thanks, looking forward to the additional testing



If you have time to conduct other testing - some other situations where DAIN/RIFE tend to do better/cleaner (I mean, besides "picket fence") - are general FG/BG/FG object boundary delineation such as objects crossing over. Even something simple such as walking. Often a person's arms/hands/legs will look like 3 arms instead of 2 (3 legs instead of 2) using traditional optical flow methods. Or dancing, rotation - as person "spins" their arms/leg look blended or become artifacts. Any one that uses mvtools2 for motion interpolation or other optical methods in the past should be familiar with this category of artifacts. Maybe zorr can find some mvtools2 settings which "solve" this scene or at least improve on the results

This one is from John Meyer. Sample link down in original thread
https://forum.doom9.org/showthread.php?t=167914

src mirror (with frc and rife @ 60000/1001 libx264 crf 16 "normal speed") in zip
https://www.mediafire.com/file/8ty526fot6jixhr/fashionshow_src,frc,rife.zip/file

YT comparison, slow down to 6fps
https://www.youtube.com/watch?v=N8LpwRiix8M

Object (person) boundaries are cleaner with DAIN and RIFE. There are negligble artifacts as the hands/arms cross over the body (FG/FG layers), and also the background vertical lines on the wall (FG/BG). No need to resort to blending coverups or other fixes




Sample walk, 23.976p with motion blur .

src, (with frc and rife @ 48000/1001 libx264 crf 16 "normal speed", and "moving window" crop comparison) in zip
https://www.mediafire.com/file/cnt8g2laplu3sae/high_heel_walk_src,frc,rife+windowcomparison.zip/file

YT comparison, slowed down to 6fps, moving crop window (HD version might take a while to show up on YT, but higher quality file in the zip anyways)
https://www.youtube.com/watch?v=7GI-Xc7cSsQ

There are typical mvtools2 artifacts when using default settings (maybe zorr can find some magical settings). Arms, legs are doubled up and BG objects like wall lines, electrical outlet move around and are not stationary. FRC with default settings does a decent job of making FG artifacts more palatable with blending instead of jarring artifacts .

RIFE is not perfect on this sample, but it produces cleaner in terms of edges, objects, BG. Note 1 frame near the end is bad because it's almost "reverse" motion





DAIN/RIFE are not the "holy grail" either. They can make mistakes on some parts of frames that mvtools2 does not. Some cue just messes up the neural net and it makes a big mistake, whereas mvtools2 breezes through some scenarios with flying colors. YMMV. Mix/match for better results

I mentioned these "cons" in another thread - but other ones - DAIN/RIFE are less flexible, fewer settings, currently only 2x multiples, 8bit RGB for official internal version only (not sure if VPY version has other conversions in the background). Scene change detection also not very good (but it looks like VPY version has sc and misc.SCDetect to repeat last frame)



Something like RIFE with secondary mvtools2 fallback (or vice-versa), then tertiary artifact blend fallback would be cool

[zorr]

Often a person's arms/hands/legs will look like 3 arms instead of 2 (3 legs instead of 2) using traditional optical flow methods. Or dancing, rotation - as person "spins" their arms/leg look blended or become artifacts. Any one that uses mvtools2 for motion interpolation or other optical methods in the past should be familiar with this category of artifacts. Maybe zorr can find some mvtools2 settings which "solve" this scene or at least improve on the results

That artifact is familiar - I did one test with a similar case (it's actually in the Zopti introduction). Perhaps not completely "solved" but much better. This was with a basic MAnalyse + MFlowFPS script so it can be improved.

https://i.postimg.cc/T2Zg6s1y/ezgif-2-85352ef164.gif

DAIN/RIFE are less flexible, fewer settings, currently only 2x multiples, 8bit RGB for official internal version only (not sure if VPY version has other conversions in the background).

That 8bit limitation could be circumvented by using MCompensate - let DAIN/RIFE create a good target for MCompensate, calculate vectors in 8bit and apply them in HBD clip.

Something like RIFE with secondary mvtools2 fallback (or vice-versa), then tertiary artifact blend fallback would be cool

Yeah I have already thought about combining them. Nothing concrete yet - still trying to comprehend MVTools. It's a monster and it takes time to tame it.

Those example clips look good test cases and I'm sure I will try them. They also don't look unreasonably hard - MVTools should be able to handle them quite well. The high heel walk clip is also an interesting special case, it has a (mostly) stationary background which can be extracted with temporal median. Once we have that we can build masks of moving objects. By making the background completely black it should be much easier for MVTools to track objects.

[poisondeathray]

That artifact is familiar - I did one test with a similar case (it's actually in the Zopti introduction). Perhaps not completely "solved" but much better. This was with a basic MAnalyse + MFlowFPS script so it can be improved.

Yes, I remember, good improvement.

Here is FRC vs. RIFE to compare, basically an even bigger improvement
https://i.postimg.cc/zBBm9pbB/handwave-frc-rife.gif



That 8bit limitation could be circumvented by using MCompensate - let DAIN/RIFE create a good target for MCompensate, calculate vectors in 8bit and apply them in HBD clip.


Interesting approach, not sure how to go about it yet.

But really 8bit is a minor issue , I'd take clean edges over 8bit RGB as a tradeoff any day of the week.

They VPY version only supports RGB float, so I'm not sure how that is navigated

I've been using the CUDA/Pytorch implementation, not Vulkan (which the VPY version is based on). CUDA version is supposed to be ~3x faster, not sure if there are quality differences. I can't get the VPY plugin to load... trying to figure out the VPY version

[MysteryX]

RIFE does look impressive on the hand! Where FRC is really good at is building artefact masks. The problem is that there wasn't a good back-up to replace them, other than Frame Blending.

The image also looks better defined with MVTools2 in the non-artefact areas.

btw I'm currently porting FRC plugins DLL to VapourSynth. It's almost done. Remains to rewrite the AVSI file...

Then can try swapping out the Frame Blending clip with RIFE.

Looking at RIFE clips, it looks like for easy linear surfaces (panning), MVTools2 is just moving the texture linearly without distortion, whereas RIFE is trying to move it in all kinds of subtle ways that blur some of the details.

[kedautinh12]

Thanks

[Selur]

btw I'm currently porting FRC plugins DLL to VapourSynth. It's almost done. Remains to rewrite the AVSI file...:goodpost:

[kedautinh12]

New mod FrameRateConverter.avsi from Dogway for spped and HBD
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi
It's request extools to work
https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi

[real.finder]

btw, https://github.com/mysteryx93/FrameRateConverter#conditionalfiltermt for last avs+ conditionalfiltermt is not needed since mt problem fixed with AvisynthNeo changes

[MysteryX]

btw how do you convert this into Expr (for VapourSynth)?
ColorYUV(gamma_y = ((1.0 / 2.2) - 1.0) * 256.0)

[MysteryX]

OK guys, here's a beta DLL working for both Avisynth and VapourSynth. The same DLL works in both. Requires FMTC (https://forum.doom9.org/showthread.php?t=166504).

>> Download here (https://mega.nz/file/iFoUmZgT#8OluWsf0hY2A2Rq43RE_ce4-0iBK14WKpLZbJRFNnVU)

StripeMask isn't currently converting the mask back to HBD when input is HBD, and more work is needed to handle errors properly, but otherwise it's usable.

As for converting the AVSI file, could someone work on converting it? I do not know Python syntax and it would take me a while to get it done. The DLL is the hard part; many people can do the scripting part.

One issue came to my attention with StripeMask: level range PC vs TV. Take the same script, and the strength of the mask applied will be very different depending on whether the input is PC or TV. Is this negatively affecting other parts of the script?

The Avisynth gamma-to-linear solution of "ColorYUV(gamma_y = ((1.0 / 2.2) - 1.0) * 256.0)" is completing ignoring the level range. For VapourSynth, I'm using FMTConv instead. It produces a different output that serves as source for mask processing. It's more accurate but considerably brighter if converting to RGB before converting to Linear. It's brighter but less if converting to Grey16 and converting that to Linear. Might have to see whether it's worth doing Linear conversion in RGB or Grey16 is fine. More importantly, PC vs TV level needs to be taken into account.

And can the same solution be done in Avisynth to have consistent output? FMTConv is only for VapourSynth I think.

If someone can work on converting the AVSI file, that would be great! You can then swap out the Frame Blending clip with RIFE and dance.

[Dogway]

This is a simple gamma expression in RPN. You need to use scale_inputs="allf" in order to work. You can also take out "1 {gamma} /" from the expression to speed it up slightly.

rangePC = tv_in ? "x ymin - ymax ymin - /" : "x range_max /"
rangeTV = tv_out ? "ymax ymin - * ymin +" : " range_max *"

Format(""+rangePC+" 1 {gamma} / ^ "+rangeTV+"")

[Selur]

New mod FrameRateConverter.avsi from Dogway for spped and HBD
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi
It's request extools to work
https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi
there are more dependencies,...
you also need Utils-r41 (http://avisynth.nl/images/Utils-r41.avsi)

[kedautinh12]

Or link github: https://github.com/raffriff42/AvisynthPlusUtilities/blob/master/Utils-r41.avsi

[MysteryX]

Dogway, do your have VapourSynth versions of your scripts?

[Dogway]

No, unfortunately I'm on Win7 but VS already has ported masktools2 and also has fmtconv. I think VS has AVX2 accelerated Convolution() function so it's already very fast.

[MysteryX]

This is a simple gamma expression in RPN. You need to use scale_inputs="allf" in order to work. You can also take out "1 {gamma} /" from the expression to speed it up slightly.

rangePC = tv_in ? "x ymin - ymax ymin - /" : "x range_max /"
rangeTV = tv_out ? "ymax ymin - * ymin +" : " range_max *"

Format(""+rangePC+" 1 {gamma} / ^ "+rangeTV+"")

I'm trying to use this, but it says "failed to convert ymin to float". How do you use this?

rangePC = "x ymin - ymax ymin - /"
rangeTV = " range_max *"
query = ""+rangePC+" 1 {gamma} / ^ "+rangeTV+""
video = core.std.Expr(clips=[video], expr=[query, ""])

[Dogway]

I don't know how vapoursynth deals with constants, you can replace them with this table (https://forum.doom9.org/showthread.php?p=1945324#post1945324)based on bitdepth.

I think you need (http://www.vapoursynth.com/doc/functions/expr.html)to replace with 8-bit values and set output format.

[real.finder]

vapoursynth depends on python for this, those keywords are avs exclusive

[MysteryX]

I'm trying to get consistent linear conversion output between Avisynth and VapourSynth.

Best way is to first convert to Y8 and then apply gamma correction, to give an Y8 output in full range.


ConvertToY()
ColorYuv(levels="TV->PC")
Levels(0, 1/2.2, 255, 0, 255)



video = core.resize.Point(video, format=vs.GRAY16)
video = core.fmtc.transfer(video, transs="709", transd="linear", fulls=0, fulld=1)
video = core.resize.Point(video, format=vs.GRAY8)


There's just a very slight contrast difference between the 2 methods. Shouldn't be an issue.

Now, if I want slightly better conversion, Levels doesn't work in 16-bit, and using ColorYUV instead of Levels gives a much darker output for some reason


ColorYUV(gamma_y=((1.0 / 2.2) - 1.0) * 256.0)


Other option is to use Expr for both and have perfectly consistent output without needing FMTC dependency.

This is a simple gamma expression in RPN. You need to use scale_inputs="allf" in order to work. You can also take out "1 {gamma} /" from the expression to speed it up slightly.

rangePC = tv_in ? "x ymin - ymax ymin - /" : "x range_max /"
rangeTV = tv_out ? "ymax ymin - * ymin +" : " range_max *"

Format(""+rangePC+" 1 {gamma} / ^ "+rangeTV+"")

Format isn't an Avisynth function, and I can't get this to work. Can you write the exact syntax that I can put into the Avisynth script?

[Dogway]

Format() (http://avisynth.nl/index.php/Internal_functions#Format) is AVS+ only I think, don't know from what version.

This is an HBD aware TV<->PC levels conversion function. link (https://github.com/Dogway/Avisynth-Scripts/blob/6963044760236bc7226a6151935b891fce060e86/TransformsPack.v1.0.RC17.avsi#L1116)

And here is an HBD aware Levels function. link (https://github.com/Dogway/Avisynth-Scripts/blob/6963044760236bc7226a6151935b891fce060e86/GradePack.v1.9.avsi#L189)

They both require ExTools found in the repo.
I'm in the middle of a rebase so I'm taking my time and preparing an elaborated post for possible solutions, so if something triggers an error let me know, or use an earlier version of ExTools (v1.6 or v1.7).

EDIT: Oh by the way, this all to linearize an image/source right? I'm also in the middle of a refactor for Transforms Pack but the gamma function (https://github.com/Dogway/Avisynth-Scripts/blob/6963044760236bc7226a6151935b891fce060e86/TransformsPack.v1.0.RC17.avsi#L580) is very functional at this state. Pick the alpha and gamma value from this list (https://github.com/Dogway/Avisynth-Scripts/blob/6963044760236bc7226a6151935b891fce060e86/TransformsPack.v1.0.RC17.avsi#L949).

[MysteryX]

I got gamma correction working with Levels. That's probably the best option.

[Dogway]

Good luck!

[MysteryX]

Releasing FrameRateConvert v2.0-Beta (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta), DLL only. Works in both VapourSynth and Avisynth.

- Ported to VapourSynth
- Fixed a bug in StripeMask
- StripeMask linear gamma correction is now more accurate
- StripeMask now accounts for PC vs TV range
- StripeMask: added fullRange parameter, set to True if input is in full range

For VapourSynth, FMTC dll is required. For Avisynth, you can use the previous AVSI file and the StripeMask will have a few corrections. Note that StripeMask thr parameter default is now 28 instead of 26, and possibly could use further adjustment.

[kedautinh12]

Thanks

[ChaosKing]

Thx for the port.
I see some warnings / problems in vsedit :D

clip = core.frc.StripeMask(clip)
Core freed but 6 filter instance(s) still exist
Core freed but 1075200 bytes still allocated in framebuffers
Plugin C:/Users/anato/AppData/Roaming/VapourSynth/plugins64/FrameRateConverter-x64.dll tried to register 'ConvertFpsLimit' more than once. Second registration ignored.
Plugin C:/Users/anato/AppData/Roaming/VapourSynth/plugins64/FrameRateConverter-x64.dll tried to register 'ConvertFpsLimit' more than once. Second registration ignored.
Plugin C:/Users/anato/AppData/Roaming/VapourSynth/plugins64/FrameRateConverter-x64.dll tried to register 'ConvertFpsLimit' more than once. Second registration ignored.

and if I want to preview the mask I get
Error on frame 0 request:
Resize error -1: invalid graph state L910: !m_state.has_chroma() || m_state.plan


core.frc.ContinuousMask(clip, 24000, 1000) #crashes the editor
Log shows this msg
setVideoInfo: The frame rate specified by ContinuousMask must be a reduced fraction. (Instead, it is 24000/1000.)

[MysteryX]

Fixed most of those bugs.

Error on frame 0 request:
Resize error -1: invalid graph state L910: !m_state.has_chroma() || m_state.plan

This error is in FMTC.transfer; but it works when run in VirtualDub.


video = core.resize.Point(video, format=vs.GRAY16)
video = core.fmtc.transfer(video, transs="709", transd="linear")
video = core.resize.Point(video, format=vs.GRAY8)


There's the option of converting to RGBS but it would be slower and would produce a slightly brighter output than in Avisynth.

I released a new beta at the same link. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

[ChaosKing]

Is 16bit int or 32bit float input supported? Output of ContinuousMask() looks strange with non 8bit input.


The parameter seems to be ConvertFpsLimit(clip, num, den, ratio) but I get this error msg when calling ConvertFpsLimit(clip, 24000, 1000, 1)
vapoursynth.Error: ConvertFpsLimit: Can only set one of the following parameters: num/den (fraction), fps (float), preset (string) or match (clip).

+ it crashes with just ConvertFpsLimit(clip)

[MysteryX]

ContinuousMask gives strange output on regular clips. It's meant to process masks. I didn't document it either -- don't remember what it did exactly :P Works in the context it's used for though. Gives same output as it did before in Avisynth.

Since I can't register ConvertFpsLimit several times like in Avisynth, the syntax is now


"clip:clip;"
"num:int:opt;"
"den:int:opt;"
"fps:float:opt;"
"preset:data:opt;"
"match:clip:opt;"
"ratio:int:opt;"

Forgot to validate if none is set; fixed it.

[ChaosKing]

ContinuousMask gives strange output on regular clips. It's meant to process masks.
I'm pretty sure this output is stranger then it should be :p
https://i.imgur.com/BzCFDvk.png

[MysteryX]

I'm pretty sure this output is stranger then it should be :p
I'll answer like Microsoft. It's by design.

Just finished porting the first function. Only problem: GaussResize doesn't exist in VapourSynth. Also is BoxBlur equivalent to Avisynth Blur?

I could use some help here :stupid:


import vapoursynth as vs
core = vs.get_core()

def frc_GaussianBlur42(input, var = None, rad = None, vvar = None, vrad = None, p = None):
{
if not isinstance(input, vs.VideoNode):
raise vs.Error('frc_GaussianBlur42: input is not a clip')

var = max(0.0, float(var or 1.0)))
rad = max(1.0, float(rad or pow(var, 0.5)))
var = pow(min(max(0.0, rad), 60.0), 1.9) ## arbitrary max radius = 60

vvar = max(0.0, float(vvar or var)))
vrad = max(1.0, float(vrad or pow(vvar, 0.5)))
vvar = pow(min(max(0.0, vrad), 60.0), 1.9)
p = p or 19

w0 = input.width
h0 = input.height
w1 = round(w0/rad)
h1 = round(h0/vrad)

B = input.resize.Bilinear( \
min(max(4, w1 + (w1 % 2)), w0), \
min(max(4, h1 + (h1 % 2)), h0))

B = B.std.BoxBlur(hpasses=2, vpasses=2)

if var < 0.01 and vvar < 0.01:
return input
elif B.Width > 8 and B.Height > 8):
return B.std.GaussResize(w0, h0, p=p) ### NOT IN VAPOURSYNTH
else:
return B.std.Bilinear(w0, h0)
}


Note: just realized that ConvertFpsLimit can be quite useful in VapourSynth since ConvertFps doesn't exist! Good thing it's already ported.

[kedautinh12]

fmtconv have GaussResize
https://github.com/AkarinVS/fmtconv/blob/4f187bf5b7fe116d98a55910d597146b6b3a79a8/doc/fmtconv.html#L1080

[MysteryX]

Thanks. Another issue. I had already ported ConvertFps, but ChangeFps (no blending) doesn't exist in VapourSynth either. What's the best solution? It's actually used several times for masks.

Also to consider: what to do in case of variable framerate input?

Finally... MaskTools2 hasn't been ported to VapourSynth??

[kedautinh12]

Mt_lutspa: https://github.com/kewenyu/VapourSynth-Draw

mt_motion: https://github.com/dubhater/vapoursynth-motionmask

[kedautinh12]

havsfunc have ChangeFPS: https://github.com/HomeOfVapourSynthEvolution/havsfunc/blob/019ad633afa2982dfee8959435b2a0deb862c9f7/havsfunc.py#L5353

[kedautinh12]

I think you need convert masktool2 from avisynth to internal functions of Vapoursynth. Here for example convert:
https://forum.doom9.org/showthread.php?p=1802951

[MysteryX]

I need:
mt_inpand
mt_lut
mt_lutxy
mt_expand
mt_binarize
mt_circle
mt_merge

Pretty much the whole suite

[Reel.Deel]

I need:
mt_inpand
mt_lut
mt_lutxy
mt_expand
mt_binarize
mt_circle
mt_merge

Pretty much the whole suite


mt_lut and mt_lutxy: http://www.vapoursynth.com/doc/functions/expr.html

mt_binarize: http://www.vapoursynth.com/doc/functions/binarize.html

mt_merge: http://www.vapoursynth.com/doc/functions/maskedmerge.html#std.MaskedMerge

mt_inpand and mt_expand: http://www.vapoursynth.com/doc/functions/minimum_maximum.html


Not sure about mt_circle.

[kedautinh12]

mt_lut and mt_lutxy: http://www.vapoursynth.com/doc/functions/expr.html

mt_binarize: http://www.vapoursynth.com/doc/functions/binarize.html

mt_merge: http://www.vapoursynth.com/doc/functions/maskedmerge.html#std.MaskedMerge

mt_inpand and mt_expand: http://www.vapoursynth.com/doc/functions/minimum_maximum.html


Not sure about mt_circle.

That's was i said

[Reel.Deel]

That's was i said


Umm I guess ... You posted a link to another thread which has a script with all kinds of things that are mostly not relevant to the question that MysteryX asked ... but hey, it's nice to see you actually writing something instead of 'thanks' on every post :p

[kedautinh12]

Umm I guess ... You posted a link to another thread which has a script with all kinds of things that are mostly are not relevant to the question that MysteryX asked ...

Are you sure???

mt_merge = maskmerge
https://forum.doom9.org/showthread.php?p=1803090#post1803090

And some functions of Mastool2 in this script:
http://avisynth.nl/images/MfToon-v0.54.avsi

Was converted to internal functions from vapoursynth here:
https://gist.github.com/Frechdachs/b3a05afe2f7d25316a10cf7239d53d0c

And you say all kinds of things that are mostly are not relevant to the question that MysteryX asked???

[MysteryX]

I also need to convert this


Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true)
Overlay(MergeRGB(Blank, EM, EM), mode="Add", opacity=0.40, pc_range=true)

[ChaosKing]

I also need to convert this


Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true)
Overlay(MergeRGB(Blank, EM, EM), mode="Add", opacity=0.40, pc_range=true)


https://github.com/HomeOfVapourSynthEvolution/havsfunc/blob/019ad633afa2982dfee8959435b2a0deb862c9f7/havsfunc.py#L5427

[MysteryX]

mt_inpand and mt_expand: http://www.vapoursynth.com/doc/functions/minimum_maximum.html


this though...

mt_expand(mode= mt_circle(zero=true, radius=8))


Good news is that everything else is "converted" and non-tested. Just missing that last function.

[kedautinh12]

I seen here, but how to use avs functions in Vapoursynth scripts???
https://github.com/darealshinji/vapoursynth-plugins/blob/614d367e826b5f6bb0af33e1f91453f3ff7999ec/scripts/edgecleaner.py#L40

[MysteryX]

Making good progress.

This gives me "failed to convert '^' to float"

core.std.Expr(EM, "x 1.09 * 1.28 ^")
to replace
mt_lut(EM, "x " + string(dct_mult) + " * " + string(dct_pow) + " ^")


Performance looks much higher in Vapoursynth so far, but there are a couple of restrictions. Analysis can only be done with blksize 4x4, 8x8, 16x16 or 32x32, whereas in Avisynth I was using 8, 12, 16, 24,32

[ChaosKing]

I seen here, but how to use avs functions in Vapoursynth scripts???
https://github.com/darealshinji/vapoursynth-plugins/blob/614d367e826b5f6bb0af33e1f91453f3ff7999ec/scripts/edgecleaner.py#L40

VS can use (some) avs plugins. But avs script functions can't be used. For that there's avsw.Eval https://forum.doom9.org/showthread.php?t=175141



Performance looks much higher in Vapoursynth so far, but there are a couple of restrictions. Analysis can only be done with blksize 4x4, 8x8, 16x16 or 32x32, whereas in Avisynth I was using 8, 12, 16, 24,32
mvtools-sf by feisty seems to support even lower blksizes. BUT it will be slower and only supports 32bit float.
https://github.com/IFeelBloated/vapoursynth-mvtools-sf/blob/master/src/MVAnalyze.hxx#L304

https://forum.doom9.org/showthread.php?t=172525

[kedautinh12]

Thanks

[MysteryX]

mvtools-sf by feisty seems to support even lower blksizes. BUT it will be slower and only supports 32bit float.
It's blksize 12 and 24 that would be useful, which it doesn't support either.

Remains the issues of core.std.Expr(EM, "x 1.09 * 1.28 ^") and mt_expand with mt_circle

[wonkey_monkey]

Remains the issues of core.std.Expr(EM, "x 1.09 * 1.28 ^")

core.std.Expr(EM, "x 1.09 * 1.28 pow")

[MysteryX]

Comparing MVTools2 in VapourSynth vs Avisynth, performance is MUCH higher in VapourSynth, and the interpolation output is slightly different (slightly better in VapourSynth). The Mask output, however, is VERY different! A lot more detailed in VapourSynth. This means I'll have to retest and rebalance everything for VapourSynth.

VapourSynth version doesn't support DCT=0. The whole CalculateDiff was between DCT=4 and DCT=0, or between DCT=1 and DCT=0. Will have to see whether it makes sense to compare DCT=4 and DCT=1.

[real.finder]

I think that because MT is on in vs by default
LWLibavVideoSource("test.avi")
RequestLinear(clim=100)
super_search = MSuper(rfilter=4)

bv2 = super_search.MAnalyse(isb = true, delta = 2, overlap= 4)
bv1 = super_search.MAnalyse(isb = true, delta = 1, overlap= 4)
fv1 = super_search.MAnalyse(isb = false, delta = 1, overlap= 4)
fv2 = super_search.MAnalyse(isb = false, delta = 2, overlap= 4)

MDegrain2(MSuper(levels=1), bv1, fv1, bv2, fv2, thSAD=300, thSADC=150)
Prefetch()

as fast as

import vapoursynth as vs
core = vs.get_core()
clip=core.lsmas.LWLibavSource(source=r'test.avi')
super_search = core.mv.Super(clip,rfilter=4)

bv2 = core.mv.Analyse(super_search, isb = True, delta = 2, overlap= 4)
bv1 = core.mv.Analyse(super_search, isb = True, delta = 1, overlap= 4)
fv1 = core.mv.Analyse(super_search, isb = False, delta = 1, overlap= 4)
fv2 = core.mv.Analyse(super_search, isb = False, delta = 2, overlap= 4)

clip=core.mv.Degrain2(clip,core.mv.Super(clip,levels=1), bv1, fv1, bv2, fv2, thsad=300, thsadc=150)
clip.set_output()

don't know about output, didn't check if there are some parameter default difference between them

[MysteryX]

Good catch about MT. DCT=0: "usual spatial blocks, do not use DCT". So I wonder why I was using that for the slowest preset.

As for masks, there are MAJOR differences. I wonder why.

[MysteryX]

Raw mask is working fine, but I'm having issues converting it into a decent mask.

Here I do Binarize, followed by Bicubic, and it gives a weird vertical stretching. Here's what it does with Point resize at the end. Binarize+Point should give pure black&white. What's happening here? Mask is in Gray8 format.


EM = EM.resize.Bicubic(round(C.width/blkSize/4.0)*4, round(C.height/blkSizeV/4.0)*4)
EM = EM.std.Maximum()
EM = EM.std.Binarize(maskThr)
EM = EM.resize.Point(C.width, C.height)

https://i.postimg.cc/5HbfvqDw/Mask-Point.png (https://postimg.cc/5HbfvqDw)

Then I also have the issue that BoxBlur makes the blur too wide. I tried havsfunc.MinBlur but it makes smaller mask areas dimmer.

Here's a good mask produced in Avisynth. How can I reproduce something similar?

https://i.postimg.cc/jWcthnrB/MaskAvs.png (https://postimg.cc/jWcthnrB)

Here's the Avisynth code


EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(MaskThr)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)

[kedautinh12]

B = input.resize.Bilinear( \
min(max(4, w1 + (w1 % 2)), w0), \
min(max(4, h1 + (h1 % 2)), h0))

B = B.std.BoxBlur(hpasses=2, vpasses=2)


Maybe replace boxblur with RemoveGrain


B = input.resize.Bilinear( \
min(max(4, w1 + (w1 % 2)), w0), \
min(max(4, h1 + (h1 % 2)), h0))

B = B.rgvs.RemoveGrain(mode=12).rgvs.RemoveGrain(mode=12) #Deprecated. Use Convolution(matrix=[1, 2, 1, 2, 4, 2, 1, 2, 1]) instead.

[MysteryX]

ah I can use this instead of Blur(1)
Convolution(matrix=[1, 2, 1, 2, 4, 2, 1, 2, 1])

but what about Blur(0.6) ?

[kedautinh12]

but what about Blur(0.6) ?

Dogway use this replaced blur(.6) with Expr, you can convert it to Vapoursynth code
https://github.com/Dogway/Avisynth-Scripts/blob/6eb7e81ea48717c648fc5140be0f3ce41d553123/EX%20mods/FrameRateConverter.avsi#L246

[kedautinh12]

Like ChaosKing say VS can use (some) avs plugins. You can use this script in VS replace EM.std.Maximum()
EM.avs.mt_expand(mode=avs.mt_circle(zero=true, radius=1))

Same here: https://github.com/darealshinji/vapoursynth-plugins/blob/614d367e826b5f6bb0af33e1f91453f3ff7999ec/scripts/edgecleaner.py#L40

[amayra]

Ported to VapourSynth


finally

[MysteryX]

I think that using Avisynth's MaskTools2 is the best option until someone decides to port it. It's really not difficult to write shared code for both platforms, I wish more people did that.

[kedautinh12]

Hope anyone port VapourSynth-RIFE-ncnn-Vulkan to avisynth

[MysteryX]

Like ChaosKing say VS can use (some) avs plugins. You can use this script in VS replace EM.std.Maximum()

Same here: https://github.com/darealshinji/vapoursynth-plugins/blob/614d367e826b5f6bb0af33e1f91453f3ff7999ec/scripts/edgecleaner.py#L40

How is he importing AVS into VapourSynth? All I can find is AvsProxy (https://forum.doom9.org/showthread.php?t=175141), which only has an Eval function.

That's not what he's using here, his syntax is much nicer. He lists his dependencies but doesn't say how he's loading them.

Then I still have the weird vertical blur issue on resize.

EM.std.Expr("x[0,1] 0.12 * x[0,0] 0.76 * x[0,-1] 0.12 * + +")
"Failed to convert x[0,1] to float." And is this exactly .6 strength?

[kedautinh12]

Here for load avs plugins in Vapoursynth
http://www.vapoursynth.com/doc/functions/loadpluginavs.html

[kedautinh12]

AvsProxy used for load scripts .avsi file in Vapoursynth not .dll. please read slowly what ChaosKing says

[kedautinh12]

Bad news
Avisynth plugins are never autoloaded. Support for this may be added in the future.
From here: http://www.vapoursynth.com/doc/plugins.html

[real.finder]

EM.std.Expr("x[0,1] 0.12 * x[0,0] 0.76 * x[0,-1] 0.12 * + +")
"Failed to convert x[0,1] to float." And is this exactly .6 strength?

cuz it use http://avisynth.nl/index.php/Expr#Pixel_addressing and seems it's only in avs+

[kedautinh12]

"Failed to convert x[0,1] to float." And is this exactly .6 strength?

Dogway say that: https://github.com/Dogway/Avisynth-Scripts/issues/13#issuecomment-867535798

[MysteryX]

Converting to VapourSynth isn't so simple..


core.std.LoadPlugin(path=r'C:\GitHub\FrameRateConverter\masktools2.dll')

"No entry point found". Note that if it doesn't find the file or it's a x86 DLL, it instead says "Failed to load".

List of pending issues:
- Blur(.6)
- Loading Avisynth DLL
- Resize weird vertical blurring

Blur(.6) may not be as fast as Blur(1), but there's a reason I put it to .6: otherwise the mask edges are too soft and makes artefacts leak through.

Oh wait, Dogway created ex_expand (https://github.com/Dogway/Avisynth-Scripts/blob/ccabc3d261a787e38f762b4976cc3e698f226880/ExTools.avsi#L531)(1,mode="circle"), does that work in VapourSynth? It's using the same Pixel Addressing feature; is that supported or not?

EDIT:
For AVS plugin, simply replace core.std.LoadPlugin with core.avs.LoadPlugin !

Now, this crashes without any message. EM format is Gray8.

EM = EM.avs.mt_expand(mode= core.avs.mt_circle(zero=True, radius=1))

[MysteryX]

OK now I'm running only mt_expand() to debug it. It "works" in YUV420P8 (with weird output?) and CRASHES in GRAY8.

Now if I run this (in YUV420P8), I get that.


video = video.avs.mt_expand(core.avs.mt_circle(zero=True, radius=1))
> Python exception: could not convert string to float: b'-1 0 0 -1 0 0 0 1 1 0 '


I don't know how others are using it, but converting Dogway's function may be the best option?

[kedautinh12]

Don't know cause Dogway still use same Pixel Addressing in his script

Edit: can you try AvsProxy??

[MysteryX]

Edit: can you try AvsProxy??
Could run the whole thing under AvsProxy; but then it isn't really a port.

Porting the Expr code supporting pixel addressing would allow to solve both problems (mt_expand and blur). Or can I run those expressions with AVS LoadPlugin? It's kind of weird if Avisynth's Expr would be more powerful than Vapoursynth's Expr.

[kedautinh12]

Porting the Expr code supporting pixel addressing would allow to solve both problems (mt_expand and blur).

You can ask Vapoursynth's developer about this porting
https://github.com/vapoursynth/vapoursynth/issues

[MysteryX]

This works to create the mask.

EM = EM.resize.Bicubic(round(C.width/blkSize/4.0)*4, round(C.height/blkSizeV/4.0)*4) \
.std.Maximum(coordinates=[0, 1, 0, 1, 1, 0, 1, 0]) \
.std.Binarize(maskThr) \
.std.Convolution(matrix=[0, 1, 0, 1, 3, 1, 0, 1, 0]) \
.resize.Bicubic(C.width, C.height)

Maximum with those coordinates is like mt_circle(radius=1). Convolution balanced in this way gives a decent output.

Blur(1) can be rewritten like this and then you can fine-tune as desired.

.std.Convolution(matrix=[10, 20, 10, 20, 40, 20, 10, 20, 10])

Apparently it also supports floats, which I never saw any mention before.

[MysteryX]

Having another issue with Overlay


# Prepare output=Over: Mask(cyan), Stripes(yellow)
FlowOver = Flow.Overlay(MergeRGB(Blank, EM, EM), mode="Add", opacity=0.40, pc_range=true)

becomes

Overlays = core.std.ShufflePlanes(clips=[Blank, EM, EM], planes=[0, 0, 0], colorfamily=vs.RGB).resize.Point(format=B.format, matrix_s="709")
FlowOver = havs.Overlay(Flow, Overlays, mode="addition", opacity=0.40)

This however causes the whole image to turn purple! Instead of adding the mask on top of it.

EDIT: if I convert Flow to RGB then it works

[MysteryX]

Here's an example of how the mask is different in VapourSynth than in Avisynth

Here's the mask with Analyze (without Recalculate)

Avisynth
https://i.postimg.cc/SXWjzsP2/MaskAvs.png (https://postimg.cc/SXWjzsP2)

VapourSynth
https://i.postimg.cc/cKqrHYFG/MaskVpy.png (https://postimg.cc/cKqrHYFG)

[kedautinh12]

Blur(1) can be rewritten like this and then you can fine-tune as desired.

.std.Convolution(matrix=[10, 20, 10, 20, 40, 20, 10, 20, 10])


But document here Convolution(matrix=[1, 2, 1, 2, 4, 2, 1, 2, 1]) same RemoveGrain (12) and blur(1)
http://www.vapoursynth.com/doc/functions/convolution.html

[kedautinh12]

Having another issue with Overlay


# Prepare output=Over: Mask(cyan), Stripes(yellow)
FlowOver = Flow.Overlay(MergeRGB(Blank, EM, EM), mode="Add", opacity=0.40, pc_range=true)

becomes

Overlays = core.std.ShufflePlanes(clips=[Blank, EM, EM], planes=[0, 0, 0], colorfamily=vs.RGB).resize.Point(format=B.format, matrix_s="709")
FlowOver = havs.Overlay(Flow, Overlays, mode="addition", opacity=0.40)

This however causes the whole image to turn purple! Instead of adding the mask on top of it.

EDIT: if I convert Flow to RGB then it works

try increase opacity in Vapoursynth

[MysteryX]

But document here Convolution(matrix=[1, 2, 1, 2, 4, 2, 1, 2, 1]) same RemoveGrain (12) and blur(1)
http://www.vapoursynth.com/doc/functions/convolution.html
It averages the value of a 3x3 square to determine its middle pixel. If you multiply all the values by 10, the weighting is exactly the same.

[MysteryX]

Great news! The VapourSynth script is now functional!

> Download Beta 3 here (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

Requires havsfunc and FMTC.

I changed all DLL parameter names to lowercase. FrameRateConverter took parameter stp to enable/disable stripe mask. Stp now specifies the mask threshold. Default was previously 23, now it is set to 35 by default. 0 to disable.

In Avisynth, DCT=0 is faster and was used for various presets. In VapourSynth, DCT=0 gives the same as DCT=4. Which brings 2 questions: what DCT to use for better performance, and what DCTs to use with the DIFF mode?

Need to play around with mask levels and thresholds. Stripe mask seemed to be working better before, might just be threshold settings. Needs more testing.

Also for now you need to set blkSize manually, as blkSize 12 and 24 aren't supported; but perhaps could still be used for StripeMask.

Right now most presets behave like "slow", with Calculate and Recalculate with DCT=4. If you want to do some testing, need to see what to offer at the various preset levels.

btw I'm getting MUCH better performance in VapourSynth. Same script preset "slow"

Avisynth: 10327ms at 29% CPU usage (100% would give 2994ms)
VapourSynth: 6442ms at 52% CPU usage (100% would give 3349ms ?)

Cheers!

[kedautinh12]

Thanks for your hardwork, MysteryX

[Selur]

Nice! Thanks!

[Selur]

Hmm, when using FrameRateConverter in Vapoursynth I get:
vapoursynth.Error: BlankClip: nodes foreign to this core passed as input, improper api usage detected
script I used was:
# Imports
import os
import sys
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
import FrameRateConverter
# source: 'G:\TestClips&Co\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="G:/TestClips&Co/test.avi",cachefile="E:/Temp/avi_9dec25d3f707eb4813d42334c7f1a8d6_853323747.ffindex",fpsnum=25,format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting frame count&rate with FramerateConverter
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8, output="auto", skipOver=210) # new fps: 50
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model (i420@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

-> Any idea what could cause this?

Okay, replacing:
core = vs.get_core()
with:
from vapoursynth import core
in FrameRateConverter.py (and deleting the Python cache files) fixed the issue.

Cu Selur

Ps.: Should such thinks better be reported here, over at github or at both places?

[MysteryX]

I cannot reproduce your problem. This works for me

import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
import FrameRateConverter as frc
import vapoursynth as vs
core = vs.get_core()

core.std.LoadPlugin(path=r'C:\GitHub\FrameRateConverter\Src\x64\release\FrameRateConverter.dll')
core.avs.LoadPlugin(path=r'C:\GitHub\FrameRateConverter\masktools2.dll')
#file='E:\AVSMeter\Motion Estimation Torture Clip.avi'
file="E:\AVSMeter\MotionTest2.mp4"
#file="E:\NaturalGrounding\Girl's Day\Original\Female President.mp4"
video = core.ffms2.Source(source=file, format=vs.YUV420P8, alpha=False)
clip = video.std.SetFrameProp("_FieldBased", intval=0)

clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
clip = frc.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8, output="auto", skipOver=210) # new fps: 50
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip.set_output()

To keep note:

- Need to validate that _FieldBased == 0 or throw an error. The weird resizing issue I had was due to this flag being improperly set.

- Need to call AssumeFPS or validate that the input is already fixed frame rate

[MysteryX]

Better report issues here. vs.get_core or import core, why is it making a difference?

[Selur]

My guess is:
get_core(..) -> seems to create a new core-instance
import core -> takes the existing core-instance
compared how havsfunc got the core and there 'import ..' was used.

[MysteryX]

I'm doing vs.get_core() and it doesn't crash.

AH! -- libraries may need to specifically import core. Is this documented anywhere? Even there... I'm not getting your error.

[Selur]

Might be related to that I use a portable Vapoursynth version,...

[Selur]

A few questions regarding FrameRateConverter in Vapoursynth:

Is it correct/intended that FrameRateConverter always outputs RGB24?
Using:
# Imports
import os
import sys
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
import FrameRateConverter

clip = core.ffms2.Source(source="G:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_6c441f37d9750b62d59f16ecdbd59393_853323747.ffindex",fpsnum=25,format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)

# adjusting frame count&rate with FramerateConverter
clip = FrameRateConverter.FrameRateConverter(clip, newNum=25, newDen=1, blkSize=8, blkSizeV=8, output="over", maskThr=100, blendOver=65, skipOver=255) # new fps: 25

# Output
clip.set_output()
the output is RGB24 (I was expecting YUV420P8, same as the input).
If this is correct, does it respect the color matrix and luma range durint the yuv->rgb conversion?
any plans for high bit depth support?


Cu Selur

[MysteryX]

That's because the Overlay function in VapourSynth only behaves as expected when done in the RGB space; gives weird result in YUV. I guess I could convert it back to the original format afterwards so that there's no matrix issue; and I should do that YUV->RGB->Overlay->YUV conversion in 16-bit.

There shouldn't be much issue with HBD, just need to try and see where it blocks.

[poisondeathray]

That's because the Overlay function in VapourSynth only behaves as expected when done in the RGB space; gives weird result in YUV.

In what way is the Overlay result "weird" in YUV ?

[Selur]

I guess I could convert it back to the original format afterwards so that there's no matrix issue; and I should do that YUV->RGB->Overlay->YUV conversion in 16-bit.
Looking forward to it. :)

There shouldn't be much issue with HBD, just need to try and see where it blocks.
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)
with a yuv420p10 clip produces:
Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2242, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 2243, in vapoursynth.vpy_evaluateScript
File "C:\Users\Selur\Desktop\test.vpy", line 30, in <module>
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8) # new fps: 50
File "I:\Hybrid\64bit\vsscripts\FrameRateConverter.py", line 178, in FrameRateConverter
EM = ToGray(C.mv.Mask(bak, ml=255, kind=1, gamma=1/gam, ysc=255, thscd2=skipOver))
File "src\cython\vapoursynth.pyx", line 2067, in vapoursynth.Function.__call__
vapoursynth.Error: Mask: input clip must be GRAY8, YUV420P8, YUV422P8, YUV440P8, or YUV444P8, with constant dimensions.
comes from https://github.com/dubhater/vapoursynth-mvtools/blob/6d805e232397cb08074dbb58cc02849f63b96714/src/MVMask.c#L328
Seems like the documentation (https://github.com/dubhater/vapoursynth-mvtools/blob/master/readme.rst) and functionality of mvtools don't match. :/
known 'missing feature' -> https://github.com/dubhater/vapoursynth-mvtools/issues/16

[MysteryX]

Actually a 8-bit mask is perfectly fine. Just need to convert it to Gray16 to make it compatible.

[MysteryX]

As an update on the discussion about Pixel Addressing, it is now supported in this VapourSynth plugin (https://github.com/AkarinVS/vapoursynth-plugin/releases/tag/v0.60).

[kedautinh12]

Time to replace masktool with ex_ family in Vapoursynth
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi

[MysteryX]

ex_ scripts haven't been ported to VapourSynth AFAIK. Plus the VPY replacement methods are highly optimized for HBD.

[kedautinh12]

About BicubicResize
https://forum.doom9.org/showthread.php?p=1946623#post1946623
https://forum.doom9.org/showthread.php?p=1946622#post1946622

[MysteryX]

DogWay, to replace Blur(.6) (https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi#L246) with a Convolution like you're doing, this convolution matrix gives a better result. If the blur is too wide, it lets artefacts leak through.

Convolution(matrix=[0, 1, 0, 1, 3, 1, 0, 1, 0])

for StripeMask giving "worse" results, I get similar results as before if doing it on PC range instead of Full range. Playing with the color range... seems like you can get better results by reducing the range. I might add a parameter to adjust the value range on top of the threshold.

[wonkey_monkey]

Wouldn't something like

Convolution(matrix=[1, 3, 1, 3, 9, 3, 1, 3, 1])

be more accurate? Ignoring the corner pixels seems wrong to me.

[Dogway]

I run some tests and yes Mysterys's kernel is more similar to either blur(0.6) or blur(0.5), actually a practical match to blur(0.5) besides the kernel is shorter so faster to compute.

Dog: Expr("x[-1,1] x[0,1] x[1,1] x[-1,0] x[0,0] 6 * x[1,0] x[-1,-1] x[0,-1] x[1,-1] + + + + + + + + 0.071428571 *","")
monkey: Expr("x[-1,1] x[0,1] 3 * x[1,1] x[-1,0] 3 * x[0,0] 9 * x[1,0] 3 * x[-1,-1] x[0,-1] 3 * x[1,-1] + + + + + + + + 0.04 *","")
Mystery: Expr("x[0,1] x[-1,0] x[0,0] 3 * x[1,0] x[0,-1] + + + + 0.1428571428 *","")
mergeluma(blur(0.6))
mergeluma(blur(0.5))

[MysteryX]

It's not about accuracy (of a pure white/black mask), but about having the right shape: not too wide, and not too square.

[MysteryX]

btw the existing StripeMask has a memory leak. I wonder why that never came up before. I cannot reproduce the memory leak in v2 beta in Avisynth. In VapourSynth, memory usage goes up from 1.3GB to 1.8GB over time but that could just be the way it handles the cache.

I've been playing with parameters to try to match previous results; the bar is pretty high, my best efforts barely come to 90% of the previous results!

Working with Luma-only (ShufflePlanes) works better than converting to Grey8. Btw, why do ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY) and resize.Point(format=vs.GRAY8) give different results? The latter gives a slightly darker output.

This is the pre-filtering that worked the best so far... yet it's not as good as the previous results! The linear gamma conversion is a bit different. I guess I got very lucky on the last released version (except that the old version assumes PC range and would behave different with Full range input)


video = core.std.ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY)
video = video.std.Levels(gamma=1/2.2, min_in=16, max_in=235, min_out=65, max_out=190)


The question is: what makes it that I can't fully match the accuracy of the last version? It was using thr=23. Now I nearly match it with thr=15, range=125 (considerably compressing the data range)

Before I was using ColorYUV to change gamma. Perhaps the difference is the way Gamma is applied, over Full or PC range? I think it was applying full-range gamma on a PC-range clip... somehow improving results. What impact did it really have? Anything to learn here in terms of better tweaking the input before processing?

Edit: I suppose converting to GRAY8 takes chroma into account during conversion. If I use ShufflePlanes, then coloured stripes with similar Luma would fail to be detected.

Note: StripeMask could very well run on an image with Width and Height reduced by half to increase performance.

[wonkey_monkey]

blur(0.6) produces the following result on a single white pixel (in RGB32, anyway; it's slightly different in RGB24 for some reason):

8 29 8
29 110 29
8 29 8

So it does spread to the corners, and these would be the best numbers if you want the closest match.

EDIT: My mistake, that's in Avisynth. Vapoursynth blur() might be different.

[MysteryX]

I could use some help here. Here's a new DLL vesion. (https://mega.nz/file/mBhCFTKK#0ZaI0jc48Aioeq3jXaOJFAsoHD7Tcy8I4S2Lj2DMu_I)

The download link contains a sample video clip. Can you try to get a more accurate stripe detection, which means, more of the fence being detected, and less false positives. You can compare to Avisynth StripeMask from before. Best I got so far is about 90% the quality of before for some reason. You can play with levels, contrasts, gamma, perhaps sharpening. Havsfunc has SmoothLevels which perhaps could be useful. Resize to Gray8 or ShufflePlane? This is time-consuming trial-and-error task that requires a lot of testing. This version has an additional parameter "skip". When set, it bypasses the pre-filter calls so you can do it manually.

Let me know what you find out!

Also for the crash about invalid core -- I made a small change. Can you test whether you still get a crash?

As for ContinuousMask, I fixed it to process all planes (instead of leaving garbage on chroma planes). It should behave more appropriately now. There really isn't much to debug in there... (https://github.com/mysteryx93/FrameRateConverter/blob/master/Src/Common/ContinuousMaskBase.cpp) Perhaps I could use a Threshold parameter so that it processes pixels above a certain value (other than 0), if it gets useful for anyone else.

[zorr]

You can compare to Avisynth StripeMask from before. Best I got so far is about 90% the quality of before for some reason.

Just to be clear - so this difference is not just between the Avisynth and Vapoursynth versions but with the current and previous Avisynth versions as well?

This is time-consuming trial-and-error task that requires a lot of testing.

Sounds like a perfect job for Zopti. Since we have a good result (the old Avisynth version) we can compare the result with it, I think a simple diff will do instead of some fancy SSIM or GMSD metric. If you want even better results you could create a mask manually (even for just one frame).

I'll try to come up with a Zopti script and do some testing. First I will need to recreate the good result and save it as lossless video.

[MysteryX]

Previous version was using ColorYUV to change gamma. It was never converting to Gray8 and was only using the Luma plane, so that's the equivalent of ShufflePlanes. I don't know whether there are other "unknown" factors or errors coming into play.

[zorr]

Is this the correct code for the good result (using FrameRateConverter v1.3)?

src = FFVideoSource("MotionTest2.mp4")
src = src.BicubicResize(src.width / 2, src.height / 2)
mask = src.StripeMask(thr=23)
return mask

[zorr]

The VapourSynth version is returning a mask with two gray values, is that expected?

https://i.postimg.cc/hvY5RgyN/vs-stripemask.png

The Avisynth version returns a mask with luma 0 or 255, VS version returns a mask with luma values 0, 100 or 200.

EDIT: Ah, that's controlled by the parameters str and strf. So I guess we should use the default values to match the Avisynth version?

[zorr]

StripeMask now has parameter range. What's it for and what is the usable value range? Should it always be max_out - max_in?

[MysteryX]

The lesser-gray is adding the next frame into it. Temporal stabilizer. Str sets the color of matching areas, strf(orward) sets the color of next frame data.

BTW......... could I improve performance 2x by caching the data of next frame instead of calculating it twice? I guess so. Or perhaps initialize the plugin twice with 1-frame offset and merge them.

Btw reducing size by half is only for performance reason because I don't need the clip to be that big; can reduce it to various sizes to test with various block sizes. Also see the impact of processing on half-resolution or quarter-resolution vs full-resolution (with half or quarter block sizes).

Range. If you want Full range, set 255. If you want PC range, set 220. To further compress the data range, I've set it to 125. Note that if skip=1, range isn't doing anything because it's only used for pre-filtering.

The ZIP contains the VapourSynth script that is working good,


import os
import sys
dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(dir)

import vapoursynth as vs
core = vs.get_core()

core.std.LoadPlugin(path=dir + r"\FrameRateConverter-x64.dll")
file="MotionTest2.mp4"
video = core.ffms2.Source(source=file, format=vs.YUV420P8)
video = video.resize.Bicubic(video.width / 2, video.height / 2)

video = core.std.ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY)
video = video.std.Levels(gamma=1/2.2, min_in=16, max_in=235, min_out=65, max_out=190) # range=125
video = core.frc.StripeMask(video, blksize=16, str=200, strf=100, thr=15, range=125, skip=1)

video.set_output()


btw I'm curious what would be the effect of sharpening.

[zorr]

I've got some interesting results.

Short version: You can use thr=23, min_out=0, max_out=208 and gamma=1/2.078 to get a pretty good result. I measured similarity against the older Avisynth version and average SSIM is 0.987 per frame (1.0 is perfect score).

Long version: Naturally I used Zopti to find the optimal values. First I needed a "reference video" so I ran StripeMask with FrameRateConverter v1.3 and saved the resulting mask as lossless video. I also created a pre-scaled lossless version of the original video to avoid unnecessary work during optimization.

I created this VapourSynth / Zopti script:

import os
import sys
from zoptilib import Zopti

dir = os.path.dirname(os.path.abspath(__file__))
sys.path.append(dir)

import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(path=dir + r"\FrameRateConverter-x64.dll")

zopti = Zopti(r'results.txt', metrics=['ssim', 'time'])

file="MotionTest2_scaled.avi"
video = core.ffms2.Source(source=file, format=vs.YUV420P8)

goodmask = core.ffms2.Source(source="StripeMask_good.avi")
goodmask = core.std.ShufflePlanes(goodmask, planes=0, colorfamily=vs.GRAY)

gamma = 220/100.0 # optimize gamma = _n_/100.0 | 1..300 | gamma
min_out = 65 # optimize min_out = _n_ | 0..100 ; max:max_out 1 - | min_out
max_out = 190 # optimize max_out = _n_ | 100..255 ; min:min_out 1 + | max_out
thr = 15 # optimize thr = _n_ | 1..100 | thr

video = core.std.ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY)
video = video.std.Levels(gamma=1/gamma, min_in=16, max_in=235, min_out=min_out, max_out=max_out) # range=125
video = core.frc.StripeMask(video, blksize=16, thr=thr, range=max_out-min_out, skip=1)

frame = 20
video = video[frame]
goodmask = goodmask[frame]

zopti.run(goodmask, video)

So there are four optimized parameters: gamma, min_out, max_out and thr. I didn't include min_in or max_in because those should be correct for this video.

Gamma range is 1 - 300 divided by 100 so effectively 0.01 - 3.0 in steps of 0.01. max_out and max_in have dependencies, max should always be larger than min so I set a min limit for max_out and max limit for min_out.

I only measured SSIM on one frame to save time and just to see if even one frame can be matched well (if we can't do that then doing it for the whole clip is hopeless). The frame number 20 contained interesting mask features so that's the one I chose. I later found out that if the result matches well with this one frame it also matches well with the rest of the frames.

I ran a couple of Zopti tests with
zopti StripeMaskTest.vpy -alg mutation -iters 1000 -pop 48 -threads 6 and then set the iterations to 10000 and population to 30. With VapourSynth you don't need that many threads to saturate all cores, with Avisynth I usually run -threads 16 or 24 (and 16 usually only when there isn't enough memory to run more).

The best result was 0.9778275 so already quite promising and I verified visually that it's pretty close. You can output the best result script with zopti -mode evaluate -scripts best

Next I wanted to see some heatmaps to better understand where the good values are. The first one was for max_out and thr.

https://i.postimg.cc/rpL5Qbf9/Stripe-Mask-Test-ex-thr-max.png

There's a nice linear area of good values. In theory you could find a good match with any thr or max_out value as long as you adjust the other one. The red dot is the best found result.

The next one is min_out and max_out.

https://i.postimg.cc/d0v4LLcv/Stripe-Mask-Test-ex-min-max.png


This revealed to me what you probably already knew - that the absolute values of min_out and max_out don't matter, only their distance (which is the range). Ok, this means I can run the next tests with min_out=0 and leaving only gamma, max_out and thr to optimize.

What about gamma + min_out?

https://i.postimg.cc/PqtQd915/Stripe-Mask-Test-ex-gamma-min.png

That's odd... there's kind of checkerboard pattern going on. Changing gamma slightly makes the result go up and down. Let's take a closer look at the gamma alone:

https://i.postimg.cc/m2YdRQKV/Stripe-Mask-Test-ex-gamma-2021-07-05-01-56-46-optimize-exhaustive-run-01.png

Oh wow, no wonder you had trouble finding a good match, the gamma behaves quite erraticly! I wonder what's going on, is the up-down swing also there in the brightness of the image? I don't know yet.

Meanwhile I ran some tests with 100 000 iterations and using more focused search ranges. I decreased the gamma step to 0.001. I had trouble finishing those runs as the execution would sometimes stop with error. Once I got

Script exceeded memory limit. Consider raising cache size.
terminate called after throwing an instance of 'std::out_of_range'
what(): _Map_base::at
even though I had plenty of memory left (about 40 gigs). Usually the script just reports Output 1 frames in 0.62 seconds (1.61 fps) or similar. Zopti has a tendency to find rare bugs so this could be a sign of that.

Anyway, two complete and one almost-complete runs later they all found the best result 0.98015726 so I think that's as good as it gets. I think giving gamma even more decimals could improve things a bit so that's something I'll try.

I looked at some heatmaps of these 100000 iteration runs and they look like they're straight out of a horror film:

https://i.postimg.cc/zBfmHdYJ/Stripe-Mask-Test2-run-01-1.png

I have no idea why there are those vertical stripes but I know they use different thr than the best found value. First I thought they are just some randomness but the stripes were found in every one of the three runs at the same locations. Investigation continues...

[MysteryX]

If you took the v1.3 results as reference, those aren't perfect at all. Perhaps you could manually draw the areas in Photoshop as a perfect reference mask, and see how close the tests get. Comparing to v1.3, there are random fluctuations of areas that are included or excluded that would randomize the test results. You need a fixed and clean reference.

As for performance, it could be optimized by doing this in a script function:
- Create 2 clips with 1-frame offset
- Output with 255 (pure black & white)
- Apply each mask over a blank clip of the desired colour
- Merge the 2 masks together, taking the highest value of each clip

Could someone write that function? It should double the performance.

Did you try sharpening just out of curiosity? Anything else that could be tried?

P.S. your last test, it's a galactic war between two fleets. The Dark Fleet tried to take over the galaxy.

The gamma could be altering the way stripe patterns are being detected or not past certain thresholds. Your white bars kind of look like the stripes we're trying to detect. If playing with the gamma considerably alters the results, there's also the option of running the mask different times with various gamma values and merging them. Or a parameter: run with 1, 2 or 3 passes.

[wonkey_monkey]

Oh wow, no wonder you had trouble finding a good match, the gamma behaves quite erraticly!

Rounding errors/artefacts?

[MysteryX]

This function gives just over 25% performance gain with strf

def StripeMask2(clip, blksize = None, blksizev = None, overlap = None, overlapv = None, thr = None, range = None, comp = None, compv = None, str = None, strf = None, lines = None, skip = None):
if not isinstance(clip, vs.VideoNode):
raise vs.Error('StripeMask: This is not a clip')

mask = clip.frc.StripeMask(blksize=blksize, blksizev=blksizev, overlap=overlap, overlapv=overlapv, thr=thr, range=range, comp=comp, compv=compv, str=str, strf=0, lines=lines, skip=skip)
if strf > 0:
maskf = mask.std.DeleteFrames(frames=[0]).std.DuplicateFrames(frames=[clip.num_frames-2])
return core.std.Expr(clips=[mask, maskf], expr=f"x {str} y {strf} 0 ? ?")
else:
return mask

[zorr]

I ran one more test to see how much even more decimals for gamma helps. Not much, best result was 0.98015726 with gamma 1/2.0778.

Rounding errors/artefacts?

I decided to take a closer look at how the gamma behaves. This is the average brightness of the source frame 20 with gammas between 1/2.05 and 1/2.1.

https://i.postimg.cc/Bnv8hB7h/Brightness-Test-2021-07-07-20-45-16-optimize-exhaustive-run-01.png

So while it's not buttery smooth it doesn't look like it's doing anything radical. If we draw a straight line from gamma 1/2.05 to gamma 1/2.1 and calculate the offset it looks like this:

https://i.postimg.cc/jdM2JJqy/Brightness-Test-nonlinear-2021-07-07-20-14-42-optimize-exhaustive-run-01.png

Not pretty but the errors are small.

If you took the v1.3 results as reference, those aren't perfect at all. Perhaps you could manually draw the areas in Photoshop as a perfect reference mask, and see how close the tests get.


Yes this is just a starting point, at least we now know that there is nothing fundamentally wrong with the VS version as it's capable of emulating the Avisynth version.

Doing one or more masks manually and letting Zopti play with the parametes is probably a good next step. With Zopti you can develop algorithms on the idea level not having to bother with the gruesome testing of whether this or that idea actually delivers. Of course setting up the test scripts and running them still takes time but it's still tremendously helpful.

Comparing to v1.3, there are random fluctuations of areas that are included or excluded that would randomize the test results. You need a fixed and clean reference.

Yes there are some differences, curiously the VS version usually makes the mask one pixel wider than Avisynth. I'm not sure what you mean by fixed and clean reference. Using the same clip and frame counts as fixed, right?

Did you try sharpening just out of curiosity? Anything else that could be tried?

Didn't try sharpening, that (and blurring) should be tested as well. My first idea on how to improve the mask was to remove the smaller areas using the RemoveSmallSpots-function I used with Delta Restore (https://forum.doom9.org/showthread.php?p=1944708#post1944708). And likewise fill the small holes of the mask using a similar FillSmallHoles -function. I also tried offsetting the image half blksize and taking another StripeMask and combining it with the original. This seems to stablilize the mask somewhat just like using strf. Here's the script (including aforementionded functions) I played with:


src = FFVideoSource("MotionTest2.mp4")
src = src.BicubicResize(src.width / 2, src.height / 2)

blksize = 16
thr=20
offset = 8
spotSize = 14
holeSize = 20
finalHoleSize = 22

src2 = src.AddBorders(offset,0,0,0)

mask = src.StripeMask(thr=thr, blksize=blksize)
mask2 = src2.StripeMask(thr=thr, blksize=blksize)
mask2 = mask2.Crop(offset,0,0,0)
mask2 = mask2.RemoveSmallSpots(spotSize).FillSmallHoles(holeSize)

mask = mask.RemoveSmallSpots(spotSize).FillSmallHoles(holeSize)
mask_final = mt_logic(mask, mask2, "and")
mask_final = mask_final.RemoveSmallSpots(spotSize).FillSmallHoles(finalHoleSize)

return src.Overlay(mask_final, opacity=0.5)


function FillSmallHoles(clip mask, int holeSize) {
return mask.mt_invert().RemoveSmallSpots(holeSize).mt_invert()
}

function RemoveSmallSpots(clip mask, int spotSize) {
mask_orig = mask

# remove too small spots
for (i = 1, spotSize) {
mask = mask.mt_inpand()
}
mask = mt_hysteresis(mask, mask_orig, chroma="-128")
return mask
}

Perhaps the area around the found mask should be allowed to match the stripes with a lower threshold, that way some of those cracks in the mask could be filled. This can be implemented by running another StripeMask with lower threshold and anding it with the original which has been expanded about half blocksize.


P.S. your last test, it's a galactic war between two fleets. The Dark Fleet tried to take over the galaxy.

:D

there's also the option of running the mask different times with various gamma values and merging them. Or a parameter: run with 1, 2 or 3 passes.

Yes, good ideas that could be tested.

[MysteryX]

Yes there are some differences, curiously the VS version usually makes the mask one pixel wider than Avisynth.

Mask areas are wider, or the output clip is wider?


I'm not sure what you mean by fixed and clean reference. Using the same clip and frame counts as fixed, right?

I mean, creating a mask manually that covers 100% of the right areas with no false positive.


I also tried offsetting the image half blksize and taking another StripeMask and combining it with the original. This seems to stablilize the mask somewhat just like using strf.

strf stabilizes in time (after all, the mask hides all generated frames between this frame and the next), whereas this will help fill spacial holes. It's actually a really good idea! Cutting the image in half, and working on a single grey plane, the performance hit will be minimal.

Small spots will be removed by ContinuousMask. mt_expand fills small holes.

[zorr]

Mask areas are wider, or the output clip is wider?

The mask, and just the right edge. It's actually two pixels, and sometimes the bottom edge is one pixel wider. Like this:

https://i.postimg.cc/26HyzjYm/vs-avisynth-mask-comparison.gif

I mean, creating a mask manually that covers 100% of the right areas with no false positive.

Oh yes. And that's what I did. Well, perhaps not 100% right areas as that depends on how MaskTools sees the frame (where the stripes are causing trouble). But I did roughly select the fence, if StripeMask can do that it's going to be a significantly improved result. I chose 4 frames, numbers 5, 20, 32 and 72. This is the mask in frame 5:

https://i.postimg.cc/8C4M6SGX/ref-mask-05.png

I've started running Zopti on this new challenge, first using the same three params gamma, max_out and thr. Initial results indicate that this is a much harder problem.

It's actually a really good idea! Cutting the image in half, and working on a single grey plane, the performance hit will be minimal.

Not sure I follow but I'm glad I was able to help! :)

Small spots will be removed by ContinuousMask. mt_expand fills small holes.

Ok, we just have to test the ContinuousMask as well. Actually we should follow the whole procedure in the .avsi and include new parameters from there.

mt_expand will also make the mask larger, if you only want the holes filled then it must be combined with mt_hysteresis. But that comes with a performance penalty (I haven't measured but since mt_hysteresis is a fill algorithm it's probably not super fast) and might not be worth it.

[MysteryX]

The mask, and just the right edge. It's actually two pixels, and sometimes the bottom edge is one pixel wider. Like this:
Is that between VapourSynth and Avisynth or between v1.3 and v2.0? I did fix a bug where sometimes the area to mark wasn't closed properly and a huge bar would appear; so I did a change to the closing of the masked area. That could be the pixel difference.

I'm not sure it's worth testing the whole procedure. The rest of the AVSI code is to give the shape I want for a mask. I do want it larger so that artefacts are properly hidden.

What I really need is just the right thr, range and gamma parameters. If doing 2 or 3 passes with different values gives a better mask, then need to find which set of params to use. Also do these parameters need to be adjusted for various videos, or are the right values right for everything?

We're better to include more and have a bit more false positive here; the small spots get removed with ContinuousMask, and the most important is for stripe areas to definitely be masked because they're UGLY otherwise.

In terms of fine-tuning the analysis, I think we've got enough data to implement something.

Unless someone wants to write machine-learning AI for stripe detection? If it's not over-kill.

[zorr]

Is that between VapourSynth and Avisynth or between v1.3 and v2.0? I did fix a bug where sometimes the area to mark wasn't closed properly and a huge bar would appear; so I did a change to the closing of the masked area. That could be the pixel difference.

Yes between v1.3 and 2.0. And that's a very logical explanation.

What I really need is just the right thr, range and gamma parameters.

Here's some results, using the default StripeMask parameters except for thr. The best SSIM found was 0.3176762 so much lower than before, especially since the maximum possible is now 4.0. I did two 100k iteration runs.

This is the best mask for each of the four test frames (target is overlaid in red).

https://i.postimg.cc/137dwDk2/Stripe-Mask-Test4-PARETO-01-0-3176762-470-vpy-0.png

The best found combo was gamma 1/1.811, max_out 247 and thr 16.

I'm leaving for a little trip for a couple of days, I'll adress the rest of your comments after that. :)

[zorr]

I'm not sure it's worth testing the whole procedure. The rest of the AVSI code is to give the shape I want for a mask. I do want it larger so that artefacts are properly hidden.
...
We're better to include more and have a bit more false positive here; the small spots get removed with ContinuousMask, and the most important is for stripe areas to definitely be masked because they're UGLY otherwise.

Perhaps the whole procedure is not needed but anything that ends up doing more than just cosmetic changes (like blurring) to the final mask should be included, otherwise we're measuring the wrong thing. For example if the ContinuousMask is left out then we won't know how much larger the mask needs to be for it to be optimal for ContinuousMask.

I actually tried ContinuousMask but seems like it's not designed to remove large areas completely. Then I did some tests with RemoveSmallSpots to see how including it will change the optimal gamma, max_out and thr. The best result was 0.32462016 with gamma=1/1.763, max_out=225, thr=13 and spotsize=20. So removing spots afterwards seems to change the optimal values.

Here's what the optimal mask looks like with RemoveSmallSpots:

https://i.postimg.cc/TwvF7FMZ/Stripe-Mask-Test5-PARETO-01-0-32462016-640-vpy-0.png

It looks much cleaner than the previous attempt.

After this I also wanted to see if changing the defaults of parameters blksize, overlap and comp will give better results. This resulted in SSIM of 0.43589097 using gamma=1/1.345, max_out=230, thr=14, blksize=27, overlap=26, comp=5 and spotsize=38.

Something I observed: optimal overlap is always the largest possible (blksize-1) and optimal comp is the largest supported 5.

After that I also included filling the small holes in the mask using FillSmallHoles and the best result was 0.45771345 using gamma=1/1.034, max_out=204, thr=9, blksize=24, overlap=23, comp=5, spotsize=61 and holesize=43.

But while the SSIM is now much better the mask looks way too large, it seems SSIM is not measuring this the way we want:

https://i.postimg.cc/15LxTJFc/Stripe-Mask-Test8-PARETO-01-0-45771345-820-vpy-0.png

So I think the next step is to replace SSIM with something better. As you mentioned it's very important to cover all of the striped areas we are better off measuring that directly - how large percentage of the target area is covered. But in addition we need another measurement to determine how much larger the mask is than what is needed to cover the target (otherwise it would be easy to get perfect score by making the mask fill 100% of the screen), this one should be minimized. Zopti does multiobjective optimization by default so this is not a problem - the results will be a pareto front with increasing target mask cover and increasing overflow area and we just need to pick the one that is the best compromise. I have already added custom props to ZoptiLib so that the similarity metric(s) can be calculated in the script.

If doing 2 or 3 passes with different values gives a better mask, then need to find which set of params to use. Also do these parameters need to be adjusted for various videos, or are the right values right for everything?

These are also interesting to test and we'll get to those once the similarity metric has been fixed. We'll need some more test clips in order to determine if one set of values works well. I think one of those could be the one I'm doing MVTools tests with (https://forum.doom9.org/showthread.php?p=1941970#post1941970).

Unless someone wants to write machine-learning AI for stripe detection? If it's not over-kill.

I'd like to try frequency domain filters such as DeFreq to create a stripe mask. The filter is not meant for creating masks but the output can be turned into a mask quite easily.

[kedautinh12]

@zorr, this is FrameRateConverter.avsi support HBD
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi

[zorr]

Sorry about the delay, I now have some new results to share.

The latest incarnation calculates the percentage of covered mask in each of the four test frames and also the percentage of StripeMask covering the area outside of the target mask. The first property we'd like to maximize and the latter we'd like to minimize. I gave gamma two decimal accuracy in these tests (I also had to rerun some tests with extended gamma range as I noticed it maxed out the initial limits, now gamma range extends all the way to 1/20.00).

I first tested the basic arguments gamma, max_out and thr. The result after 312600 iterations (I used dynamic iteration count) was a pareto front with 516 different results (the red dots below).

https://i.postimg.cc/63qqzkVz/Stripe-Mask-Test13-2021-07-18-22-27-34-optimize-spea2-pop-100-mutcount-60-1-mutamount-0-3-0-01-cros.png

The maxCover means the percentage of target mask covered, and the maximum is 400 because we have 4 frames. The minOutside is the percentage of area outside of the mask, the max is 400 also.

The largest target percentage is about 354% but then also about 324% outside of the mask is selected which is way too much.

It's no fun checking each result one by one so I turned the results into a video, one result per frame. You can download it here (https://drive.google.com/file/d/1VsRYrt9E1mlU7lI3hiR1orArYoAg5JiB/view?usp=sharing). NOTE: the preview sucks, download the video for good quality.

Second test was running StripeMask twice and combining the masks. I tried combining them both as union and intersection and union turned out much better. The results are below:

https://i.postimg.cc/sxZXC5Nh/Stripe-Mask-Test12-2021-07-17-12-56-15-optimize-spea2-pop-100-mutcount-60-1-mutamount-0-3-0-01-cros.png

Results were better which was to be expected. Now the maximum coverage is about 387%. I have a comparison chart later in this post. Download a video of the results here (https://drive.google.com/file/d/1ckyzV6Tsdmgo1Agh7WennZ01E0JbE9FD/view?usp=sharing).

Next I tried adding the rest of StripeMask parameters blksize, overlap and comp but with a single StripeMask. Results were:

https://i.postimg.cc/NGW89DYZ/Stripe-Mask-Test15-2021-07-21-13-24-15-optimize-spea2-pop-100-mutcount-60-1-mutamount-0-3-0-01-cros.png

Video here (https://drive.google.com/file/d/1rlImnzezyQBM0EO1SY0PS9rHODUUwtrI/view?usp=sharing). This is already pretty good result, for example this is a result with about 356% mask covered and only 11% outside.

https://i.postimg.cc/zXvmvJzx/Stripe-Mask-Test15-x264-00.png

Finally I tried the previous StripeMask parameters and two masks combined.

https://i.postimg.cc/7ZNSRy7y/Stripe-Mask-Test16-2021-07-22-19-18-46-optimize-spea2-pop-100-mutcount-60-1-mutamount-0-3-0-01-cros.png

Video here (https://drive.google.com/file/d/1BL4BYMQCGSZzPj5JSodGv8P3R4YODIrc/view?usp=sharing). Results were still better but not by a whole lot.

Here are the first three runs together in the same chart.

https://i.postimg.cc/3NJb6ZfK/Stripe-Mask-Test-run-03-0.png

Pareto 0 is the basic run, pareto 1 added a second mask and pareto 2 has one mask but more StripeMask parameters.

And this is a comparison of the last two runs:

https://i.postimg.cc/s2dHyzHH/Stripe-Mask-Test-run-04-0.png

So adding another mask was not nearly as beneficial than when it was done with the basic parameters.

The results can be improved further for example by adding RemoveSmallSpots or ContinuousMask.

[sofakng]

Does anybody have an example of how I can use this with mpv + vapoursynth + SVP?

[Selur]

btw. any news regarding incoporating rife into FrameRateConverter?

[ChaosKing]

It looks like this could be the new standard in video interpolation:

https://github.com/uzh-rpg/rpg_timelens
https://www.youtube.com/watch?v=dVLyia-ezvo&feature=youtu.be
https://www.youtube.com/watch?v=G00A1Fyr5ZQ&feature=emb_title

[takla]

Cuda only. Yikes.

[real.finder]

Cuda only. Yikes.

it can be ported to c++ (for cpu and others) I think

[zorr]

That TimeLens-method relies on event data which is generated by custom hardware.

[Dogway]

As I see it it's only a dataset that you can use for AI based slomo splash sequences.

[takla]

That TimeLens-method relies on event data which is generated by custom hardware.

Imagine if in the future, streaming bandwidth for videos is cut in half or more, since frames will be reconstructed via hardware. (the stream provider sends an extra file for reconstruction to the client) simply amazing.

Eventually, I'd expect a similar thing to happen for resolution, too.

[Milardo]

Hi, does anybody have the current FrameRateConverter.avsi for the latest beta release? It only comes with vapoursynth script.

[kedautinh12]

Hi, does anybody have the current FrameRateConverter.avsi for the latest beta release? It only comes with vapoursynth script.

Still use old .avsi, or if you want use in HBD can use here
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverter.avsi

[MysteryX]

Sorry had been travelling 6 weeks across Mexico.

It looks like this could be the new standard in video interpolation:

https://github.com/uzh-rpg/rpg_timelens
https://www.youtube.com/watch?v=dVLyia-ezvo&feature=youtu.be
https://www.youtube.com/watch?v=G00A1Fyr5ZQ&feature=emb_title

So this method isn't applicable here, correct? Requires extra hardware while recording.

[MysteryX]

Zorr, very interesting research! But after all this, it's still not clear at all, what are the optimal parameters when using 1 mask, and when combining 2 masks?

RemoveSmallSpots is a great idea, but searching about it returns nothing. Where do you find that plugin?

I wouldn't use the max overlap for performance reasons. I don't expect much quality difference between half-block size and blksize-1. Setting Comp to 5? If it's consistently better, then perhaps. As for BlkSize, it entirely depends on the video size, and is set to the same as the motion estimation blksize. It's 8, 12, 16, 24, 32, 48 or 64. No point in testing that with other values. Actually no point in testing it here, because it should be set based on motion estimation results.

[StainlessS]

but searching about it returns nothing. Where do you find that plugin?
https://forum.doom9.org/showthread.php?p=1944487#post1944487

[zorr]

Zorr, very interesting research! But after all this, it's still not clear at all, what are the optimal parameters when using 1 mask, and when combining 2 masks?

There is no one optimal result. The pareto front is the set of optimal results, each better than the others in some way. You have to use your own judgement as to which one of them suits you best. For example if you think that it's acceptable to have no more than 5% error rate (at most 5% of area outside of the stripes selected) then you can look for the result with largest maxCover that has minOutside less or equal to 20 (400*0.05 = 20), that would be 367.02933 19.190443 gamma=364 max_out=143 thr=19 blksize=16 overlap=15 comp=5 gamma2=105 max_out2=219 thr2=44 blksize2=25 overlap2=24 comp2=5) in the previous results. But I think it's easier to look at the video with all the results and choose one that looks like a good compromise. Even better would be to test at least some of those results with FrameRateConverter and see the resulting video.

RemoveSmallSpots is a great idea, but searching about it returns nothing. Where do you find that plugin?

S*ssS already provided a link to it, it's from a larger set of mask filters called Delta Restore. I got the idea from some old message of Didee.

I wouldn't use the max overlap for performance reasons. I don't expect much quality difference between half-block size and blksize-1.

Performance was not the focus of these tests but it should be considered, of course. I could add the runtime as another variable to optimize but then the results would be a set of points in three dimensions so that would be even more difficult to decipher. Perhaps there could be a runtime limit, anything slower than x milliseconds would be rejected.

Some people don't care about the runtime that much, perhaps you could add a preset for those that want maximum quality?

Setting Comp to 5? If it's consistently better, then perhaps.

It seems to be a very popular choice among the pareto front. And its impact on performance can also be tested just like with the overlap.

As for BlkSize, it entirely depends on the video size, and is set to the same as the motion estimation blksize. It's 8, 12, 16, 24, 32, 48 or 64. No point in testing that with other values. Actually no point in testing it here, because it should be set based on motion estimation results.

I don't see where the motion estimation (or even the video size) enters the equation. StripeMask doesn't take any motion vectors as input so it doesn't care about those. I believe the optimal blksize depends on the general size of the stripe patterns in the video, in this video it's about 18. It remains to be seen what the optimal size is in other videos.

By the way I did another test where I compared ContinuousMask and RemoveSmallSpots, I'll post those results next. :)

[MysteryX]

This does give pretty good resuts!

video = frc.StripeMask2(video, blksize=16, str=200, strf=100, thr=17, range=205, gamma=2.87)


Setting Comp=5 makes the image a lot noisier; if set, it seems to move the pareto front and everything needs to be configured differently.

Default Overlap is 1/4 (4) of blksize. Setting it to half (8) does give better results. Setting it to 3/4 (12) gives even better results in hard-to-detect edge cases. Setting to 15 gives even better results indeed. Rarely makes much difference but edge cases are more clearly detected. Adjusting between 1/4, 2/4 or 3/4 of blksize might be a reasonable tuning.

[MysteryX]

First of all, here's a new build with Range and Gamma parameters. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta) I confirm: Range min doesn't matter, all that matters is the range between min and max, so we only need to specify RangeMax.

Second, there remains a difference between Avisynth and VapourSynth versions. VapourSynth converts to grey with ShufflePlanes, keeping the first plane intact. Avisynth uses ConvertToY. I believe it weights the colors to give a different output, considering red more visually important or something? Whatever it does, these 2 methods produce an output with a different brightness. Which will give inconsistent output between platforms. Which approach is best here, ShufflePlanes or ConvertToY? TODO: in VapourSynth, I'll need to convert to YUV in case it's in something else.

Third, I'm currently mostly testing VapourSynth. If you test in Avisynth with the current discrepancy, the results you'll get won't be fully valid for VapourSynth.

I've been having a hard time getting consistent results with the settings you came up with. Particularly the settings with weird extreme values, it works well in some places and less in others, thus I'd rather avoid extreme settings.

As for Gamma, higher (2.8) gives better output in some places... whereas lower (1.4) gives better output in other places, depending on the stripe shades. The very first frame has one part better defined with 2.8 and another part better defined with 1.4 ... so overall 2.2 is more consistent.

BlkSize depends on the particular video. The tests are for the general algorithm, not for a specific video, thus we can't include that in the variations. What I *could* do is merging 2 masks with different blksize so that patterns of various sizes get better detected.

Comp=5, from my tests, causes certain areas to be more clearly defined, whereas it misses other finer details. If merging 2 masks, perhaps it could be used with the larger block size.

I'm still unsure of the best settings to use, but I'm inclined to use gamma 1.8, 2.0 or 2.2 for single-pass and 1.4/2.8 for double-pass.

First need to deal with the inconsistency between ShufflePlanes and ConvertToY; how to do ShufflePlanes in Avisynth. Experts?

[Reel.Deel]

First need to deal with the inconsistency between ShufflePlanes and ConvertToY; how to do ShufflePlanes in Avisynth. Experts?

For regular 8-bit YUV you can use ConvertToY8/UToY8/UToY8/ to extract and YToUV (http://avisynth.nl/index.php/Swap) to merge. For 8-bit RGB there's ShowRed/Green/Blue (http://avisynth.nl/index.php/ShowAlpha) to extract and MergeRGB (http://avisynth.nl/index.php/MergeRGB) to merge. In AviSynth+ there's ExtractX/PlaneToY/ShowY/U/V (http://avisynth.nl/index.php/Extract) to extract and CombinePlanes (http://avisynth.nl/index.php/CombinePlanes) to merge. Anything in particular that you're trying to do?

[MysteryX]

For regular 8-bit YUV you can use ConvertToY8/UToY8/UToY8/ to extract and YToUV (http://avisynth.nl/index.php/Swap) to merge. For 8-bit RGB there's ShowRed/Green/Blue (http://avisynth.nl/index.php/ShowAlpha) to extract and MergeRGB (http://avisynth.nl/index.php/MergeRGB) to merge. In AviSynth+ there's ExtractX/PlaneToY/ShowY/U/V (http://avisynth.nl/index.php/Extract) to extract and CombinePlanes (http://avisynth.nl/index.php/CombinePlanes) to merge. Anything in particular that you're trying to do?

hum. There's no difference between ConvertToY and ExtractY, so that clarifies that. Now I'm comparing with ShufflePlanes, getting same output. Had issues in the past... not sure what it was. Must have been the TV-PC conversion methods that caused discrepancy.

In that case, VapourSynth and Avisynth should be consistent in theory. Haven't re-tested it yet.

[Reel.Deel]

hum. There's no difference between ConvertToY and ExtractY, so that clarifies that. Now I'm comparing with ShufflePlanes, getting same output. Had issues in the past... not sure what it was.

In that case, VapourSynth and Avisynth should be consistent in theory. Haven't re-tested it yet.

No difference in ConvertToY and ExtractY provided that the input clip is YUV, otherwise ConvertToY does an RGB to YUV conversion. ExtractY would just throw and error in that case.

Combined, Extract and CombinePlanes do everything that ShufflePlanes does. Although ShufflePlanes having arrays does make some task a bit easier, scripting wise.

[zorr]

I've been having a hard time getting consistent results with the settings you came up with. Particularly the settings with weird extreme values, it works well in some places and less in others, thus I'd rather avoid extreme settings.

That could very well be because only a few frames of one video was used for optimization - the settings will work very well for this video but probably not for others. Selur recently posted (https://forum.doom9.org/showthread.php?p=1951704#post1951704) a couple of interesting clips with stripe patterns - perhaps we should do another round of tests but take frames from multiple videos and see what settings work for all of them.

[MysteryX]

For Gamma, putting it higher or lower gets better results in specific scenes and worse in others -- so IMO linear light is best (2.2)

For Levels adjustment in Avisynth, I needed to add Coring=false. Without it, it was giving funky results rendering all the tests useless... I released a new version that sets coring=false. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

This is giving the best results so far... marginally better than in v1
video = frc.StripeMask2(video, blksize=16, str=200, strf=100, thr=26, range=241, gamma=2.2)

Now I'm still struggling with slight difference between Avisynth and VapourSynth.

This pre-processing should give the same output

#VapourSynth
video = core.std.ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY)
video = video.std.Levels(gamma=1.0 / 2.2, min_in=16, max_in=235, min_out=0, max_out=241)

#Avisynth
ConvertToY()
Levels(16, 1.0 / 2.2, 235, 0, 241, coring=false)


It's *almost* the same, brightness is the same, but white contrasts are different for some reason. If I copy/paste both clips into a photo editor, then the image looks the same, making me think it could be just an interpretation issue, but I manually set both to full-range Rec.601 in VirtualDub and still see a difference. Then when I run StripeMask, the output is slightly different. What could it be?

Finally, if doing 2-pass detection, to set a different block size for the 2nd pass... are the missing details generally smaller or larger stripes? Could multiply blksize by 1.25 or 1.5 for a 2nd pass. If setting comp=5, then larger block size probably makes sense. Unless it's more struggling with small details.

[poisondeathray]

Now I'm still struggling with slight difference between Avisynth and VapourSynth.

This pre-processing should give the same output

#VapourSynth
video = core.std.ShufflePlanes(clips=video, planes=0, colorfamily=vs.GRAY)
video = video.std.Levels(gamma=1.0 / 2.2, min_in=16, max_in=235, min_out=0, max_out=241)

#Avisynth
ConvertToY()
Levels(16, 1.0 / 2.2, 235, 0, 241, coring=false)


It's *almost* the same, brightness is the same, but white contrasts are different for some reason. If I copy/paste both clips into a photo editor, then the image looks the same, making me think it could be just an interpretation issue, but I manually set both to full-range Rec.601 in VirtualDub and still see a difference. Then when I run StripeMask, the output is slightly different. What could it be?


I get the same Y levels for both, checked with makediff . At first with colorbars, but also on a 0-255 8it420 perfect gradient

You can import avs script into vpy script using core.avisource.AVISource , or import vpy script into avs script using VSImport


clip = core.avisource.AVISource(r'8bit420_greyscale_gradient_256x256_utvideo.avi', pixel_type="YV12")
vpylevels = core.std.ShufflePlanes(clip, planes=0, colorfamily=vs.GRAY)
vpylevels = core.std.Levels(vpylevels, gamma=1.0 / 2.2, min_in=16, max_in=235, min_out=0, max_out=241)

avslevels = core.avisource.AVISource(r'levels_grad.avs')

d = core.std.MakeDiff(avslevels, vpylevels, planes=[0])
da = core.std.Levels(d, min_in=127, max_in=129, gamma=1, min_out=0, max_out=255, planes=[0])
da.set_output()

da.set_output()


where levels_grad.avs is


AVISource("8bit420_greyscale_gradient_256x256_utvideo.avi", pixel_type="YV12")
ConvertToY()
Levels(16, 1.0 / 2.2, 235, 0, 241, coring=false)


I can upload test video if you want
https://www.mediafire.com/file/0nk29m6sqnvak55/8bit420_greyscale_gradient_256x256_utvideo.avi/file

[MysteryX]

Considering copy/pasting it into a photo editor removes the visual difference, I can assume it's just a display issue?

If that's the case, what else could be the difference? The platform-specific code is very simple. It only does the range/gamma conversion and then calls shared code. VapourSynth (https://github.com/mysteryx93/FrameRateConverter/blob/master/Src/VapourSynth/StripeMaskVpy.cpp) and Avisynth (https://github.com/mysteryx93/FrameRateConverter/blob/master/Src/Avisynth/StripeMaskAvs.cpp) code. Nothing in there to justify any difference... Any idea?

[poisondeathray]

How are you viewing the "output" ? What application ?

Exactly what you do mean by "white contrasts are different" ? How are they different ?

Copy/pasting into photo editor means you've converted to RGB somewhere, so that introduces other variables, including how you converted it. Limited range equations conversion of YUV to RGB will clip data and all values Y <16, >235 will look the same in a photo editor.

MakeDiff compares Y channel directly, it should be more reliable , or at least is a direct measure of Y

[MysteryX]

I'm using VirtualDub for both Avisynth and VapourSynth. Ctrl+1 copies the output to clipboard.

Here's the full code for both.

LoadPlugin("FrameRateConverter.dll")
file="MotionTest2.mp4"
LWLibavVideoSource(file, cache=False)
ConvertToYV12(matrix="Rec709")
StripeMask(blksize=16, str=200, strf=100, thr=26, range=241, gamma=2.2)


import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
import FrameRateConverter as frc
import vapoursynth as vs
core = vs.get_core()

core.std.LoadPlugin(path=r'FrameRateConverter.dll')
file=r"MotionTest2.mp4"
video = core.ffms2.Source(source=file, format=vs.YUV420P8)
video = video.std.SetFrameProp("_FieldBased", intval=0)
video = video.std.SetFrameProp("_ColorRange", intval=0)
video = core.frc.StripeMask(video, blksize=16, str=200, strf=100, thr=26, range=241, gamma=2.2)
video.set_output()

I tried with a different video file that is more standard, the difference remains.

[poisondeathray]

I'm using 2.0 beta-6

The stripemask shape is slightly different between avs and vpy, but for the most part the corresponding values are the same. Is the shape difference what you are referring to ?

I used one of Selur's test clips, "Fensterladen.mkv" and used Lsmash for both avs and vpy to be consistent
https://drive.google.com/drive/folders/1hG4-vEDn4l_B0QTlsrUnyl8JrOAuRgBy


You can view pixel values with vapoursynth editor color picker; eg. so if G=200 in the vpy version, you can see that the corresponding "block" in the avs version is G=200

a.avs

LWLibavVideoSource("Fensterladen.mkv")
StripeMask(blksize=16, str=200, strf=100, thr=26, range=241, gamma=2.2)


vpy

import FrameRateConverter as frc
import vapoursynth as vs
core = vs.get_core()

video = core.lsmas.LWLibavSource(r'Fensterladen.mkv')
video = video.std.SetFrameProp("_FieldBased", intval=0)
video = video.std.SetFrameProp("_ColorRange", intval=0)
video = core.frc.StripeMask(video, blksize=16, str=200, strf=100, thr=26, range=241, gamma=2.2)

avs = core.avisource.AVISource(r'a.avs')

stack = core.std.StackHorizontal([avs, video])

stack.set_output()



The mask values seem binarized to 100 or 200 (or 0)

This is what the output of that stack script looks on frame 66
https://postimg.cc/jD6NvnBD

[MysteryX]

Ya. Output is similar but different. What am I missing?

[poisondeathray]

Ya. Output is similar but different. What am I missing?

Can you clarify what you're referring to specifically ?

Did you mean the difference in the general mask (areas included/excluded) ?

Or did you mean "It's *almost* the same, brightness is the same, but white contrasts are different for some reason" ?

Because I don't see the difference in brightness, and the gradient ramp test should detect that if there was a fundamental math difference between "levels" filter in avs vs. vpy

[Selur]

btw. I added another test clip called 'bicycle wheel' which shown a spinning animated wheel that RIFE with it's anime-Model can handle fine, but from the looks of it FrameRateConverter can't handle (just inserts dublicates).
->setting dctRe=2 helps, a lot, but result is still behind RIFE.

---

Also when using Vapoursynth version with 'stp=False':
# Imports
import os
import sys
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
import FrameRateConverter
# source: 'C:\Users\Selur\Desktop\bicycle wheel.mp4'
# current color space: YUV420P10, bit depth: 10, resolution: 840x760, fps: 29.97, color matrix: 709, yuv luminance scale: limited, scanorder: progressive
# Loading C:\Users\Selur\Desktop\bicycle wheel.mp4 using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/bicycle wheel.mp4", format="YUV420P10", cache=0, fpsnum=30000, fpsden=1001, prefer_hw=0)
# making sure input color matrix is set as 709
clip = core.resize.Bicubic(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting color space from YUV420P10 to YUV444P8 for vsFrameRateConverter
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# adjusting frame count&rate with FramerateConverter
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="anime", blkSize=8, blkSizeV=8, maskThr=40, maskOcc=45, skipThr=25, stp=False, dctRe=1) # new fps: 59.94
# adjusting output color from: RGB24 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="709", range_s="limited")
# set output frame rate to 59.940fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Output
clip.set_output()
it fails with:
Failed to evaluate the script:
Python exception: local variable 'EMstp' referenced before assignment

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2242, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 2243, in vapoursynth.vpy_evaluateScript
File "E:\Temp\tempPreviewVapoursynthFile15_54_12_915.vpy", line 31, in <module>
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="anime", blkSize=8, blkSizeV=8, maskThr=40, maskOcc=45, skipThr=25, stp=False, dctRe=1) # new fps: 59.94
File "I:\Hybrid\64bit\vsscripts\FrameRateConverter.py", line 264, in FrameRateConverter
EMstp = havs.ChangeFPS(EMstp, newNum, newDen)
UnboundLocalError: local variable 'EMstp' referenced before assignment

Cu Selur

Ps.: Also added "jump_1440x1080i" QTGMC bob brings it to 50fps using RIFE looks fine, haven't found good values for FrameRateConverter so far.
PPs.: Is there any downside to use dctRec=2 and dec=2 aside from the speed drop? (haven't seen any example where it had a negative effect on the results, only none or positive effects)

[MysteryX]

Can you clarify what you're referring to specifically ?

Did you mean the difference in the general mask (areas included/excluded) ?

Or did you mean "It's *almost* the same, brightness is the same, but white contrasts are different for some reason" ?

Because I don't see the difference in brightness, and the gradient ramp test should detect that if there was a fundamental math difference between "levels" filter in avs vs. vpy
You're seeing that the input (after pre-processing) of StripeMask is the same for both yet the mask output is different. That's the problem. Most of the code is shared except pre-processing and parsing parameter values. Unless there's a bug in the way parameters are passed in either version?

Selur, the script isn't finished yet.

[MysteryX]

An idea that just came to my mind. Instead of just saying "values above thr will have value 200", what if I say "values between thr1 and thr2 will have values between 0 and 200"? Let's say thr1=17 thr2=20, value 17 gives mask 50, 18 gives 100, 19 gives 150 and 20 gives 200. Would that better represent details or not? Will have to think of how the lighter tones would affect the overall mask.

EDIT: Never mind this makes no sense. Threshold is to determine contrast lines; and from those lines, I detect patterns.

[MysteryX]

Found the problems and released beta 7. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

In Avisynth, the new Gamma parameter was assigned to an int instead of double, and Coring wasn't passed to Levels because although the function had 7 parameters defined, I was then passing only 6 parameters to the call.

Now both versions produce the same output :D

[MysteryX]

Released beta 8. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

Renamed StripeMask to StripeMaskPass and removed strf parameter.

VapourSynth script now contains function StripeMask(blksize, blksizev, str, strf). I think you'll be pleased with the results! It kicks v1 out of the water. Thanks Zorr for your research.


blksizev = blksizev or blksize
mask1 = clip.frc.StripeMaskPass(blksize=blksize, blksizev=blksizev, overlap=blksize//2+1, overlapv=blksizev//2+1, thr=29, range=241, gamma=2.2, str=str)
blksize *= 1.25
blksizev *= 1.25
mask2 = clip.frc.StripeMaskPass(blksize=blksize, blksizev=blksizev, overlap=blksize//2+1, overlapv=blksizev//2+1, thr=42, range=214, gamma=2.2, comp=5, str=str)


Perhaps it can be further tweaked, but it's working pretty well. Now need to look at the mask post-processing...

The changes will break the Avisynth script. That will be released later.

EDIT: updated the beta 8 package with proper post-processing. using output="stripe" now gives pretty good results!

[MysteryX]

I'm having an issue with VapourSynth distorting the colors; not related to my plugin, but simply loading the video clip. Compared to playing the source video, it's fine in Avisynth, but the color matrix (?) is wrong in VapourSynth but even setting _Matrix property doesn't fix it. Both of these don't work correctly. What a I missing?


video = core.ffms2.Source(source=file, format=vs.YUV420P8)
video = core.lsmas.LWLibavSource(file, cache=False)


Btw where is LSMashWorks syntax for VapourSynth documented? It took me a long while to find the correct function and had to dig through source code.

[ChaosKing]

Just for the function names (parameters will follow soon™) vsdb can be helpful https://vsdb.top/plugins/lsmas
Readme: https://github.com/AkarinVS/L-SMASH-Works/tree/master/VapourSynth

[kedautinh12]

For new ver here:
https://github.com/AkarinVS/L-SMASH-Works/blob/master/VapourSynth/README

[Selur]

I always use something like:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# source: 'G:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="G:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_6c441f37d9750b62d59f16ecdbd59393_853323747.ffindex",format=vs.YUV420P8,alpha=False)

# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
to make sure color matrix, luma range and fps are set to the values I expect.

[MysteryX]

I see my _ColorRange was set to 0 instead of 1, now it opens fine. Great, now it shifts the colours again when I run FrameRateConverter(output="flow") ...

[Selur]

Using 2.0beta8 and:
# Imports
import os
import sys
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
import FrameRateConverter
# source: 'G:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="G:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_6c441f37d9750b62d59f16ecdbd59393_853323747.ffindex",fpsnum=25,format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting frame count&rate with FramerateConverter
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8, output="flow") # new fps: 50
# adjusting output color from: RGB24 to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 50.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=50, fpsden=1)
# Output
clip.set_output()
colors dont' seem shifted here.

[MysteryX]

I'm looking at Dogway's version. Output="diff" shows nothing. Masks are very different than they should be and the output is completely different. Output hasn't been compared with the base script for equivalence.

v2.0 is nearly working.

Quick question. How do I access the clip's _Matrix property (and format) to convert it back and forth correctly?

video = video.resize.Bicubic(format=vs.RGBS)
video = video.rife.RIFE(uhd=False)
video = video.resize.Bicubic(format=vs.YUV420P8, matrix_s="709")


and...... RIFE doesn't have any parameter to specify the frame rate??

[MysteryX]

Beta 9. Looks functional! (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

Test it and let me know if you find problems. It contains both VapourSynth and Avisynth scripts.

TODO:
- implement RIFE interpolation with format conversion to RGBS back-and-forth, then you can call FrameRateConverter(rife=True)
- add fullRange parameter in the Avisynth version.

[Dogway]

I'm waiting on 2.0 so I can finish my mix mod.
I normally compare the outputs to the T so they match, and on any bitdepth, this has been so with ExTools and other mods.
As for FrameRateConverter there were a few odd things:

Was this really necessary?
.mt_expand(mode= mt_circle(zero=true, radius=8))

This is utterly slow, even in ExTools is slow so I used an approximation, so yes is not mathematically exact, if it was me personally I would resize down (further) and use rad=4.

Another thing, FRC_GaussianBlur42(2.8).
Again the concept is good but the values represent nothing, you are blurring on the downscaled clip and hence breaking the blur linearity.
I made ex_GaussBlur() based on this, removed the explicit blur() and parametrized rad to respond to an approximated gaussian standard deviation, so again not mathematically exact to your output, but things make more sense. If you want to be mathematically accurate to a gaussian use vsTCanny() in mode=-1, but it will never match FRC_GaussianBlur42().

Other than that I don't recall changing anything else.

[poisondeathray]

How do I access the clip's _Matrix property (and format) to convert it back and forth correctly?



format=clip.format

Not sure about matrix, but doesn't really matter, because you're using the same one both ways . So if you randomly choose "709" it "reverses" when you go back and forth. Also if input clip is "unspec" or unassigned, it might cause problem so just picking one should work



RIFE doesn't have any parameter to specify the frame rate??

RIFE only does powers of 2, you need to also use mvtools2 or other for intermediate rates

There was talk on the official implementation page of eventually including other rates, but it hasn't materialized yet

[MysteryX]

Dogway, I'm getting entirely different mask output -- doesn't even compare. Wrong dependency perhaps?


FrameRateConverter(60, preset="slower", blkSize=16, output="auto", debug=True)


Using
FrameRateConverter-Dogway 1.4.2 (28-06-2021)
ExTools v2.7 (01-07-2021)
Utils-r41 v0.41 (2017-05-13)

RIFE only does powers of 2, you need to also use mvtools2 or other for intermediate rates

There was talk on the official implementation page of eventually including other rates, but it hasn't materialized yet
...

Well that's a "slightly" limiting factor.

Still, could very well double the frame rate before using the exact same frame blending.

[kedautinh12]

Why you don't test new ver??
Frame Rate Converter mix mod Version: 1.5.0 (06-09-2021)
ExTools v5.8 (18-09-2021)

[poisondeathray]

RIFE only does powers of 2, you need to also use mvtools2 or other for intermediate rates

There was talk on the official implementation page of eventually including other rates, but it hasn't materialized yet




Well that's a "slightly" limiting factor.

Still, could very well double the frame rate before using the exact same frame blending.

It's still beneficial in scenes where RIFE works well. I posted some examples, such as 23.976=> 47.952 RIFE => 59.94 MVTools2. The "distance" that mvtools2 has to interpolate in the 2nd step is smaller, and the end result has minimal/negligible artifacts. But if you use MVtools2 directly from 23.976 => 59.94, there are the usual artifacts

Having the option to use RIFE I think is still a plus overall. Also consider implementing a switch for model version (anime model, vs. 2.4, vs. 3.8 etc...)

[MysteryX]

Beta 10. (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

The moment you've all been waiting for: RIFE support. Set rife=1 to run it once, or rife=2 to run it twice on the frame blending clip used for artefact masking. This will reduce frame blending in artefact areas.

It however doesn't have as huge of an impact as one might think, but definitely helps in some frames. You can also lower the mask threshold to see more of the RIFE clip.

First running RIFE and then running FrameRateConverter on top of it would also be an option.

btw I'm unable to set gpu_id, how do I set it to my 2nd NVIDIA graphic card? Haven't found how to use list_gpu param either.

TODO: when preset="anime", set model=2

btw I had to comment this validation in the source code to avoid an error when doubling framerate twice. Any idea what this validation was for? (https://github.com/mysteryx93/FrameRateConverter/blob/master/Src/Common/ConvertFpsLimitBase.cpp#L16) That was copy/pasted from "somewhere" LOL (isn't that how doom9 works?)

[MysteryX]

Playing with it, couple of issues.

When I use RIFE in 2 separate VirtualDub windows, sometimes junk comes through (in artefact areas).

As for difficult frames marked as full-frame blending, RIFE isn't doing any magic so it's not giving a good output.

More tweaking will be required.

[MysteryX]

Good news. After keeping full frame blending on difficult frames, and simply adding Maximum() to the mask when using RIFE, it gives a pretty good result.

Here's a sample frame with Blend / Rife / 2x Rife

https://i.postimg.cc/hJLrXVB0/Blend.png (https://postimg.cc/hJLrXVB0) https://i.postimg.cc/R6vTKqCx/Rife1.png (https://postimg.cc/R6vTKqCx) https://i.postimg.cc/G8zxQr98/Rife2.png (https://postimg.cc/G8zxQr98)

Can do further tweaking on mask thresholds when using RIFE.

There are however some corner cases where RIFE gives worse results, such as quick spotlights. You might get 1 worse frame for 20 better frames.

[StainlessS]

MX,
Not totally sure, but think it was you who posted a K-Pop "Girl's Day - Female President" clip,
I loved that and have went out of my way to find more of GD. Also I found I liked "Itzy - Wannabee",
plus more of them too. but my favourite is probably "BlackPink - Kill this Love" [and their entire collection],
I also liked the "Making Of, Kill This Love" where they get showered with boulders [not the Boulder on this site]
during the explosion. Great stuff, and thanks for putting me on the this K-Pop thing, love it.

BlackPink - Kill This Love:- https://www.youtube.com/watch?v=2S24-y0Ij3Y&list=RD2S24-y0Ij3Y

EDIT: Yep I thought so, your triple image above this post is from "Female President":- https://www.youtube.com/watch?v=xF3MC8PWgJE

[MysteryX]

StainlessS, that's what the Natural Grounding Player is for, attuning to the energy of those videos. Although that software now has broken download feature and it will require a lot more work before the next version is ready. I recently published the whole database of videos here. (https://docs.google.com/spreadsheets/d/1jpjKgEDgcKxxzSdtDG2gXO59xMsDX_Yx/edit?usp=sharing&ouid=101904282551013923658&rtpof=true&sd=true) Long list, and great stuff in there. Try this one (https://www.youtube.com/watch?v=wCPZTjqvN0Q) and that one (https://www.youtube.com/watch?v=V2hlQkVJZhE).

[poisondeathray]

Yes, RIFE can sometimes produce worse results on seemingly "simple" content that mvtools2 breezes through without issues

I've been doing quite a bit of testing and there are frames where 2.4 does better job than 3.1 or 3.8. I'm finding 2.3 seems the best model overall on varied content (but it's not included in vapoursynth version - But YMMV as usual . Scene changes are not handled well either - probably the same underlying issue with those flashing lights

Also , often you can get a "clean solve" on some frames by pre manipulations such as turnleft , or flipping (then reverting the change post rife) ; so I asked zorr about zopti maybe figuring combinations out on a per frame basis

It would be pure awesomeness if an automatic tiered choice algorithm could choose what works based on least artifacts, then fallback on some of the masking, or full blending if none work . MVTools2 is much faster still, and if it works - great . That's my preference. But if that doesn't work, replace that frame with RIFE 2.4 , and if that doesn't work replace with xyz etc...

[MysteryX]

It would be pure awesomeness if an automatic tiered choice algorithm could choose what works based on least artifacts, then fallback on some of the masking, or full blending if none work . MVTools2 is much faster still, and if it works - great . That's my preference. But if that doesn't work, replace that frame with RIFE 2.4 , and if that doesn't work replace with xyz etc...
That's pretty much what I'm doing; except that RIFE doesn't provide any info as to what artefacts it produces. Full-frame blending or skipping is done based on the raw masks from MvTools. Which handles scene changes correctly.

If there remains specific situations causing trouble, the only way would be to have a specific plugin to scan for that problem, just like StripeMask is doing.

BTW can someone explain this. I run twice with output="raw" debug=true in VapourSynth, and although the raw mask visually looks the same, the raw value varies randomly. Why does Vpy MvTools have this "random" factor?

I confirm that model=1 gives better results. The random junk that sometimes appear when running multiple instances does make it harder to test. Also, VirtualDub process doesn't terminate correctly. It does have a few bugs that need to be rounded up.

[StainlessS]

Thanks for that MX, I'm aware of some of them.
Love this [Japanese] one from Kill Bill II [closing credits], "Urami Bushi" by Meiko Kaji:- https://www.youtube.com/watch?v=yT9zJ2V9lfw

EDIT: Didn't realize it but Meiko Kaji has serveral more tracks from Kill Bill I.

[MysteryX]

Beta 11 (https://github.com/mysteryx93/FrameRateConverter/releases/tag/v2.0-beta)

Added "rife" and "rifeanime" presets
Preset "rife" sets rife=2 and maskThr=80. If doubling framerate, rife is set to 1-pass.
Rife uses model=1 (v2.4)
Changed maskOcc from 105 to 125, and blending with .7 opacity instead of .4
Avisynth version now has FullRange parameter

Using diff mode with RIFE is counter-productive, so preset "rife" uses preset "slow".

Remains a weird randomness to the masks. Run the same clip twice (with preset="normal") and you'll get slightly different masks.

I created a thread for RIFE problems here. (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan/issues/7) Reported the MvTools randomness here. (https://github.com/dubhater/vapoursynth-mvtools/issues/53)

This now gives seriously good results!! Please test it and post your reports.

Pay particular attention to the occlusion mask. VapourSynth occlusion mask is a lot narrower than in Avisynth where it marks large areas. If you find that occlusions aren't being detected enough, yet show on the raw mask, let me know and we can tweak either maskOcc or the merge opacity. Also do a bit of testing about StripeMask as to whether it includes too much.

So please test these 3 areas and let me know whether it feels weak, just right or too strong.
- Occlusion mask
- Stripe mask
- Overall mask

[poisondeathray]

Thanks for the updates


Rife uses model=1 (v2.4)



That was for v1, which is loaded with a .dll
model: Model to use.

0 = rife-v3.1
1 = rife-v2.4
2 = rife-anime



v1.1 uses model_ver
3.1
3.5
3.8

You can check __init__.py

v1.1 is accessible when you use pip install method


To access all the models for the vapoursynth version, I've been both loading the old R1 .dll and importing the R1.1 py. I've been using stackvertical/horizontal to check versions, also other non vapoursynth versions to check other model versions


core.std.LoadPlugin(r'PATH\RIFE.dll') #RIFE-r1
from vsrife import RIFE as RF3 #RIFE r1.1

r24 = core.rife.RIFE(clip , model=1)
ranime = core.rife.RIFE(clip , model=2)
r31 = RF3(clip , model_ver=3.1) #3.1,3.5,3.8
r35 = RF3(clip , model_ver=3.5)
r38 = RF3(clip , model_ver=3.8)



Remains a weird randomness to the masks. Run the same clip twice (with preset="normal") and you'll get slightly different masks.

Did you check source filter, maybe try threads=1

[poisondeathray]

I can't get vapoursynth version to run, never seen this error before


File "FrameRateConverter.py", line 170, in FrameRateConverter
Blank = core.std.BlankClip(C.resize.Point(format=vs.GRAY8))
File "src\cython\vapoursynth.pyx", line 2067, in vapoursynth.Function.__call__
vapoursynth.Error: BlankClip: nodes foreign to this core passed as input, improper api usage detected


The py says Version: 2.0 (2021-09-18) beta 9 , not beta 11 - could it be wrong bundled version ?

EDIT - NM , some "weirdness" going on , but you need to specify output mode and blkSize, but sometimes that error message comes back... weird . You have to close vsedit and restart and message goes away. It's really flaky right now, maybe it's related to that weird mask randomness. Similar thing happens in vdub2 - 1st time works, but if you push f2 (reopen video file) that error message pops up

Selur had the same issue a few pages back
https://forum.doom9.org/showpost.php?p=1946115&postcount=402

Might be related
https://forum.doom9.net/showthread.php?p=1801028#post1801028
https://forum.doom9.net/showthread.php?p=1801033#post1801033

If I edit the FrameRateConverter.py to

from vapoursynth import core


instead of

core = vs.get_core()

the issue goes away

[MysteryX]

Ah yes there was that. Probably related to the way I'm playing with Core instance in StripeMask code. I remember doing a hack I wasn't sure about.

vsrepo gives me v1.0 I believe? Where do I find that v1.1?

[ChaosKing]

I belive he means https://github.com/HolyWu/vs-rife
These are 2 different plugins

vsrepo currently only has https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan
Not sure if vs-rife is worth adding if you still have have many post install steps.

[poisondeathray]

Yes, 2 versions . The Pytorch/CUDA version is significantly faster than Vulkan if you have a Nvidia GPU. But Vulkan is accessible to everyone - so that version should definitely be included for general use

The other issue is model availability - I'm finding 2.3 and 2.4 generally better than the 3.x versions . But we can probably ask HolyWu to make other models available for the Vapoursynth Pytorch version

There was some discussion on why original Vulkan version was slower, but apparently it just is
https://github.com/nihui/rife-ncnn-vulkan/issues/22

[MysteryX]

I can't get vapoursynth version to run, never seen this error before
I thought I was storing Core from the initializer and using it elsewhere; perhaps from wrong thread or something. I looked through the code. There was 1 instance where I was storing Core in a field, and it wasn't used at all. So it's not that.

I don't know then...

[ChaosKing]

vs-rife was just updated with 1.8, 2.3 and 2.4 models. https://github.com/HolyWu/vs-rife

[Selur]

Nice! btw. is there some info about what the differences are between the models? (I mean is model xy better suited for content wz?)

--
I also think it would be nice to choose whether
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-RIFE-ncnn-Vulkan
or
https://github.com/HolyWu/vs-rife
would be used.

Cu Selur

[Selur]

As a side note, using Beta11:
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="rife", blkSize=8, blkSizeV=8, rife=1, rifeModel=1, rifeTta=False, rifeGpu=0) # new fps: 50
causes blending while
clip = core.rife.RIFE(clip)
does not with this (https://drive.google.com/file/d/1xchtRU6uZyLm9xJWh4HgNxDrSqC70zhl/view?usp=sharing) file (Frame 359).
Neither does 'FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)'.

Cu Selur

[Selur]

Any news on RIFE incorporation into FrameRateConverter?

[poisondeathray]

As a side note, using Beta11:
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, preset="rife", blkSize=8, blkSizeV=8, rife=1, rifeModel=1, rifeTta=False, rifeGpu=0) # new fps: 50
causes blending while
clip = core.rife.RIFE(clip)
does not with this (https://drive.google.com/file/d/1xchtRU6uZyLm9xJWh4HgNxDrSqC70zhl/view?usp=sharing) file (Frame 359).
Neither does 'FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)'.

Cu Selur


You can disable some or all of the frc features . Adding output="flow"gives you interpolation only.

You can set individual masking features you don't want to "0" , such as maskThr, maskOcc, skipThr, blendOver, skipOver



There is an issue with stp ; stp=0 to disable strip detection produces error, but values >0 produce no error msg


File "PATH\FrameRateConverter.py", line 285, in FrameRateConverter
EMstp = havs.ChangeFPS(EMstp, newNum, newDen)
UnboundLocalError: local variable 'EMstp' referenced before assignment

[Dogway]

MysteryX, what's the point of having different mask settings for different clip sizes as shown here (https://forum.doom9.org/showthread.php?p=1945840#post1945840)? Does it have to do with the MAnalyse vectors?
I ask because it would make sense to have the same mask for the same clip whatever its size is.

This was my test code:

msk="C.BicubicResize(Round(C.Width()/blk/4.0)*4, Round(C.Height()/blk/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(thr,u=-128,v=-128)
\ .Blur(.6)
\ .BicubicResize(1280, 720)"

a=bicubicresize(640,360,-0.5,0.25) # 12 blksize
b=bicubicresize(854,480,-0.5,0.25) # 12 blksize
c=bicubicresize(1280,720,-0.5,0.25) # 16 blksize
d=bicubicresize(1920,1080,-0.5,0.25) # 24 blksize

a=Eval(ReplaceStr(msk,"thr,","120-10").ReplaceStr("blk","11").ReplaceStr("C.","a."))
b=Eval(ReplaceStr(msk,"thr,","120-10").ReplaceStr("blk","12").ReplaceStr("C.","b."))
c=Eval(ReplaceStr(msk,"thr,","120" ).ReplaceStr("blk","16").ReplaceStr("C.","c."))
d=Eval(ReplaceStr(msk,"thr,","120+10").ReplaceStr("blk","24").ReplaceStr("C.","d."))

StackVertical( \
StackHorizontal(a,b), \
StackHorizontal(c,d))

[pintcat]

Tried a very simple script on beta 11 and got this error:
Avisynth open failure:
Script error: Invalid argument to function "StipeMaskPass"
(FrameRateConverter.avsi, line 376)
(FrameRateConverter.avsi, line 219)

Script was something like
directshowsource(test.mp4)
framerateconverter()
Nothing fancy and it used to work with FrameRateConverter v1.3.

[manolito]

The correct function name should be "StripeMaskPass"...

[pintcat]

Yeah, that's a typo on my side. I did not copy/paste the error message. Yet, I have no idea what's wrong here. Am I the only one getting this error?

[Floatingshed]

Just downloaded version 1.3 zip, but the avsi file states that it is version 1.21...

[manolito]

This is just a small oversight by MysteryX. The script is really version 1.3.

[Floatingshed]

Great, thanks for letting me know.

[MysteryX]

MysteryX, what's the point of having different mask settings for different clip sizes as shown here (https://forum.doom9.org/showthread.php?p=1945840#post1945840)? Does it have to do with the MAnalyse vectors?
Not sure I understand your question; but at the link you pointed to, I take blkSize into the equation so that the resulting frame is resolution-independent; meaning a "mask zone" will be about let's say 1/20th of the width on a 480p clip and 1/20th of the width on a 4K clip. Will look similar when looking at that clip full-screen without regards to input resolution.

[MysteryX]

I got tired of Windows and switched to Linux.

For when I get back to this; does RIFE work under Linux? It will be a bit more of an adventure to get all the plugins working in Linux.

[manolito]

I got tired of Windows and switched to Linux.


Alright, I guess you will continue support for the upcoming Linus version in the Linux section at Doom9. Just one question:

Will you publish a final stable Windows version 2.0, or will you just abandon it and refer Windows users to the old stable version 1.3 ?

Whatever, thank you very much for your work on FrameRateConverter.

Cheers
manolito

[StainlessS]

or will you just abandon it and ...
Of course he would not just abandon it, MX is a very respectable person
with very strong principles and would not ever do that,
a final stable ver$ 2.0 is the least you can expect from the noble gentleman. :)

[MysteryX]

I also have Windows running within Linux with near-native performance with QEMU+KVM with GPU passthrough; enough to play games with native Windows performance.

Been busy with lots of other stuff. Including porting some of my software to Linux by switching UI from WPF to Avalonia (https://avaloniaui.net/), and porting MvvmDialogs (https://github.com/mysteryx93/mvvm-dialogs) to support Avalonia and other UI platforms. Oh, and converting the Media Player UI to work on Avalonia/Linux (https://github.com/mysteryx93/MediaPlayerUI.NET) was a pain too (and not completed).

There's FrameRateConverter to finish, and xClean to finish.

Not counting my usual work in the astral realms.

[StainlessS]

There's FrameRateConverter to finish, and xClean to finish.
"Complete", better than 'finish".
And Good Luck in the "Astral Realms", [EDIT: whatever the hell they are {I'm guessin' close friend of Mystic Meg}],
also see U at some point in the Linux Realm :)
(Thank you MX)

[kedautinh12]

I think xclean avisynth version still don't finish

[MysteryX]

Oh and I spent a full week to get GPU passthrough to work for my Windows Virtual Machine... this is what I had to go through... (https://github.com/mysteryx93/GPU-Passthrough-with-Optimus-Manager-Guide)

yes xClean Avisynth is what needs to be finished. Also a comparison video I want to produce. Got a long list of things to do...

[real.finder]

Any news on RIFE incorporation into FrameRateConverter?

don't know if it possible or not but I think most mvtools problems come from bad motion vectors that MAnalyze do, so I wonder if MAnalyze can be updated to use some ideas from RIFE (neural network) to make perfect solution not only for FrameRateConverter but for any thing use MAnalyze like Motion Compensated and Temporal Denoising (like SMDegrain)

[johnmeyer]

don't know if it possible or not but I think most mvtools problems come from bad motion vectors that MAnalyze do, so I wonder if MAnalyze can be updated to use some ideas from RIFE (neural network) to make perfect solution not only for FrameRateConverter but for any thing use MAnalyze like Motion Compensated and Temporal Denoising (like SMDegrain)Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

[real.finder]

Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

yes I know it will not be perfect improved

with "perfect solution" I mean as "RIFE incorporation into FrameRateConverter" plus improve mvtools in general

[zambelli]

Any plans for 10-bit YUV support?

[takla]

Probably not. I say that because I've been watching a lot of "restored" old USA TV shows that were shot on film (e.g., "Perry Mason"). Many of them have been "improved" to show at 60 fps instead of how they were shot (24 fps). Since I've done so much of this work, I instantly see all the artifacts and, even when using professional motion estimation tools, all the artifacts you see when using MVTools are there: broken vertical objects, warping at edges when a foreground objects moves across the screen, and "broken legs" when people walk across the frame at a certain distance from the camera. I also see it on establishing shots when the camera pans up across a tall building that has lots of small windows.

Interpolating 60 fps from 24 fps is just moronic.

[Selur]

Any updates on Vapoursynth&RIFE support? (adding support for RIFE-v4 model and the new skip&skip_threshold might be a good idea)

[Selur]

Any news?

[MysteryX]

Updates? Released a few software for Windows/Linux/MacOS

https://sourceforge.net/projects/converter432hz/
https://sourceforge.net/projects/player432hz/
https://sourceforge.net/projects/yangdownloader/
https://sourceforge.net/projects/powerliminals-player/
plus xClean script

Haven't got back to FrameRateConverter yet.

[Selur]

Haven't got back to FrameRateConverter yet.
:( fingers crossed this will change soon, really like the idea of FrameRateConverter using RIFE properly :)

[kedautinh12]

Yeah, RIFE support avs and vs now. Maybe need new ver of FrameRateConverter using RIFE

[MysteryX]

if RIFE supports AVS, it should be easy to do.

It's porting it to VapourSynth that is more complicated; particularly that the MvTools2 masks are completely different under VapourSynth!

I'm now on Linux and not currently setup for Avisynth development, but if someone wants to do it...

Just edit the frame blending clip like this: run RIFE once or twice before frame blending and voilà!

Could have a parameter "rife" set to -1 for automatic, 0 to disable, 1 or more to manually specify how many passes of doubling frames. We still need frame-blending to reach the exact desired framerate.

[kedautinh12]

:( fingers crossed this will change soon, really like the idea of FrameRateConverter using RIFE properly :)

Dogway add RIFE to FrameRateConverter now
https://github.com/Dogway/Avisynth-Scripts/blob/master/MIX%20mods/FrameRateConverterMIX.avsi

[MysteryX]

How is RIFE working out?

[Dogway]

I did some tests and found that DCT=1, DCTRe=4 and CalcDiff=true (so 'Slowest' preset with DCTRe=4) made improvements for blending and some distortions.
Only tested on one shot but compared to no-RIFE is simply at another level. RIFEAnime stays at 'Slow' preset. Maybe you can check if implementation is correct.

[johnmeyer]

DCT=1 sometimes (not always) makes improvements in motion estimation artifacts, but at a 5x speed penalty! It is definitely worth testing on each video before proceeding.

[Dogway]

I think RIFE is also very slow so maybe it can be thought as a 'Placebo' preset, despite not being placebo at all.
I will try to run it through some popular clips like the one with the fence or the famous one @zorr used for zopti.

[takla]

Script error: the named argument "str" to StripeMask had the wrong type
FrameRateConverter.avsi, line 219 (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi#L219)

And
with Dogways FrameRateConverterMIX.avsi (https://raw.githubusercontent.com/Dogway/Avisynth-Scripts/master/MIX%20mods/FrameRateConverterMIX.avsi) I get

Evaluate: operands of "%" must be integers
ExTools.avsi, line 7509 (https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi#L7509)