Does converting a video to 24 fps allow for more smooth motion (frame interpolation)?

[Milardo]

Does converting a video to 24 fps allow for more smooth motion (frame interpolation)? I've tried bluesky frame rate converter with my rx 570 and DmitriRender as well and svp. I have most videos in 29.97 fps and when i play them with any of those programs with like mpc-be or mpc-hc they don't look that different than standard playback.

I have a video though that is 23.976 fps and it plays very smoothly when using bluesky, svp, DmitriRender. I was wondering why? I see in bluesky frame rate converter that there is a option when playing back 24 fps videos that states "Mode 2: Created 1 original frame and 4 interpolate frames when converted 24p to 60p". When this is used on that 23.976 video it plays very smoothly. The other mode is not as smooth.

Also on a 29.97 video, playback is not that smooth and there is no option for playing back those 29.97 videos in "Mode 2: Created 1 original frame and 4 interpolate frames when converted 24p to 60p".

So i thought what if i could play a 29.97 video and somehow convert it to 24 fps in realtime and use svp or bluesky or dmitriRender?

I haven't found too many ways to do this. I have settled on using FDecimate 2.

Script is something like below used in like ffdshow

LoadPlugin("C:\smoothmotion\FDecimate2_x86.dll")
FDecimate2(24)

I have this filter running before bluesky, svp or DmitriRender in mpc-hc or similar program.

In svp avs script I have it placed before svp avs parameters take place.

Could anybody try this or reply with a different method to achieve this? I can provide a sample video clip if necessary.

http://www.mediafire.com/file/oc8790evvimdu5k/bluegender3.asf/file

[FranceBB]

Does converting a video to 24 fps allow for more smooth motion (frame interpolation)?


Not really, no (generally).
But I gotta say that there are a few catches.

If your original file is 23.976 and is interpolated to 60p, then fine, but if you don't see the 29.970 one smoother than the 23.976 one it might be because it's a 23.976 hard-telecined to 29.970 so your motion interpolation would still have dups. In that case and only in that case I would suggest to inverse telecine it to 23.976:

tfm(mode=1,pp=5,slow=2,micmatching=2,clip2=tdeint(mode=2,type=3))
tdecimate()

Or, if it's a 29.970p with dups:

TDecimate(mode=2, rate=23.976)

I didn't really take a look at your sample that much, but it looks like a 29.970p with dups and occasional blended frames. Something kinda crappy I would say.
Try to play around with either TDecimate or SRestore.
I gotta go back to work now, my boss is watching me xD

[Milardo]

Not really, no (generally).
But I gotta say that there are a few catches.

If your original file is 23.976 and is interpolated to 60p, then fine, but if you don't see the 29.970 one smoother than the 23.976 one it might be because it's a 23.976 hard-telecined to 29.970 so your motion interpolation would still have dups. In that case and only in that case I would suggest to inverse telecine it to 23.976:

tfm(mode=1,pp=5,slow=2,micmatching=2,clip2=tdeint(mode=2,type=3))
tdecimate()

Or, if it's a 29.970p with dups:

TDecimate(mode=2, rate=23.976)

I didn't really take a look at your sample that much, but it looks like a 29.970p with dups and occasional blended frames. Something kinda crappy I would say.
Try to play around with either TDecimate or SRestore.
I gotta go back to work now, my boss is watching me xD


Hi and thanks for your reply.

I will try the above.

A question, though what about a source from a video capture card that has a 29.97 framerate or so? What should I use for that?

[FranceBB]

what about a source from a video capture card that has a 29.97 framerate or so? What should I use for that?

It depends. I mean, it doesn't really matter at which framerate you're capturing, but rather at which framerate the source you're capturing was recorded. In other words, if the source is 29.970p and you're capturing at 29.970p then fine, but if your source is at 23.976p and you're capturing at 29.970p then you're gonna end up with Dups and you'll definitely have to use TDecimate().

[feisty2]

Hi and thanks for your reply.

I will try the above.

A question, though what about a source from a video capture card that has a 29.97 framerate or so? What should I use for that?

motion blur (MFlowBlur)

[Milardo]

motion blur (MFlowBlur)

How do I use this, an example?

[Milardo]

It depends. I mean, it doesn't really matter at which framerate you're capturing, but rather at which framerate the source you're capturing was recorded. In other words, if the source is 29.970p and you're capturing at 29.970p then fine, but if your source is at 23.976p and you're capturing at 29.970p then you're gonna end up with Dups and you'll definitely have to use TDecimate().

TDecimate script above doesn't seem to work for me.

Is there a way to use avisynth plugin in vapoursynth? Specifically in linux?

[johnmeyer]

Changing a a 29.97 interlaced video to 24 fps progressive will make it much less smooth.

If you want to interpolate frames in an interlaced video in order to go to a higher frame rate and/or to to from 29.97 interlaced to 59.94 progressive, you have to first separate the fields and perform the motion estimation on the fields and then re-weave. You can do this in one step with QTGMC:

QTGMC( Preset="Slower", SubPel=2, EdiThreads=4,Blocksize=32,Overlap=4)

I have the script to do it directly with MVTools2, but it is much longer.

[Milardo]

Changing a a 29.97 interlaced video to 24 fps progressive will make it much less smooth.

If you want to interpolate frames in an interlaced video in order to go to a higher frame rate and/or to to from 29.97 interlaced to 59.94 progressive, you have to first separate the fields and perform the motion estimation on the fields and then re-weave. You can do this in one step with QTGMC:

QTGMC( Preset="Slower", SubPel=2, EdiThreads=4,Blocksize=32,Overlap=4)

I have the script to do it directly with MVTools2, but it is much longer.


Here is my sample script that i mentioned with svp from original post.

# This script was generated by SmoothVideo Project (SVP) Manager.
# Check http://www.svp-team.com for more details.

SetMemoryMax(1024)

global threads=6


LoadPlugin("C:\Users\Milardo\FirefoxDownloads\test\FDecimate2.dll")
LoadPlugin("C:\Program Files (x86)\SVP\plugins\svpflow1.dll")
LoadPlugin("C:\Program Files (x86)\SVP\plugins\svpflow2.dll")

ConvertToYV12()

FDecimate2(24)

SetFilterMTMode("DEFAULT_MT_MODE",2)

global crop_params=""
global resize_string=""
global super_params = "{pel:2,gpu:1,scale:{up:2,down:2},rc:true}"
global analyse_params = "{gpu:1,vectors:3,main:{search:{coarse:{distance:-8,bad:{sad:2000,range:24}},type:2}},refine:[{thsad:250}]}"
global smoothfps_params = "{gpuid:0,rate:{num:144,den:1,abs:true},algo:21,linear:true,block:false,scene:{mode:0,blend:false}}"

global demo_mode=0
stereo_type=0
stereo_left_selection=""
stereo_right_selection=""

########## BEGIN OF MSMoothFps.avs ##########
# This file is a part of SmoothVideo Project (SVP) 3.1
# This is NOT the full AVS script, all used variables are defined via
# JavaScript code that generates the full script text.

function interpolate(clip src)
{
input = crop_params=="" ? src : eval("src.crop("+crop_params+")")
input = resize_string=="" ? input : eval("input."+resize_string)

super=SVSuper(input, super_params)
vectors=SVAnalyse(super, analyse_params, src=input)
smooth=SVSmoothFps(input, super, vectors, smoothfps_params, mt=threads, url="www.svp-team.com")

return demo_mode==0 ? smooth : demo(input,smooth)
}

input=last

stereo_type==0 ? eval(""" interpolate(input)
""") : stereo_type==1 || stereo_type==3 ? eval("""
lf = interpolate(input.crop(0,0,input.width/2,0))"""+stereo_left_selection+"""
rf = interpolate(input.crop(input.width/2,0,0,0))"""+stereo_right_selection+"""
StackHorizontal(lf, rf)
""") : stereo_type==2 || stereo_type==4 ? Eval("""
lf = interpolate(input.crop(0,0,0,input.height/2))"""+stereo_left_selection+"""
rf = interpolate(input.crop(0,input.height/2,0,0))"""+stereo_right_selection+"""
StackVertical(lf, rf)""") : input

########### END OF MSMoothFps.avs ###########
Prefetch(threads)



Svp seems to work better smoothing videos with that FDecimate parameter there. However the smoothness is not consistent throughout the video and not always consistent if video is played/stopped ff or rewind and played again. What can I do to my script to make sure the consistency is fluid for smoothness?

[manono]

An interlaced source was blend deinterlaced. I tried but can't fix it. Whoever did this is an idiot. And who uses ASF these days? What's the source of this thing? I suggest you find another source.

[qyot27]

Changing a a 29.97 interlaced video to 24 fps progressive will make it much less smooth.

If you want to interpolate frames in an interlaced video in order to go to a higher frame rate and/or to to from 29.97 interlaced to 59.94 progressive, you have to first separate the fields and perform the motion estimation on the fields and then re-weave. You can do this in one step with QTGMC:

QTGMC( Preset="Slower", SubPel=2, EdiThreads=4,Blocksize=32,Overlap=4)

I have the script to do it directly with MVTools2, but it is much longer.
What the OP is describing/wondering aloud is roughly:

24fps content that's been tweened to 30 (and then frame doubled to 60, but only because it's just matching the display sync rate of the monitor) appears 'smoother'* than native 30fps content. Because of course it would: the interpolation is creating more motion than would naturally occur in a real 30fps capture. And it's usually going by too quickly for your eyes to see the artifacts, unless it's really bad.

*read: the unnaturally fluid motion and pseudo-3D depth faking of the 'soap opera effect'.

So they're asking if using frame decimation to go from 30->24 and then doing the 24->'60' interpolation will make it 'smoother'. I get the feeling that IVTC or deinterlacing as we know it was never the point of the question here.


RE: the FDecimate talk above:
Decimation filters aren't needed for this, either. Regular old ChangeFPS will do the job too, considering it won't matter once the tweening happens. And on Linux, I'm not aware of any of the decimation filters having been ported yet, so ChangeFPS inside the AviSynth+ core is the only option if you insist on using AviSynth+ to drop the framerate (mpv's vapoursynth filter can be made to act as an interpolator through the use of vapoursynth-mvtools and https://gist.github.com/phiresky/4bfcfbbd05b3c2ed8645; I was never able to/cared about getting SVP to function on Linux, but I also stuck with stock Intel UHD Graphics). But mpv's --vf filtering could also be set to decimate before giving it to vapoursynth, or the motioninterpolation.vpy bridge could be configured to do so.

[johnmeyer]

What the OP is describing/wondering aloud is roughly:

24fps content that's been tweened to 30 (and then frame doubled to 60, but only because it's just matching the display sync rate of the monitor) appears 'smoother'* than native 30fps content. Because of course it would: the interpolation is creating more motion than would naturally occur in a real 30fps capture. I don't understand how that could be. If the result of tweening, via motion estimation, results in 30 fps, and if the tweening is perfect, which it can be under various circumstances, then there is no "more motion" (your words) in the tweened version than in native 30 fps: they BOTH contain 30 frames every second.

There is therefore zero difference. In fact, if the tweening is in fact perfect, the two videos would be indistinguishable.

If you are saying that there are more than 30 frames per second in the tweened version, then that version is NOT 30 fps video.

At the risk of stating a tautology: 30 fps is the same as 30 fps.

[qyot27]

Yeah, I think I typed that out too fast to add in the big honking exception, because we are talking about 60p output: it matters when and which frames get doubled. A 24p->interpolate to 30p->double to 60p is less strenuous on the interpolation software, and it will be subtle unless there's weird timing decisions involved, but the effect wouldn't be completely invisible (I just tried it on a 30p source video with very low motion; the 30->24->interpolated30 output was still subtly motion smoothed in spots compared to the original, likely because of where it decided to insert new frames, not to mention that the artifacting was also very noticeable).

The OP's issue is that 30->60 isn't 'smooth' like 24->60 is. But then they quoted whatever option that is in the software that talked about 24->60 being '1 original frame and 4 interpolated frames', which would add up to 29, not 60. That math doesn't add up. I didn't mention the more likely scenario: 24->double to 48->interpolate to 60 will create a much more dramatic effect compared to 24->interpolate to 30->double to 60. And 30->60 wouldn't exhibit it nearly at all because it's probably just displaying every real frame twice, leading them to wonder if going 30->24 first would trick the software into creating the soap opera effect on video that it normally would not.

[johnmeyer]

The OP's issue is that 30->60 isn't 'smooth' like 24->60 is.I can tell you, with 100% certainty, based on 20+ years of using motion estimation, that 24->60 will always be worse than 30->60. It has nothing to do with whether it is done in one step or two, but instead is a function the temporal difference between frames when you only have 24 frames each second. The lower the frame rate, the more impossible it becomes for the motion estimation to figure out where all the pixels should be in the intermediate frames it is creating.

I deal with really low frame rate amateur film which can be as low as 12 fps and believe me, if I ever tried to create 60p from 12p, it would look really ugly. I've posted this clip many times before, but it is from 16 fps 1940s 16mm film that I converted to 30 fps in order to reduce the considerable judder when the camera panned. I succeeded in reducing that artifact, but at the cost of horrible morphing artifacts on some frames. As for smoothness, it is exactly what you'd expect from 30 fps progressive.

https://www.youtube.com/watch?v=t8HjRN0rw5M

[Milardo]

Yeah, I think I typed that out too fast to add in the big honking exception, because we are talking about 60p output: it matters when and which frames get doubled. A 24p->interpolate to 30p->double to 60p is less strenuous on the interpolation software, and it will be subtle unless there's weird timing decisions involved, but the effect wouldn't be completely invisible (I just tried it on a 30p source video with very low motion; the 30->24->interpolated30 output was still subtly motion smoothed in spots compared to the original, likely because of where it decided to insert new frames, not to mention that the artifacting was also very noticeable).

The OP's issue is that 30->60 isn't 'smooth' like 24->60 is. But then they quoted whatever option that is in the software that talked about 24->60 being '1 original frame and 4 interpolated frames', which would add up to 29, not 60. That math doesn't add up. I didn't mention the more likely scenario: 24->double to 48->interpolate to 60 will create a much more dramatic effect compared to 24->interpolate to 30->double to 60. And 30->60 wouldn't exhibit it nearly at all because it's probably just displaying every real frame twice, leading them to wonder if going 30->24 first would trick the software into creating the soap opera effect on video that it normally would not.

Just to mention I had set the framerate to 144 not 60

[qyot27]

I can tell you, with 100% certainty, based on 20+ years of using motion estimation, that 24->60 will always be worse than 30->60. It has nothing to do with whether it is done in one step or two, but instead is a function the temporal difference between frames when you only have 24 frames each second. The lower the frame rate, the more impossible it becomes for the motion estimation to figure out where all the pixels should be in the intermediate frames it is creating.
Right. I wasn't disputing that at all. I was using 'smooth' in air quotes precisely because the result of 24->60 looks alien and unnerving due to what you elaborate on here concerning the temporal difference between frames at 24fps.

What I was getting at is that, stuff like SVP is meant to emulate precisely that strange look (what TVs call various things in their internal settings: Auto Motion Plus, TruMotion, MotionFlow...reaction against it is why some manufacturers are now offering a Filmmaker Mode or equivalent). It's not intended for correctness or video restoration.

Just to mention I had set the framerate to 144 not 60
So is it that 30fps looks jerky on your monitor and you want to smooth it out, or that it just looks flat in comparison to whatever 24fps looks like on the same monitor?

It also means that the '1 original frame and 4 interpolated ones' could mean it's targetting 120 Hz (5x24=120). In which case, there shouldn't be a discrepancy with 30fps because 120 is also a multiple of 30.

[johnmeyer]

What I was getting at is that, stuff like SVP is meant to emulate precisely that strange look (what TVs call various things in their internal settings: Auto Motion Plus, TruMotion, MotionFlow...reaction against it is why some manufacturers are now offering a Filmmaker Mode or equivalent). It's not intended for correctness or video restoration.OK, I get it now and totally agree. The detractors call it the "soap opera effect" because back in the 1960s when almost all shows were still shot on film and then broadcast using a film chain, soap operas were shot on video which made them look much different.

Unfortunately, even though there has been a backlash I have noticed that some old TV shows that were shot on film have now been treated to this "improvement." I tuned past a broadcast of the 1970s "M*A*S*H" TV show that had been interpolated to 60i and it looked like video. It had a totally different feel and different impact. It most definitely was not "better."

[Milardo]

Right. I wasn't disputing that at all. I was using 'smooth' in air quotes precisely because the result of 24->60 looks alien and unnerving due to what you elaborate on here concerning the temporal difference between frames at 24fps.

What I was getting at is that, stuff like SVP is meant to emulate precisely that strange look (what TVs call various things in their internal settings: Auto Motion Plus, TruMotion, MotionFlow...reaction against it is why some manufacturers are now offering a Filmmaker Mode or equivalent). It's not intended for correctness or video restoration.


So is it that 30fps looks jerky on your monitor and you want to smooth it out, or that it just looks flat in comparison to whatever 24fps looks like on the same monitor?

It also means that the '1 original frame and 4 interpolated ones' could mean it's targetting 120 Hz (5x24=120). In which case, there shouldn't be a discrepancy with 30fps because 120 is also a multiple of 30.

No, it is just that my monitor is 144 hz. My goal is to play almost any video i have with smooth motion soap opera effect in realtime. Don't want to have to convert the video to get that effect, just playback in realtime to acheive the desired effect.

[FranceBB]

Unfortunately, even though there has been a backlash I have noticed that some old TV shows that were shot on film have now been treated to this "improvement." I tuned past a broadcast of the 1970s "M*A*S*H" TV show that had been interpolated to 60i and it looked like video. It had a totally different feel and different impact. It most definitely was not "better."

Yep... Besides, even when you do the right thing and preserve the original framerate, the amount of people that have linear interpolation turned on by default on their TVs is staggering... The fact that TV manufacturers have been promoting this as "True Motion", advertising it as something incredible that makes everything look better hasn't helped the cause either... :( I'm pretty sure that nowadays, the vast majority of people see everything interpolated somehow by their TVs with every kind of artifacts without even noticing it or knowing what their TV is doing... :(