Are framerate conversions always jerky?

[bugmenot]

A general question I know, but, after weeks of converting files in a variety of framerates into ones which fit the various standards, using Tmpgenc and Virtual Dub Mod and a range of guides and tips found on the net, I have yet to find a method that doesn't give me a slightly jerky end product.

Is jerkiness is an unavoidable end product of framerate conversion?

[jggimi]

No, it doesn't have to be that way, at least, not when I've done 29.97->25 interlaced->interlaced conversion, which is the only conversion I've ever done.

The video I used this for was talking heads, and the result looked fabulous. There's no jerky hand or head motions, nor jerky lips. It's very professional looking, if you consider analog capture from VHS "professional" to begin with.
:cool:

I used Xesdeeni's AviSynth/Smoothdeinterlacer methodology. Its complex, and I used AviSynth 2.5 so I had to aquire the Smoothdeinterlacer for AviSynth 2.5 ... but it worked very well for me.

Though I must say, framerate conversion metholdology discussions are not for the Newbie. I'm including a recently used script... it has Xesdeeni's comments in it unchanged, though the Smoothdeinterlace plugin for Avisynth 2.5 is found at www.avisynth.org/warpenterprises/.

The filters at the very end ... from the Letterbox() filter on down were for blacking out head switching interference, adjusting color, and for noise reduction, and were not applicable to the framerate adjustment.

I'll PM Xesdeeni, and see if he has any additional comments / recommendations for you.
######################################################################
#
# Poor man's video standards conversion (NTSC to PAL and PAL to NTSC)
#
# This script converts one INTERLACED video format to another
# INTERLACED video format.
#
# NOTE: This script is NOT meant to convert telecined films (that is,
# films that have been transferred to video). There are much better
# ways to convert that type of content (see the guides at
# www.doom9.net and www.vcdhelp.com). This script is best for
# INTERLACED content like HOME MOVIES shot with camcorders or live
# events. It is also good for mixed content (film + video).
#
#---------------------------------------------------------------------
#
# >>> Tools <<<
#
# This script is for use with AVISynth version 2.0.6 or above,
# available from http://www.avisynth.org.
#
# This script uses my AVISynth port of Gunnar Thalin's smooth
# deinterlacer, which is available at
# http://home.bip.net/gunnart/video/A...hDeinterlacer/.
# Place the plugin in an AVISynth plugin directory, so it can be
# accessed below.
#
#---------------------------------------------------------------------
#
# For comments/suggestions email me (Xesdeeni2001) at Yahoo dot com.
#
######################################################################
#
# >>> How It Works <<<
#
# This script works by first converting the input video from
# interlaced fields to progressive frames using an adaptive
# deinterlacer. Then the progressive frame rate is converted.
# Finally the progressive frames re-interlaced for output. Scaling
# is also performed where appropriate.
#
######################################################################
#
# >>> To Use <<<
#
# To use this script, first modify the lines below as specified.
# Then save the script with any name ending in .AVS.
#
#---------------------------------------------------------------------
#
# Set PluginPath equal to the path for your AVISynth plugins. Be sure
# to end the path with a backslash. Note that if you put the plugins
# in the system directory, you can leave this path completely empty.
# ex.:
# PluginPath = "C:\AVISynth\PlugIns\"
#

loadplugin("SmoothDeinterlacer.dll")

#---------------------------------------------------------------------
#
# Set Input equal to the load directive for the input file. Also add
# any plugins necessary to load the video file. Note that if the clip
# contains audio, it is fed straight through without being modified,
# because the output video will have the same length as the input
# video.
# ex.:
# LoadPlugin(PluginPath + "MPEG2DEC.dll")
# Input = MPEG2Source("Input.mpg")
#

Input = AVISource("cap.avi").Trim(120,14246).crop(4,0,-4,-0)

#---------------------------------------------------------------------
#
# Set InputTopFieldFirst to either true or false, depending on the
# format of your input file. DV files are normally bottom field
# first, so set this value to false. DVDs and most other sources are
# normally top field first, so set this value to true.
#

InputTopFieldFirst = true

#---------------------------------------------------------------------
#
# Set OutputFrameRate to the desired frame rate of your output video.
# For PAL, this is normally 25. For NTSC, this is normally 29.97.
# The input frame rate is derived directly from the input video.
#

OutputFrameRate = 25

#---------------------------------------------------------------------
#
# Set the OutputWidth and OutputHeight to the desired width and
# height of your output video. In most cases, the width of the
# output should match the width of the input. For PAL, the height
# is normally 576. For NTSC, the height is normally 480. The input
# width and height are derived from the input video.
#

OutputWidth = 720
OutputHeight = 576

#---------------------------------------------------------------------
#
# Set OutputTopFieldFirst to either true or false, depending on the
# desired format of your output file. See InputTopFieldFirst above.
#

OutputTopFieldFirst = true

#---------------------------------------------------------------------
#
# Set ConversionType to your desired type of frame rate conversion.
# The choices are:
# 0 - Replication/Decimation: Frames are repeated to increase the
# frame rate; frames are dropped to decrease the frame rate.
# This type of conversion is the fastest, but may show visible
# stuttering on motion when decreasing the frame rate (i.e.
# NTSC to PAL).
# 1 - Temporally Interpolate: Output frames are created by
# temporally interpolating between adjacent input frames. This
# type of conversion can show a "jutter" effect on motion, but
# is best when decreasing the framerate to ensure every input
# frame is at least partially shown in the output.
# 2 - Asynchronous: The conversion is done by showing the
# portions of the input frames that correspond to the time
# during which the output frame is visible. When decreasing
# the frame rate, this can cause some areas of some frames to
# never be seen, and can cause "broken" vertical edges on
# horizontal pans.
#

#ConversionType = (OutputFrameRate <= Input.framerate) ? 1 : 0

ConversionType = 1
#
######################################################################

#LoadPlugin(PluginPath + "SmoothDeinterlacer.dll")

vpro = Input.SmoothDeinterlace(tff=InputTopFieldFirst, \
doublerate=true)
vinfps = Input.framerate < OutputFrameRate ? \
vpro.BilinearResize(OutputWidth, OutputHeight) : \
vpro
vfps = ConversionType == 2 ? \
vinfps.ConvertFPS(OutputFrameRate * 2, zone = 80) : \
ConversionType == 1 ? \
vinfps.ConvertFPS(OutputFrameRate * 2) : \
vinfps.ChangeFPS(OutputFrameRate * 2)
voutfps = OutputFrameRate <= Input.framerate ? \
vfps.BilinearResize(OutputWidth, OutputHeight) : \
vfps
vfields = voutfps.SeparateFields()
vlace = OutputTopFieldFirst ? \
vfields.SelectEvery(4, 1, 2) : \
vfields.SelectEvery(4, 0, 3)
# The ConvertToRGB() below is to work around a problem with the YUV to
# RGB conversion caused by a bug in one of the Microsoft DLLs. The
# bug makes the colors look bad. Your destination may bypass this
# conversion, so you may be able to remove this conversion in some
# cases.
#vout = vlace.Weave().ConvertToRGB()
vout = vlace.Weave()
#return(vout)

last = letterbox(vout,0,10)
Temporalsoften(2,3,3,mode=2,scenechange=6)
mergechroma(blur(1.3))
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\FluxSmooth.dll")
FluxSmooth(5,7)
ColorYUV(levels="PC->TV")
ColorYUV(off_y=-15, gain_y=30)

[Xesdeeni]

Ech!!!! Old stuff lives forever on the net :-)Is jerkiness is an unavoidable end product of framerate conversion?Well, the short answer is "no."

The medium answer is "probably on a PC."

The really long answer is that to do a good framerate conversion, you have to create frames where there were none before. Forget interlacing or the resolution differences between PAL and NTSC for a moment, and imagine a 50 fps video of something moving across the screen. Each frame is captured 1/50 (0.02) second from the one before and after it. So the first few frames were captured at:

0.00 sec
0.02
0.04
0.06
etc.

To create a 60 fps video (forgetting the weird 59.94 frame rate of NTSC for a moment as well), you need frames that look like they were captured every 1/60 (0.01666...) second. So you need frames that look like they were captured at:

0.00 sec
0.01666...
0.0333...
0.05
0.0666...
etc.

The first frame is easy. The first PAL frame is already at 0.00. But what about the second?

Ideally, you'd like to magically create a frame that looks like it was captured at 0.016 seconds. The most expensive standards conversion devices do this by doing motion estimation on the source frames around this time and they attempt to create just such a frame. These devices can cost into 6-digits ($s). On a computer, it might be possible for software to do the same thing. But it's going to be pretty slow even on today's machines (the hardware version will use lots of parallel processing). And if someone does all that work, they'll probably charge a pretty hefty price for it. I know I would ;)

Which comes back to the rest of us. The simplest is just to use one of the PAL frames. Let's say you use the closest in time. So you do the following:

0.00 => 0.00
0.02 => 0.01666...
0.04 => 0.0333...
0.04 or 0.06 => 0.05
0.06 => 0.0666...
etc.

No matter which PAL frame you use for the fourth frame, it will be a repeat of the frame before or after it. This will introduce a noticable stutter in the motion of the object.

So you could try interpolation:

0.00 => 0.00
1/6 * 0.00 + 5/6 * 0.02 => 0.01666...
2/6 * 0.02 + 4/6 * 0.04 => 0.0333...
3/6 * 0.04 + 3/6 * 0.06 => 0.05
4/6 * 0.06 + 2/6 * 0.08 => 0.0666...
5/6 * 0.08 + 1/6 * 0.10 => 0.08333...
0.10 => 0.10
etc.

That's very similar to what the more affordable standards conversion devices do. But this has problems as well. As you can see, most every source frame shows up in two destination frames. And most destination frames have parts of two source frames. The result is a "reverse rippling" effect (the best I can describe it) on smooth motion.

You can take a look at both using the script below on your own PAL footage.

I personally prefer the first for PAL to NTSC conversion. I find the occasional hitch less annoying than the ripple. And since film, which is 24 fps, is converted to video using a similar frame repetition (called telecine or 3:2 pulldown), I'm used to seeing this on TV.

NTSC to PAL is a bit different. Using decimation (the opposite of replication), you actually discard an entire frame (I'll leave the details as an exercise for the reader ;) ). In this case, interpolation is probably the better choice.

DETAILS
-------

I still use AVISynth for this, but of course now we all use version 2.5 (http://avisynth.org). I prefer the MMX enabled KernelBob() (http://forum.doom9.org/showthread.php?s=&threadid=81322) to SmoothDeinterlacer(). Also, for MPEG sources, I prefer DGIndex/DGMPGDec (http://neuron2.net/fixd2v/decodefix.html) to DVD2AVI/MPEG2Dec.dll.

First you must determine the polarity of your input video. For some reason the automatic detection of this doesn't work, and in at least one case I encountered, the automatic polarity detection worked for part of the video and switched in mid-video! So explicitly setting the polarity is a must.

Start by using a script (.AVS extension text file) like this (stolen from KernelDeint's help):xxxSource("PAL.xxx")
AssumeTFF()
SeparateFields()Load this script into VirtualDub (http://virtualdub.org) and single-step through it to be sure the motion is correct (all in the same direction; not jumping back and forth). If it's wrong, try AssumeBFF() instead.

Then modify your script:xxxSource("PAL.xxx")
AssumeTFF() # or AssumeBFF()
KernelBob(order=1) # order=1 for TFF, order=0 for BFF
LanczosResize(720,480)
ChangeFPS(60.0 / 1.001) # ChangeFPS()=repl; ConvertFPS()=interp
SeparateFields()
SelectEvery(4,0,3) # (4,0,3)=same field order, (4,1,2)=reverse order
Weave()
Xesdeeni

[jggimi]

:goodpost: No, let me change that, it was a great post!:thanks: :thanks: :thanks:

"...Ech!!!! Old stuff lives forever on the net :-)..."

Yes, it does. I discovered your code when searching specifically for a fields based conversion method. And, I discovered that it worked extremely well. But my source was an edited BetacamSP recording that was delivered to me in standard VHS form.

I'd already captured and encoded the interlaced NTSC VHS source six different ways: fixed quantizer DivX for download, low bitrate WMV and RM for streaming, and NTSC DVD/SVCD/VCD discs for standalones. I needed to create PAL versions of the video discs, and your older methodology, while a little less streamlined, worked wonders.