24p to 60i, still has chunky motion

[Blue_MiSfit]

Hey folks,

So this is a followup thread to an older one, where I requested help converting 24p DV material to NTSC 60i DV for a Final Cut system. The results I achieved were good, but when compared to another camera angle (this time properly shot in 60i), the motion is unacceptably shuttery.

Obviously perfect interpolation is impossible, but hopefully there's some way to clean things up so that they layman watching the DVD at home won't be stunned by the very harsh transition. Intercut, this footage looks dreadful.

Here (http://www.digitalmediafactory.net/clients/pub/dv_conversion_help.zip) are some samples

There are three files, one is a sample from an alternate angle (and scene), shot in true 60i. The second is a sample of 24p footage, with only the standard 3:2 pulldown, which is applied in-camera. The third is my attempt at removing the pulldown to recover 24p, and using MVTools to convert the footage to 59.94p, and then reinterlacing to output proper 60i, finally outputting in the Apple DV codec.

As you can tell, the sample that originated in 60i has much cleaner motion. Is there something we can do to fix up this footage?

Here is the AviSynth script I used to do the conversion:

setmemorymax(256)

LoadPlugin ("d:\archives\video software\avisynth plugins\common\mvtools.dll")
LoadPlugin ("d:\archives\video software\avisynth plugins\common\QTSource.dll")

# Movie : H:\Fit Beginnings\MOVs 2\20030802-045837-01.mov

# Store parameter in variable named "Clip"
Clip="20030802-045837-01"

# Variables for target frame rate (60000/1001 = 59.94)
num = 60000
den = 1001

# Set clip as Source
QTInput("H:\Fit Beginnings\MOVs 2\20030802-045837-01.mov", audio=true)

#Recover 24p
TFM()
TDecimate()

#Color Correct, Denoise, and Sharpen
levels(0, .9, 255, 0, 255)
fft3dgpu(plane=3, sigma=1.5)
fft3dgpu(plane=0, sigma=1, sharpen=.8)

#Frame-Rate Conversion
backward_vec = last.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1, blksize=8)
forward_vec = last.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1, blksize=8)
MVFlowFps(backward_vec, forward_vec, num, den, ml=100, idx=1)

#Reinterlace
assumebff()
separatefields().selectevery(4,0,3).weave()

#Encode to Apple DV Codec
QTOutput("h:\20030802-045837-01.mov", format="dvc")


~MiSfit

[foxyshadis]

Maybe you could fieldblend it. Say:
Average(last, 0.5, temporalsoften(1,255,255), 0.5))
or
Average(last, 0.4, temporalsoften(1,255,255), 0.4, temporalsoften(2,255,255), 0.2))
as possibilities. Just to see whether it flows any better, even if it'll totally kill compressibility.

[wonkey_monkey]

I think I got marginally better results using mvflowfps2, but the wobbles are still visible.

David

[Didée]

Blue_MiSfit: are you aware of the "Alchemist" thread (http://forum.doom9.org/showthread.php?t=113256)? It's about framerate conversions & scripts that try to improve ...

[Blue_MiSfit]

@foxyshadis
I don't have the function Average(...). Which DLL is this a part of?

@Didee:
Yep, thanks for reminding me though! I got some good insight into the workings of MVTools, and even tried out the new salFPS3, but it wasn't what I needed...

@davidhorman:
Same results here. Even when I really crank up MVAnalyze (search=3, searchparam=4, blksize=4), I still get choppiness in certain high motion areas... :(

Any other ideas? We're tearing our hair out in great swaths over this one...

Thanks..

~Misfit

[Blue_MiSfit]

A quick update:

Here (http://www.digitalmediafactory.net/clients/pub/first_six_months.zip) is another sample which clearly illustrates the difference between the 24p and the 60i footage. The head-on shot is 24p converted to 60i with my previous script, and the side shots are true 60i.

Note the "liveliness" of the 60i vs the 24p. Our goal is to make the differences transparent to the average viewer.

Thanks again for the help.

~MiSfit

[tateu]

The head-on shot is 24p converted to 60i with my previous script, and the side shots are true 60i.
Are you sure about that? The head on shots are 3 progressive frames followed by 2 interlaced...standard 3:2.

[Blue_MiSfit]

I think you're right :)

Sorry, I guess I uploaded the wrong sample. It gets confusing when you have 3 people editing a project, and they all use different organizational and naming systems. Grrr..

Check the other samples? :)

~MiSfit

[foxyshadis]

Average would be part of Average.dll (http://forum.doom9.org/showthread.php?t=100626). =p

It'd have to go right before reinterlacing. I don't know if it will actually work for what you need, but blending tends to smooth out motion quite a bit (sometimes too much). If the stuttering is caused because it's already blending instead of compensating well, it won't help. The alchemist thread might, but it's not perfect either.

[Blue_MiSfit]

@ foxyshadis:
Thanks, I was sure I looked for that exact file, but I guess not well enough :p

@ Didee:

I got your PM, and constructed the following script:

SetMemoryMax(256)

LoadPlugin("D:\archives\video software\avisynth plugins\common\MVTools.dll")

QTInput("C:\first_six_months.mov", audio=false)
trim(46,261)

tfm()
tdecimate()

source=last
num=60000
den=1001

sharpened=source.sharpen(.8)

backward_vec = sharpened.MVAnalyse(isb = true, truemotion=false, pel=2, idx=1, lambda=256, search=2, searchparam=10)
forward_vec = sharpened.MVAnalyse(isb = false, truemotion=false, pel=2, idx=1, lambda=256, search=2, searchparam=10)

cropped = sharpened.crop(4,4,-4,-4)
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=false, pel=2, idx=2, lambda=256, search=2, searchparam=10)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=false, pel=2, idx=2, lambda=256, search=2, searchparam=10)

return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num, den)


I think I was following your general idea, though I'm not sure about all the parameters for MVTools, I'm still a newbie with it!

It looks nice, but I do get some very nasty artifacts in the high motion areas.

Here (http://www.megaupload.com/?d=ZXZ0QD31)is a sample of what I'm talking about. It's the 59.94p result of the script.

Thanks for all your help everyone!

~MiSfit

[dbzgundam]

Hey folks,

So this is a followup thread to an older one, where I requested help converting 24p DV material to NTSC 60i DV for a Final Cut system. The results I achieved were good, but when compared to another camera angle (this time properly shot in 60i), the motion is unacceptably shuttery.
~MiSfit

I have to ask, why can't you use the straight 24p with Final Cut? Final Cut is more than capable of using 24p footage (and 24p DV material is actually just edited in 60i throughout the program), and with 24p "Advance" (just the 2:3:3:2 cadence if I remember), Final Cut can do the pulldown removal on its own to edit in an actual 24p (23.976) timebase.

You'll only want AVISynth to IVTC back to 24p (as in, no cadence present) if you shot in normal 3:2 pulldown and FCP cannot do the IVTC for you (it requires Cinema Tools to do this, and I find that without timecode, etc, it's just a pain in the ass... Actually Cinema Tools seems unable to field-match from what I can tell, it merely field matches based on the information you give it, unlike AVS field-matchers that can do this with practically just knowing the field order)

[Blue_MiSfit]

Well, editing isn't the problem. The edit is done already as a matter of fact. The issue is the look of the footage. The true 60i stuff looks great - the typical video look that comes with 60i. The 24p stuff does not have very smooth motion, because of the decreased temporal resolution. This is a multi-cam shoot, and cutting from one camera to another looks terrible.

And yes, you are totally correct that Cinema Tools is very unintelligent. You have to tell it everything about how to remove pulldown, it can't detect anything. After Effects has a pretty good pulldown remover, but it's not adaptive, so if you have any dropped frames, the cadence pattern changes and you're totally screwed :) The one really nice thing about using these apps is that they preserve the quicktime metadata (timecode tracks etc). Doing it all through AviSynth destroys this data :| Still, TFM() + TDecimate() handles any dropped frames in a standard 3:2 pattern perfectly. It was a lifesaver for this indie film I worked on that had some dropped frames in their live uncompressed HD capture.

~MiSfit