Canon Vixia HF 24p pulldown interlacing artifacts

[Efenstor]

I though this may be very useful for those who also own this type of camcorder (HF100 more specifically).

If you're a quality hunter (like I am) you are probably aware that with Vixia HF100 (and probably the other HF's) simple 24p pulldown is not enough to get rid of all the combing. The reason for this is that the two subframes contain one and the same progressive frame but with slightly different motion estimation. There are two methods of fighting this:

1. By scharfis_brain (best but slow). You will need ITVC and nnedi2 (http://web.missouri.edu/~kes25c).
tfm(pp=0).tdecimate(cycle=5)
nnedi2(field=-2)
merge(selecteven(), selectodd())
sharpen(.35)

2. By Efenstor (a little or no combing but jaggies may be present, faster). You will need ITVC (http://web.missouri.edu/~kes25c) and Smooth Deinterlacer (http://www.guthspot.se/video/AVSPorts/SmoothDeinterlacer):

tfm().tdecimate(cycle=5)
smoothdeinterlace(lacethresh=5, edgethresh=40, staticthresh=20, staticavg=20)

Updated on January 5, 2010

[Didée]

What about Vinverse() ?
# TFM(mode=0)
# TDecimate(cycle=5)
ImageSource("651_pulldown_cr.jpg").ConvertToYV12()

SeparateFields().Vinverse().weave() http://img30.imageshack.us/img30/3167/651vinversecr.th.jpg (http://img30.imageshack.us/i/651vinversecr.jpg/)

Seems pretty OK, but ... something is fishy. Your image shows combing with the laces being 2 pixels in height (which is why separate/weave was needed for vinverse), and that is something that should not be.

[Efenstor]

Your image shows combing with the laces being 2 pixels in height
I have zoomed it by 2 intentionally. I'll try Vinverse.

[scharfis_brain]

I have zoomed it by 2 intentionally.
Please don't zoom your screenshots. We can do this ourselves ;)

Also, can you provide an unprocessed sample video straight off the cam?

[Efenstor]

Also, can you provide an unprocessed sample video straight off the cam?
http://www.efenstor.net/external/00066.MTS

[Efenstor]

Nope, for this particular problem Vinverse seems to be worse than SmoothDeinterlace (zoom and compare details on the swaying spruce at the right of the man):

Original (pulldowned):
http://s3.sendpic.ru/big/1015/t/e7.jpeg (http://s3.sendpic.ru/big/1015/i/e7.png.html)

SmoothDeinterlace:
http://s3.sendpic.ru/big/1015/t/dN.jpeg (http://s3.sendpic.ru/big/1015/i/dN.png.html)

Vinverse:
http://s3.sendpic.ru/big/1015/t/Fy.jpeg (http://s3.sendpic.ru/big/1015/i/Fy.png.html)

[Didée]

Use whatever fits you the best. For my eyes, the result of Vinverse is closer to the IVTC'ed source: it just removed the combing, and apart from that the result reflects the input more truly. OTOH, SmoothDeint caused some subtle changes in the shadings and faint tonalities of moving image parts, see e.g. on the trousers of the man, and such.

From the overall appearance and perceived amount of detail, I don't see any difference anyway. Flipping forth/back under 200% zoom, there are some tidbits, sure. But if you'd play them side by side, most probably I could not tell which is which. :)

And frankly, I see worse problems than this. For example, chromatic aberration ...

[wonkey_monkey]

I only looked at the "original" PNG above, but I got fine results just by doing the Photoshop equivalent of (dumb) bobbing, merging adjacent frames, and applying a small amount of vertical sharpening:

http://img402.imageshack.us/img402/5012/bobmergeshapen.th.png (http://img402.imageshack.us/img402/5012/bobmergeshapen.png)

Guaranteed no residual combing ;) Doing an eedi upsize of separatedfields (however one does that) before a merge might get rid of the need for sharpening.

David

[Efenstor]

I only looked at the "original" PNG above, but I got fine results just by doing the Photoshop equivalent of (dumb) bobbing, merging adjacent frames, and applying a small amount of vertical sharpening:

Guaranteed no residual combing ;) Doing an eedi upsize of separatedfields (however one does that) before a merge might get rid of the need for sharpening.

David
Of course, but don't forget it's 24p, sharpness and details are lost after such brute methods.

[Efenstor]

Use whatever fits you the best. For my eyes, the result of Vinverse is closer to the IVTC'ed source: it just removed the combing, and apart from that the result reflects the input more truly. OTOH, SmoothDeint caused some subtle changes in the shadings and faint tonalities of moving image parts, see e.g. on the trousers of the man, and such.

From the overall appearance and perceived amount of detail, I don't see any difference anyway. Flipping forth/back under 200% zoom, there are some tidbits, sure. But if you'd play them side by side, most probably I could not tell which is which. :)

And frankly, I see worse problems than this. For example, chromatic aberration ...

Thanks. Chromatic aberration doesn't bother me at all. Everyone has its own paranoia. :D

[scharfis_brain]

That's what I'd strongly suggest:

loadplugin("c:\x\tivtc.dll")
loadplugin("c:\x\nnedi2.dll")
directshowsource("00066.MTS")
tfm(pp=0).tdecimate()
nnedi2(field=-2)
merge(selecteven(), selectodd())


It completely removes every bit of remaining mice teeth and also alvoids remaining stairstepping.
Also you should do the pulldown removal within avisynth (tfm + tdecimate).
And be sure to get another source filter than directshowsource() I just used it as quick'n'dirty preview filter.
Use DGDecode / DGNV instead.

Furthermore the source only contains detail up to 720p. So reducing the resolution during processing could be a very clever idea for speeding up the processing and editing as well as for getting a much better compression result for the final movie.


loadplugin("c:\x\tivtc.dll")
loadplugin("c:\x\nnedi2.dll")
directshowsource("00066.MTS")
bicubicresize(1280, height) #downscale horizontally, which doesn't affect the stairstepping and interlacing
tfm(pp=0).tdecimate() # remove pulldown
nnedi2(field=-2) # bob individual fields
merge(selecteven(), selectodd()) # merge them
bicubicresize(width, 720) # downsize vertically as it is now save to do, cause interlacing is completely gone.

[Efenstor]

Furthermore the source only contains detail up to 720p.
Quite interesting, never noticed that, may be the source is too fast-moving? The manufacturer clearly states that the camcorder has a 3.1-megapixel full HD CMOS sensor (http://www.amazon.com/Canon-HF100-Definition-Camcorder-Stabilized/dp/B00114162K) and on steady shots that is pretty obvious:

Original:
http://s3.sendpic.ru/big/1015/t/a2.jpeg (http://s3.sendpic.ru/big/1015/i/a2.jpeg.html)

Downsized to 720p then upsized back to 1080p (with Lanczos):
http://s3.sendpic.ru/big/1015/t/32.jpeg (http://s3.sendpic.ru/big/1015/i/32.jpeg.html)

[Didée]

Scharfi - Good suggestion. The merging of two NNEDI-interpolated fields reminds me of an AA-script I once had come up with for Chainmax.

Though, if speed is relevant of any sort, and seeing you suggest a catmull-rom bicubic resize at the end, you could as well just omit the NNEDI thingy: bicubicresize(1280,720) is soft enough to remove the combing on its own ... ;)

[Efenstor]

Thank you, scharfis! Your method is actually much better, yet it's slow. And a slight sharpen (.35 or like that) is also needed after processing as nndi2 smooths the image a bit.

Is there a way to speed-up nnedi2 besides SetMTMode?

[Efenstor]

Updated the first post.

[Didée]

Hmh, NNEDI2 should use all cores by default, shouldn't it? I'm still on a singlecore so can't tell ... but the readme says so.

Regarding (NNEDI2+NNEDI2)/2 * sharpen - based on my old script here (http://forum.doom9.org/showthread.php?p=916871#post916871) (hey, it's been 3 years since)
o = last
AssumeTFF().SeparateFields()
dbl = mt_Average( SelectEven().EEDI2(field=1,maxd=4),
\ SelectOdd() .EEDI2(field=0,maxd=4), U=3,V=3 )
dblD = mt_MakeDiff(o,dbl,U=3,V=3)
shrpD = mt_MakeDiff(dbl,dbl.RemoveGrain(11),U=3,V=3)
DD = shrpD.Repair(dblD,13)

dbl.mt_AddDiff(DD,U=3,V=3)

Quickly ported to NNEDI2:
o = last
o.NNEDI2(field=-2)
dbl = Merge(SelectEven(),SelectOdd())
dblD = mt_MakeDiff(o,dbl,U=3,V=3)
shrpD = mt_MakeDiff(dbl,dbl.RemoveGrain(11),U=3,V=3)
DD = shrpD.Repair(dblD,13)

dbl.mt_AddDiff(DD,U=3,V=3)

For Full-HD, perhaps RemoveGrain(20) works better (since stronger) than RG(11).

[Efenstor]

I have 4.5 fps with Sharfis' method. No good for mass-processing. :( Frankly I don't know yet to which method I will stick personally.
Didée, your method is going to be even slower...

[wonkey_monkey]

Of course, but don't forget it's 24p, sharpness and details are lost after such brute methods.

I'd argue that there isn't really enough sharpness and detail present in the original to warrant anything more CPU-intensive.

David

[Didée]

I have 4.5 fps with Sharfis' method.
You guys have problems. That's my usual speed for source -> no filters -> encode, on standard definition. :eek:

Didée, your method is going to be even slower...

Experiment: try replacing "NNEDI2(field=-2)" with "Yadif(mode=1)". It'll be lightyears faster ... for the quality difference, I'm not sure. Just try.

[Blue_MiSfit]

@ Didee

/hug

I'm so so sorry :(

I'd send you a dual i7 rig if I had some spare coin. No joke.

~MiSfit

[WorBry]

Why not shoot 30p (or more correctly, 30PsF) if these 24p pull-down interlace artifacts are so problematic?

BTW - I see that Canon state that their new HFS series AVCHD camcorders (HFS21, HFS200) will (set for release April 2010) offer 'native 24p' recording, in addition to the '60i-wrapped' 24p (Cinema) mode. About time too. I wonder if its still PsF though.

http://www.usa.canon.com/consumer/controller?act=ModelInfoAct&fcategoryid=177&modelid=19829#ModelTechSpecsAct

[Efenstor]

Well, true cinematic look worths it. :) Also, while in 24 fps moderately fast pans look "stuttery", in 30 fps they are already too "jittery", and I personally have problems following them. I tried 30p and don't like it much, 24p is still the best to me, and 60i comes second.

[shoopdabloop]

what kind of 24p camera taints the footage like that during interlacing?

[um3k]

One that encodes with hard pulldown and a not-very-well-implemented codec. I've seen it occasionally on my HV20 HDV camera, as well.