Inconsistent IVTC / Artifact Removal

[thecoreyburton]

Hey guys,

I have a DVD that I'm not sure how to go about decimating. For the most part, the source is telecined and TFM/TDecimate can easily handle the pulldown, but for a few of the scenes I get jerky motion. At first I just thought perhaps those scenes were supposed to be at 29.97fps but investigation revealed that rather than a consistent 3:2 telecine there are sections which have four combed frames in a row. QTGMC and SRestore don't seem to provide any better results.

Is this something anyone's come across before?

Sample @ Mediafire (http://www.mediafire.com/download/ckgucmsw3sjlcke/sample-source.zip)

[bxyhxyh]

Well, I don't know if deinterlacing is correct choice.

It worked ok on your sample
QTGMC("very slow")
TDecimate(cycle=9,cycler=5)

But I would use
QTGMC("very slow")
TDecimate(cycle=2)
#decimate again to make VFR

Instead of deinterlacing you can use ConditionalFilter to choose good field match from different order and field values.

[ChiDragon]

The proper way to examine the cause of combing is by viewing the fields, not the frames. SeparateFields() or Bob().

The pan is actually jerky in your source. There are orphaned fields. Even viewed on a CRT, it wouldn't be smooth.

I wonder whether it aired on TV the same way. Perhaps they screwed up the DVD transfer.

[johnmeyer]

I looked at the video one field at a time (see ChiDragon's post). The pattern is irregular; it doesn't repeat exactly; and there are definitely a few fields that are orphaned (i.e., they are from a different point in time than either of the adjacent fields, something that shouldn't happen when film is properly telecined.

In addition, there are huge spatial jumps at some points. I don't normally bother looking at animation because the cadence can be so strange, so I don't know if the big spatial jumps between some fields are normal. Here is what I was seeing during one portion of the film. "3" indicates three fields in a row from the same moment in time; "2" indicates two fields from the same moment in time (i.e., a standard progressive frame) and "1" indicates a field that is from a different moment in time than either of its adjacent neighbors:

3 2 2 2 3 2 3 1 3 2 3 2 2 2 3 2 2 2 3 2 2 2 3 2 3 1 3 2 3 1 3 2 3 2 2 2 3 2 2 3 2

This is not meant to be a complete cycle, although if you start with the first "3 1 3" and go forward and back from that point, you will see some similarity to the point where "3 1 3" happens the next time. However, the pattern doesn't repeat exactly.

I can think of all sorts of convoluted ways to make this work, including the script that Didée helped me create several years ago, where I had to decimate a lot of duplicates, but then look for big spatial gaps and synthesize (using motion estimation) frames to fill in those gaps.

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?t=161758&highlight=duplicates)

I don't think there is any easy solution for this one.

[LemMotlow]

And it looks like the field order was reversed in goofy processing.

[johnmeyer]

I didn't see any field reversals, but I didn't have time to walk through every field in the clip.

[thecoreyburton]

Hmm, that's unfortunate. Thank you for all your help so far. I might have a look at the PAL DVDs for this show (as I own them, too). From what I recall they're not interlaced (except for the first season) so provided I'm right and there isn't any blend-removal required they might be the superior source to work with in this case.

Edit: Alright, got two samples of the PAL DVDs here here (https://mega.nz/#!SN9CSBxZ!oECQL0JIC7SGr7WZ8K-vAW-t6N-q7AfZtkPd8AtaA74). Definitely a much smoother source (for instance, the scene from the NTSC sample I posted earlier has no problems whatsoever). The issue now is the artifacts present in clip 1 (particularly noticeable on the lines on the roof, and then moreso on the tables in the following scene). A quick pass of QTGMC reduces the effects, but doesn't eliminate them. I believe that's because upon inspection, the animation is done at 12.5fps and the fluctuation occurs every two frames because it plays at 25fps. I'd just halve the frame rate and do it like that, but some scenes (particularly panning) is done at the full frame rate, that's why I included the second clip. What's the best way to remove the artifacts in the first clip without altering the flow of the motion in the second (the same script will end up being used for both as they're sections from the same episode)? I'm terrible at explaining so hopefully the samples do a better job. You're welcome to ask any questions or request anything you need.