480i to 1080i/p with Film Mode detection

[Lyris]

Hi forumers,

When I receive material that's 480i and is a mix of 60i video camera content and 3:2 pulldown film clips, right now I'm running everything through QTGMC to generate a nice 60p version, then going to 1080i/60 on Blu-ray to preserve the video camera aesthetic:

qtinput("bonusfeature.mov") #60i n
qtGMC(preset="fast") #60p output
convertTOyv12
crop(8,0,-8,-0)
spline64resize(1440,1080)
addborders(240,0,240,0)
assumeTFF #so the output field order is correct
SeparateFields() # 60 frames becomes 120 fields
SelectEvery(4, 0, 3) # 120 fields to 60 fields (selects 2 from 4)
Weave()

However, what I'd like to do is detect the segments that are 3:2 pulldown from film, inverse 3:2 them, and then regenerate new 3:2 pulldown in 1080i. In other words, I'd like to avoid doing video deinterlacing on the film segments.

Most functions (animeIVTC, etc) seem to be geared towards getting a 23.98fps output with optional handling of video-generated content like ending credits or fades. What I'm looking for is the other way around, priority towards video camera material with intercut film content.

Does anyone have any pointers as to how I would do this?

[DarkSpace]

Note that this is completely untested, but I believe it should work.

source = blah()
deint = source.QTGMC()
even30 = source.TFM(PP=2, clip2=deint.SelectEven())
odd30 = source.TFM(PP=2, clip2=deint.SelectOdd())
Interleave(even30, odd30)

What it does:
TFM tries to reconstruct the original progressive frames from the source. When it decides that after the attempt at reconstruction there is still interlacing left in the frame, it replaces the current frame with the frame from clip2. Because that produces only 30p video from a 60i source, it will have half the number of frames that the QTGMC output has (60p). Therefore, I'm doing this twice, once replacing the interlaced frame with the deinterlaced first field of the source, the other replacing the same frame with the deinterlaced second field from the source. That results in 2 30p clips, which are then interleaved into one 60p clip.

I know that my explanations can be very confusing, so if you have questions, please ask.

[Lyris]

Yup, that works - many thanks! I would never have thought of that!

Although, this is one half of the problem. This produces a clean 60p clip. While still an improvement, there's still the case of going back to 1080i/60.

[colours]

You just need AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave(), which you already had in your original script.

In addition, you might want to disable TFM's micmatching, which can cause transitions between telecined and 60i content to not be frame-accurate.

Also, Blu-ray allows for 1440x1080 content (according to Wikipedia), in which case you don't really need to pad it with borders to make it 1920x1080.

[DarkSpace]

Originally Posted by colours

In addition, you might want to disable TFM's micmatching, which can cause transitions between telecined and 60i content to not be frame-accurate.

I did mention that it was untested and should work in principle?
I didn't put that much thought into it, and I didn't test it either... so I'll just assume that you're more knowledgeable about that than I am.

Lyris, I thought that your problem was just the getting-a-60p-clip part, so I left the other stuff out. I also left the QTGMC arguments aside (though you only specified a preset).
I'm glad it works!

[ChiDragon]

Shouldn't one of the TFM lines be set to order=0 and the other to order=1? The TFM hackaround method has never worked properly for me when I've tried it (extra duplicate frames), but perhaps I forgot to disable micmatching as well. I would also recommend setting mode=0,slow=2.

Originally Posted by Lyris

Most functions (animeIVTC, etc) seem to be geared towards getting a 23.98fps output with optional handling of video-generated content like ending credits or fades. What I'm looking for is the other way around, priority towards video camera material with intercut film content.

That's what TDeint(1,tryWeave=true) does. TDeint's own interpolation can be replaced with external filters, but I'm not sure how well it would play with QTGMC. There's probably some way to use masks so that field-matched frames are used from TDeint while the rest are from QTGMC but that may end up being more complex than getting the TFM thing working 100%.

Originally Posted by colours

Also, Blu-ray allows for 1440x1080 content (according to Wikipedia), in which case you don't really need to pad it with borders to make it 1920x1080.

All HD video on Blu-ray needs to be 16:9 formatted; the 1440x1080 is the deprecated anamorphic form.

[Lyris]

^ Yeah, the 1440x1080 is for "HDV-style" 1.33 pixel aspect ratio stuff.

Thanks for the pointers, all. I'll give this a look!

[DarkSpace]

Originally Posted by ChiDragon

Shouldn't one of the TFM lines be set to order=0 and the other to order=1?

No, why? The field order doesn't change, so why should the matching change?
Assume you've got a 30p-as-60i clip, and apply this "double TFM" to it. What you want, in this case, is to have each frame repeated once, while your way of changing the matching order possibly generates two different frames...

[ChiDragon]

Originally Posted by DarkSpace

No, why?

On 2:3 material, your script produces 2:2:2:4.

[DarkSpace]

Originally Posted by ChiDragon

On 2:3 material, your script produces 2:2:2:4.

Which is exactly what you would want, isn't it? Smoothness isn't the issue, after all, because the video has to be re-interlaced with pulldown in the progressive areas later. If the script doesn't produce this output, the pulldown pattern will have changed. And because of the re-interlacing, half the frames (literally ) will be dropped anyway...

[ChiDragon]

Originally Posted by DarkSpace

Smoothness isn't the issue, after all, because the video has to be re-interlaced with pulldown in the progressive areas later. If the script doesn't produce this output, the pulldown pattern will have changed.

In re-interlacing, the pulldown pattern present in the 59.94-frame video gets carried over 1-for-1 to the pattern present in the 59.94-field video. Your script changes this pulldown pattern. When re-interlaced the output becomes 4 progressive + 1 dupe instead of 3 progressive + 2 interlaced.

Originally Posted by ChiDragon

I would also recommend setting mode=0,slow=2.

Slow=2 is good but I was wrong about mode=0. It needs to be set to 1 (which is the default) for this bobbing function. [EDIT: Read post #16 for corrections to this.]

One other thing: I think instead of placing ConvertToYV12 before the upscale, more chroma would be preserved from a 4:2:2 source by placing ConvertToYV12(interlaced=true) at the end.

[DarkSpace]

Originally Posted by ChiDragon

In re-interlacing, the pulldown pattern present in the 59.94-frame video gets carried over 1-for-1 to the pattern present in the 59.94-field video. Your script changes this pulldown pattern. When re-interlaced the output becomes 4 progressive + 1 dupe instead of 3 progressive + 2 interlaced.

You're right, I didn't think of that. Now I'm embarrassed.

Originally Posted by ChiDragon

One other thing: I think instead of placing ConvertToYV12 before the upscale, more chroma would be preserved from a 4:2:2 source by placing ConvertToYV12(interlaced=true) at the end.

I completely agree, but I thought that there must be a reason for this, so I didn't comment on it...

On another note, it may be more efficient to just scale the fields/frames without deinterlacing, with a function that scales each frame based on whether it is interlaced (scale fields separately) or not (scale as progressive).
Yet another thought: Is it feasible to just separate the 3:2 pulldown content from the 60i content, and process those separately, including encoding the 24p segments as 24p with soft pulldown to 60i?

[ChiDragon]

Originally Posted by DarkSpace

On another note, it may be more efficient to just scale the fields/frames without deinterlacing, with a function that scales each frame based on whether it is interlaced (scale fields separately)

I believe this wouldn't work, due to the spatial positioning of the fields relative to each other (see the Bob documentation's explanation of why SeparateFields.Resize is bad).

[TheSkiller]

Originally Posted by DarkSpace

On another note, it may be more efficient to just scale the fields/frames without deinterlacing, with a function that scales each frame based on whether it is interlaced (scale fields separately) or not (scale as progressive).

Interlaced resizing is possible and fast yes (but there's no film/video detection), but not recommendable at all for upsizing. A 480i -> 1080i interlaced resize would look horrific. Even if you just use plain and ultra fast Bob() you get much better results because there are twice as many lines of video to apply a resize on (even if half of them are just interpolated).


@ ChiDragon
This is a function which does interlaced resizing with correct field alignment and chroma.

However, like I've said, Bob() provides better quality than field based resizing and it's still really really fast.

[colours]

Using Bob() prior to resizing isn't very much different from just using an interlaced resize directly. You still get all the ugly aliasing since Bob() is a linear filter; possibly even more since you have two resizing steps instead of just one. Having "twice as many lines of video" is a complete red herring because those extra lines don't have any extra detail.

To get rid of aliasing, what you need is a nonlinear filter. The QTGMC+TFM combination DarkSpace suggested (with corrections as ChiDragon pointed out) works perfectly for that.

[ChiDragon]

Playing with this further recently, it came to my attention that order isn't the correct option to toggle for the even and odd fields. It technically works, but causes match failures in some instances. Swapping the field parameter instead is better, and allows mode=0 to be selected as I originally suggested. This should be more accurate and slightly faster.

Here is the current script I'm using for BFF material. For TFF, the field numbers need to be swapped or motion will stutter back-and-forth very obviously.

deint  = QTGMC(preset="slow") #set however you prefer
even30 = TFM(field=0,mode=0,slow=2,micmatching=0,PP=2,clip2=deint.SelectEven())
odd30  = TFM(field=1,mode=0,slow=2,micmatching=0,PP=2,clip2=deint.SelectOdd())
Interleave(even30,odd30)

Small combs still remain, like on 60Hz fades to black as well as 60Hz dissolves (between shots, or credits that dissolve in) and as always there are frames that TFM simply fails to match correctly. Depending on your level of perfectionism/OCD you can always tweak the comb detection and/or use an ovr file to specify individual frames that it got wrong.