Treating duplicated fields

[MarcN]

Hello!

I have recently captured a movie from TV that seems to have some strange interlacing artefacts. In general the film is progressive but sometimes it looks like it was stretched in time by adding an extra field. That of course results in interlaced looking frames. Now I want to encode the movie with CCE. The easy way would be to treat it as interlaced and don’t worry about it being progressive most of the time.

But I wonder, is there a way to restore the movie somehow, so that it is really progressive again? Searching the forum I stumbled across tritical’s TFM. Using the default mode removes the interlaced frames but now I have duplicate frames in the video here and there. TDecimate needs a pattern to work, right? Unfortunately there seems to be none. Any Ideas?

Thanks a lot, Marc!

[hartford]

Is your capture NTSC or PAL? Knowing that helps.

[Guest]

Assuming it's NTSC, it sounds like hybrid video and 3:2 pulled-down material. If you field match the 3:2 sections to make them progressive, you'll get duplicates there. Now you have the Devil's Dilemma of deciding whether to render the output at video rate or film rate. If you leave it at video rate, you'll still have the duplicates in the film section. If you decimate the duplicates, you'll have to decimate the video, too, to keep audio sync, thereby making the video jerky.

You can go for some kind of VFR solution if your output device permits it. Otherwise, choose your poison. To me, the duplicates are less objectionable than jerky video sections.

[MarcN]

Oops, sorry forgot to mention that. It's PAL :eek:

The problem is, that the extra fields are only present in some sections of the film. Most of the time the picture is fine. I guess the TV station wanted to lengthen the movie a little bit to squeeze in an extra commercial break.

I think it would help if I could identify those regions. TFM allows this via outputC options. Now I would like to know what is the fastest way to process the video. That’s running the video through TFM while not creating/encoding the video.

Thanks, Marc.

[scharfis_brain]

Provide an unprocessed sample video, that shows several duplicated fields.
at least 10 seconds.

[MarcN]

Unfortunately I’m on Dial-Up at the moment, so I can’t upload big videos right now. :(

I will do some test encodings over night to help me with my decision. I’m also going to use Graphedit and TFM to find out where the extra fields in the video are.

[Guest]

I guess the TV station wanted to lengthen the movie a little bit to squeeze in an extra commercial break. Wouldn't they shorten the movie for that?

What makes you think there are extra fields? Maybe it is just field-blended. Do separatefields() and then see if any of the fields are blends of two pictures. If so, scharfis_brain is the guy to help with that.

[Didée]

Wouldn't they shorten the movie for that?
Not if MarcN captured from a German TV channel.

Dunno for other countries, but here it is like that, basically:
If a movie's running time*) is shorter than 'X' minutes, then it is allowed to put e.g. 20 minutes of advertisement in it. If the running time is more than 'X' minutes, then e.g. 30 minutes are allowed.

*) running time in whatever form the TV station puts the show on air ... !


So, if a movie would run just slightly shorter than X minutes, its running time is stretched slightly to just exceed 'X', and ... ta-daaa ... 10 or 15 minutes more advertisement.
Depending on the actual TV channel, this stretching is done either by duplicating fields, or by fieldblending.

Another possibility is that the movie was cutted (violence etc.) to be allowed to show it at an earlier time of day. In this case, they often stretch the remaining movie to still reach the un-cutted running time, so that noone can see already in advance "oh, it's a cutted version they're showing."

It's a very nasty problem, and an automated solution to get back throughout progressive content is not possible, but requires pretty much manual work
If I capture something with this sort of deformation, I usually just delete it, and wait for the next time it is shown ...

[LocalH]

There are two ways to present 24fps material through PAL video. The first way is to speed up the material by 4%, making it run at 25fps with no extra fields (each original frame gets two fields). The second is to borrow from NTSC's 3:2 pulldown and add some extra fields here and there to keep the video the same length, although the number of extra fields would be much lower than on NTSC. It sounds like this was given the latter treatment. Basically, it sounds like each original frame is given at least two fields (or one full frame) which then makes 48 fields per second, then the 24th and 48th fields would be duplicated, giving those two original frames three fields and ending up with a total of 50 fields, which is exactly 25 frames PAL. I would think that every time this happens, mice teeth suddenly alternate between being present or not being present. And as long as the pattern is consistent (which it probably isn't, any more than broadcast 3:2 pulldown is), it wouldn't be too hard to whip up an Avisynth script to discard those two extra fields per second. That's also assuming that the station (or distributor) didn't elect to use time-compression technology to shorten the air time (some US cable channels are notorious for this).

[MarcN]

Hello

Sorry for responding so late. :o

What makes you think there are extra fields? Maybe it is just field-blended. Do separatefields() and then see if any of the fields are blends of two pictures.
separatefields() shows three consecutive field that are more or less the same, but definitely not blended.

As I said, there is no pattern with these extra fields. Large sections (up to 20 minutes) are perfectly progressive. But I think I have found a solutions that produces quite good results:

Telecide(order=1)
Decimate(mode=1,threshold=3.0)

That way I get mostly progressive frames and only here and there a blended one. What do u think? To me it looks quite good on DVD.

Marc

[Guest]

Yes, that's a perfectly reasonable way to handle hybrid video/film. Short of some VFR solution, your options are limited.

[MarcN]

I'm encoding for DVD, so VFR is not an option. :D

Thanks for your help!

[scharfis_brain]

Telecide(order=1)
Decimate(mode=1,threshold=3.0)

will destroy your PAL-movie because decimates kicks out every 5th frame, resulting in an 20 fps-video with PAL-input.

Do NOT use it.

but since you are Aiming for DVD, I suggest to leave it interlaced if you won't spend time into manual pattern finding and scene-wise processing.

[MarcN]

Telecide(order=1)
Decimate(mode=1,threshold=3.0)

will destroy your PAL-movie because decimates kicks out every 5th frame, resulting in an 20 fps-video with PAL-input.
Are you sure? :confused:

I thought Decimate does not throw away any frames in mode 1.
Here is what the documentation says:

If mode=1, instead of discarding the most similar frame, Decimate() will either replace it with a frame interpolated between the current frame (usually a duplicate of the preceding frame) and the following frame, or it will pass the frame through as is. The choice between these two depends on two things: 1) whether the cycle of frames is from 3:2 pulldown material or from pure video, and 2) the threshold parameter setting and on how different the frame is from its preceding frame (see below). The first requires guide=1 for Telecide (to declare the pulldown pattern).

Decimate(mode=1) is useful for hybrid clips having mostly video. It allows you to leave the frame rate at video rates (to get smooth video sequences) but also to ameliorate the effect of duplicate frames that are emitted by Telecide() for film sequences (frames that are normally removed with mode=0).

But I guess neuron2 can clarify that. :)

[Guest]

Oh, it's PAL? :)

I need to re-read threads. OP's last post implied he had an NTSC hybrid. If not, who knows what he has?!

I've been cogitating on YV12 upconversions. Sorry for not paying attention properly.

Assuming it really is progressive PAL with extra fields thrown in, then you can fix the combing with just Telecide. If you want to make the result a bit smoother but at the expense of some blending, then yes, Decimate in its blending/nondecimating mode can be used. So the script appears OK.

[MarcN]

If not, who knows what he has?!A very, very badly treated movie, I captured from TV. :angry:

Without decimate the movie looks jerky so it has to stay in. And I think my bitrate is too low for a good interlaced encoding with CCE. ProCoder's field based encoding might do a better job with that. I guess I have to do some more testing. :D