Forced film decimation algorithm?

[Katie Boundary]

What decimation algorithm do DVD2AVI/DGIndex use when processing 30fps content in Forced Film mode?

[Sharc]

From the manual:

Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1. Also do not use this option on streams containing frame repeats.

Don't know whether this answers your question.

[Katie Boundary]

It doesn't.

[StainlessS]

Basically it just switches off the soft pulldown (does not insert extra frames made up from 2 repeat fields from different frames).
Why it says "inserting/removing", I dont know (so far as I know, it just does not insert extra ones, and only plays the
frames/fields from the original source).
Perhaps someone can clarify the 'insert' of "inserting/removing frames" thing.

[Sharc]

"insertion" as a last resort for badly edited sources with broken pulldown patterns? Just guessing.
The source code which is available for download would probably clarify this, if somebody has the skills and patience ....

[StainlessS]

That sounds like good a guess to me

[Katie Boundary]

Originally Posted by StainlessS

Basically it just switches off the soft pulldown

There is no soft pulldown in the 30fps parts.

[StainlessS]

Forced Film should not be used on hybrid source, which if it contained '30fps parts' and telecine film parts, it would be (hybrid).

EDIT:
DGIndex, does not support output of Variable Frame Rate, dont know about DVD2AVI but suspect also not.
Using ForcedFilm, is saying that the entire clip is soft telecine, what result you would get from hybrid is to me a mystery.

EDIT:

Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1. Also do not use this option on streams containing frame repeats.

So for hybrid;- Soft Telecine parts, will output the encoded frames, 30fps parts will have fields thrown away to achieve 23.976 output in sync with the audio.

[videoh]

That's correct.

A running tally is kept of the output rate versus 23.976 (after ignoring soft pulldown). If it exceeds that rate then fields are dropped. If it falls short, fields are added. It's reminiscent of Bresenham's algorithm for line drawing. It was conceived originally by jackei, of course. It's generally useful only for relatively small deviations from pure 3:2 pulldown, e.g., when the main movie is 3:2 but intros and credits are not. That is the reason for the conventional wisdom of when to use force film (90-100% film).

[Katie Boundary]

Originally Posted by StainlessS

Forced Film should not be used on hybrid source, which if it contained '30fps parts' and telecine film parts, it would be (hybrid).

Yes, I know. However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.

Originally Posted by videoh

A running tally is kept of the output rate versus 23.976 (after ignoring soft pulldown). If it exceeds that rate then fields are dropped.

Yes but I'm asking how it chooses which frames to drop.

[hello_hello]

Originally Posted by Katie Boundary

Yes, I know. However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.

You'll have frames with combing.

Originally Posted by Katie Boundary

Yes but I'm asking how it chooses which frames to drop.

I only tried this because the question was asked and I was curious and thought I might learn something. Katie will probably still be too busy ignoring.

I indexed an NTSC DVD for a look. It was actually PAL converted to NTSC using field blending, but it was handy and that should mean it's 100% video (no pulldown flags) so I created a d2v file using force film and again without it. The d2v files declared it to be 100% video.
I compared the two by opening them with Avisynth and stepped through the frames. The 23.976fps (force film) version obviously had every fifth frame missing relative to the 29.97fps version. It doesn't seem any more clever than that.

I've never been a fan of force film myself. I recall having problems when I was quite an NTSC novice with a particular DVD because it had a few small sections with hard-coded pulldown while the majority used pulldown flags and there was combing in the sections with hard-coded pulldown. Since then I've always honoured pulldown flags and taken the TFM().TDecimate() route and more recently, for hybrid sources, I've been using TIVTC for VFR encoding. x264 is VFR aware if you give it a timecodes file so it's not that hard. Harder for mpeg2/Xvid. Not VCD compatible.

[manono]

Originally Posted by Katie Boundary

However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.

If you're lucky the video portions will be some black frames at the beginning or end or perhaps some black during a fade and nothing bad happens.

However, there are a lot of cases where a DVD might drop to video briefly during some live action and in such cases you'll get some brief interlacing. For example, the Criterion DVDs drop to video around chapter points often, and they'll give you a couple of very noticeable interlaced frames.

To be safe, it's not a good idea to use Forced Film unless the D2V shows it to be 100% Film.

[StainlessS]

Originally Posted by Katie Boundary

Yes but I'm asking how it chooses which frames to drop.

Each field will have a time stamp (I think), and the ones that are closest to the required output time (@23.976) will be kept, and the ones furthest from req time will be dumped. (methinks)

NOTE, fields (top and bottom) dropped rather than frames.

[videoh]

Originally Posted by StainlessS

Each field will have a time stamp (I think), and the ones that are closest to the required output time (@23.976) will be kept, and the ones furthest from req time will be dumped. (methinks)

That is not true. The source code is available if anyone wants to understand the algorithm.

[Katie Boundary]

Originally Posted by manono

To be safe, it's not a good idea to...

Well, that just guarantees that I'll try to do it.

Originally Posted by videoh

That is not true. The source code is available if anyone wants to understand the algorithm.

Perhaps you could be kind enough to explain it to those of us who don't speak C++?

[videoh]

For 3:2 pulldown you expect as you traverse the stream to see a certain rate of RFF flags. If the rate falls below, then you mark the current field to be repeated. If the rate gets too high, then you mark a field to be discarded. As I mentioned earlier, it is reminiscent of Bresenham's line drawing algorithm. MPEG2Source() creates a table this way and then uses it to decide what to return based on the requested frame number. Most of the magic is in the table and how you actually implement pulling down or deleting fields to generate the output frames.

https://www.cs.helsinki.fi/group/goa...s/bresenh.html

Typically, the RFF rate gets too low, e.g., where you have a video section, or where you have hard pulldown instead of soft. But it is possible to have a too-high RFF rate, and that is why the algorithm allows for it.

[Katie Boundary]

Does this mean that, if a source was originally 100% film but it gets transferred to DVD as a mix of "hard" and "soft" telecine, Forced Film will deduce the correct fields to drop based purely on the disruptions in the pulldown pattern?

[manono]

Originally Posted by Katie Boundary

Does this mean that, if a source was originally 100% film but it gets transferred to DVD as a mix of "hard" and "soft" telecine, Forced Film will deduce the correct fields to drop based purely on the disruptions in the pulldown pattern?

No, it means no such thing. That kind of source is where you find film percentages of less than 100% in the D2V. And if the film percentage is less than the 95% often given as the point below which you shouldn't use 'Forced Film' (although I think it's an arbitrary and misleading figure), you're almost guaranteed to wind up with some interlaced frames and some jerky-playing places.

What's the big deal? Just make the D2V using 'Honor Pulldown Flags' and IVTC it if it's a mix of hard and soft telecine, no matter the percentage. That's much better than going through the whole conversion process and later noticing a few interlaced frames here and there.

[Katie Boundary]

Originally Posted by manono

What's the big deal?

* no field-matching filter is perfect.
* decimation filters are even less perfect
* Some TV shows on DVD will be encoded in an odd way wherein some episodes will be pure film and others will be 98%-99.99% film, and it's annoying to have to check DGindex results to see which field operation I should have used and which avisynth scripts need the extra TFM(mode=0,pp=0,micmatching=0).tdecimate() and which ones don't.

As my sig says... "Katie reasons"

[manono]

Originally Posted by Katie Boundary

* no field-matching filter is perfect.

They can be tweaked in those rare instances when the defaults don't work perfectly. It's still way better than using Forced Film when you shouldn't have and only finding out later after having done all the work. And you can parse the D2V to find out where the video portions are and decide if you should handle it differently (if, for example, the little bit of video is during harmless black frames and not some active video).

* decimation filters are even less perfect

Same as before.

...and it's annoying to have to check DGindex results to see which field operation I should have used...

The D2V is easily edited so you don't have to run the whole thing all over again.

As my sig says... "Katie reasons"

And that justifies your sometime irrationality?

With 'quirky' sources, yes, deciding what to do can be a pain in the ass. Sometimes there's no 'right' way. You've fooled with enough of the space shows with the CGI that messes everything up to know this already.