Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > General > DVD2AVI / DGIndex

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th April 2017, 06:17   #1  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Forced film decimation algorithm?

What decimation algorithm do DVD2AVI/DGIndex use when processing 30fps content in Forced Film mode?
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 14th April 2017, 08:05   #2  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,263
From the manual:
Quote:
Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1. Also do not use this option on streams containing frame repeats.
Don't know whether this answers your question.
Sharc is offline   Reply With Quote
Old 15th April 2017, 03:40   #3  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
It doesn't.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 15th April 2017, 03:59   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Basically it just switches off the soft pulldown (does not insert extra frames made up from 2 repeat fields from different frames).
Why it says "inserting/removing", I dont know (so far as I know, it just does not insert extra ones, and only plays the
frames/fields from the original source).
Perhaps someone can clarify the 'insert' of "inserting/removing frames" thing.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 15th April 2017, 07:55   #5  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,263
"insertion" as a last resort for badly edited sources with broken pulldown patterns? Just guessing.
The source code which is available for download would probably clarify this, if somebody has the skills and patience ....
Sharc is offline   Reply With Quote
Old 15th April 2017, 08:40   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
That sounds like good a guess to me
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 19th May 2017, 22:55   #7  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Quote:
Originally Posted by StainlessS View Post
Basically it just switches off the soft pulldown
There is no soft pulldown in the 30fps parts.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 19th May 2017, 23:07   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Forced Film should not be used on hybrid source, which if it contained '30fps parts' and telecine film parts, it would be (hybrid).

EDIT:
DGIndex, does not support output of Variable Frame Rate, dont know about DVD2AVI but suspect also not.
Using ForcedFilm, is saying that the entire clip is soft telecine, what result you would get from hybrid is to me a mystery.

EDIT:
Quote:
Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1. Also do not use this option on streams containing frame repeats.
So for hybrid;- Soft Telecine parts, will output the encoded frames, 30fps parts will have fields thrown away to achieve 23.976 output in sync with the audio.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 19th May 2017 at 23:54.
StainlessS is offline   Reply With Quote
Old 20th May 2017, 21:01   #9  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 326
That's correct.

A running tally is kept of the output rate versus 23.976 (after ignoring soft pulldown). If it exceeds that rate then fields are dropped. If it falls short, fields are added. It's reminiscent of Bresenham's algorithm for line drawing. It was conceived originally by jackei, of course. It's generally useful only for relatively small deviations from pure 3:2 pulldown, e.g., when the main movie is 3:2 but intros and credits are not. That is the reason for the conventional wisdom of when to use force film (90-100% film).
videoh is offline   Reply With Quote
Old 24th May 2017, 04:26   #10  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Quote:
Originally Posted by StainlessS View Post
Forced Film should not be used on hybrid source, which if it contained '30fps parts' and telecine film parts, it would be (hybrid).
Yes, I know. However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.

Quote:
Originally Posted by videoh View Post
A running tally is kept of the output rate versus 23.976 (after ignoring soft pulldown). If it exceeds that rate then fields are dropped.
Yes but I'm asking how it chooses which frames to drop.
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

Last edited by Katie Boundary; 24th May 2017 at 04:42.
Katie Boundary is offline   Reply With Quote
Old 24th May 2017, 10:35   #11  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,484
Quote:
Originally Posted by Katie Boundary View Post
Yes, I know. However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.
You'll have frames with combing.

Quote:
Originally Posted by Katie Boundary View Post
Yes but I'm asking how it chooses which frames to drop.
I only tried this because the question was asked and I was curious and thought I might learn something. Katie will probably still be too busy ignoring.

I indexed an NTSC DVD for a look. It was actually PAL converted to NTSC using field blending, but it was handy and that should mean it's 100% video (no pulldown flags) so I created a d2v file using force film and again without it. The d2v files declared it to be 100% video.
I compared the two by opening them with Avisynth and stepped through the frames. The 23.976fps (force film) version obviously had every fifth frame missing relative to the 29.97fps version. It doesn't seem any more clever than that.

I've never been a fan of force film myself. I recall having problems when I was quite an NTSC novice with a particular DVD because it had a few small sections with hard-coded pulldown while the majority used pulldown flags and there was combing in the sections with hard-coded pulldown. Since then I've always honoured pulldown flags and taken the TFM().TDecimate() route and more recently, for hybrid sources, I've been using TIVTC for VFR encoding. x264 is VFR aware if you give it a timecodes file so it's not that hard. Harder for mpeg2/Xvid. Not VCD compatible.
hello_hello is offline   Reply With Quote
Old 24th May 2017, 10:41   #12  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,098
Quote:
Originally Posted by Katie Boundary View Post
However, there are a LOT of discs out there that are almost but not quite pure film - they'll be 99.78% film or 99.12% film or whatever, and I'd like to know just what kind of butchery might be performed on the video if I run such content through forced film.
If you're lucky the video portions will be some black frames at the beginning or end or perhaps some black during a fade and nothing bad happens.

However, there are a lot of cases where a DVD might drop to video briefly during some live action and in such cases you'll get some brief interlacing. For example, the Criterion DVDs drop to video around chapter points often, and they'll give you a couple of very noticeable interlaced frames.

To be safe, it's not a good idea to use Forced Film unless the D2V shows it to be 100% Film.
manono is offline   Reply With Quote
Old 24th May 2017, 11:35   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Quote:
Originally Posted by Katie Boundary View Post
Yes but I'm asking how it chooses which frames to drop.
Each field will have a time stamp (I think), and the ones that are closest to the required output time (@23.976) will be kept, and the ones furthest from req time will be dumped. (methinks)

NOTE, fields (top and bottom) dropped rather than frames.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 24th May 2017, 11:58   #14  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 326
Quote:
Originally Posted by StainlessS View Post
Each field will have a time stamp (I think), and the ones that are closest to the required output time (@23.976) will be kept, and the ones furthest from req time will be dumped. (methinks)
That is not true. The source code is available if anyone wants to understand the algorithm.
videoh is offline   Reply With Quote
Old 25th May 2017, 07:20   #15  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Quote:
Originally Posted by manono View Post
To be safe, it's not a good idea to...
Well, that just guarantees that I'll try to do it.

Quote:
Originally Posted by videoh View Post
That is not true. The source code is available if anyone wants to understand the algorithm.
Perhaps you could be kind enough to explain it to those of us who don't speak C++?
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 25th May 2017, 16:09   #16  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 326
For 3:2 pulldown you expect as you traverse the stream to see a certain rate of RFF flags. If the rate falls below, then you mark the current field to be repeated. If the rate gets too high, then you mark a field to be discarded. As I mentioned earlier, it is reminiscent of Bresenham's line drawing algorithm. MPEG2Source() creates a table this way and then uses it to decide what to return based on the requested frame number. Most of the magic is in the table and how you actually implement pulling down or deleting fields to generate the output frames.

https://www.cs.helsinki.fi/group/goa...s/bresenh.html

Typically, the RFF rate gets too low, e.g., where you have a video section, or where you have hard pulldown instead of soft. But it is possible to have a too-high RFF rate, and that is why the algorithm allows for it.

Last edited by videoh; 25th May 2017 at 16:18.
videoh is offline   Reply With Quote
Old 26th May 2017, 16:22   #17  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Does this mean that, if a source was originally 100% film but it gets transferred to DVD as a mix of "hard" and "soft" telecine, Forced Film will deduce the correct fields to drop based purely on the disruptions in the pulldown pattern?
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 26th May 2017, 19:32   #18  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,098
Quote:
Originally Posted by Katie Boundary View Post
Does this mean that, if a source was originally 100% film but it gets transferred to DVD as a mix of "hard" and "soft" telecine, Forced Film will deduce the correct fields to drop based purely on the disruptions in the pulldown pattern?
No, it means no such thing. That kind of source is where you find film percentages of less than 100% in the D2V. And if the film percentage is less than the 95% often given as the point below which you shouldn't use 'Forced Film' (although I think it's an arbitrary and misleading figure), you're almost guaranteed to wind up with some interlaced frames and some jerky-playing places.

What's the big deal? Just make the D2V using 'Honor Pulldown Flags' and IVTC it if it's a mix of hard and soft telecine, no matter the percentage. That's much better than going through the whole conversion process and later noticing a few interlaced frames here and there.
manono is offline   Reply With Quote
Old 27th May 2017, 23:59   #19  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 493
Quote:
Originally Posted by manono View Post
What's the big deal?
* no field-matching filter is perfect.
* decimation filters are even less perfect
* Some TV shows on DVD will be encoded in an odd way wherein some episodes will be pure film and others will be 98%-99.99% film, and it's annoying to have to check DGindex results to see which field operation I should have used and which avisynth scripts need the extra TFM(mode=0,pp=0,micmatching=0).tdecimate() and which ones don't.

As my sig says... "Katie reasons"
__________________
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.
Katie Boundary is offline   Reply With Quote
Old 28th May 2017, 01:24   #20  |  Link
manono
Moderator
 
Join Date: Oct 2001
Location: Hawaii
Posts: 7,098
Quote:
Originally Posted by Katie Boundary View Post
* no field-matching filter is perfect.
They can be tweaked in those rare instances when the defaults don't work perfectly. It's still way better than using Forced Film when you shouldn't have and only finding out later after having done all the work. And you can parse the D2V to find out where the video portions are and decide if you should handle it differently (if, for example, the little bit of video is during harmless black frames and not some active video).
Quote:
* decimation filters are even less perfect
Same as before.
Quote:
...and it's annoying to have to check DGindex results to see which field operation I should have used...
The D2V is easily edited so you don't have to run the whole thing all over again.
Quote:
As my sig says... "Katie reasons"
And that justifies your sometime irrationality?

With 'quirky' sources, yes, deciding what to do can be a pain in the ass. Sometimes there's no 'right' way. You've fooled with enough of the space shows with the CGI that messes everything up to know this already.
manono is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:56.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.