Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-2 Encoding

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th June 2010, 05:23   #321  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by mp3dom View Post
Yes, it's within the dvd specs as long as the flags/encode are made in the proper way. Note that also picture structure must be fixed to 'frame' and not change over time. I'm in the PAL land and I've made about 100+ dvd titles in that way and all passes the EclipseSuite verifier (the verifier used by replication facilities just before the glassmastering process).
Thanks for that clarification. Yes, all the frames are encoded as frames. No fields.

So, DVD FAQ #3.4: http://www.dvddemystified.com/dvdfaq.html#3.4
where it says: "MPEG-2 progressive_sequence is not allowed"

Can we debunk this FOR CERTAIN?


Also:

"In the case of 24 fps source, the encoder embeds MPEG-2 repeat_first_field flags into the video stream to make the decoder either perform 2-3 pulldown for 60Hz NTSC displays (actually 59.94Hz) or 2-2 pulldown (with resulting 4% speedup) for 50Hz PAL/SECAM displays"

Can we lay this to rest, at least for PAL?
Koutsoubos is offline   Reply With Quote
Old 14th June 2010, 15:22   #322  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 991
Quote:
Originally Posted by Koutsoubos View Post
Can we debunk this FOR CERTAIN?
To say 'for certain' you must have the dvd-video specification from the DVDForum. What I can say is that I've made a lot of these dvd and no one complain about strange incompatibility with standalone players. Also, EclipseSuite (which is very reliable) validate the verification.

Quote:
"In the case of 24 fps source, the encoder embeds MPEG-2 repeat_first_field flags into the video stream to make the decoder either perform 2-3 pulldown for 60Hz NTSC displays (actually 59.94Hz) or 2-2 pulldown (with resulting 4% speedup) for 50Hz PAL/SECAM displays"
Can we lay this to rest, at least for PAL?
If your source is 24p and you want to convert to PAL there are only 3 ways:
- Encode as 25p with 4% speedup
- Encode as 50i with 4% speedup (this is the 2:2 pulldown)
- Encode as 24p with 2^12:3 euro-pulldown

As for the third option, there's no progressive-scan players able to reconstruct the original 24p and display at 25p since the cadence is quite uncommon. You'll see it always as interlaced with 2 stutters every second. This is different with software dvd decoders that will output true 24p. I don't know if this happends even for dvd players with HDMI output. If the player can output 24p via HDMI it could be that the hdtv will display the original 24p... but I've never tried it.
mp3dom is offline   Reply With Quote
Old 15th June 2010, 09:03   #323  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Point One is quite settled in my opinion, and to draw it back to the topic of this thread, the Prog_frame=ON TFF=off Prog_sequence=ON was the default behaviour in HCEnc when doing progressive encodes.

I think it'd be good to have that default behaviour back? But anyway the choices are now clear.


As to the second point of 2:2 pulldown again! Apparently this is a nullity. Are there any real world applications of 50i, where 2:2 pulldown happens, when encoding from progressive sources (film or not, it has been sped up to 25fps frame-wise)?

Is this another possibility for HCEnc? 50i?
Koutsoubos is offline   Reply With Quote
Old 15th June 2010, 16:30   #324  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 889
Quote:
Originally Posted by Koutsoubos View Post
Are there any real world applications of 50i, where 2:2 pulldown happens, when encoding from progressive sources (film or not, it has been sped up to 25fps frame-wise)?

Is this another possibility for HCEnc? 50i?
50i = 25fps interlaced. The number differs only because 50i measures the total fieldrate per second, rather than relating it in terms of how many complete frames per second - interlaced or not - are being displayed.

So yes, there are plenty - anything that broadcasts or displays interlaced material uses it.
qyot27 is offline   Reply With Quote
Old 15th June 2010, 17:07   #325  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
I was responding to mp3coder's observation that with a progressive source we could choose to "Encode as 50i with 4% speedup (this is the 2:2 pulldown)".

It may seem like splitting hairs, but what I was driving at is whether that would entail interlacing the progressive source and causing the MPEG stream to be hard-encoded at 50i (i.e. picture structure = field)

If, when fed a truly progressive source, HCEnc or any other EN-coder splits the frame into two fields, it would be hard 50i encoding and then we'd probably have a case where "2:2 pulldown" is an apt description.

Of course, interlacing a progressive source is at the EN-coder level is programmmatically simple. But should we do it? Is it done this way? Perhaps. I don't know.

Right now it seems that HCenc (or any other encoder) does not do that to a truly progressive source. Instead the frames are encoded (25fps nominally) and it is the job of the DE-coder to interlace the output auto-magically. There is no "pulldown", in the strict sense of the term, by the EN-coder.

And even if such an option were available, I still think it would be cleaner to encode as progressive stream.

Any thoughts?
Koutsoubos is offline   Reply With Quote
Old 15th June 2010, 17:36   #326  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by mp3dom View Post
I'm in the PAL land and I've made about 100+ dvd titles in that way and all passes the EclipseSuite verifier (the verifier used by replication facilities just before the glassmastering process).

You know, I was pretty convinced. But just a few minutes ago I came across this statement:
"Although Eclipse Image Analysis is crucial, it does not check the VIDEO_TS folder for DVD Spec compliance (IFO files, VOB data structure, navigation commands, etc)."
http://www.dvdverification.com/public/119.cfm


I honestly wish that this basic information became public instead of staying subject to some hefty licence fee and NDA.

Last edited by Koutsoubos; 15th June 2010 at 17:43.
Koutsoubos is offline   Reply With Quote
Old 15th June 2010, 18:17   #327  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 991
Quote:
Originally Posted by Koutsoubos View Post
It may seem like splitting hairs, but what I was driving at is whether that would entail interlacing the progressive source and causing the MPEG stream to be hard-encoded at 50i (i.e. picture structure = field)
Picture structure field is not the only choice for encoding interlaced. Very often, for interlaced encoding the picture structure is still 'frame'. Generally, the field structure is optimized for high-motion interlaced scene, whereas the frame structure work best for progressive source or moderate-motion interlaced scene. Frame structure is also more compatible. You can also mix the structure on a per-frame basis (so use frame structure for normal motion and switch to field when the encoder detects high-motion). The latter method is quite incompatible since it's a situation not covered by the specs (the specs says that both field and frame structure must be supported by mpeg decoder but doesn't says anything regarding the fact that the structure could be switched over time on the same segment. So some player support it, and other doesn't).
Returning in topic, if you encode your progressive source as interlaced (top field, frame or field structure, alternate scanning order etc etc), you're making a '2:2 pulldown encode'.

Quote:
If, when fed a truly progressive source, HCEnc or any other EN-coder splits the frame into two fields, it would be hard 50i encoding and then we'd probably have a case where "2:2 pulldown" is an apt description.
Exactly, but this depends by your settings and your needs.

Quote:
Of course, interlacing a progressive source is at the EN-coder level is programmmatically simple. But should we do it? Is it done this way? Perhaps. I don't know.
A lot of dvd were encoded in that way (with interlaced flags). This also prevent some dvd players to output in progressive-scan (if they rely completely to the flags embedded in the mpeg2 stream)

Quote:
But just a few minutes ago I came across this statement:
I don't remember which part of the suite were used, but I've received a report from the replicator (made with the EclipseSuite) with a warning regarding the dts audio: "Since dts is not mandatory, some dvd players could be not able to decode it correctly". So I think that at least one tool of the suite have indeed read the IFO/VOB.
mp3dom is offline   Reply With Quote
Old 15th June 2010, 19:33   #328  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,175
That might be a lawyer-worded way of saying that while they'll check for their idea of conformance, they do not guarantee that their idea is a perfect complete test of the spec. Otherwise, lawsuits and yada yada.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 15th June 2010, 19:56   #329  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,027
First of all a small bug report for HC 0.24 beta 04-04-2010:
In 1-pass VBR mode the HC log file always reports an average quantizer which is too small by a factor of 10 (0.400 instead of 4.00). Haven't tested it in 2-pass VBR. Peanuts...

@Koutsoubos
Interesting discussion, but I still think you should follow neuron's advice and post a sample. If your source is purely progressive, but your display chain still shows combing artifacts, a phase shift in your source is pretty much the only explanation.

Just assume for a moment that I (and neuron) are right and your source is phase shifted by 1 field:

Play it through a standard DVD player (no progressive scan) on an interlaced display (CRT), no combing would be visible as long as the field order is right.

Play it on a progressive scan player fed to a progressive display device, and say that either the player or the display has turned on their built-in deinterlacer: No combing artifacts, but certainly a loss of quality. And according to your first post not setting the TFF flag must have triggered the deinterlacer which is not good...

Play it in a progressive chain where neither the player nor the display have turned on their deinterlacer (which would be correct) then you would see combing artifacts.


So please provide a sample...

Cheers
manolito
manolito is offline   Reply With Quote
Old 16th June 2010, 02:05   #330  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Hey, it's not that I do not want to heed the advice of the likes of neuron2 or you. I see that both of you have a very rich bank of experience behind you.

Just that the source is very big (even a 4 second Lagarith-compressed snippet is 80 MB) and copyrighted. I am not authorized to release the material.

And it cannot be a field shift. For starters, there are no fields. The source is progressive.

In my "case 1 encoding" (HCEnc's new behaviour), in two progressive test setups (progressive player and display), when I actually force the DVD standalones to output progressive, things are fine. When I leave them in "standard" interlaced mode, perceptible combing appears, shall we say they do not seem smart enough to produce a kosher 50fps interlaced image from the 25p encoding. Commercial interlaced DVDs play fine.

In a plain-vanilla non-progressive scan DVD player and interlaced CRT, the decoder-interlaced image appears just fine.


So... that led to the long discussion about flags and all and whether "case 3" is the way to go ...


I would like to pose a question to all. Given a nominally 25 fps (i.e. forget about speed-up or not, it's now 25fps) progressive source bound for PAL, would you encode it as 25p or go ahead and encode it as 50i? Why or why not? How would you do it?

Last edited by Koutsoubos; 16th June 2010 at 04:45.
Koutsoubos is offline   Reply With Quote
Old 16th June 2010, 02:12   #331  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by mp3dom View Post
I don't remember which part of the suite were used, but I've received a report from the replicator (made with the EclipseSuite) with a warning regarding the dts audio: "Since dts is not mandatory, some dvd players could be not able to decode it correctly". So I think that at least one tool of the suite have indeed read the IFO/VOB.

This from an individual known as Trai, expressed in the Creative Cow forums (http://forums.creativecow.net/thread/155/869917):

The EclipseSuite of tools is crucial, no doubt; I've been licensed with them since September 2002, and use them everyday. But these tools make sure that what's presented to you as a replicator, i.e., the DVD Image, and that it can be mastered. Only a very few checks of the contents of the build folder (DVD-Video file structure in the VIDEO-TS folder) are included that pertain to mastering the Image; things like proper placement and flagging of layer break cell (which I and my partner working on DVDAfterEdit got Eclipse to implement during 2003/2004), making sure the DSI (data search information) has the correct number of sectors listed as are actually on the disc (more to it than that), and just a very few others...

And yes, it's true I'm afraid, Eclipse, as good as it is, doesn't check for DVD for spec compliance. The MEI DVD-Video Verifier does that. Eclipse (worth it's weight for the crucial tasks it performs) verifies if the Image presented is configured properly for your mastering equipment, and that the Image follows certain file system specifications - but again (and again) does not check the DVD for DVD-Video spec compliance.

Last edited by Koutsoubos; 16th June 2010 at 04:44.
Koutsoubos is offline   Reply With Quote
Old 16th June 2010, 09:17   #332  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 991
Quote:
Originally Posted by Koutsoubos View Post
Given a nominally 25 fps (i.e. forget about speed-up or not, it's now 25fps) progressive source bound for PAL, would you encode it as 25p or go ahead and encode it as 50i? Why or why not? How would you do it?
Personally I would encode it as 25p. Just to be sure, have you tried to encode your footage with same/similar settings on other encoders? Does it changes something?

Quote:
This from an individual known as Trai, expressed in the Creative Cow forums
From that thread, the answer is not that clear. Other users says that the ImageMapper tool is able to check dvd compliancy. Maybe it depends by the version too. Anyway, if you're so worry, stay on the safe side by using conservative settings.
Regarding your problem about 'combing' artifacts on progressive image, it could be that some flags are switching over time. Even the scanning order (from Alternate to Zig-Zag and viceversa) could led to some strange artefacts on some standalone players.
mp3dom is offline   Reply With Quote
Old 16th June 2010, 17:57   #333  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,027
Maybe to avoid all possible issues with different hardware player setups you could do what virtually all commercial authoring houses do:

Flag progressive PAL DVDs as interlaced TFF

Out of curiosity I just pulled 20 commercial progressive PAL DVDs from my collection and checked their flagging with ReStream.

Result:
None of them had progressive flags whatsoever, all were flagged as interlaced TFF. Nine of them showed zig-zag scan order, eleven had alternate scan order.

This leads to a feature request for the next HC version:
I would like to have an additional setting called *NO_PROGRESSIVE_FLAGS (or similar)
This way I could correctly encode progressive content with zig-zag scan order, but there would still be no progressive flag in the stream. This is currently not possible with HC (CCE SP can do this...) In AutoInterlaced mode this would also avoid the constantly alternating progressive flags which has been reported to cause trouble in some hardware configurations.


Cheers
manolito

Last edited by manolito; 16th June 2010 at 18:00.
manolito is offline   Reply With Quote
Old 17th June 2010, 07:36   #334  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by mp3dom View Post
Just to be sure, have you tried to encode your footage with same/similar settings on other encoders?
I have only tried HCEnc.


Quote:
Originally Posted by mp3dom View Post
Other users says that the ImageMapper tool is able to check dvd compliancy. Maybe it depends by the version too.
ImageMapper seems to be a software used to mount DDP images as if they were DVDs: www.eclipsedata.com/PDFs/ImageMapper.pdf



Quote:
Originally Posted by mp3dom View Post
Regarding your problem about 'combing' artifacts on progressive image, it could be that some flags are switching over time. Even the scanning order (from Alternate to Zig-Zag and viceversa) could led to some strange artefacts on some standalone players.
I don't know about switching flags. Would HCEnc do that sort of thing? What could I use to check this? Restream?
Koutsoubos is offline   Reply With Quote
Old 17th June 2010, 07:37   #335  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by manolito View Post
Out of curiosity I just pulled 20 commercial progressive PAL DVDs from my collection and checked their flagging with ReStream.
Out of curiosity, how would you know that the DVDs are "progressive PAL"? I have never seen a PAL DVD labelled as "progressive" or otherwise, in fact I have a few that seem to be encoded interlaced. Did you inspect the stream and the picture structure?



Quote:
Originally Posted by manolito View Post
Result:
None of them had progressive flags whatsoever, all were flagged as interlaced TFF. Nine of them showed zig-zag scan order, eleven had alternate scan order.
Do I correctly understand that the streams are flagged as: Progressive_frame=OFF; TFF=ON; Progressive_sequence=OFF ?



Quote:
Originally Posted by manolito View Post
This is currently not possible with HC (CCE SP can do this...)
How is this achieved in CCE SP?
Koutsoubos is offline   Reply With Quote
Old 17th June 2010, 09:17   #336  |  Link
mp3dom
Registered User
 
Join Date: Jul 2003
Location: Italy
Posts: 991
Quote:
Originally Posted by Koutsoubos View Post
I have only tried HCEnc.
It could be a good thing to test other encoder if you can.

Quote:
I don't know about switching flags. Would HCEnc do that sort of thing? What could I use to check this? Restream?
Maybe, other encoders can do that too if you tell to. With ReStream you can check only the first header, not the whole file. If the scanning order switch over time, you'll see only the starting scanning order.

Quote:
I have never seen a PAL DVD labelled as "progressive" or otherwise, in fact I have a few that seem to be encoded interlaced.
I've seen quite of them but in general they don't have the progressive_sequence set to ON (because in general the encoder used for dvd is CCE which doesn't support the progressive_sequence). They set progressive frame = ON and Top Field First set to ON, with constant ZigZag scanning order. This is probably the most compatible settings since it allows anyway to output progressive in a progressive-scan player.

Quote:
How is this achieved in CCE SP?
Set ZigZag scanning order, leave unchecked 'progressive frame', set offset line to 0 and check Top Field First
mp3dom is offline   Reply With Quote
Old 17th June 2010, 12:48   #337  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,027
Quote:
Originally Posted by Koutsoubos View Post
Out of curiosity, how would you know that the DVDs are "progressive PAL"? I have never seen a PAL DVD labelled as "progressive" or otherwise, in fact I have a few that seem to be encoded interlaced. Did you inspect the stream and the picture structure?
To determine if a movie is progressive or interlaced you have to step through single frames (in a scene with horizontal motion) and look for combing. DGIndex or VirtualDub-MPEG2 are perfect for this, do not use a player which automatically deinterlaces. If there is no combing then the film is progressive no matter if it was encoded interlaced.
The probable reason why most commercial progressive PAL DVDs are encoded (or just flagged) as interlaced is that the authoring folks are too lazy or just too ignorant to flip a couple of switches on their hardware encoders.

Quote:
Originally Posted by Koutsoubos View Post
Do I correctly understand that the streams are flagged as: Progressive_frame=OFF; TFF=ON; Progressive_sequence=OFF ?
Yes!

Quote:
Originally Posted by Koutsoubos View Post
How is this achieved in CCE SP?
mp3dom already answered this.

Quote:
Originally Posted by mp3dom
They set progressive frame = ON and Top Field First set to ON, with constant ZigZag scanning order.
Unfortunately not. None of the 20 DVDs I tested had progressive frame = ON, and not even half of them had been encoded with ZigZag scanning order.


Cheers
manolito
manolito is offline   Reply With Quote
Old 18th June 2010, 03:52   #338  |  Link
Koutsoubos
Registered User
 
Join Date: May 2010
Posts: 24
Quote:
Originally Posted by mp3dom View Post
Set ZigZag scanning order, leave unchecked 'progressive frame', set offset line to 0 and check Top Field First
In CCE, assuming a 25fps progressive source, is the "Rate Conv" or "Rate Conversion" option selected in the GUI?



Quote:
Originally Posted by manolito View Post
The probable reason why most commercial progressive PAL DVDs are encoded (or just flagged) as interlaced is that the authoring folks are too lazy or just too ignorant to flip a couple of switches on their hardware encoders.
How would YOU encode a 25fps progressive source PAL DVD, Manolito? What settings would you use (assuming spec compliance I guess).



I really want to thank you all, guys. You were a revelation.

I'm off to do more tests. I hope this discussion has been fruitful for the HCEnc developers too. It's a great program.
Koutsoubos is offline   Reply With Quote
Old 18th June 2010, 06:48   #339  |  Link
Lyris
Registered User
 
Join Date: Sep 2007
Location: Europe
Posts: 588
Quote:
Originally Posted by manolito View Post
The probable reason why most commercial progressive PAL DVDs are encoded (or just flagged) as interlaced is that the authoring folks are too lazy or just too ignorant to flip a couple of switches on their hardware encoders.
I also thought that the reason for encoding interlaced was simply to avoid any cases where small Interlaced segments were hiding in progressive content. (Or what you said, worded less strongly )

The Cinema Craft line of encoders (at the very least the high-end versions) are designed for DVD compressionists, and the manual tells you to set PAL Progressive as Progressive if the source is such. So I would be surprised if Cinema Craft a) even allowed you to produce non-compliant output when its "DVD" mode is enabled and b) would recommend you to do so in their user manual!

FWIW, this is the UK PAL version of "Howl's Moving Castle" which is one of the only PAL discs I've seen that's flagged Progressive. I believe the same version is on sale in Australia:



I've only done one PAL DVD (sold in the UK) and encoded it Progressive where applicable. I would hate to encode it Interlaced due to the decline in compression efficiency/quality. Has anyone ever seen any issues with doing this?

Last edited by Lyris; 18th June 2010 at 07:16.
Lyris is offline   Reply With Quote
Old 18th June 2010, 15:15   #340  |  Link
Emulgator
Big Bit Savings Now !
 
Emulgator's Avatar
 
Join Date: Feb 2007
Location: close to the wall
Posts: 644
Some posts back:
http://forum.doom9.org/showthread.ph...63#post1367663

I encode all my PAL-Speedup25p-from-film-24p DVDs as 25p with HC since then
and am happy with the outcome on different setups.

To flag Progressive Sequence as: no
seems to be the vital part to be DVD compliant.

Quote:
So, DVD FAQ #3.4: http://www.dvddemystified.com/dvdfaq.html#3.4
where it says: "MPEG-2 progressive_sequence is not allowed"
Yes, I would follow this.
Given the fact that PAL and NTSC are field-based
and the main video connection 1995 was the yellow Cinch connector
carrying the composite video signal in interlaced sequence
there was no need to allow an encoding parameter to be set to an untransmittable value.

Otherwise, DVD players should simply skip this value if set to Progressive sequence.
But it made a visible difference on my setup...
All pulldowns, 3:2 (23.976p Film on 29.97i NTSC),
Euro (2:2:2:2:2:2:2:2:2:2:2:2:3 24p Film on 25i PAL) come out a bit borked on this particular combination.

To encode and flag the progressive frame as progressive frame came out beautiful,
player will have to perform 2:2 pulldown to output as fields via Composite anyway.
A progressive-capable player may probably output such encodes as progressive if this is allowed via HDMI.
I can't tell exactly, but reflagging tests showed an advantage in certain player/TFT-combinations.

To encode and flag progressive material as interlaced (no matter if tff=1 or 0)
came out less optimal encodingwise.
On playback player will follow tff/rff flags anyway.
__________________
Die toten Augen von Friedrichshain: "Data reduction ? Yep, Sir. We're working on that issue. Synce invntoin uf lingöage..."
"To bypass shortcuts and find suffering"...no, it was somehow different...

Last edited by Emulgator; 18th June 2010 at 15:39.
Emulgator is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.