bt.709 and CCE

[fjhdavid]

Dear all,

I use DVD-RB and CCE with the "colormatrix" conversion command because CCE requires bt.709 coeff in order to re-encode (like HC).

Then I read the burnt DVD on my DVD player (an oppo hd981) and I choose "RGB output over hdmi".
I assume that the player does the conversion between internal YV12 data stored on the DVD to RGB for the hdmi output.

which matrix does the player use, bt.601 or bt.709? or Am I totally wrong as there no need for conversion?

thanks
Francois

[jdobbs]

The default for DVD is Rec.709 -- so your player should play it back correctly.

[fjhdavid]

I thought the same things, but oppo told me that they use bt.601 for the internal YCB to RGB conversion.

after reading the CCE owners guide, it seems that CCE itself work in bt.601.

where does come the assumption that CCE work with 709 coeff?
(the CCE owner's manual is speaking about bt.601, (see p.51 and p.52)
and this link too:
http://forum.doom9.org/showthread.php?s=&threadid=53770
(Question.12)

[Video Dude]

CCE outputs Rec.709.

Q12 refers to "Luminance Level".
Since DVD-RB already inputs YUY2 into CCE, that conversion is not done by CCE.

[fjhdavid]

DVD-RB inputs YUY2 into CCE, then it means that it will input the "DVD orginal color matrix" and it tells only that, it could be rec.601 or rec.709

BUT, where did you read that CCE output Bt.709? because if it's true, we don't need "colormatrix avisynth filter" anymore.....

I think CCE doesn't write any color matrix in its header and by default, we are thinking it is bt.709

but the DVD compliant color matrix has always been rec.601 as DVD is only SD

anyway, give a try with getgray calibration disc (which is 601), and you will see if CCE output bt.601 or bt.709

[jdobbs]

Here's some text from the Colormatrix documentation (my boldfacing):

There are several ways to convert a YUV stream to RGB. The most well known one, uses Rec.601 coefficients. It is for example used in the color conversion routines of AviSynth, VirtualDub and XviD/DivX. When playing back a XviD/DivX the stream is converted to RGB using Rec.601 coefficients. The main issue is that sometimes other coefficients are used for the YUV to RGB conversion (the other two are Rec.709 coefficients and FCC coefficients). A problem arises if a stream is encoded using one set of coefficients (Rec.709 coefficients for many dvd streams for example), and somewhere in the reencoding-processing-playback chain a different set of coefficients is assumed (Rec.601 coefficients for the XviD/DivX decoder or FCC coefficients for TMPGEnc/QuEnc or Rec.709 coefficients for CCE). You will get a slightly color distortion, which looks like a change in brightness (it's not really a change in brightness, the colors are just slightly off).

How do you know what set of coefficients are using when encoding a MPEG-2 stream? Sometimes the coefficients are stored in the header of the MPEG-2 file (the "matrix coefficients" field in the "sequence display extension"). Newer versions of GSpot will be able to read this part of the header, but also DGDecode (with Mpeg2source(info=1)) can be used to view them. If this extension field is not present in the header of the MPEG-2 file, the specs say we are supposed to use the default Rec.709 coefficients (0.2126, 0.7152, 0.0722). The Colormatrix filter is applied before the encoding.

CCE's site spells out that it outputs in MPEG-2 in accordance with ISO/IEC13818-2, ITU-T Rec.H262 (MP@ML only). CCE doesn't add a sequence_display_extension.

The specification for 13818-2 spec page 53 states:

"In the case that sequence_display_extension() is not present in the bitstream or colour_description is zero
the chromaticity is assumed to be that corresponding to colour_primaries having the value 1."

and

the value 1:

"1 Recommendation ITU-R BT.709"

If CCE were not outputting in Rec.709, and was not setting the sequence_display_extension(), it would be writing a non-compliant stream... which I highly doubt.

In writing/testing DVD-RB I can assure you that the vast majority of DVD (SD) movies either don't have sequence_display_extension present or have it with a matrix coefficients value set to "1" -- meaning it uses Rec.709. The exceptions I see on a common basis are seen in menus and extras and video from television -- which often are Rec.601 -- and have sequence_display_extension present with a value of "6".

[fjhdavid]

I think we are facing a real problem.

I completely agree with you that in mpeg2 stream if no header is present, it is bt.709 assumed and that CCE output doesn't exhibit any color matrix header (but maybe it is a mistake!)

BUT, all the DVD spec says that DVD MUST be bt.601 compliant because they are SD (I found many place on the WEB and datasheet where it is written)

In addition, a lot of DVD player use bt.601 (I asked OPPO developper and Philips and if you read the DVD re-encoded by CCE in this case, you will face a problem)

last, if you look at the page 51 and 52 of the CCE owner guide it is written that RGB to YCB is bt.601-5 compliant and the equation gives coeff 601)
this link:
http://forum.doom9.org/showthread.php?s=&threadid=53770
tends to say the same things that CCE is expecting bt.601 as input , so you must not use any colormatrix conversion before input in CCE...

maybe a mail to CCE will answer our question?

PS: a lot of PAL DVD commercial are also bt.601...and getgray which is a wellknow calibration DVD is bt.601 also (I asked its author)

[dragongodz]

(but maybe it is a mistake!)

do you realise how long CCE has been around and used by professionals ? i think someone would have noticed before now if this was the case.

BUT, all the DVD spec says that DVD MUST be bt.601 compliant because they are SD (I found many place on the WEB and datasheet where it is written)

please give links that say this.

In addition, a lot of DVD player use bt.601 (I asked OPPO developper and Philips

use how ? if they are looking at a dvd with no sequence_display_extension and saying that means its 601 then they are the ones that have it wrong. jdobbs post already showed that.

if you look at the page 51 and 52 of the CCE owner guide it is written that RGB to YCB is bt.601-5 compliant and the equation gives coeff 601
this link:
http://forum.doom9.org/showthread.php?s=&threadid=53770
tends to say the same things that CCE is expecting bt.601 as input

sorry but no it does not. it is specifically talking about the conversion of RGB input to YCbCr when done by CCE and the luminance range of the conversion. it does not specify what the output is etc.

a lot of PAL DVD commercial are also bt.601

do you mean commercial PAL dvds ? if so i call B.S. to that. i tested that already and the vast majority are 709, NOT 601. i even found 1 that used 420m.

[fjhdavid]

a link to DVD player from oppo:

http://www.avsforum.com/avs-vb/showthread.php?t=760726&pp=30&page=144
(response 4308, 4317 and 4320)
and

http://www.avsforum.com/avs-vb/showthread.php?t=760726&pp=30&page=145
(response 4324)

look to neuromancer post which is working in oppo

you can also go to getgray site where it is explained that SD DVD MUST be bt.601

for the CCE owner guide manual:
it is said: if you enter the CCE encoder with a RGB signal, it will convert it according to bt.601, then it means that we must NOT use the colormatrix filter if the dvd is bt.601 before entering and that we have to use a bt.709 to bt.601 conversion for a dvd bt.709...(i said that because I use a green correction avisynth filter)

THEN why would CCE use bt.601 for input and then bt.709 to output (double conversion)? wouldn't it have been easier to leave everything in bt.709 from the input to the output (especially if all DVD were bt.709)?

in addition, I don't prefer bt.601 over bt.709 (you will see in the link I provided that I am a strong advocate of bt.709.....), I just want to know, because I have a lot of DVD which are bt.601 and you have one that are bt.709....

maybe if you own CCE, a quick mail should answer the question...

[Wilbert]

a link to DVD player from oppo:

http://www.avsforum.com/avs-vb/showthread.php?t=760726&pp=30&page=144
(response 4308, 4317 and 4320)
Not convincing:

response 4308:
DVD is YCbCr 4:2:0 BT.601 IIRC.
No evidence here.

response 4317:
"main level (ML) - A range of proscribed picture parameters defined by the MPEG-2 video standard, with maximum resolution equivalent to ITU-R BT.601 (720 x 576 x 30)."
I'm not sure what this statement means here (which parameters are they referring to?). In any case it is referring to that maximum resolution 720 x 576 x 30. Does it also refer to which coefficients are used? I don't know. Perhaps others know or can look it up?

response 4320:
DVDs are encoded like Neuromancer said. Upconverted SD DVDs should be output with 709 because when a TV sees a 720p, 1080i or 1080p signal it assumes its HDTV standard which uses REC 709.
More statements without evidence.

response 4324:
The MTK solution decodes at BT.601. Once this signal is sent to the Faroudja chipset, the Faroudja chipset will then sample it at the native BT.601 (480p) or convert it to BT.709 (720p, 1080i, 1080p).

This chipset will then output the resulting signal to the one which is most appropriate to the native decoding of your display.
Well ok, this chipset works that way. The main question still remains, why ????

or the CCE owner guide manual:
it is said: if you enter the CCE encoder with a RGB signal, it will convert it according to bt.601, then it means that we must NOT use the colormatrix filter if the dvd is bt.601 before entering
This can be either true and false. It all depends on how the final YCbCr->RGB conversion is done upon display. If it's done correctly (ie using Rec.709, at least that's my opinion and many others in this thread), then their statement is false.

I've added a reply in that thread. Let's wait if they come up with any credible evidence.

[45tripp]

Is there a tool to insert color matrix coefficients into the mpeg2 header?

I know you can strip the color coefficients with restream,
but what about addition?

ty

[dragongodz]

because I have a lot of DVD which are bt.601 and you have one that are bt.709

actually what i said is i have many that are 709 and i have 1 that is 420M. of the ones i checked i found exactly 0 to be 601.

a link to

i asked for links to show the dvd specs say dvds MUST be 601 as you claimed. none of those links do that. opinions do not count as fact.

[dragongodz]

maybe this will help

http://www.dvddemystified.com/dvdfaq.html#3.4

Pictures are subsampled from 4:2:2 ITU-R BT.601 down to 4:2:0 before encoding, allocating an average of 12 bits/pixel in Y'CbCr format.

note it doesnt say its subsampled TO bt.601 Y'CbCr. so for RGB or 4:2:2 footage then it would appear to expect bt.601 to sample down. now if CCE is only going to encode to bt.709 surely it knows to sample down to the correct Y'CbCr to do so.

[fjhdavid]

Ok, I am sure we will find the answer, but remember, DVD-RB is a tool to re-encode DVD in order to play them with a PC or with a DVD player.

For the DVD player, the MTK pre-processor is used in a lot of products (the output of the MTK chip is then going to a faroudja or an anchorbay or an HQV chip for de-interlacing or scaling, and then for scaling they use bt.709 for 720p and 1080p as it is specified in the HD spec but not SD)

and the fact is that MTK is 601 compliant!

so even if CCE is bt.709, you won't read correctly your DVD in a lot of commercial or standalone player....as today, this is a fact, and it should be better to ask CCE guys to be bt.601 compliant!

this a problem, don't you think? do you think that CCE guys don't know the fact about MTK is 601? so why would they go to bt.709?...surprising if it is true

guys from DVE essential and Getgray told me also that DVD is bt.601 and they are (at least for DVE) wellknown in the DVD calibration tool (and i am sure they bought the DVD specs even if it is 5000 dollars)

[dragongodz]

The MTK solution decodes at BT.601. Once this signal is sent to the Faroudja chipset, the Faroudja chipset will then sample it at the native BT.601 (480p) or convert it to BT.709 (720p, 1080i, 1080p).

and the fact is that MTK is 601 compliant!

is it decoding using bt.601 though or is it decoding to bt.601(thus a conversion if not bt.601 already) ? the second would be basically a reverse of the bt.601 RGB conversion i mentioned earlier.

guys from DVE essential and Getgray told me also that DVD is bt.601

you will have to excuse me for having more faith in the evidence of all the commercial dvds out there and real viewable specs ,such as the mpeg2 13818-2, than just the word of these people. especially when you said
BUT, all the DVD spec says that DVD MUST be bt.601 compliant because they are SD (I found many place on the WEB and datasheet where it is written)

now if you could get someone from a major studio or such to say categorically that dvd must be encoded USING bt.601, not just in RGB colour conversion, then you may have something.

[fjhdavid]

maybe you are right, guys from MTK and Joe Kane from DVE are wrong...but let me have a doubt about that....

DVE is used by professional calibrator and studio broadcaster to calibrate their SD and HD stuff (and DVE is available in two different disc, one for SD in bt.601 and the second for HD in bt.709)

MTK is used in philips, Oppo ...

Cirrus Logic (an another wellknown and wide used DVD player decoder chip also use bt.601)

but have a look to:
http://www.pinnaclesys.com/files/MainPage/Professional/TopTabItems/products/dc1000/WhitePapers/DC1000-DVD1000MPEG2whitepaper.pdf

Page 4 and page 15

and
http://www.dvddemystified.com/dvdfaq.html#1.1

section 3.4

but, I have an idea: I will re-encode tonight a DVD bt.601 and look with DGindex if CCE write something in the header. If it writes bt.601, it means that CCE simply use the matrix color of the original DVD and dosen't change it...
Then for your DVD bt.709, CCE will write nothing as it is bt.709 by default....

an another point is that you refer to 13818-2, but which version? 1995, 1996, 2000 with 2002 correction included or with 2007 corection...
if it is the 1995 version, DVD didn't exist at this time, and I am not able to download the other versions

[dragongodz]

MTK is used in philips, Oppo ...

Cirrus Logic (an another wellknown and wide used DVD player decoder chip also use bt.601)

which doesnt answer the question i asked at all.

Page 4 and page 15

which only talks about resolution. so no proof there that output must be bt.601 colormetry.

section 3.4

so you missed i already posted a link to that and commented on it ?

I will re-encode tonight a DVD bt.601 and look with DGindex if CCE write something in the header. If it writes bt.601, it means that CCE simply use the matrix color of the original DVD and dosen't change it...

and what if it does convert it to bt.709 and then doesnt write the header, so should be assumed bt.709 per mpeg2 specs, what then ?

[fjhdavid]

if the CCE re-encoded DVD show a bt.601 with DGindex, it means that there is no bt.709 in CCE !

(you didn't answer to my question about the 13818-2 version)

[Wilbert]

if the CCE re-encoded DVD show a bt.601 with DGindex, it means that there is no bt.709 in CCE !
It doesn't. Other people have checked that.

an another point is that you refer to 13818-2, but which version? 1995, 1996, 2000 with 2002 correction included or with 2007 corection...
if it is the 1995 version, DVD didn't exist at this time, and I am not able to download the other versions
That's a good point. Mine is "ISO/IEC 13818-2: 1995 (E)". Do you have a more recent one?

[kumi]

By the way, if the OP's goal is to see if his player is correctly displaying CCE-encoded video, why doesn't he simply encode 2 clips in CCE:

A) SMTPE colorbars, Rec.709
B) SMTPE colorbars, Rec.601

...then burn them to DVD and examine the output on his standalone player. Colorspace conversions using the wrong set of coefficients (Rec.709 vs. Rec.601, or vice-versa) are easy to spot, aren't they?

[jdobbs]

if the CCE re-encoded DVD show a bt.601 with DGindex, it means that there is no bt.709 in CCE !

(you didn't answer to my question about the 13818-2 version)DGIndex won't show Rec.601 unless there is a sequence_display_extension() calling it out. If it is missing, DGIndex assumes Rec.709 (as it should).

[fjhdavid]

Ok, I did some experiment tonight and results add nothing more (Wilbert and JDobbs are right):

I took a bt.601 film and dgindex stated it is "bt.601 colorimetry" (the header is written)

I did a CCE re-encode and dgindex stated it is "bt.709 colorimetry" (the header is written?)

what does it mean exactly? I don't know but it seems that there are three possibilities:

1- DGdecode use a bt.601 matrix to decode and CCE apply a sort of colormatrix (bt.601 -> bt.709) before re-encoding

2- DGdecode use a bt.709 matrix to decode and CCE keep it

3- DGdecode use a bt.601 to decode, CCE use a bt.601 to re-encode but doesn't write the header and DGindex assumes it is bt.709

what is your opinion?

For Wilbert I also have the "ISO/IEC 13818-2: 1995 (E)" as you have to pay for the others.

For Kumi, I don't know if it is easy to see if the two pictures are not projected side by side
but I will try to find something to re-encode and try to detect color shift on the PC


second part of the night:

I took mires from calibration tools (gray at 20%, 40%, 60%) which are bt.601

I do a convertToRGB (bt.601)
I do a RGBAdjust in order to add red only on the red channel
I do a convertToYUY2 (bt.709)

I do a re-encode with CCE
I play the re-encoded DVD with Cyberlink PowerDVD and take a photo

and there is a problem, the others channels (green and blue) had changed

I think that if PowerDVD uses bt.709, we wouldn't see any change in the green and blue channels?

[manono]

If you'd like to see what happens to the colors if you do it wrong, scroll down to the bottom of this page, start hitting the "Next" button, and study the color changes:

http://www.sigmadesigns.com/public/Support/chromaticity.html

For you (and me) starting with some form of a Rec.601 DVD and using CCE, but without the ColorMatrix filter applied correctly, I think the results you'll get will be the BT.601 encoding, BT.709 decoding. Someone correct me if I'm wrong on that. The differences in the first set of pics are hard to make out, The differences become more obvious with later sets in the sequence.

[dragongodz]

a thought about this statement

The MTK solution decodes at BT.601. Once this signal is sent to the Faroudja chipset, the Faroudja chipset will then sample it at the native BT.601 (480p) or convert it to BT.709 (720p, 1080i, 1080p).

now bt.601 specs are limited to SD resolutions and BT.709 added HD resolutions. so would that above statement be just saying that the MTK just decodes to SD resolution and then the Faroudja will convert it to the required resolution for your output ?

if so, and that what it does appear to be really saying to me, then that is all just about resolutions. it has nothing to do with colormetry at all.

so saying

MTK is 601 compliant!

can be true but does not cover the question being asked, that of colormetry. really there needs to be some specifity to these questions and answers or they can be construed to mean more than they actually say.

[fjhdavid]

I do my best!

which dvd player are you using?

because if CCE is bt.709 and dvd players as said by Oppo guys are bt.601 (and they know how they built their players at least), do you think that you get the solution?

secondly, I don't own CCE but if you do, can you post a mail to CCE guys, it would be grateful for all of us and we will have definitively the answer

thanks

[jdobbs]

That seems a little awkward. It's saying:

"Hi. Your documentation says you are ISO/IEC13818-2, ITU-T Rec.H262 compliant. I have no reason to doubt you, but are you really compliant?"

I trust their documentation and their product and if I were going to question it I'd need some reason... and I really don't see one in all this discussion. I am certain that Rec.709 is the most common matrix in use on standard definition DVDs and that it is within the DVD standard -- why would I suspect that the output of a high-end product like CCE, in a profile designed for DVDs, would be anything other than Rec.709 when a scan of the M2V files says that it is?

[fjhdavid]

the reason should be that one of the major DVD player manufacturer, like Oppo (which get many rewards) or guys from DVE like Joe Kane which are also high end product are saying that DVD is SD and should follow the bt.601colorimetry (and MTK and Cirruslogic manufacturers are saying the same)

remember also that CCE itself in their owner guide mentions bt.601 and don't speak even one time of bt.709

and also that we are, here, refering to a document ISO that was written in 1995 when the DVD didn't even exist

but anyway:

If CCE guys says "of course we are bt.601", then we will have learn something
and if they say "of course we are not", Oppo and DVE guys have learn something

isn't it the aim of our quest?
but if everybody is completely sure....you didn't show me evidences...

[dragongodz]

dvd players as said by Oppo guys are bt.601

but that was my point, that could just mean 601 resolution limit. the statement does not say it decodes using 601 colormetry irrelevant of what the mpeg streams information says(or doesnt say). you would need to get that information from them, from an actual tech not a PR guy, and then it may have relevance.

[fjhdavid]

me and Wilbert asked Neuromancer which is working in Oppo and I also asked them directly with their support team because I own a DVD player from Oppo and they are talking of bt.601 colorimetry compliancy

and what about Joe Kane? its DVE disc are used by professional broadcaster...you can go to DVE essential site

http://www.videoessentials.com/

IMPORTANT info:

PowerDVD (everybody knows PowerDVD), uses also bt.601 colorimetry to decode DVD. If you have it, go on this site and download the 75% red bt.601 and the 75% red bt.709....take a photo of each and look with photoshop...

http://www.w6rz.net/

but maybe guy from cyberlink (PowerDVD manufacturer) also doesn't know their job for 6 or 7 years

if you can't read your DVD on PC with PowerDVD and with DVD player from Oppo, where will you read your DVD bt.709?

[Mtz]

What I think: bt.609 is for SD and bt.709 is for HD.
(Update, 1/23/2007) As I mentioned last week, OPPO has been working on a firmware update to allow control of the colorspace setting. That work has led to a new beta firmware (not yet official). You can find the firmware on the 981HD's support page. This beta should only be necessary if you are encountering an uncorrectable "green push" (green tint to all video), as the problem may be due to the display improperly using the BT.601 color conversion matrix (intended for SD video, YCbCr sources) instead of the BT.709 color conversion matrix that should be used with HD sources. Manually selecting "RGB" will instruct the 981HD to apply BT.601 even at HD resolutions. If you are having no problems and elect to install this firmware, you should select "Auto" from the options of Auto, RGB, and YCbCr 4:4:4 on the General Setup Page of the setup menu.
Source: http://www.prillaman.net/oppo981_review.html

The problem with Oppo maybe is because of the Faroudja chipset for HD upscaling which needs bt.709, not the MTK chipset:
Adding the color space selection menu allows the user to specify which color space to use. This feature may potentially be used to work around some color accuracy issue with certain TVs. For example, the HDTV specification requires the use of BT.709 color conversion matrix for YCbCr signal in high definition, and the DV-981HD correctly implemented this requirement. However there are some TVs on the market incorrectly applies the BT.601 color conversion matrix, which is intended for SD YCbCr signal, to HD signals. When this incorrect color conversion matrix is applied by the TV, it will result in so called "green push" - green becomes too saturated or the image has a green tint. By selecting the RGB color space, a user can avoid the TVs color conversion matrix and work around this problem."

To achieve desired color reproduction on a LCD TV is a complex task, particularly under competitive cost constraints. As color reproduction devices, LCDs have unique problems that require careful characterization and calibration. For media display applications, LCD TVs are demanded to handle various video encoding standards and viewing environments. Color management tools are a system-wide solution, aiming to provide a common protocol for accurate color reproduction among various imaging devices. In this workshop, we will review (1) color imaging science as applied to display devices; (2) video color encoding and interface standards, such as NTSC, EBU, ITU-R BT.709, sRGB, xvYCC, and HDMI
Hsien-Che Lee received his B.S. from National Taiwan University, M.S. from the Ohio State University, and Ph.D. from Purdue University. He now works as a full-time consultant to MediaTek Inc., Hsin-Chu, Taiwan.
Source: http://www.di.nctu.edu.tw/IDMC07/PresentationsWL.htm

enjoy,
Mtz

[mc2man]

I also have an oppo DV-981HD which at the moment is mainly used for pal titles. It plays both cce backups and orig. .601,.701's fine. The colormatrix dll only affects whether a backup is accurate to a .601 source. Proper playback is an issue between the player and display. I think your comparing apples to oranges.
a simple view can be found here
http://www.gadgetbench.com/colormatrixerrors/

oppo did offer a firmware update as in mtz's post several weeks ago

[jdobbs]

Interestingly I found this link (http://www.bjpace.com.cn/data/tec/tec-DVB/DVB%20BlueBooks%20Standards/Specifications%20and%20Standards/mpeg/dvb-mpeg/A001r5.pdf) that contradicts the specification quotes in earlier posts.
It's the "Implementation Guidelines for the use of MPEG-2 Systems, Video and Audio in Satellite, Cable and Terrestrial Broadcasting Applications DVB DOCUMENT A001 rev 5 June 1999"

It says:

Within 25Hz SDTV bitstreams, if the sequence_display_extension() is not present in the
bitstream or colour_description is zero, the chromaticity shall be implicitly defined to be
that corresponding to colour_primaries having the value 5, the transfer characteristics
shall be implicitly defined to be those corresponding to transfer_characterstics having the
value 5 and the matrix coefficients shall be implicitly defined to be those corresponding
matrix_coefficients having the value 5. This set of parameter values corresponds signals
compliance with ITU-R Recommendation BT.470-3 System B,G,I. The value "5", of course, is the equivalent of Rec.601... so the most I can say is "WTF? Will the prevailing standard please stand up?" Of course this document is talking about Digital Video Broadcasting Standards -- but you'd think MPEG-2 would be MPEG-2, wouldn't you?

[Mug Funky]

FWIW, it's possible that oppo, MTK etc are simply outputting 601 (which would make sense as it applies to analog component as well as digital IIRC), and just making sure colorimetry is handled correctly on input. if the DVD is flagged as 709, then the decoder will presumably know what to do with it to get a 601 output.

[dragongodz]

it's possible that oppo, MTK etc are simply outputting 601 (which would make sense as it applies to analog component as well as digital IIRC), and just making sure colorimetry is handled correctly on input. if the DVD is flagged as 709, then the decoder will presumably know what to do with it to get a 601 output.

well i asked on the first page

s it decoding using bt.601 though or is it decoding to bt.601(thus a conversion if not bt.601 already) ?

meaning the matrix was detected and as you say decoder TO 601.

me and Wilbert asked Neuromancer which is working in Oppo and I also asked them directly with their support team because I own a DVD player from Oppo and they are talking of bt.601 colorimetry compliancy

well i saw you asked and the response of

The MTK solution decodes at BT.601. Once this signal is sent to the Faroudja chipset, the Faroudja chipset will then sample it at the native BT.601 (480p) or convert it to BT.709 (720p, 1080i, 1080p).

doesnt say it ignores what colormetery is in the stream at all. decodes at ? at ? as i said that can mean the resolution. that is really no answer at all.

Of course this document is talking about Digital Video Broadcasting Standards -- but you'd think MPEG-2 would be MPEG-2, wouldn't you?

yes and no. they are doing is limiting what parts of the mpeg2 specs they use. just as dvd has limits to what frame rates and resolutions are usable etc. its when they decide to go against the mpeg2 specs altogether that it becomes a "WTF?" moment. such as when they say when the header is not present use BT.470-3.
though you have to remember that that is a limitation they have built in. if dvd had such a limit why does professional hardware and software encoders not enforce it ? why do the majority of dvds use bt.709 ? doesnt make sense to me.

[dragongodz]

refering to a document ISO that was written in 1995 when the DVD didn't even exist

from here
http://en.wikipedia.org/wiki/DVD_authoring

DVD authoring software must conform to the specifications set by the DVD Forum group in 1995.

here is something else interesting from that page

The DVD specifications were written in Japanese and then translated to English for use in America with prose that often rivals the complexity of legal documents. To this day, many companies interpret various parts of the specifications in different ways. This is the reason DVD players from different manufacturers do not always conform to the same rules – each developer understands the specifications in a slightly different way.

scary huh.

[fjhdavid]

as I asked before, we are all intrested in first place with CCE (input and output)

please ask CCE guys, and we will know for sure which color matrix they implemented for DVD re-encoding (independently from the norm...if one does exist...)

then we will be able to find the best prameters for our players (PC or standalone DVD player) and display (plasma, videoprojector...)

[fjhdavid]

sorry for this second post but I think I have the answer to our question: CCE is 601 input and 601 output

the evidence:

I took a video sequence with a perfect 75% red from two different calibration disc which are 601.

I read it with powerDVD and took a picture. the RGB value is as supposed (189,0,0) (189 is 75% of 255 and in the same time this is an evidence that PowerDVD is 601).

then I did a CCE re-encoding with NO colormatrix filter

then I read it with PowerDVD, took the picture and look at the value which is still (189,0,0)

as I know that powerDVD is bt.601 compliant and the original video are 601, CCE MUST be 601 compliant to get the same result before and after re-encoding

do you agree?

PS: I made the same process with, in addition a colormatrix (Bt.601->bt.709) before re-encoding and this time the RGB showed (171,0,0)

I made also the same experiment with 75% green and 75% blue which confirm

[kumi]

PowerDVD could be mangling your video in all sorts of creative ways. To judge CCE's behaviour, you need to control all intermediary colorspace conversions.



First, let's see what the SMPTE color bars are supposed to look like. Use this script: ColorBars(720,480,"YV12").ConvertToRGB24(), and you get:
http://img523.imageshack.us/img523/2532/smptecolorbarsuo4.th.png (http://img523.imageshack.us/img523/2532/smptecolorbarsuo4.png)

OK? Now encode that Rec.601 clip into CCE. If you convert the CCE output clip to RGB24 using Rec.709 coefficients, it looks wrong:
http://img219.imageshack.us/img219/2195/rec601torgbwithrec709ix8.th.png (http://img219.imageshack.us/img219/2195/rec601torgbwithrec709ix8.png)

On the other hand, if you convert the CCE output clip to RGB24 using Rec.601 coeffiecients, the colorbars look right:
http://img151.imageshack.us/img151/1698/rec601torgbwithrec601hu3.th.png (http://img151.imageshack.us/img151/1698/rec601torgbwithrec601hu3.png)

Conclusion? Rec.601 input is not converted to Rec.709 internally by CCE, yet the CCE output clip is (according to DGDecode) flagged as Rec.709.



Now let's feed CCE a SMPTE colorbars clip encoded with Rec.709 coefficients. You can get this with colorbars(720,480,"YV12").ColorMatrix(mode="Rec.601->Rec.709", clamp=false).

Convert the CCE output clip to RGB24 using Rec.709 coefficients, and it looks right:
http://img48.imageshack.us/img48/8062/rec709torgbwithrec709hb3.th.png (http://img48.imageshack.us/img48/8062/rec709torgbwithrec709hb3.png)

But convert the CCE output clip to RGB24 using Rec.601 coefficients, and it looks wrong:
http://img48.imageshack.us/img48/7002/rec709torgbwithrec601ix3.th.png (http://img48.imageshack.us/img48/7002/rec709torgbwithrec601ix3.png)

Conclusion? Rec.709 input is not converted to Rec.601 internally by CCE, and the CCE output clip is AGAIN (according to DGDecode) flagged as Rec.709.



See? CCE requires Rec.709 input coefficients. Anything else will be incorrectly flagged.

[fjhdavid]

You said exactly the same thing than me in the first conclusion of your post and you give a second evidence of what I said; but you omit that DGdecode always give rec.709 when the header is not written and it doesn't mean that it is rec.709 (it is also the purpose of the main discussion of this thread) and CCE dosn't write anything in the header

PowerDVD is very good as you select "original" in the settings.
I tested it with many calibration disc and it gives exactly what was expected: it is 601 compliant

look back to my post and give me the evidence that I am wrong somewhere as my evidence is pure logic

PS: I didn't say that CCE convert anything from rec 601/709 to rec709/601 (I would be surprised if it does)
I just say CCE give correct results when feed by rec.601 and output using a player 601 compliant, it is sufficient for me....not for you?

[jdobbs]

I think kumi's post makes good sense. What it says is that CCE doesn't do any internal conversion at all -- it simply encodes what was given to it. So by feeding Rec.709 into it you get Rec.709 output -- and that's the only way to make it meet the 13818-2 callout without insertion of a sequence_display_extension(). It also means that by default all the existing commercial DVDs (that are marked as Rec.709 based upon 13818-2) are being correctly encoded -- without the need for ColorMatrix() -- as has been previously stated.

Either way, this is a CCE discussion and should probably be moved to the CCE forum -- as I'm sure it really has nothing to do with DVD-RB itself -- as interesting as it is.

[fjhdavid]

Jdobbs, this thread is also releant for DVD-RB as you use dgdecode.

but let's have this summary (which is OK, according to Kumi and myself tests, and with Wilbert, Jdobbs, Oppo guys and other posts) and works with 709 DVD and 601 DVD:


First CCE does NOT do colorimetry conversion and is expecting nothing as input.

If your original DVD is 709, dgdecode will decode it as 709, and you have to input directly into CCE, and CCE will output 709.
The only warning is to be sure that your original DVD is 709 (and dgdecode seems not sufficient) and check that your DVD player or DVD software player is 709 compliant. Your display must also expect 709.

If your original DVD is 601, dgdecode will decode it as 601, and you have to input DIRECTLY into CCE (without any colormatrix filter) and CCE will output 601.
The only warning is to be sure that your DVD player or DVD software player is 601 compliant (it is true for Oppo and PowerDVD ("original" settings) and that your display is expecting 601.

DVD 601 seems easier to handle because when dgdecode identifies them as 601, you are sure they are 601.

PS: all that was said, is for SD output. For HD ouput the scaler or DVD player scaler has to handle properly the 709 and do the conversion for 601 DVD.

my two cents.
Maybe if you all agree, Jdobbs could do a sticky post from this one?

[manono]

Sorry, but I can't resist:
Maybe if you all agree, Jdobbs could do a sticky post from this one?
Speaking strictly for myself, the last place your post should wind up is as a sticky, as I don't agree with much of anything you said in your post. It's almost as if the posts by others have gone in one ear and right out the other. Maybe there's an English comprehension problem here, I don't know.

[fjhdavid]

maybe my english is not as good as yours, I am sorry about that (but I am sure my french is better....smile)

but remember that the aim of DVD-RB is to do the best DVD possible, and if you are wrong in matrix color, it is a pitty, but one more time if someone can demonstrate something different, I would like to hear (I don't care about document from 1995 or 2007 (becaude jdobbs showed us maybe two contradictory one (post 32)),

I don't have any 709 video so I trust Kuni (post 38), I just said be careful with dgdecode as it seems it say 709 even when there is no header extension written

But about 601 DVD, I am sure that you don't need colormatrix() and I have evidences about that (post 37), I have also evidence that PowerDVD and Oppodigital players are 601 compliant (post 29 part2)

But please be free to sending me a 601 video footage I can check with my PC which show me I am wrong with 601 DVD

[kumi]

be careful with dgdecode as it seems it say 709 even when there is no header extension written
Take a deep breath, and read my last post again. If you feed Rec.601 to CCE, the output will not have the sequence_display_extension() set, as it should. Do you REALLY think CinemaCraft would overlook such a thing? Of course not. CCE expects ONLY Rec.709 input.

It's obvious that CinemaCraft engineers are comfortable with encoding Rec.709 WITHOUT the sequence_display_extension(). Thanks to that, you can easily argue that DGDecode's assumption of Rec.709 coefficients on input streams without explicit flagging is reasonable and correct.

[fjhdavid]

maybe you are right Kuni,

but, to convince me, can you just do a simple test for me:

take a 601 pure red showing RGB = (X,0,0) (or a part of a picture)
convert it with DVD-RB and CCE WITHOUT any colormatrix filter

then read the output and tell us the re-encoded output RGB value

now do the conversion with DVD-RB and CCE and a colormatrix (bt601->bt.709) before encoding as it "should be" if I understood your purpose

then read the output and tell us the re-encoded output RGB value

thanks for your help

[Wilbert]

Let me respond on two points:

as I know that powerDVD is bt.601 compliant and the original video are 601, CCE MUST be 601 compliant to get the same result before and after re-encoding

do you agree?

No. Suppose you original video is 601, and you feed it as YCbCr to CCE. CCE doesn't add the "sequence display extension" field to the resulting MPEG-2 file. Hence IF (*) we follow the MPEG-2 specs, Rec.709 coefficients should be assumed upon display, and the displayed clip has wrong colors.
But assuming that you are correct by your statement that PowerDVD uses Rec.601 upon display, then your encoding will look the same as your original.

About (*), which brings me to the following point:
but remember that the aim of DVD-RB is to do the best DVD possible, and if you are wrong in matrix color, it is a pitty, but one more time if someone can demonstrate something different, I would like to hear (I don't care about document from 1995 or 2007 (becaude jdobbs showed us maybe two contradictory one (post 32)),
What we really know is what the DVD-specs say about this. None of us seems to know, and in the AVS forum they are silent about it. The DVD-specs are supposed to be a subset of the MPEG-2 specs, but it wouldn't be the first time if two specs would contradict each other, when they should not.

About your last post:
but, to convince me, can you just do a test for me:

take a 601 pure red showing RGB = (X,0,0)
convert it with DVD-RB and CCE WITHOUT any colormatrix filter

then read the output and tell me the RGB value
Such tests are pointless. Your assumption here is that the player/renderer is correctly displaying the video, but at the same time we question *how* it should be displayed.

[kumi]

then read the output and tell us the re-encoded output RGB valueLike Wilbert said, that's meaningless. What coefficients do you choose for the YV12-to-RGB conversion? Rec.709? Rec.601?

Look, it's already proven that CCE does not flag it's output stream, even when it's fed Rec.601 input. This fact alone demolishes your entire argument about how "601" is the "SD standard". I now retire myself from further participation in this thread (it's not going anywhere.)

[dragongodz]

and dgdecode seems not sufficient

i have to disagree. dgdecode is following the mpeg2 specs and reporting exactly what they say it should when the extension is missing. the only other solution would be a complex mess of decode tests trying to determine minor colour variances etc. hardly practical.

The only warning is to be sure that your DVD player or DVD software player is 601 compliant

actually that would be 601 forced as it is ignoring what the mpeg2 specs says it should doing.

but, to convince me

and here is the real problem i think. its been shown there is plenty of doubt about what you think it all should mean. i do not think you will be totally happy or convinced no matter what anyone says. so

I now retire myself from further participation in this thread (it's not going anywhere.)

i am doing the same. i have other more imprtant things to deal with at the moment.

it wouldn't be the first time if two specs would contradict each other, when they should not.

true. also go back and read the quote that i commented "scary huh" to. :)

[fjhdavid]

Wilbert and Jdobbs thanks for your point about my post. I will go further in testing if I can.

Kumi and Dragongodz, I am sorry you lost your time with me.

My purpose was just to try to understand why:

people from PowerDVD and WinDVD assume 601 for DVD SD (powerDVD and winDVD are nb 1 and 2 of PC DVD player)

people like Joe Kane from DVE and guys from getgray are doing wrong DVD calibration tools for SD DVD

why MTK and Cirruslogic, wellknown chip maker are using wrong colormatrix

why, we don't have any info from CCE guys to tell us their specs

I am also wondering how you are playing your DVD with all this people doing wrong bt.601 compliant stuff?
but as you said it is maybe not important...

[Wilbert]

I'm reading "ITU-T Rec.H262 (2000 E)". All the ITU-T docs are available for free since a short while:

http://www.itu.int/rec/T-REC-H.262/en

The relevant quote (page 44) says something different (than the 1995 version):

In the case that sequence_display_extension() is not present in the bitstream or colour_description is zero the matrix
coefficients are assumed to be implicitly defined by the application.

What's that supposed to mean?

[jdobbs]

It means "if the sequence_display_extension() isn't there -- your guess is as good as mine." they were just afraid to write that into the spec. ;)

It also says this:

NOTE 2 – In applications which may have signals with more than one set of colour primaries, transfer characteristics, and/or matrix coefficients, it is recommended to transmit a sequence display extension with colour_description set to one, and to specify the appropriate values for the colorimetry parameters.

[fjhdavid]

look at this address even if they are talking about SDTV:

(sigmadesigns is also a wellknown decoder chip maker used in a lot of DVD player)

http://www.sigmadesigns.com/public/Support/chromaticity.html

But for Wilbert an Jdobbs, I read also ITU-T Rec.H262 (2000 E)".
The application should be for example SD (or HD), or SD TV, or HD TV, as the player would recognize by the height and width which application the mpeg2 stream are talking about?
It could be the same with PAL and NTSC as the player would recognize the application with the refresh rate (25hz and 30hz)?

PS: I tested DGindex and it is a 709 compliant DVD player...

[hank315]

In the case that sequence_display_extension() is not present in the bitstream or colour_description is zero the matrix coefficients are assumed to be implicitly defined by the application.
This is a mess, not a standard anymore :(

BTW, thanks for the ITU link.

Suppose you original video is 601, and you feed it as YCbCr to CCE. CCE doesn't add the "sequence display extension" field to the resulting MPEG-2 file. Hence IF (*) we follow the MPEG-2 specs, Rec.709 coefficients should be assumed upon display, and the displayed clip has wrong colors.
But assuming that you are correct by your statement that PowerDVD uses Rec.601 upon display, then your encoding will look the same as your original.
True, but what if the "sequence display extension" with 601/709 flags is added to the stream.

I just added this header to HCenc to see what happens for different input/output combinations, the next table shows the RGB values for the red, green and blue colorbars using PowerDVD.

input output red green blue
601 601 RGB=187, 0, 0 RGB= 2,187, 2 RGB= 1, 0,187
601 709 RGB=187, 0, 0 RGB= 2,187, 2 RGB= 1, 0,187
709 601 RGB=171, 0, 2 RGB= 17,220, 8 RGB= 1, 0,179
709 709 RGB=171, 0, 2 RGB= 17,220, 8 RGB= 1, 0,179
Seems DVD has its own rules and is not a subset of the MPEG2 standard regarding colorimetry, PowerDVD always outputs 601 regardless if the stream is flagged 601 or 709, the info in the sequence display extension is ignored.

Now lets see how DGIndex handles it:
input output red green blue
601 601 RGB=188, 1, 1 RGB= 3,188, 3 RGB= 1, 1,188
601 709 RGB=204, 19, 0 RGB= 0,160, 0 RGB= 0, 12,195
709 601 RGB=172, 0, 2 RGB= 18,221, 9 RGB= 1, 0,179
709 709 RGB=188, 1, 1 RGB= 4,189, 3 RGB= 0, 1,187
According to the MPEG2 standard this is correct, the info in the sequence display header is read and processed.

What does this mean for DVD-RB using CCE?
IMHO nothing, DVD content already has 601 characteristics, CCE encodes the YUY planes to YV12 planes and doesn't write the sequence display header so DGIndex correctly sees it as 709 but DVD players will ignore it and use 601 for the conversion.
However if you use Colormatrix to convert the DVD from 709 to 601 you will end up with the wrong colorimetry, and DGIndex will still report it as 709...

For those who are interested the testfiles used can be downloaded here (http://rapidshare.com/files/67173505/colors.zip.html)

[jdobbs]

So here are the questions, since the standard can't apparently be trusted:

1. Can we assume DVDs with no sequence_display_extension() are Rec.709 or Rec.601?

2. What conversion is DGDECODE doing when it frame serves it to HC or CCE?

3. Why is it that when I do see sequence_display_extension() used on a DVD it is always used to specify Rec.601? It would at least give you the impression that the default is other than Rec.601, wouldn't it?

Since DVD-RB scans the file and creates the D2V file itself, I could very easily make the D2V default to Rec.601 -- but is that right?

Sigh...

[hank315]

As long as nobody here has the DVD specification it's all speculation...

1. Can we assume DVDs with no sequence_display_extension() are Rec.709 or Rec.601?My guess is all DVDs are rec.601, there's too much info on the Inet saying SD is 601.
The testfiles also show wrong colorimetry using 709 as input.

2. What conversion is DGDECODE doing when it frame serves it to HC or CCE?As far as I can see in the code: none.
The sequence display info is only parsed, not processed.

3. Why is it that when I do see sequence_display_extension() used on a DVD it is always used to specify Rec.601? It would at least give you the impression that the default is other than Rec.601, wouldn't it?Sounds logical indeed but PowerDVD (WMP does the same) seems to ignore the settings in the sequence display header completely, outputting the header seems just a waste of bits?

[45tripp]

My guess is all DVDs are rec.601,
I agree


Sounds logical indeed but PowerDVD (WMP does the same) seems to ignore the settings in the sequence display header completely, outputting the header seems just a waste of bits?
I think you'll find all pc players will do the same,
use 601, ignore headers.


2. What conversion is DGDECODE doing when it frame serves it to HC or CCE?

it does nothing, the encoders do nothing.


3. Why is it that when I do see sequence_display_extension() used on a DVD it is always used to specify Rec.601? It would at least give you the impression that the default is other than Rec.601, wouldn't it?
It gives me the impression that it's an encoder error not setting the display extension. A shortcoming.

btw, restream can add matrix coeffs to a stream,
it sets to I470.
Don't know if there's a tool to set the header coeffs to 709.
I like that Hank added the feature to add the header in a test version of HC,
I'm sure it will make it through to the next HC.

anyway,
i think it's good to know what behaviour to expect with conversions to rgb so basically i gave this answer to the op:

consider 4 sources:

A
colorbars(720,480)
killaudio()
converttoyv12()
trim(0,120)

Feed to HC/CCE/mencoder, get mpeg2, 601 coeffs used.
No sequence_display_extension present.

B
Take stream A and use restream to add sequence_display_extension with colour,
restream sets matrix coeffs to 'I470'.


C
colorbars(720,480)
killaudio()
converttoyv12(matrix="rec709")
trim(0,120)

Feed to HC/CCE/mencoder, get mpeg2, 709 coeffs used.
No sequence_display_extension present.

D
the same as C but set matrix coefficients in header to 709.
Like I said I don't know of a tool to do it,
but Hank's colorbar input709to709output stream linked to above fits the bill.

slap the 4 sources on a disk or use them for input to your apps.

Scenario 1:
Mpeg2 is converted to rgb using 601 always.
Source A = Normal (N) ; B = N ; C = Red Push (RP) ; D = RP

Scenario 2:
Mpeg2 is converted to rgb respecting the header,
in the absence of the header 709 is assumed.
A= Green Push (GP) ; B = N ; C = N ; D = N

Scenario 3:
Mpeg2 is converted to rgb respecting the header,
in the absence of the header 601 is assumed.
A = N ; B = N ; C = RP ; D = N

Scenario 4:
Mpeg2 is converted to rgb using 709 always. (nicely hypothetical)
A = GP ; B = GP ; C = N ; D = N


That way you know what's happening on the end part.
and effectively what to do before you get there.

If Scenario 2 is actually the standard,
and is abided to,
then what CCE (and other encoders) does is wrong by not setting
color matrix coeffs in the header,
and I would run them through restream to add a header.

As for knowing what you are working with...
I'd treat dvds with no header as though they are 601.
Of course you can just wing it visually, deciding if you're ok,
or if you've been pushed either red or green,
and revert with colormatrix.

anyway,
that's what I think.

gl

[fjhdavid]

I am happy that Hawk315 found same results as my post 37 with the RGB values, even if Hawk has been able to do more things than me thanks to its knowledge, and as I said in my post 41: "CCE does NOT do colorimetry conversion and is expecting nothing as input"

As I said many many times in this thread, we don't care about the norm (and maybe we never know if there is a followed norm) but we must be pragmatic:

Even if assuming that 709 DVD were true, NO DVD players on the market (as long as I know) would play the 709 DVD without chromacity error (as DVD players use only two or three different chip decoder that are 601 - see my posts)), and ONLY Dgindex for PC (as I know (see my post 52)) would be able to read the re-encoded 709 DVD!!!

IT IS NOT an EVIDENCE of something, it is a FACT.

Now, everyone can do what he wants but it is a non sense to use a colormatrix (bt 601->bt709) with 601 DVD.

I know my english is bad (someone else, like Manono can write it), but I think we need a sticky post about this subject.

But even if there is nothing more on the subject, lets me thank, one more time, Jdobbs for its work, as I used it everyday.

[kumi]

we don't care about the norm (and maybe we never know if there is a norm) but we must be pragmatic:I agree.


NO DVD players on the market (as long as I know) would play the 709 DVD without chromacity error, and ONLY Dgindex for PC (as I know see my post 52) would be able to read the re-encoded 709 DVD!!!

It is not an evidence of something, it is a fact.I'm not sure why you are so sure about this. Can you explain?



Since the standards are murky, what if we blind test the users on Doom9? Carefully author a DVD with a few test patterns, have users view them, and then answer some questions about their visual impressions. The questions would be designed to determine which of Immersion's four 'Scenarios' is in effect for a particular SAP (see post #56). If enough users participate, we might be able to arrive at some real-world conclusions regarding the desirability of different colorimetry/header combinations.

But... is this type of blind study possible? Or rather, given a test DVD with a few sample clips: does there exist a series of simple-to-answer questions that will tell us which of Immersion's "Scenarios" a given SAP is using?

[jdobbs]

2. What conversion is DGDECODE doing when it frame serves it to HC or CCE?
it does nothing, the encoders do nothing. Uhh... it does something as it very obviously decodes it and presents it to the encoder as a picture.

[fjhdavid]

All the DVD players on the market are using a DVD decoder chip, and major chip manufacturer (MTK, cirruslogic, sigmadesign) are 601 compliant (see my previous posts).

then if 709 DVD was the norm, all this DVD player would play 709 DVD with a chromacity error (independantly from DVD-RB or HC/CCE)....do you think that all these engineers are wrong? I don't know but I doubt...

this is why Hawk and myself said that DVD must be 601 compliant as there is so much info on the net according to this fact.

[kumi]

All the DVD players on the market are using a DVD decoder chip, and major chip manufacturer (MTK, cirruslogic, sigmadesign) are 601 compliant (see my previous posts).I just looked through your posts, and I only see a clear answer about the MTK chipset.


this is why Hawk and myself said that DVD must be 601 compliant as there is so much info on the net according to this fact.Yes, I agree that Hank's word is valuable, but his tests were limited to PowerDVD. We still don't know the answer to what SAPs do, IMHO.

[fjhdavid]

No Kumi, look at my post 52 for sigmadesign...

and earlier post for cirruslogic (I don't remember where)
Remember in Post 37 I told you about PowerDVD (which is nr 1 for PC)

but we have to be pragmatic

for 601 flagged DVD the problem is solved (that was my original question!) as I know that all my 601 are working without colormatrixfilter

for "709 Dgindex flagged DVD" if you are ok with your DVD and your DVD player, it is the more important (and it seems to because I think your "709 DVD" are in reality 601 DVD with no header extension....)
(everything was ok for "dgindex 709 flagged DVD" from the beginning...the real question was just for 601 flagged DVD)

but this thread is useful as we learn many things...

[Sharc]

Hi experts

I came across the following:

1) Original DVD, Sequence Display Extension:
color_primaries=5 (ITU-R recommendation 624-4 System B,G)
transfer_characteristics=5 (ITU-R recommendation 624-4 System B,G)
matrix_coefficients=5 (ITU-R recommendation 624-4 System B,G)

2) DGindex info shows this as:
BT 470-2 B,G

3) My question:
What conversion - if any - should I include in the DVD-RB avisynth script? (xxx, yyy parameters?).
c:i:ColorMatrix(mode="rec.xxx->rec.yyy",interlaced=true)
c:p:ColorMatrix(mode="rec.xxx->rec.yyy").
For CCE, for HCenc?

I don't know what my DVD player expects, or how to find out.

Sorry if my question should be out of the scope of this thread.

[fjhdavid]

It seems that you have "flagged 601 DVD" (recorded from a Satellite box or an adsl box I presume?), then you should include NO conversion filters (colormatrix or others conversion filters) in the DVD-RB avisynth script

It is true for CCE and HCenc (maybe for others encoder, but I did tests only for these two decoders)

your standalone DVD player expect bt.601 and will output 601 in SD resolution (576i, 576p, 480i,480p)
your DVD software player for PC expect 601 and in your case (flagged 601 DVD), even dgindex expect 601


if your DVD is an upscaler player, then it will do, itself, the proper conversion from 601 to 709 to deliver HD resolution (720p, 1080i and 1080p)

you have to check also what your display is expecting, as sometimes, even if they are native HD, they expect 601 (but it was true for old plasma or videoprojector)

[45tripp]

it does nothing, the encoders do nothing.

Uhh... it does something as it very obviously decodes it and presents it to the encoder as a picture.
Then quite obviously the encoder takes the pictures and compresses them to a new set?
Am I on the right track?


2. What conversion is DGDECODE doing when it frame serves it to HC or CCE?

Nothing with regard to the color considerations being discussed.
At least not in it's common use.

it very obviously decodes it and presents it to the encoder as a picture.
not to rgb,
at least not in it's common use.


1.Take the following:
colorbars(720,480)
killaudio()
converttoyv12(matrix="rec709")
trim(1,60)

2.feed to cce, encode.

3.take output and index with dgindex

4.Add some conversion in decode, for fun
LoadPlugin("C:\Program Files\AviSynth 2.5\dgdecode.dll")
mpeg2Source("colorbar709.d2v", upconv=1)

5.encode with cce

709 down the line.
3 and 5 are the same
(apart from worsening of the vertical chroma shift that is)


btw,
as far as I can see,
DGindex display is scenario3: respects headers and uses 601 in header absence (!)
and decode with upconv=2 is scenario2: respects headers and uses 709 in header absence.

gl

[kumi]

I don't know what my DVD player expects, or how to find out.

Burn a DVD ("Disc1") with three clips on it:
Clip "A" = Hank315's in601out601.m2v
Clip "B" = Hank315's in601out709.m2v
Clip "C" = colorbars(720,480,"YV12").trim(0,25) (Encoded with CCE)

Burn another DVD ("Disc2") with three clips on it:
Clip "A" = Hank315's in709out601.m2v
Clip "B" = Hank315's in709out709.m2v
Clip "C" = colorbars(720,480,"YV12").ColorMatrix(mode="Rec.601->Rec.709", clamp=false).trim(0,25) (encoded with CCE)


Now check for the following conditions on both DVDs. Both discs should match:

If "B" and "C" do not look the same:
-> Your player respects colorimetry header info, and assumes Rec.601 when it is not present.

If "B" and "C" look the same and "A" and "B" do not look the same:
-> Your player respects colorimetry header info, and assumes Rec.709 when it is not present.

If "B" and "C" look the same and "A" and "B" look the same:
-> Your player does NOT respect colorimetry header info, and uses arbitrary coefficients to convert colorspace. In this case, I don't think it's possible to know for certain which coefficients it uses unless you have a calibrated test pattern for comparison.

Although it is possible to guess! Compare any of the clips on Disc1 and any of the clips on Disc2 with the reference SMPTE colorbars:
http://img523.imageshack.us/img523/2532/smptecolorbarsuo4.th.png (http://img523.imageshack.us/img523/2532/smptecolorbarsuo4.png)

If the clips on Disc1 (they should all look identical) are closer to the reference pattern than Disc2, your player uses Rec.601 coefficients, all the time.

If the clips on Disc2 (they should all look identical) are closer to the reference pattern than Disc1, your player uses Rec.709 coefficients, all the time.

[Sharc]

Thanks kumi.
Where do I get clips A and B from?

[kumi]

Link to the .zip is at the bottom of this post: http://forum.doom9.org/showpost.php?p=1061520&postcount=53

If you do try that out on your SAP, it'd be great to hear what your results are ;)

[fjhdavid]

may I know Sharc what is your DVD player brand and model and display?

if the decoding chip inside is coming from cirrus logic, MTK or sigmadesign, your DVD player is 601 (you can know the decoder chip by reading review on the net) and as you can read on the net almost (if not all) DVD players are 601 compliant...

BUT if you want to play and find the DVD player compliancy, you must know also what is your display compliancy (709 or 601 and regarding to which resolution SD or HD)...

secondly, are you always using SD or HD output as results should be completly different...

thirdly, if you are not watching the two prictures side by side, or if you have to swap your DVD in the player, or wait 2 or 3 seconds between each pictures, you won't be able to see anything, as the difference are sometimes invisible (in the blue) or hard to see (a colorimeter like a "spyder" for example is a must have for this kind of exercise...)


I think you will loose your time except if you watch always your DVD with a DVD player software on PC (as you can take picture you can check with photoshop)...

for info, differences are going from 0 to 15% (max!) in RGB level and eyes is not sensitive the same way to all wavelength and is in NO way an absolute detector but a relative one...

do a favor for me: take your halogen in your living room and look at the luminosity (medium level), then close your eyes 5 seconds, and in the same time ask a friends to change the intensity in a very small way, then open your eyes and say if the luminosity as decreased or increased....I did it and I am wrong 2 times out of 5....end of story

[kumi]

you must know also what is your display compliancy (709 or 601 and regarding to which resolution SD or HD) if you want to test only the DVD player
That's not entirely correct. Yes, players that do not respect the sequence display extension field are difficult to test. But if your player DOES respect this field, you can easily determine what default coefficients it uses to decode streams missing the sequence display extension! You do not need to know about your display, possess a colorimeter, calibrate any equipment, or compare to reference test patterns.

Just answer a few "does this look the same" questions, as in my last post.

[fjhdavid]

Kumi, did you see my post on the halogen luminosity? do it yourself and tell me if you are right all the time.

the problem is that except for DGindex, players DON'T seem to respect the sequence display extension field (and the norm itself is a mess)...

Sharc, I don't want to start the thread from the beginning, I think we have now the answers (or at least a very very good statistical guess) to our questions and my post 64 is just a summary for your case and question

one more time we have to be pragmatic...maybe we never know for sure....but according to our knowledge...the post 64 is highly likely

[kumi]

Yes, I read your post, but you didn't read mine ;) I'm only talking about players that respect the sequence display extension. Halogen luminosity is not important for these types. Let's break it down so it's easy to see. Consider 6 sample clips:

Clip "A": Rec.601 input, Rec.601 coefficients
Clip "B": Rec.601 input, Rec.709 coefficients
Clip "C": Rec.601 input, missing coefficients
Clip "D": Rec.709 input, Rec.601 coefficients
Clip "E": Rec.709 input, Rec.709 coefficients
Clip "F": Rec.709 input, missing coefficients



And four possible player behaviors:

Behavior 1: Sequence display extension not respected, Rec.601 conversion always
Behavior 2: Sequence display extension respected, Rec.601 coefficients assumed if extension missing
Behavior 3: Sequence display extension not respected, Rec.709 conversion always
Behavior 4: Sequence display extension respected, Rec.709 coefficients assumed if extension missing



Each Behavior will have a distinct fingerprint, in the form of how different clips are displayed with respect to each other:

"A" "B" "C" "D" "E" "F"
+----------------------------------------------------------------------
Behavior 1 | Normal Normal Normal Green-push Green-push Green-push
Behavior 2 | Normal Red-push Normal Green-push Normal Green-push
Behavior 3 | Red-push Red-push Red-push Normal Normal Normal
Behavior 4 | Normal Red-push Red-push Green-push Normal Normal



IF
"B" and "C" (or "E" and "F") look different
...we know our player MUST HAVE Behavior 2.


IF
"B" and "C" (or "E" and "F") look the same
AND
"A" and "B" (or "D" and "E") look different
...we know our player MUST HAVE Behavior 4.



P.S.: I'm NOT talking about Behavior 1 or 3. Knowing which of those a player is operating under involves real work! Determining the player's default coefficient in these two cases will require calibration and actual color measurement. Perhaps "halogen luminosity" is a factor here as well, I don't know.

But Behavior 2 and Behavior 4 do not require fancy steps.

[Sharc]

fjhdavid, kumi:

Thanks so much for your comments. I didn't want to start a new debate, and I am now more than happy to learn that the differences are minor, so there is little risk for my backups to be "noticeably wrong" because a colorimetry conversion is missing or superfluous.

Anyway, I may try kumi's test some time later this week to satisfy my curiousity, and report the results when available.

Added:
..and fjhdavid:
May DVD player is "no name / low cost" buy from a special offer .....

[fjhdavid]

Kumi,

I will try to do it too, but I have two questions:

In all the behaviour you talked about, will I have to compare different pictures that are not displayed in the same time or am I wrong?

secondly, my display is a sanyo Z3 projector which have a particularity (reported my many display calibrators): it is a 720P panel that expects 709 HD signal, but when you feed it with 601 SD, it also expect 709 even if it scaled the picture from SD to HD...so will all your GP or RP combinaison change or not?

thanks

[kumi]

will I have to compare different pictures that are not displayed in the same time or am I wrong?
That's right. "Is 'A' the same as 'B'" means you have to cycle between "A" and "B" with your remote. Author a DVD-Video with some simple navigation to cycle through the 6 different clips.


when you feed it with 601 SD, it also expect 709 even if it scaled the picture from SD to HD...
Please clarify: your projector will apply a conversion using Rec.709 coefficients on all types of SD input? Are there any input jacks or display modes that don't have this problem? If not, maybe you need to find a different TV to test your DVD player equipment...

[Sharc]

Here my results:
Disc 1:
A1 = B1, C1 slightly different (blue colors at the bottom of the picture

Disc 2:
A2=B2, C2 slightly different (blue colors at the bottom of the picture).

There seem to be very subtle differences between Disc1 and Disc2, ie between A1/A2 and B1/B2.

Added:
For the C clips produced with CCE I had to convert to YUY2() because CCE does not accept YV12.

[kumi]

A1=B1 and A2=B2?

Then it seems your player does not respect colorimetry stream parameters. Although I'm not sure what to infer from that slight difference in C1/C2.

[Sharc]

For clarification:
A1: Clip A of Disc 1
B1: Clip B of Disc 1
C2: Clip C of Disc 2
etc.

Yes. A1=B1, A2=B2
C1 is different from (A1,B1).

Same for Disc 2.
Disc 2 appears to be very slightly different from Disc 1 (?)

Here my test DVD for download:
http://rapidshare.com/files/67713605/VIDEO_TS.ZIP.html

[45tripp]

There seem to be very subtle differences between Disc1 and Disc2, ie between A1/A2 and B1/B2.


Of course.
Is the green of disk 1 too dark?
or is the green of disk 2 too light?

I say two too light,
and have money firmly on 601.

Although I'm not sure what to infer from that slight difference in C1/C2.

luma range.



I edited a correction in my last post,
dgindex display.... peculiar.

gl

[Sharc]

Is the green of disk 1 too dark?
or is the green of disk 2 too light?

I say two too light,
and have money firmly on 601.


Yes, exactly.
Disk1: green is darker than on Disk2
Disk2: Green appears too light.

Btw: For the C clips avisnyth didn't accept the clamp=false.
Clamp is unknown. Are there different versions of the colormatrix plug-in?
Edit: Solved with version 2.2 (I had to update from 2.0)

=> New test DVD here:
http://rapidshare.com/files/67775503/VIDEO_TS.ZIP.html

[ToS_Maverick]

thank you all for the whole lot of information in this thread! i think you got almost all information together ;)

now what i'm wondering is, why dosesnt ffdshow respect the colorimetry flags? even the very robust MPV decoder filter doesn't?!

[Insomniak4700]

Now this thread is interesting, but the more I read about colormatrix and bt709 vs 601 for dvd players, the more I get confused.... If I try to resume all this, Am i correct in assuming that there is no real standard, but that usualy all SD programs are 601?

A lot of people here have recommended the colormatrix filter to go from 601 to bt 709 for CCE and other encoders. Is this still considered true?

Or should I just remove this from my filter chain and let DVD-RB and CCE do their stuff? (I believe CCE will output 601 for 601 input, and 709 for 709 input?)

Thanks everyone, I thought I had all this color matrix stuff sorted out, but now i'm confused as h*ll

[kumi]

A lot of people here have recommended the colormatrix filter to go from 601 to bt 709 for CCE and other encoders. Is this still considered true?
The standard for SD MPEG-2 may actually be Rec.601, according to Hank and others. If that's the case, then your Rec.601->Rec.709 conversion is incorrect.


Or should I just remove this from my filter chain and let DVD-RB and CCE do their stuff?
If you believe that Rec.601 is the standard for SD MPEG-2, you should make sure you always feed CCE with Rec.601, i.e.
ColorMatrix(dest=2, hints=true)

[dragongodz]

The standard for SD MPEG-2 may actually be Rec.601

i disagree. the only specs we have say you either have the information in the mpeg or if it is not there then assume 709. what the question here is about is dvd playback which really hasnt been helped by all these posts about things like sd transmission etc which could just as easily have its own limitations etc. lets not try to confuse mpeg2 in general with other "standards".

Sharc, I don't want to start the thread from the beginning, I think we have now the answers (or at least a very very good statistical guess) to our questions and my post 64 is just a summary for your case and question

actually you came here with a pre-conceived idea in your head and have pushed that non stop. not wanting to start this thread ? well you started first in HC's thread, then this one and then another in the "mpeg2 encoding" section. cross post city.
seems you think you have proved your point so you are happy.

[fjhdavid]

insomniak,

if you are not sure about your source, Kumi and others gave some tests to guess.

What is sure is that PowerDVD is 601 (it doesn't care about the header extension)
a lot of DVD player (like Oppo, or players using MTK, cirruslogic or sigmadesign chip) seems also to be 601

most of the time DVD doesn't have info in their header extension and were supposed to be 709 (this is why many talked about (and used) the colormatrix (601 ->709)).

but it seems now that the norm is not so clear and that we must consider DVD to be 601 by default (then you must NOT use the colormatrix anymore for this purpose).

another info, is that my DVD are 601 (because they are recorded with a DVD recorder which writes 601 in the header extension).
In this case, the use of colormatrix (bt 601->bt 709) give me wrong results. I tested it with DVD calibration tools so I don't use colormatrix anymore in my case.

[Boulder]

I'm not sure what to think. I just did a small test once again, and the screenshots are here to see:

I used HC022 to encode, setting the flag appropriately. The screenshots are from VDubMod. DGIndex v1.4.9 used.
Original, not flagged (marked as unknown in DGIndex).
http://img131.imageshack.us/img131/486/orgcj4.th.png (http://img131.imageshack.us/my.php?image=orgcj4.png)
No ColorMatrix, no flag set in HC
http://img111.imageshack.us/img111/5707/nocmhb7.th.png (http://img111.imageshack.us/my.php?image=nocmhb7.png)
Rec.709->Rec.601, flagged as BT.470-2M
http://img136.imageshack.us/img136/4712/rec601uq7.th.png (http://img136.imageshack.us/my.php?image=rec601uq7.png)
Rec.601->Rec.709, flagged as BT.709
http://img136.imageshack.us/img136/7584/rec709nf0.th.png (http://img136.imageshack.us/my.php?image=rec709nf0.png)

In my DVD player (an old Pioneer DV-343), all of them look different. The one that is converted from Rec.601->Rec.709 does look a lot closer to the original though, but it is still not the same (except for the one that has no CM nor the flag set).

EDIT: I have never used ColorMatrix in the case where I have an unknown colorimetry shown. I'll test videos that have a flagged stream as soon as I've got the time.

[kumi]

i disagree. the only specs we have say you either have the information in the mpeg or if it is not there then assume 709. what the question here is about is dvd playback which really hasnt been helped by all these posts about things like sd transmission etc which could just as easily have its own limitations etc.
Fair enough. Do you agree with Hank that DVD has it's own rules, and that Rec.601 is more appropriate? http://forum.doom9.org/showthread.php?p=1061520#post1061520

[45tripp]

i disagree. the only specs we have say you either have the

information in the mpeg or if it is not there then assume 709.

And what good is that info?
If you know your app/player doesn't comply to this rule when converting to rgb?
If your source carries 601 coefficients but isn't flagged?


@Boulder
Vdubmod has known behaviour,
it complies to the spec drogongodz quotes,
respects flags, assumes 709 in absence.
You can easily predict the result without seeing actual output.

You might expect they look different.
You're forcing a push with colormatrix.

But,
Actually they shouldn't be that different.
You must have done something wrong.

image1 will always equal image2.
In vdub with what you did you should get:
image1=image2=image3,
image4 to have lighter greens/darker reds than the others.

your image3 is wrong,
I assume you didn't flag it correctly, either not setting a flag or setting
a 709 equivalent.

Deciding which set is 'correct',
can stem from assumption, comparison to external imagery, or just plain taste.

I don't like the quality of the image nor the selected frame enough to venture a guess.

You only need one of image1 and image2,
image3 is not required,
the correct way to produce image4 is to not apply colormatrix and encode applying 601 flags.

I think colormatrix should be limited in use.
A good example of use would be if you assumed/thought/liked your source as 709,
and knew your player was to ignore flags and use 601 coeffs,
then you could use mode="rec.709->rec.601"

gl

[dragongodz]

Do you agree with Hank that DVD has it's own rules

if you reread the statement you quoted that is EXACTLY what it says. it infact goes even further to say that you can not say specs for any other "standard" such as transmissions etc can be used as any sort of conclusive proof of what the dvd specs are as they may have their own set of limits.
you however said the standard for SD mpeg2 and NOT dvd. so i disagreed since plain mpeg2 specs we do actually have some documentation for.

and that Rec.601 is more appropriate?

that i am holding my judgement on for now. lets just say there are reasons i have doubts thats totally correct and when i have more information either way then i will respond.

And what good is that info?

please read the above response and take it in the actual context. i can easily ask what a lot of posts in this thread have to do with actual dvds aswell.
too much guessing and assuming based on other sources.

[dragongodz]

ok i have got some more information back about this. please find my latest post in this thread

http://forum.doom9.org/showthread.php?p=1089979

[hkazemi]

I've compiled a summary to hopefully consolidate the threads and address the confusion over colorspace and the usage of Colormatrix. It is available at http://forum.doom9.org/showthread.php?t=133982#post1090068

[jdobbs]

Just as a short add-on to this discussion -- I've been doing a lot of analysis on this lately as I'm backing up DVDs. It is not wise to make any assumptions on colorimetry based upon resolution (SD vs. HD). For example, while it is apparent that the default for DVD is BT.601 based upon the most current ITU docs and general consensus -- there are a lot of DVDs where the colorimetry is explicitly called out to be BT.709 in the SEQUENCE_DISPLAY_EXTENSION even though the source is SD. Whether that flag is honored by players (according to dragongodz link) isn't clear (at least to me).

[hkazemi]

Just as a short add-on to this discussion -- I've been doing a lot of analysis on this lately as I'm backing up DVDs. It is not wise to make any assumptions on colorimetry based upon resolution (SD vs. HD). For example, while it is apparent that the default for DVD is BT.601 based upon the most current ITU docs and general consensus -- there are a lot of DVDs where the colorimetry is explicitly called out to be BT.709 in the SEQUENCE_DISPLAY_EXTENSION even though the source is SD. Whether that flag is honored by players (according to dragongodz link) isn't clear (at least to me).

Do you have some examples of these DVDs that apparently are flagged BT.709? (Examples were requested in one of the threads.)

Note: There was an assumption in DGDECODE/DGINDEX versions before 1.5.0 beta 12 defaulted to reporting BT.709 when colorimetry information was not specified by 'sequence_display_extension'. If you were identifying DVDs as BT.709 based on output of DGDECODE/DGINDEX, you very well could have been seeing BT.709.

DGMPGDec/DGINDEX versions starting with 1.5.0 beta 12 (released Nov 9, 2007)(including the current 1.5.0 RC2), report BT.709* for HD video and BT.470-2* [aka BT.601] for SD video when colorimetry information is not specified by 'sequence_display_extension'. The asterisk (*) means the stream did not declare the colorimetry.

[Boulder]

Many PAL DVDs are flagged as BT.709.

[jdobbs]

Do you have some examples of these DVDs that apparently are flagged BT.709? (Examples were requested in one of the threads.)

Note: There was an assumption in DGDECODE/DGINDEX versions before 1.5.0 beta 12 defaulted to reporting BT.709 when colorimetry information was not specified by 'sequence_display_extension'. If you were identifying DVDs as BT.709 based on output of DGDECODE/DGINDEX, you very well could have been seeing BT.709.

DGMPGDec/DGINDEX versions starting with 1.5.0 beta 12 (released Nov 9, 2007)(including the current 1.5.0 RC2), report BT.709* for HD video and BT.470-2* [aka BT.601] for SD video when colorimetry information is not specified by 'sequence_display_extension'. The asterisk (*) means the stream did not declare the colorimetry.I'm not relying on DGDECODE/DGINDEX. I've written software that actually scans for the header and tells me the settings (so its more than apparent -- it's fact). You can easily open them up in VOBEDIT and see the same thing. There are lots of examples -- but here's one from a backup I did today: "The Wire, Season 4", Region 1, Discs 3 & 4 (but probably all). Even the menus are flagged as Rec.709 on those.