FAQ: How to correct luminance levels

[clsid]

@madshi

I have adjusted to first post to mention what to do in case of a CRT TV (method 2 and 3). Please correct me if I wrote anything wrong.

[tetsuo55]

I'm suggesting autodetection in fact.

Officially there is no such thing as full range 601, however every low resolution FRAPS video is in fact full range.

This is why the conversion needs to test both the chroma and the luma for ranges.
More info here:
http://en.wikipedia.org/wiki/YCbCr

But you understand correctly that we are missing the pal/ntsc difference in 601.

Also, i believe this has not been discussed before but:
Monitors and HDTV displays are all calibrated to sRGB/709.
However both pal and ntsc SD and broadcast signals live in slightly different colorspaces that do not fit in sRGB/709, so the conversion will result in either clipping or distortion of a small portion of the colors

[leeperry]

Originally Posted by tetsuo55

every low resolution FRAPS video is in fact full range.

because games are full range in the first place.

Originally Posted by tetsuo55

you understand correctly that we are missing the pal/ntsc difference in 601.

SD uses REC.601 coeffs for RGB conversion, whatever PAL or NTSC.
it's the gamut that is different, depending on the mastering but the YCbrCr>RGB coeffs are identical in both cases...just like REC.709 for HD.

[madshi]

Originally Posted by clsid

I have adjusted to first post to mention what to do in case of a CRT TV (method 2 and 3). Please correct me if I wrote anything wrong.

Reads fine to me now - thanks!

[mark0077]

This is excellent. So I have just calibrated my new Samsung Series 9 using settings from a guy on another forum, and I think he calibrated it with the target 709. I have my new TV set to accept input as full range.

If this type of auto detection could be done for all of the combinations you talked about, what way should our displays be calibrated? Does any calibration technique incorporate all of the standards like 601 and 709?

[tetsuo55]

Originally Posted by leeperry

because games are full range in the first place.

SD uses REC.601 coeffs for RGB conversion, whatever PAL or NTSC.
it's the gamut that is different, depending on the mastering but the YCbrCr>RGB coeffs are identical in both cases...just like REC709 for HD.

Yes thats true, but we need to correct the gamut as the display will be sRGB not PAL/NTSC, seeing as the gamut conversion can be incorporated in the YCbrCr>RGB as a slight offset we should do so.
This is not needed for REC709 because the gamut already matches sRGB.

Originally Posted by mark0077

This is excellent. So I have just calibrated my new Samsung Series 9 using settings from a guy on another forum, and I think he calibrated it with the target 709. I have my new TV set to accept input as full range.

If this type of auto detection could be done for all of the combinations you talked about, what way should our displays be calibrated? Does any calibration technique incorporate all of the standards like 601 and 709?

the default target should be sRGB/709.
This is the factory setting for all digital 720p capable displays and all monitors

[mark0077]

709 is the way to go then, thanks! Maybe you should present your findings / suggestions to an ffdshow developer. I am sure they would be delighted to read about the PAL/NTSC 601 aswell as the different methods for colorspace conversion.

[tetsuo55]

I'm not sure the story is finished yet.

I want to get some more feedback first.

-----------

Complete (automated) Colorspace, gamut and level conversion for digital video:

#check luma and chrome for clipping to determin the used levels for either
Adjust expansion parameters accordingly
#check if the video is SD or HD
if resolution is XXX or higher = HD
else SD
# in case of SD check for PAL/NTSC
If framerate is not 25/50 assume NTSC ?
((not sure how to handle 24fps))
((might also want to check resolutions))

-in case of HD use 709 conversion
-In case of PAL-SD use 601-PAL conversion
-In case of NTSC-SD use 601-NTSC conversion

The entire process leads to a single formula (not multiple conversions)

To reduce rounding errors a higher bitdepth can be used(like 16) for the processing.

NOTES:
-This method preserves all BTB/WTW data for those non-standard videos that contain them.
-HD video can contain full-range without breaking spec.
-This conversion is valid for all PC based systems, even if the videocard supports 16-235 it will do this conversion(or re-convert) itself resulting in crushed blacks

I will have to find the exact primaries for NTSC/PAL they where posted here on doom9 or at the avsforum

[leeperry]

Originally Posted by tetsuo55

Yes thats true, but we need to correct the gamut as the display will be sRGB not PAL/NTSC, seeing as the gamut conversion can be incorporated in the YCbrCr>RGB as a slight offset we should do so.
This is not needed for REC709 because the gamut already matches sRGB.

the default target should be sRGB/709.
This is the factory setting for all digital 720p capable displays and all monitors

well most big LCD screen/LCD projectors have wide gamuts, and hardly any display has a perfect sRGB gamut....and you need to measure the primaries coordinates with a colorimeter in order to fix them.

also european/russian/continental chinese BD's are either 23.976/24fps but they use the EBU gamut

I don't think you can let ffdshow make wild guesses on the gamut.

[madshi]

Originally Posted by tetsuo55

Complete (automated) Colorspace, gamut and level conversion for digital video:

#check luma and chrome for clipping to determin the used levels for either
Adjust expansion parameters accordingly

I'm not sure if you can reliably detect this in every frame of every movie. What if this algorithm sometimes gives us wrong results? Then black / white level will fluctuate in the middle of the movie. I don't like the idea of doing this automatically, unless we find a solution which is 100% fail safe.

Originally Posted by tetsuo55

NOTES:
-This method preserves all BTB/WTW data for those non-standard videos that contain them.

BTB/WTW is not "non-standard". A DVD may officially contain BTB and WTW information. Of course the DVD makers must live with the fact that a properly calibrated TV will supress most of the BTB/WTW information. But this doesn't make the use of BTB/WTW illegal or non-standard.

[leeperry]

Originally Posted by madshi

I'm not sure if you can reliably detect this in every frame of every movie. What if this algorithm sometimes gives us wrong results? Then black / white level will fluctuate in the middle of the movie. I don't like the idea of doing this automatically, unless we find a solution which is 100% fail safe.


BTB/WTW is not "non-standard". A DVD may officially contain BTB and WTW information. Of course the DVD makers must live with the fact that a properly calibrated TV will supress most of the BTB/WTW information. But this doesn't make the use of BTB/WTW illegal or non-standard.

some encoders create "padding" data in the BTB/WTW
also, Seb.26 has added a dynamic levels option in ffdshow filters(select "Seb's BTB&WTW"), it will dynamically show you these levels...if they ever existed in the first place.
you can of course set the detection threshold, the time hold etc etc
some guys on HCFR are hooked to it!

[tetsuo55]

Originally Posted by leeperry

well most big LCD screen/LCD projectors have wide gamuts, and hardly any display has a perfect sRGB gamut....and you need to measure the primaries coordinates with a colorimeter in order to fix them.

also european/russian/continental chinese BD's are either 23.976/24fps but they use the EBU gamut

I don't think you can let ffdshow make wild guesses on the gamut.

those wide gamuts are not used, the computer can only output 8bit sRGB(you could use the wide gamut at the cost of banding though)

I might be wrong, but according to the mastering article i read about bluray, no matter what the source was it gets converted to REC709 before encoding to disk.

But the editing does indeed often happen with EBU or SMPTE-C. Not all movies seem to fall into this category it seems, i have seen a few lists here and there but they seem fairly short.

This process does not need to happen in FFdshow(which should have a lot more user options on top of the auto-detec tion)

[leeperry]

Originally Posted by tetsuo55

those wide gamuts are not used, the computer can only output 8bit sRGB(you could use the wide gamut at the cost of banding though)

if you output vanilla sRGB on a wide gamut display, the picture will be oversaturated to death...reason why some ppl do gamut conversion so the input/output tints match

[tetsuo55]

Originally Posted by madshi

I'm not sure if you can reliably detect this in every frame of every movie. What if this algorithm sometimes gives us wrong results? Then black / white level will fluctuate in the middle of the movie. I don't like the idea of doing this automatically, unless we find a solution which is 100% fail safe.

We will have to find a 100% safe method.
Keep in mind though, that at default settings almost everyone is seeing an incorrect image at this point.
Only the people who tweak stuff get an accurate image.

Originally Posted by madshi

BTB/WTW is not "non-standard". A DVD may officially contain BTB and WTW information. Of course the DVD makers must live with the fact that a properly calibrated TV will supress most of the BTB/WTW information. But this doesn't make the use of BTB/WTW illegal or non-standard.

I doublechecked the mpeg2 specs.
According to the calculation on page44 anything below 16 will get clipped. If a device is capable of showing lower values it can decide to ignore this calculation.

[madshi]

Originally Posted by tetsuo55

I doublechecked the mpeg2 specs.
According to the calculation on page44 anything below 16 will get clipped. If a device is capable of showing lower values it can decide to ignore this calculation.

Then how come every good DVD player outputs full BTB/WTW information?

[tetsuo55]

Originally Posted by leeperry

if you output vanilla sRGB on a wide gamut display, the picture will be oversaturated to death...reason why some ppl do gamut conversion so the input/output tints match

I have yet to see a commercialy available wide gamut capable display that is not factory calibrated to 709/sRGB (cheap misscalibrations with blue push does not count)

Originally Posted by madshi

Then how come every good DVD player outputs full BTB/WTW information?

Because they are smart, as it saves money and they can actually use it for promotion purposes

[leeperry]

Originally Posted by tetsuo55

I have yet to see a commercialy available wide gamut capable display that is not factory calibrated to 709/sRGB (cheap misscalibrations with blue push does not count)

did you look at the CIE charts of projectors/LCD screens ?
you can't get a factory calibrated display with a perfect sRGB/HDTV(they're identical) gamut.
even professional broadcast equipment used in mastering houses is weekly recalibrated by color engineers with Minolta CS sensors(that are also recalibrated every 6 months).

this is the native wide gamut of a JVD HD750 :



offering generic gamut conversion in ffdshow is not possible

still I guess most smart displays will use the following YCbPr>RGB decoding depending on the resolution...or will let you select it at least :
Rec. ITU-R BT.601-5 => PAL / SECAM / NTSC
Rec. ITU-R BT.709-4 => HD

Originally Posted by tetsuo55

Because they are smart, as it saves money and they can actually use it for promotion purposes

video equipment works in TV range, xvYCC will finally introduce the 0-255 range for video content(increasing the gamut as well)
standalone DVD players output 16-235 video data in 0-255..if I remember well it was chosen to go 16-235 to avoid crosstalking for airwaves broadcasting.

[tetsuo55]

the problem with this wide gamut is bitdepth.

Storing all those values in 8bit will cause a lot of banding, secondly, if that display does not do a near-lossless 8bit source to 10bit wide gamut conversion all colors will be displayed incorrectly.

These wide gamuts are intended to be used with 10bit or higher signals, even 10bit has enough space to store a very wide gamut without banding.
The industry is well aware that we have 8bit-random-colorspace/gamut images and video. For this reason as much as possible is stored in REC709/sRGB. These colorspaces can be losslessly converted to the display's gamut in 8bit using ICC profiles.(Every display device should have the option to display limited range/sRGB(not necessarily accurately calibrated))

You could display the image completely in the wide gamut, however this could result in banding.

When i say generic conversion i mean converting the ANY to 1, the standard in use by the entire computer industry and HD video is sRGB. From this standard either a 3D-lut or preferably a ICC v4 profile can make the last minute adjustments for the end user display(losslessly).
For those people that do not care/know about this, they will at least have an as close to sRGB image as possible, instead of random and guaranteed wrong colors/luminance.
In the end the computer/display will always be trying to display sRGB, no matter how badly it does so.

----
PS, about the JVD HD750 chart
The first thing any professional calibrator will do in an end user situation is try to get those primaries as close as possible to sRGB.

----

I hope that when windows7 is released i will be able to buy a HDMI 1.3+ videocard and use 10bit xvYCC(my current card already does 10bit but not xvYCC, and even if it did windowsxp/vista does not allow more than 8bits per channel over hdmi)

[leeperry]

Originally Posted by tetsuo55

These wide gamuts are intended to be used with 10bit or higher signals, even 10bit has enough space to store a very wide gamut without banding.

well they are used to display 8 bits video atm.

Originally Posted by tetsuo55

The industry is well aware that we have 8bit-random-colorspace/gamut images and video. For this reason as much as possible is stored in REC709/sRGB.

US/ASIAN movies are stored in REC.709, but their mastering gamut is SMPTE-C, european movies use the EBU gamut(whatever SD or HD)...you won't be able to find a commercial movie using the sRGB gamut(only videos shot on consumer HDV I guess).

Originally Posted by tetsuo55

These colorspaces can be losslessly converted to the display's gamut in 8bit using ICC profiles.(Every display device should have the option to display limited range/sRGB(not necessarily accurately calibrated))

too bad this will only work in color managed applications(like softproofing in Photoshop)....you won't find a color managed video renderer

Originally Posted by tetsuo55

You could display the image completely in the wide gamut, however this could result in banding.

most displays work internally in 10/12 bits, so they won't create more banding than there was in the 8 bits source.....just all the colors will be oversaturated.

Originally Posted by tetsuo55

When i say generic conversion i mean converting the ANY to 1, the standard in use by the entire computer industry and HD video is sRGB. From this standard either a 3D-lut or preferably a ICC v4 profile can make the last minute adjustments for the end user display(losslessly).
For those people that do not care/know about this, they will at least have an as close to sRGB image as possible, instead of random and guaranteed wrong colors/luminance.
In the end the computer/display will always be trying to display sRGB, no matter how badly it does so.

well you've got no idea what their display CIE chart looks like, providing them with SMPTE-C/EBU>sRGB conversion won't help at all, as their native gamut is prolly a far cry from the HDTV/sRGB gamut in the first place...and a PC works in 8 bits so each conversion is VERY much lossy.



before deciding anything, please refer to yesgrey3 as he's very much aware of all I'm saying(he did the gamut conversion mathlab stuff for the PS gamut conversion script)...besides I don't think the ffdshow coders care too much about gamuts

Originally Posted by tetsuo55

about the JVD HD750 chart
The first thing any professional calibrator will do in an end user situation is try to get those primaries as close as possible to sRGB.

but DVD/BD are mastered in SMPTE-C or EBU gamuts, what would you care for sRGB exactly ?
besides it's your lucky day, the HD750 does have built-in gamut mapping options....but the RS1/HD1 does not.

here's a HD1 before/after SMPTE-C conversion using yesgrey3/JohnAd PS script in MPC :
http://www.avsforum.com/avs-vb/showp...4&postcount=90

as you can read, the movie colors are not grossly oversaturated anymore and the guy is very happy w/ the gamut corrected picture

Originally Posted by tetsuo55

I hope that when windows7 is released i will be able to buy a HDMI 1.3+ videocard and use 10bit xvYCC(my current card already does 10bit but not xvYCC, and even if it did windowsxp/vista does not allow more than 8bits per channel over hdmi)

well there's no xvYCC movies, and there won't be before a long while....it will only be useful for gamut conversion as 8 bits input/output is very lossy..

we should stop threadcrapping clsid's sticky topic anyhow

[chros]

Originally Posted by chros

Is there any registry entry for Nvidia cards for the "full range"?
Because my setup doesn't want to save this setting ...

I have find that there are values which are saved under:
HKEY_CURRENT_USER\Software\NVIDIA Corporation\NVControlPanel\RegisteredServers\Video_TVServer\AdjustVideoColorSettings\Gpu:0_Head:0

HKEY_CURRENT_USER\Software\NVIDIA Corporation\NVControlPanel\RegisteredServers\Video_TVServer\AdjustVideoColorSettings\Gpu:0_Head:1

I've got only 2 values:
AdjustVideoColorSettingsLastSelectedGammaState
AdjustVideoColorSettingsLastSelectedRGBLockStatus

Can someone see which value correspond to this?

Thanks

Setup: fresh WinXP SP3, latest whql driver, 9600GT

I have installed the new 181.20 whql drivers, but the problem still reamins.
Can someone look it for me, please? It drives me crazy ...

Thanks