yCMS - Color Management System

[madshi]

These are my thoughts (if I am thinking as a "general" engineer):
If a standard has an exact math function which defines the encoding transfer function but the standard says nothing about the decoding, then:
- It is obvious that we have to use the inverse function for decoding. (I think that any engineer who doesn't know anything about displays but she/he used any kind of standards for her/his jobs will say the same, as a general rule and first opinion.)
Generally agreed. *However*...

- The makers of the Rec709 standard could assume that everybody will follow the old "rule of thumb" that any customer displays will be calibrated to gamma 2.2 (the simple power function with two substantial value (Where "calibrated" means that it is either properly calibrated by the user/factory/ect. or it was supposed to be calibrated... Or it is a mess but it won't matter then because we design for reasonable conditions.)), or any chosen gamma value (for the simple power function) to honor the environment view conditions.
In this case, they could design the encoding transfer function with internal corrections for this situation.
The BT.709 transfer function is the same as the stone age BT.601 transfer function, which was defined when dinosaurs (eh, CRTs) roamed the earth. As far as I understand, Charles Poynton (who is kind of a guru in video processing matters) claims that the transfer function of BT.601 (and BT.709) was made so that CRTs could display a good image without having to do any funny processing. And according to Poynton, CRTs have a natural pure power curve with a gamma value of about 2.3-2.4. The BT.709 transfer function has a linear segment in it which is supposed to hide sensor noise. Now if we used the exact inverse for decoding, we would undo the purpose of the linear segment. Furthermore we would also behave very different to how CRTs behaved. In other words: We would behave very different to the display technology BT.709 was targetted at.

These arguments make me believe that the correct way to watch BT.601 and BT.709 content is to simulate the behaviour of the technology BT.601/709 were designed for.

The key argument for me is that the linear segment in the BT.709 transfer function has a very specific purpose. And if we use the direct inverse of the BT.709 transfer function for decoding, we are invalidating the very purpose of the linear segment.

[janos666]

These arguments make me believe that the correct way to watch BT.601 and BT.709 content is to simulate the behaviour of the technology BT.601/709 were designed for.


This sounds very reasonable for me.
But on the other side, this linear segment can be expounded many ways:
- A correction was required for CRTs with their native characteristics. But it is useless with a theoretically perfect power function.
There could be a strange segment on the native CRT curve. They had to assume that consumers will use uncalibrated CRTs. (There was no any digital signal processing in the consumer level CRT TVs in the past. So, the correction was used on the encoding side).
- Old sensors could produce more noise on dark areas. And new digital medias can use post-process effects to reduce this noise.
- The combination of these things together.

Here comes the testing...

I tried to do some graphical iterations to find some values for the output_transfer_function command. I think that I should use 7.5 0.099 0.373 0.018 for my LCD display (which was calibrated to gamma 2.2 - the simple power function).
But I am not sure about the result. May be I should try to skip the correction on the linear segment or I chose a bad slope value.

[yesgrey]

True. I'm not sure if he thought his answer through. But in any case he clearly said that he calibrates to a pure power curve of 2.2 - 2.35, which is a lot darker than what yCMS currently does.
A lot darker is not the same as more real or more natural looking.

I'm not saying that current yCMS output is how it should look like, because I agree that there isn't no fit all solution, but some differences are expected. My question is: do you feel the image to look unnatural?

We've asked him several questions in the past, and he replied to both you and me in separate threads that in his experience it's a useful shortcut to forget about the linear segment.
Yes, but only for simplicity (faster processing) and in case we didn't want to adapt the coefficients on the transition point.

What is your opinion of the EBU's and Charles Poynton's recommendation to use a system gamma of 1.2 or 1.25 (so you would have to multiply your gamma value with 1.2)? You seem to be totally ignoring that. Do you think the EBU and Charles Poynton got it wrong?
No, but remember that we don't watch in our displays the image directly captured by the cameras. We watch those images previously graded to video by studios. It's there, in the studios, where the 1.2 relation should be applied. So, if we had in our homes the exact studio conditions all we would need would be the same transfer function used in their monitors, hence the bt.709 with a 2.2 gamma. However, since our viewing conditions would be different from studios, another 1.0-1.2 factor needs to be used, depending on the conditions each user has. So, in the end, the final factor between the image we are watching and the original images captured by the camera might be something between 1.2-1.44.

I think you need to check your definition of "flawed mode" because I believe you got that wrong. Sorry. You seem to think that there's only one correct viewing gamma curve/value and that yCMS has currently nailed it.
Sorry, that expression was supposed to be funny but I realize now it wasn't (I've edited it). I prefer to have yCMS base working mode to be the mathematical accurate one, and for that there is only one: using the exact formulas of the standard. Having said this, I do realize that the real world viewing is much more than just some mathematical expressions, hence why I always considered the possibility of changing gamma. All this discussion has been good because now I realize that it might be preferable, in some situations, to have a pure power function as the display output, so I will also consider that in the future command settings.

Are you saying that so many of the professionals are wrong, Calman's help system doesn't know what it's talking about, and only you got it right?
Of course not.

These are my thoughts (if I am thinking as a "general" engineer):
If a standard has an exact math function which defines the encoding transfer function but the standard says nothing about the decoding, then:
- It is obvious that we have to use the inverse function for decoding. (I think that any engineer who doesn't know anything about displays but she/he used any kind of standards for her/his jobs will say the same, as a general rule and first opinion.)
That's what I thought too.

What I'm trying to say, and it seems people still didn't get it, is that current yCMS mode (that I call reference mode) is not the ULTIMATE/BEST/MIRACULOUS/FIT ALL solution, but simply the theoretical mode. It might be the best solution for some situations, but I know it's only that.

In the EU you can throw out any standards (like the Euro Code) when you feel them useless (until some additional country specific law or your actual contract says something else about it).
Yes, I know, and yCMS would allow users to break the "rules" and make their own decisions. yCMS is not meant to be a strict standards follower, but it has to have that always in his core base, or else it would start to be considered as some kind of "snake oil" solution.

I watched tons of movies with uncorrected gamma. How can't I feel it strange at the first time with the correction? (And I didn't try to watch a complete movie because here is the bug yet.)
Yes, it's natural, but only for curiosity I would like to ear what people feel while watching to the current mode, and no one but me have posted about it, and I can't be considered as an independent and trustful reviewer...:D

I used to say "transfer function" or "Rec709 curve", ect, when I talk about a complex math function and I used to write "gamma curve" or "gamma 2.2" when I talk about a simple power function. Is it theoretically right?
No. Theoretically it's not right because both type of curves include the "gamma" value. Gamma is the designation of the exponent value used in the power function, and both functions have a power function in it.

In pure mathematical terms we can consider that both the BT.709 and the pure power function belong to the same class, with the pure power curve being the case when no offset is used in the formula (you can look at the general formula inside yCMS manual).

The key argument for me is that the linear segment in the BT.709 transfer function has a very specific purpose. And if we use the direct inverse of the BT.709 transfer function for decoding, we are invalidating the very purpose of the linear segment.
Agreed, but we don't know if that would be bad... Soon we will know.;)

[janos666]

What do you think about a support for the XYZ format type of input values to use it for every type of corrections (gamut and gamma together)?
My new sensor (ColorMunki) isn't supported by HCFR or too much third-party softwares. But I can use ArgyIICMS which provides me measurements in this format:

SAMPLE_ID RGB_R RGB_G RGB_B XYZ_X XYZ_Y XYZ_Z
...
69 0.00000 50.0000 0.00000 7.87530 15.5280 2.29060
70 12.5000 62.5000 12.5000 13.5690 25.6430 5.72250
71 0.00000 50.0000 25.0000 8.87860 16.0140 7.25910
72 12.5000 62.5000 37.5000 15.4420 26.4640 15.2890
73 0.00000 50.0000 50.0000 12.0260 17.4260 23.3070
...

So, I would be able to copy-paste the full measurement data from the *.ti3 files. I have 238 measures right now but I can use more or less and I can define user points too. It has some redundant values at the beginning, because it takes 4 measures on the 100% white. (It is an instrument agreement test, I guess.) But every other measures was taken on various shades. Of course, it includes 0% black and 100% white points.)
But my old EyeOne provided sensor data in this XYZ format too. So, it requires a conversion to get xyY format for yCMS. This conversion should be done by your software to ensure the quality and to make the life of the users easier. :)

Yes, may be I can do an excel document to convert these values into copy-paste and yCMS friendly format. But read my arguments above. :cool:

[yesgrey]

What do you think about a support for the XYZ format type of input values to use it for every type of corrections (gamut and gamma together)?
OK, added to my todo list.

[yesgrey]

yCMS v1.6 released

http://yesgrey.com/ycms.html

- Fixed bug that caused corruption of blue at very low levels.
- Fixed bug that caused clipping of BTB and WTW video data with RGB_Video.

[yesgrey]

What do you think about a support for the XYZ format type of input values to use it for every type of corrections (gamut and gamma together)?
This would be done at a later stage, when yCMS would start to receive xyY data for both gamut and gamma correction. For now, if you want to, you can use part of your XYZ data into the Grayscale_Measurements command. The "Y" values in "XYZ" and in "xyY" are exactly the same, so use only the "Y" values from XYZ and copy them into the Grayscale_Measurements command - yCMS will use the same values for all color channels, but the error should not be significant.;)

[janos666]

The new version is good. I couldn't find any bugs (yet).

But I think I won't use it: We aren't sure about the theoretically perfect gamma handling and my subjective before/after tests won't let me decide squarely (sometimes I want to choose the corrected but other times I would prefer the uncorrected). So, I think I simply choose the uncorrected state because it let the display to show all the available shades. (And I think it is theoretically acceptable to watch Rec709 content on a display with pure power gamma 2.2 tonal response.) I think that my eyes adapts to an offset gamma easier than I can forget about the lost shades (or dithering noise, ect...).

And there is a little weird thing here: It was said earlier that the assumed end-user characteristic is a pure-power gamma 2.35. Well, I watch it on a display with pure-power gamma 2.2, so the image should be brightened up a bit. But I think it is either perfect or a little darker than it should be (but never brighter!). The full inversion with the Rec709 power function would brighten it up too much.
So, may be the correct state is between them. I think one of them would be correct:
1: A full Rec709 inversion and a correction between 2.35 and 2.20 pure-power gamma functions. (Inversion and "darkening".)
2: Inversion with a scaled ("darkened") Rec709 function (with the assumption that Rec709 meant to be shown on displays with gamma 2.35)
How can I figure out the correct output_transfer_function values for the second one? I would scale the full function with the slope as well. (The linear-exponential point would remain the 0.018 input value but anything else would be scaled...)

[yesgrey]

The new version is good. I couldn't find any bugs (yet).
Good. :)
How can I figure out the correct output_transfer_function values for the second one? I would scale the full function with the slope as well. (The linear-exponential point would remain the 0.018 input value but anything else would be scaled...)
I wouldn't advise going that route, it's not worth it. I'm already working on a new command for specifying the gamma curve desired, so it would be preferable to just wait a little more for it...;)

[j5627429]

My new sensor (ColorMunki) isn't supported by HCFR or too much third-party softwares. But I can use ArgyIICMS which provides me measurements in this format:


Argyllcms gives you Yxy values too.

I use "spotread -e -x" and it gives me something like this:

Result is XYZ: 14.345926 15.519101 15.986478, Yxy: 15.519101 0.312878 0.338464

I like that it gives you a lot of sigificant digits, but I now use NEC SpectraView instead. Only three decimal places but MUCH faster and easier to use than argyllcms command line.

Still, both of these are far inferior to the $200 version of Calman that supports Colormunki if you're actually going to perform a full calibration on your display. But since my projector's controls dont allow me to do that... it's yCMS to the rescue!! Thanks yesgrey :D

[yesgrey]

Argyllcms gives you Yxy values too.
It's good to know. So, for now, I will keep yCMS only with Yxy support, this way I will avoid to add another setting.

[janos666]

Argyllcms gives you Yxy values too.

I use "spotread -e -x" and it gives me something like this:

I think that "spotread" was developed for printer calibration (to measure the printed test spots). :cool:

To be specific, I used the DispcalGUI software. I think it uses the "dispcal" function for display calibration (through the VGA LUT) and the "dispread" function to get some measurement data for the display profiling function.
I want to use the post-calibration measurements (made by dispread) because I want to keep the calibration LUT loaded for 24/7. (And I posted the data format above.)

But I am limited to the sRGB mode on this display right now. (Nobody should consider to buy a U2410 for general purpose PC usage!) If I would be able to use a native mode (Which simply doesn't exist on this display: Every mode is factory calibrated but they screw up every mode expect the sRGB and that one is not perfect either...) then I want to use yCMS with these dispread measures to get a perfect gamut and gamma correction for movies. :) (And I would use the sRGB mode for PC games only. This was my plan, but...)

I think we would be able to create a little bat file for ArgyllCMS and yCMS to produce a 3DLUT as the final output. (Some kind of automatic calibration, like the ICC files for CMS supported PC softwares...)

I like that it gives you a lot of sigificant digits, but I now use NEC SpectraView instead. Only three decimal places but MUCH faster and easier to use than argyllcms command line.

You should check your instrument's precision on you display. I think that three substantial digits are enough.

Still, both of these are far inferior to the $200 version of Calman that supports Colormunki if you're actually going to perform a full calibration on your display.

I don't think so. As I know it won't make any calibration LUT. It let you configure your hardware with instrument support and it is aided by a cool GUI but nothing is automated.
It was developed for high-end HDTVs and projectors with internal hardware CMS features. ArgyllCMS is much better for any PC attached displays (until you have some high-end display with rewritable hardware (3D)LUTs or other kind of internal hardware CMS...)

But since my projector's controls dont allow me to do that... it's yCMS to the rescue!! Thanks yesgrey :D

I agree. yCMS is very good for software CMS. But software CMS can't compete with a good hardware CMS. (At least until we get DeepColor support for madVR. :))
For example, this U2410 produces better gamut emulation than yCMS and it would be a "perfect" display with rewritable internal LUTs (which are exist but I can't access them...).

I think I will be mad enough to buy an expensive EIZO (with rewritable internal 3DLUTs) when they sell some models with H-IPS panels. (Because this is the best LCD panel technology I ever saw --on quality/price side and may be generally too-- but current displays lack of software support for proper calibration...) I think they should make a Foris II with a H-IPS panel. I would consider to buy one of them! :cool:
(Until then... I will search for some hackers to access these LUTs in the U2410. Haha, nice dream... :devil:)

[j5627429]

I think that "spotread" was developed for printer calibration (to measure the printed test spots). :cool:

The -e option is for emmisive readings, which of course takes the measurements without the little LED. It also lets you options for measuring a CRT or LCD, so I don't think it is just for printed things. See for yourself here:
http://www.argyllcms.com/doc/spotread.html

To be specific, I used the DispcalGUI software. I think it uses the "dispcal" function for display calibration (through the VGA LUT) and the "dispread" function to get some measurement data for the display profiling function.
I want to use the post-calibration measurements (made by dispread) because I want to keep the calibration LUT loaded for 24/7. (And I posted the data format above.)

Sorry I'm a little confused here. Doesnt the calibration profile you created with DispcalGUI get disabled when you play a movie using MadVR? Wouldn't you want to take your yCMS 3dlut creation measurements with your display profile disabled?

Why would spotread not be able to measure your post-calibration profile?



I don't think so. As I know it won't make me any calibration LUT. It let you configure your hardware with instrument support aided by a cool GUI but nothing is automated.
It was developed for high-end HDTVs and projectors with internal hardware CMS features. ArgyllCMS is much better for any PC attached displays (until you have some high-end display with rewritable hardware LUTs or other type of internal hardware CMS...)

Well if even if your display doesn't have a full CMS, is it not better to calibrate as much as you can on the display-end before you use software such as yCMS to rein in the colors and/or greyscale the last little bit?

Obviously Calman requires SOME type of controls on the display, and your computer monitor sounds like it is very limited in its built-in calibration capability, so calman probably isnt ideal for you. I believe the software does walk you through the process and teaches you how to obtain a good result using whatever controls your display has.




I think I will be mad enough to buy an expensive EIZO (with rewritable internal 3DLUTs) when they sell some models with H-IPS panels. (Because this is the best LCD panel technology I ever saw --on quality/price side and may be generally too-- but current displays lacks of software support for proper calibration...) I think they should make a Foris II with a H-IPS panel. I would consider to buy one of them! :cool:

Do you watch movies with the lights on? Have they fixed the backlight bleed on PC monitors yet? Last I checked, most computer monitors couldn't measure up to the (black level) performance of a good TV or home theater projector...especially the TVs that have locally dimming LED backlights that can turn completely off in certain areas.

[janos666]

The -e option is for emmisive readings, which of course takes the measurements without the little LED. It also lets you options for measuring a CRT or LCD, so I don't think it is just for printed things.

I can believe that spotread could be used for LCD measurements but dispread was developed for display readings. Why do I want to use the spotread then (which was originally developed for reflective measurements)?

None of them is better or worse. Dispread produces XYZ and spotread produces Yxy. I can convert between these formats manually and the Y value is identical.

My suggestion (for yesgrey) was to accept this kind of input format. So, we can simply copy the numerous measurements from the dispread output file to achieve a color management like we have with ICC supported applications. -> Gamut, gamma and white balance correction from numerous and various measurements, instead of 100% RGB chromacity and some grayscale luminance.

And I still prefer dispread because:
- I believe that most of the instruments produce XYZ format and anything else is a converted value. (And we like to avoid the rounding errors...)
- The DispcalGUI uses this function, so it is easier to use.
- It is the one which was developed for display readings...

Sorry I'm a little confused here. Doesnt the calibration profile you created with DispcalGUI get disabled when you play a movie using MadVR? Wouldn't you want to take your yCMS 3dlut creation measurements with your display profile disabled?

No. I don't know where this rumor was born but the calibration LUT is permanent. It sits at the end of the line, right before the display output. This is the last data manipulation before the DVI/VGA/HDMI/DP/ect encoding. It is always applied until something purposely overwrites the VGA LUT data.

May be there is some D3D or OpenGL software which play with this LUT for some reasons. May be it wants to use it for gamma correction. -> You can find gamma correction option in some games... But that is a very bad act! This LUT is supposed to be untouched because we use it for device calibration. But I couldn't see anything like that since I calibrate my displays.

Why would spotread not be able to measure your post-calibration profile?

May be it can. But I talked about it above...


Well if even if your display doesn't have a full CMS, is it not better to calibrate as much as you can on the display-end before you use software such as yCMS to rein in the colors and/or greyscale the last little bit?

I have weird RGB Level graphs (the RGB color curves are crossing each other), so it is much better to leave the RGB Gains at 255 (in the service menu, because the user OSD doesn't let me change it...) and let the VGA LUT to correct the white point.

I currently have 10 bit/color DP connection but I think I would be able to achieve 12 bit/color through HDMI (the HDMI input's EDID sasy that 12 bit mode is supported but I can't get any image through HDMI with any display mode -> May be I bought a wrong cable or my third U2410 is erroneous as it's predecessors had various problems as well...). So, this is not a problem but a good method. I think that everybody should keep the RGB Gains at their maximum levels with DeepColor supported devices and let the VGA LUT to do it's job!
You can usually set the white point with 0-100 or may be 0-255 integer values. The VGA LUT uses values between 0-1024 with 10 bit/color, and it would be able to use 0-4096 values with 12 bit/color (HDMI 1.3 -> But I don't know what is wrong with DP now. It should be able to handle 12 bit/color as well. But this display is weird anyway, so I won't ask it if HDMI would work...).


My real problem is not the white point but the tonal response. There is a lot of preset modes with various gamuts and tonal response characteristics but none of them offers me a pure power gamma 2.2 curve. They are far away from that.
The sRGB mode is the only one which lies close enough to the gamma 2.2 tonal response. So, I can calibrate it without serious banding issues.

But I can't understand this behavior. The sRGB mode is close to gamma 2.2 and it has no banding while the other modes (with lower gamma values) have banding issues (out-of-box, before any manipulation through the VGA LUT!). This let me believe that a native mode would be very cool with software CMS (coupled with DeepColor support).

But the funny thing is that there is no native mode. There is a lot of preset modes but every one of them will be processed by weird real-time image manipulation or at least by a factory calibrated LUT (which are faulty in same way or other...).

Obviously Calman requires SOME type of controls on the display, and your computer monitor sounds like it is very limited in its built-in calibration capability, so calman probably isnt ideal for you.

Yes, it very limited. I had to use the service menu to gain a little access.
On the other hand... it has internal LUTs and I think these (or at least two of them) are 3DLUTs because the gamut emulation is very good in the sRGB preset.

So, (with the assumption that the native mode is close to my targets) if I could access those LUTs... I think I would be able to make a "perfect" display. :)

Do you watch movies with the lights on? Have they fixed the backlight bleed on PC monitors yet? Last I checked, most computer monitors couldn't measure up to the (black level) performance of a good TV or home theater projector...especially the TVs that have locally dimming LED backlights that can turn completely off in certain areas.

Yes, my PC room is perfectly dark (until I open the balcony door to let some fresh air to come in).

Yes, the black level is not perfect but I preferred the higher black level instead of the noise and black crush. I considered a Full-HD plasma display (with moderate sizes -> and cost) as a PC monitor but low black levels causes noise on dark areas with current displays.

I think we need materials with higher bitdepth and displays with more higher bit depths and good CMS systems to avoid this effect.
Or this will require a different transfer function as well. (But it should be in sync with the content mastering.)

I am not a fan of the projectors. I newer really liked them and they are much more expensive. (To be honest, I prefer to watch movies at home because I don't like the cinema projectors either. Even the IMAX projector can't make me fully satisfied. -> And the polarized 3D is a very big joke on IMAX! I hated to watch the Avatar movie there...)

So, I think that H-IPS is the best available choice now. (With good factory calibration or with the ability for hardware calibraton.) And I doubt they will consider this issue (noisy dark areas) for the next HD standard...


Here is my post-calibration report:
Black level = 0.14 cd/m^2
White level = 122.00 cd/m^2
Aprox. gamma = 2.19
Contrast ratio = 873:1

But the black level was increased by the calibration LUT. This panel would hit the 1000:1 contrast ratio in native mode (and may be with a proper hardware calibration as well.) It should be enough and I think that it is better than any S-PVA with higher contrast but with black crush and noisy dark tones as well. (And IPS has slightly better color reproduction too...)


Yes, some LG H-IPS panel is very inhomogeneous (the famous "tinting issue", but the real one. People mixed it with the homogeneous tinting which was caused by the faulty factory calibration...) but I am lucky because I couldn't see this problem personally. My replaced hardwares had other kind of issues. (And may be this one has problems with it's HDMI input. I think it will never end. Or may be I will wait until they release the U2411 and I will get one of those as a U2410 replacement. :D)

[madshi]

This let me believe that a native mode would be very cool with software CMS (coupled with DeepColor support).
What do you need DeepColor for? Thanks to dithering, yCMS/madVR introduce *zero* additional banding, even with only 8bit RGB output. The only improvement DeepColor would bring (when using madVR/yCMS) is that the dithering's white noise level would be lower.

[yesgrey]

The VGA LUT uses values between 0-1024 with 10 bit/color, and it would be able to use 0-4096 values with 12 bit/color (HDMI 1.3 -> But I don't know what is wrong with DP now. It should be able to handle 12 bit/color as well. But this display is weird anyway, so I won't ask it if HDMI would work...).

What do you need DeepColor for? Thanks to dithering, yCMS/madVR introduce *zero* additional banding, even with only 8bit RGB output. The only improvement DeepColor would bring (when using madVR/yCMS) is that the dithering's white noise level would be lower.

I agree. IMHO it probably would also be preferable the madVR/yCMS 16 bit dithered to 8 bit over the VGA LUT with 10 or 12 bit... People shouldn't look only to the number of bits, because that could be misleading.;)

[yesgrey]

yCMS v1.7 released

http://yesgrey.com/ycms.html

- New command Gamma_Curve to set the desired display calibration gamma curve.


This version allows changing the gamma curve to get after the display calibration.

I will perform today further comparisons, but from some quick tests I still prefer the standard BT.709 curve with the standard gamma value. When using a pure power function the gamut correction will be less accurate than when using the input standard function, so, for now, I still recommend sticking with curveType as 1.0. The gammaValue has less influence in the accuracy loss of the gamut correction, so you may try changing it.

Please let me know if it's working fine. If someone could measure the final gamma curve with several different settings and post the results would be great, because I cannot test it with my current setup...

[janos666]

I think I need some help with this Gamma_Curve function to be sure about the (theoretically) correct values. I can't understand how it works.
Gamma_Curve 0 2.2 darkens the image but the 0 2.35 values gives me even darker image. So, I think this exponent shouldn't be equal with my current display gamma or the function is faulty. (2.35 means a darker display, so this command should produce a brighter image for a compensation). So, I have to assume that this command tries to simulate the given values. But what was your assumption about the theoretically perfect display for this? A display with pure power gamma 2.2 or 2.35?
What exponent value should be applied for a display with pure power gamma 2.2 to simulate the theoretically (pure power response) display?

Will this gamma correction mess with the gamut or white balance when I don't use any gamut correction commands?

I you want to hear my subjective opinions:
- For the first look, the image looks obviously dark. I watched some random scenes from different movies and I asked myself: "Are there always night, clouded sky or solar eclipse in every movies I have?" and I realized that I watched some outdoor scenes with sunrise. :)
- On the other hand, the reverting of the changes caused the revert effect. :D "Where are the nice colors now? Why does the image look washed-out after I reverted all changes?"

The uncorrected image feels more natural (mostly for outdoor scenes), but the corrected image looks more like a well printed image (like HQ printings on photo paper -> but we should consider the visual illusion that darker colors always look nicer.)
And you can call it to placebo but I think I can notice the rounding errors on the very dark shades (even with the dithering enabled on madVR -> I didn't forget the question, I will answer it sometime).

[madshi]

I think I need some help with this Gamma_Curve function to be sure about the (theoretically) correct values. I can't understand how it works.
Gamma_Curve 0 2.2 darkens the image but the 0 2.35 values gives me even darker image. So, I think this exponent shouldn't be equal with my current display gamma or the function is faulty. (2.35 means a darker display, so this command should produce a brighter image for a compensation). So, I have to assume that this command tries to simulate the given values. But what was your assumption about the theoretically perfect display for this? A display with pure power gamma 2.2 or 2.35?
What exponent value should be applied for a display with pure power gamma 2.2 to simulate the theoretically (pure power response) display?
It seems that you think that you should use the Gamma_Curve command to tell yCMS how you manually calibrated your display? That is not at all what Gamma_Curve is for. yCMS already knows how your display is calibrated through the Gamma_Measurements command.

You should use the Gamma_Curve command to tell yCMS which gamma curve and gamma value you want to see on screen! Basically Gamma_Curve defines the target you want yCMS to achieve. So if you feed Gamma_Curve higher gamma values, the image gets darker...

I you want to hear my subjective opinions:
- For the first look, the image looks obviously dark. I watched some random scenes from different movies and I asked myself: "Are there always night, clouded sky or solar eclipse in every movies I have?" and I realized that I watched some outdoor scenes with sunrise. :)
In that case try decreasing the Gamma_Curve gamma value. Try 2.1 or 2.0. Or try using the BT.701 curve type, which is generally brighter, when using the same gamma value.

There is no "correct" or "incorrect" here, because there is no defined standard about which gamma curve/value your display should be calibrated to. Even the experts don't 100% agree how to calibrate your display exactly. Furthermore the ideal gamma value depends on your ambient light level, the brightness of your projector, whether your walls are white or not etc etc. So just play with the gamma curve type and gamma value in yCMS and use the setting which works best for you. Then please let us know which gamma curve and which gamma value you ended up with, and what your room conditions are...

And you can call it to placebo but I think I can notice the rounding errors
The math used by yCMS + madVR does not produce any rounding errors.

[yesgrey]

I will perform today further comparisons, but from some quick tests I still prefer the standard BT.709 curve with the standard gamma value.
Last night I watched a movie with a gammaValue of 2.35 instead of the standard 2.222. In the middle of the movie I switched back to the 2.222 value, but after some minutes I returned to the 2.35 value, the image looked slightly more contrasted. Of course the dark parts of the image are darker, but I did not detect any significant loss in shadow detail. So, for now, I will keep it at 2.35.
This was using the BT.709 curve (curveType 1.0). I will also test with a pure power function curve and then post my results.
My room is totally light controlled, and my walls and ceiling are fully oak covered, so the color would be a light brown.

Gamma_Curve 0 2.2 darkens the image but the 0 2.35 values gives me even darker image.
Try with Gamma_Curve 1.0 2.35. The pure power function gives a very dark image, so you might prefer using the BT.709 instead.

Will this gamma correction mess with the gamut or white balance when I don't use any gamut correction commands?
No. The gamut correction accuracy is affected only when is performed by yCMS.

try using the BT.701 curve type
You've a typo there...;)

[janos666]

yCMS already knows how your display is calibrated through the Gamma_Measurements command.

Well, I didn't supply any measurement data. This was the problem. Can I use the output_transfer_function command to define my display's tonal response and then refine the result with the Gamma_curve?

The math used by yCMS + madVR does not produce any rounding errors.

Do you want to tell me that there won't be any rounding errors between the source and the visualized image?
For example: Take a 8 bit gray scale image with a 8 bit display. With a correct display you will be able to see every little steps. (If you ask me, these are not so "little" steps, but this is another topic.) Can you make any effective gamma correction without loosing some information? I strongly doubt that. (You won't be able to recognize every little steps. Some of the original steps will band together. - But, of course the overall gray scale will be closer to the desired characteristics. So, it may worth it... or not...)

These (experimental) corrections will do relatively big changes (It would cause heavy banding effects without dithering).

Dithering, yes... You should know that dithering is more like an illusion than some kind of black magic. There are some side effects and some limitations as well. A smart dithering technique can improve the overall result, but still there are some rounding errors or additional noise.



Now, lets see my example:

Take a LCD panel with native 8 bit plus built-in dithering for 10 bit visualization (they call it A-FRC, I think it uses static dithering). It works with a 12 bit controller chip (and 12 bit LUTs) which also uses dithering to feed the 10 bit LCD panel (the documentation talks about "high-quality dithering" but I don't know the specifics).

So, I already have two dithering steps which I can't control. But I can feed the controller with up to 12 bit/color to reduce the rounding errors a bit (I mean the internal rounding errors through the factory LUTs and other built-in processings).

The difference between 8 and 10 bit/color (rounded from the 16 bit VGA LUT data- of course, there is no dithering here) can make hard differences when I calibrate my display. The 10 bit/color connection allows me to avoid any banding issues.

This experience made me believe that a proper DeepColor support (10+ bit/color from the frame-buffers) would be able to make significant differences when we apply relatively heavy image manipulations.


I think that madVR uses static dithering because dynamic FRC and temporal dithering cannot be too effective with 1080p24 display mode an 24 fps materials.

So, I would like to replace the dithering in madVR with DeepColor support. I have some fears about these many dithering steps together.

But may be I will change my mind if you would clarify this dithering for me. What bit depth does it try to simulate? How does it work (Is it static, temporal, dynamic, or...)?



Otherwise, I think that higher effective bit depth is always better or equal but it can never hurt (when it is properly applied).
And you can agree that higher bit depth may help in this situation (Rec709 YCC source --processings--> "some kind of display").
So, I can't understand your hard statement that "We really don't need DeepColor, we are fine with dithering."
You can still keep the dithering with DeepColor if you wish. I would disable it with DeepColor support to avid some unpredictable side effects. (With the tons of dithering steps between the source and visualization.)

[yesgrey]

Can I use the output_transfer_function command to define my display's tonal response and then refine the result with the Gamma_curve?
Yes, but the final result accuracy will depend highly of how accurately your display follows your output_transfer_function settings...;)

It's preferable to use the Gamma_Curve command when also using the Grayscale_Measurements command.

[madshi]

Do you want to tell me that there won't be any rounding errors between the source and the visualized image?
There will be no rounding errors between source and 8bit madVR output (provided the GPU doesn't stretch the data behind madVR's back).

For example: Take a 8 bit gray scale image with a 8 bit display. With a correct display you will be able to see every little steps. (If you ask me, these are not so "little" steps, but this is another topic.) Can you make any effective gamma correction without loosing some information? I strongly doubt that. (You won't be able to recognize every little steps. Some of the original steps will band together. These (experimental) corrections will do relatively big changes (It would cause heavy banding effects without dithering).
There will be no banding added on top of what the source already contains, no matter how big the yCMS changes are.

Dithering, yes... You should know that dithering is more like an illusion than some kind of black magic.
You obviously don't have a clue at all about dithering. Sorry to be so blunt. Of course it's not black magic, but it's very far from an illusion. The objective benefits of dithering can be measured very easily.

There are some side effects and some limitations as well. A smart dithering technique can improve the overall result, but still there are some rounding errors or additional noise.
There's only one limitation = side effect, which is a slightly higher noise floor.

may be I will change my mind if you would clarify this dithering for me. What bit depth does it try to simulate? How does it work (Is it static, temporal, dynamic, or...)?
TPDF dithering. "Infinite" bitdepth.

@janos666: Let me show you 2 screenshots (taken from the madVR thread):

http://madshi.net/madVR/ATI%20-%20VMR9%20-%20smallramp.png
http://madshi.net/madVR/madVR%200.4%20-%20smallramp.png

This is a 8bit (256 step) gray ramp image, scaled up to a higher resolution. If you look at the ATI image, you should see faint vertical bars in the image. These bars show you the steps between the 256 steps in the source. These bars proof that 8bit is not enough for banding-free ramps. Now look at the madVR image. The gray ramp was upscaled in floating point with a linear filter, which means that every column in the image now has a different value. There are not any longer 256 steps, but now one step for every column (= 1680 steps). That's a result of linear scaling in floating point. Do you see any steps in the picture? No, you don't. Why is that? Why don't you see any steps, although the image is stored as 8bit PNG? So the PNG only has 256 steps! That's because madVR has dithered the floating point data down to 8bit. There are no steps, only a higher noise floor. Basically the image has maintained its full floating point bitdepth. So here you have proof that madVR is able to show at least 1680 different steps without *any* visible banding. I could easily zoom that image up to 10000 steps and you would still not see any banding at all.

If that is all too hard to believe for you, simply install the madTestPatternSource filter shipping with madVR and "play" the smallramp.ytp test pattern on your display. Zoom it up to your full resolution. That should give you 1920 different gray steps, which is about 11bit true resolution. Then check if you see banding with and without yCMS calibration. If you already see banding without yCMS then either your GPU stretches the data behind madVR's back, or your display is not able to resolve 8bit. If you don't see banding with madVR without yCMS calibration, then you won't see any banding with madVR + yCMS, either, regardless of how big the yCMS changes are...

[janos666]

This is the same test pattern which I use for subjective before/after comparisons in the EIZO monitor test utility. I didn't notice that there is a pattern like this (smallramp.ytp) in the madVR test pattern directory. I will use it in the future. ;)


Thanks, this post helped me a lot to understand your method.
In my barbarian words (no offense, I know about my poor language and I am not an IT professional...), I would call it "static dithering" (because it works on still, discrete images, it is not carried across the video images). MadVR processes the full (discrete) image to add the required noise across the image (at the correct points, with a well defined method, of course, not like a "noise filter").

And yes, I call it to an illusion because you can clearly see the noise from a closer distance and the nice image is only a visual illusion which exists only from a greater distance (of course, the required distance depends on the pixel size).
This is a very obvious example for the visual illusions. This is an attribute of the human cognition system (eyes and brain).

Infinite bit depth? This is what I would call nonsense! There has to be a measurable/guessable "effective bit depth". You are still using 8 bit/color data and fixed physical resolutions. There are some existing physical limitations, even if you play with more than one pixel and more than one color channel at once.
I can accept if this effective bit depth is far over the human cognition abilities. But 1000 bit/pixel is better answer than "infinite", I think. :p
I would guess that it is equal with 16 bit/color if you applied this dithering across the full range of 8 bit integers. (They say that this is the limit of the human cognitive system. So, it is cool. :cool:)



And I am still resist that this kind of dithering can help a lot but it is not a "magical" solution for everything. There has to be some lost informations! This is how the world works, nothing is ever free! (But may be favorable...)

Lets forget about the test pattern now. You usually have a display with the same resolution of the Blu-Ray content's luminance map (and the Chroma map can be scaled well to this resolution as well as the sharpness depends mostly on the luminance map - these are of course, not accidents...).

So, after we forget about the YCC 4:2:0 trick, you have an image resolution which matches with the display resolution. You don't have any extra place to apply these dithering dots. You have to apply the noise on the whole image. So, you have to overwrite the original pixels to get the final, dithered image.
The test pattern builds it's gray bars from more than one pixels. So, there is a room for dithering. You don't have it with FullHD sources and FullHD displays...
Yes, yes, the final, overall image may looks much better through the human perception! I don't doubt that, I won't ever doubt that benefit!!! (You don't need to explain this!)
But still... The dithered image is only a nice looking illusion. It is not as per-pixel coherent, or exact from the theoretical chroma/luma and image information perspective as an original, native, highly detailed image.
And I didn't say anything else earlier. (May be I should place the "theoretically" words more often. :rolleyes:)



After all, I agree that I will have exactly the same amount of noise on dark areas with 10 bit/color output because this LCD panel uses the same dithering method.

My biggest concern here that I am unable to disable the display's built-in dithering algorithm.

Is it theoretically correct if madRV outputs 8 bit, the VGA LUT produces (modified) 12 bit, the display controller produces (modified) 10 bit (dithered back from the interpolated 12 bit) and the panel uses static dithering again?
May be it is not a problem but a bunch of cool features. As I imagine this, it can lead to a very serious mess! I won't know until I can test it with DeepColor support.


But there should be a real benefit from the DeepColor support (at least for me, right now). As I mentioned, my display works with 12 bit internal 3DLUTs (and I think this is the future -> the only problem is the lack of the support for home calibration). It would be able to produce less rounding errors when I feed it with 10+ bit input (instead of current 8 bit).
Of course, it should be tested for proofs as well (and I don't have any tools yet).


EDIT:
And some final thoughts: I think I have some banding issues with madVR in PC output mode and the smallramp.ytp pattern. :eek::eek::eek:
I didn't use this pattern earlier. It has to be the same problem which was mentioned by a forum member in the madVR thread! I will test his recommended yCMS settings.

People talked about some ATI HDMI PC/TV level problems there but it is the other one. It is an ATI card with DP port and this issue exists only with video renderers (EVR is a garbage anyway! :p So, never forget that I respect your work, no matter what I say as an "inspiration" for you to make it even better. :cool:). The is no banding with the EIZO test utility.

But he uses a projector with RGB limited and I need RGB Full for correct gray levels. So, I guess I have to find the correct values for myself.


EDIT2:
So, the issue: Some of the gray levels bands into full black with PC setting, while the TV setting gives me perfect result, expect that the blackest black is gray (lifted to RRG 16-16-16 as the zero point, as usually).
yCMS behaves exactly the same way with the default PC settings. There is a nice set of equally black bars together.

But may be it was designed to do it. May be the test pattern produces a Full range RGB gray scale and Blu-Ray sources uses limited range. So, the limited range is mapped into the full range and only one limited range value will produce the blackest black.

Is the limited range "absolutely" mapped into the full range or is it a bug which produces banding? I can't judge about that before I hear the developer's opinions.

If that is the case, I would request a smallramp_limited test pattern which behaves exactly as the Blu-Ray sources. :)

[madshi]

And yes, I call it to an illusion because you can clearly see the noise from a closer distance and the nice image is only a visual illusion which exists only from a greater distance (of course, the required distance depends on the pixel size).
This is a very obvious example for the visual illusions. This is an attribute of the human cognition system (eyes and brain).
You still don't understand the concept at all. Sure, the human cognition system helps here, but dithering is also objectively and measurably much superior to simple rounding. You need to (finally) understand that dithering is not a trick, not an illusion, but a mathematically much more correct way of reducing bitdepth of digital data.

Look, let's say you have this floating point data:

(1) 0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.6

With simple rounding you would get:

(2) 1, 1, 1, 1, 1, 1, 1, 1, 1, 1

With proper TPDF dithering applied you get something like this:

(3) 1, 0, 1, 1, 0, 0, 1, 1, 0, 1

Now do the math: Calculate the mean average of (1), (2) and (3). I hope that you will now finally understand that dithering is not an illusion at all. It produces mathematically correct results. You can measure dithered data and the measurements show that the data is virtually identical to the full bitdepth (e.g. floating point) original. The only side effect is an increased noise floor.

And I am still resist that this kind of dithering can help a lot but it is not a "magical" solution for everything. There has to be some lost informations! This is how the world works, nothing is ever free!
Dithering is not "free", it comes at the cost of a raised noise floor, as I have already explained to you several times.

So, after we forget about the YCC 4:2:0 trick, you have an image resolution which matches with the display resolution. You don't have any extra place to apply these dithering dots.
Once again you don't understand the concept of dithering *at all*. Adding extra dithering pixels is very far from how dithering works. Dithering is simply a replacement algorithm for rounding, which produces much more correct results. Simple rounding throws away heaps and tons of information. Dithering preserves the full information, but raises the noise floor.

As I imagine this, it can lead to a very serious mess!
Not really, unless the processing in your display is completely broken. But in that case you will have a serious mess, anyway, no matter what madVR/yCMS do.

This is my last post on the topic of dithering, since it's really OT here. I've explained to you how it works. Either you trust in that I know what I'm saying and doing, or not. In any case, I've also given you the instructions to double check for yourself with your display whether what I'm claiming is actually true. So either try it out, or don't.

[janos666]

I tried to test it (read the EDIT parts) and I tried to explain that I believe in the fact that the final, overall, perceived, virtual result is much better.
I think the misunderstanding is only belongs to the question that what is a "trick", "true math", or "illusion" in our sentences and dictionaries.

We are just talking next to each other, so yes, we should stop it. This is my last post in this topic and I don't wait for more (repeated) answers.
(But you are welcome to answer the EDIT sentences.)

I call it as a trick because every dithering method is based on cognitive illusions.
What is wrong with this statement and why do you feel it as an insult against the scientific facts?
It is not a mistake and it won't make it look "cheaper" or "worse". These are the existing scientific fats.
The nicely placed illusions and tricks are as much scientific as anything else which was engineered with a purpose. These are smart tricks and illusions, developed with a scientific purpose. But they are what they are: tricks and illusions.

Every single display is a big illusion generator! There are no true, living people and worlds built in there. This is not a warm-hole which shows other universes. This is only an illusion!

Period, end, closed (for me now).



In your example: 1, 0, 1, 1, 0, 0, 1, 1, 0, 1 instead of 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 would require dynamic changes (FRC, if you wish). The different variations of the same static video image (0 and 1) have to be displayed 4 and 6 times (in this example, but, of course, less repeats are enough), so the cognitive system can average it to one nicely perceived static image.
But you can show every video frame only in one time (with 24 fps materials and 1080p24 display mode).

For static images, when you need 0.6 for a discrete pixel but you have only 0 and 1 available, you can change the neighboring pixels to produce an overall, dithered static image which simulates higher bit dept.
Of course, the final, perceived image looks perfectly nice but in my dictionary, it is a mathematical trick which utilizes visual illusions. This is not "the image", it is some dithered data which looks like "the image" (more like the simply rounded data).

So, I thought you had to do one of these:
- You changed the neighboring pixels as well on a static image (for a nice overall image).
- You used dynamic FRC which won't work with 1080p24 display mode and moving 24 fps videos. (There is no way to show two different static images which can be averaged as one.)

Or yes. I can't understand the true nature of this dithering. But I can understand three kind of existing dithering methods. If you method is a unique method, well, I think I won't understand it easily and it would take too much effort. So, let it be uncleared then. :o

[Fer]

Thank you very much yesgrey for your great work.

Let me show you some results. My display is a Sony VW60 and the calibration is made with the HCFR software and a Eye-One meter.

First I show the Gamma and the CIE Diagram of the display:

http://a.imageshack.us/img571/2184/displaygammar.th.jpg (http://img571.imageshack.us/i/displaygammar.jpg/)

The Gamma curve is obtained as a "Display gamma" in HCFR.

http://a.imageshack.us/img821/2104/displaygamut.th.jpg (http://img821.imageshack.us/i/displaygamut.jpg/)

It can be seen that the gamut is far away from the BT.709 standard, typical in the Sony projectors with oversaturated colors.


From this measurements, I have applied yCMS with this configuration:


Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Gamut_Measurements 0.694 0.303 0.301 0.691 0.140 0.053 0.314 0.329

Grayscale_Measurements
10 0.216
20 0.850
30 1.975
40 3.411
50 5.259
60 7.605
70 10.355
80 13.643
90 17.414
100 21.713

Gamma_Curve 0.0 2.20


The results are:

http://a.imageshack.us/img291/5199/gammatf0g220.th.jpg (http://img291.imageshack.us/i/gammatf0g220.jpg/)

The Gamma curve is obtained as a "Display Gamma" in HCFR. The Gamma_Curve make a great job since the measured gamma is very near to 2.20.

http://a.imageshack.us/img821/5986/gamuttf0g220.th.jpg (http://img821.imageshack.us/i/gamuttf0g220.jpg/)

It can be seen that the gamut correction is good but not perfect yet.


Now we change to a BT.609 transfer function in the Gamma_Curve command in the last line of the configuration file


Gamma_Curve 1.0 2.35


and the results are:

http://a.imageshack.us/img717/1483/gammatf1g235.th.jpg (http://img717.imageshack.us/i/gammatf1g235.jpg/)

The Gamma curve is obtained as a "Camera Gamma" in HCFR. One more time, the Gamma_Curve make a great job since the measured gamma is very near to 2.35 (with the exception of 20 IRE and below).

http://a.imageshack.us/img821/8149/gamuttf1g2350.th.jpg (http://img821.imageshack.us/i/gamuttf1g2350.jpg/)

In this case the gamut correction is clearly better compared with the case of a pure power transfer function. It is better both in the x, y coordinates and the Y value. The delta E is below 5 for all the primaries and secondaries. Nevertheles the Y ratios Red/White = 0.197, Green/White = 0.701 and Blue/White = 0.070 are lower than the values of the BT.609 standard Red/White = 0.21, Green/White = 0.71 and Blue/White = 0.08. It would be brilliant if yCMS could manage also the Y value as FoLLgoT say before, since the colors seems something dull due to the low Y values, although my brain color perception has been severe altered in years of oversaturated colors...

So, it seems that with the pure power transfer function the gamut correction could be better.

My card is an ATI 2600XT, so it is difficult to me evaluate the image quality with both transfer functions (I have severe stuttering with 3dlut enabled). Nevertheless it seems to me that both gammas gives a very good image and is a very difficult decision say which is better. The next week will change to an ATI 4850 card and I will perform further comparisons with smooth playback (I hope so).


:thanks::thanks::thanks:

[yesgrey]

Let me show you some results.
Thank you very much for the detailed test and report.:)
If you could also upload me the HCFR files I would appreciate it, because it would let me analyze the results with a little more detail. If you prefer you could send them by e-mail (it's in the manual).

Gamma_Curve 0.0 2.20
It can be seen that the gamut correction is good but not perfect yet.
The accuracy of the gamut correction is highly dependent on gamma. So the highest accuracy gamut correction would be without Gamma_Curve, or using the 1.0 curveType with a gammaValue not much different from the standard.

For achieving a higher gamut correction accuracy when using a pure power function you would need to use it as the input transfer function, and then not use the Gamma_Curve command. This would give you higher accuracy when using the test patterns, but if the videos were created using the bt.709 transfer function you would have lower accuracy when watching them, even though the test patterns would look perfect.

Perfect test patterns measurements only guaranty the same accuracy when watching the videos WHEN the test patterns are mastered exactly the same way the videos are... ;)


Gamma_Curve 0.0 2.20
Gamma_Curve 1.0 2.35

When comparing different gamma curves you should keep one of the values, or the comparison would be much harder...
First you should decide which curveType you prefer, and only after that you should try different gammaValues.

It would be brilliant if yCMS could manage also the Y value as FoLLgoT say before, since the colors seems something dull due to the low Y values, although my brain color perception has been severe altered in years of oversaturated colors...
It's on my todo list, but I don't think the colors might seem dull due to that...

Nevertheless it seems to me that both gammas gives a very good image and is a very difficult decision say which is better. The next week will change to an ATI 4850 card and I will perform further comparisons with smooth playback (I hope so).
According to some people the mastering is performed with robustness in mind, to make the image look good within a certain range of gamma. So, unless you are using a very wrong gamma curve, you should get a pleasant result with several settings. You only need to decide which you will prefer, but it would not be that easy. Furthermore, it would also depend on your display, because a display with a higher contrast ratio will work better with higher gammaValues than a projector with a lower contrast ratio, so this will always be a personal setting.
And yes, you should judge the gamma curves when watching the movies. Screenshots are not a good way of doing it...

[madshi]

When comparing different gamma curves you should keep one of the values, or the comparison would be much harder...
First you should decide which curveType you prefer, and only after that you should try different gammaValues.
The problem is that when you only change the curveType, the image automatically gets brighter/darker, because the BT.709 curveType produces a brighter image than the pure power curve (or am I wrong?). So if people want to know which curveType they prefer, it might make sense for them to use a different gamma value for both curveTypes which produces a somewhat similar looking result. What do you think?

[yesgrey]

So if people want to know which curveType they prefer, it might make sense for them to use a different gamma value for both curveTypes which produces a somewhat similar looking result.
Yes, you're right, I forgot that the difference between both curve types with same gammaValues is too big, so the comparison would be meaningless...

Maybe a better approach would be something like this:
(1) set curveType to 0.0 and try with several different gammaValues and decide which you prefer.
(2) set curveType to 1.0 and try with several different gammaValues and decide which you prefer.
(3) Compare then the two preferred curves (from (1) and (2)) and decide which is the number one.

I know that in (3) we could end up with two curves with very different brightness levels, but on the other hand we would be comparing the top 2.

It's just my opinion, though, not a scientific method.;)

[madshi]

Just out of interest: From a mathematical point of view, which gamma value would one have to choose for curveType 1.0 to get a somewhat similar picture to curveType 0.0 (or vice versa)? There's probably a mathematical relationship?

[yesgrey]

From a mathematical point of view, which gamma value would one have to choose for curveType 1.0 to get a somewhat similar picture to curveType 0.0 (or vice versa)?
There isn't a single gamma value for that. To each IRE will correspond a different gamma value. Remember the previous gamma graphs from HCFR when not using the "camera gamma" setting? Though, it might be a good start if people calculate the gamma value for 75 IRE, or any other value.
I can post the formula for that, if it would be helpful...

[janos666]

You should change your example in the manual.txt. The empty lines counts as an IRE. I ended up with this input file:
# Source video format
Input_Format HD YCbCr 8

#3DLUT output format
Output_Format HD RGB_PC 16

Gamut_Measurements 0.677588 0.312901 0.201814 0.696344 0.151118 0.056602 0.312457 0.330534

Grayscale_Measurements 0 0.13498
0.39216 0.14294
0.78431 0.15466
1.1765 0.16328
1.5686 0.17958
1.9608 0.19381
2.3529 0.21409
2.7451 0.22772
3.1373 0.25247
3.5294 0.27298
3.9216 0.29483
4.3137 0.3216
4.7059 0.34783
5.098 0.38904
5.4902 0.41443
5.8824 0.4547
6.2745 0.48645
6.6667 0.52568
7.0588 0.57565
7.451 0.61425
7.8431 0.66877
8.2353 0.71852
8.6275 0.78184
9.0196 0.82864
9.4118 0.89781
9.8039 0.95886
10.196 1.0197
10.588 1.0952
10.98 1.1573
11.373 1.2497
11.765 1.3216
12.157 1.4103
12.549 1.485
12.941 1.5677
13.333 1.6781
13.725 1.7567
14.118 1.8674
14.51 1.9613
14.902 2.0875
15.294 2.1794
15.686 2.3047
16.078 2.4104
16.471 2.5222
16.863 2.6529
17.255 2.7607
17.647 2.9162
18.039 3.0388
18.431 3.1884
18.824 3.3103
19.216 3.4353
19.608 3.615
20 3.742
20.392 3.9147
20.784 4.0513
21.176 4.2475
21.569 4.3872
21.961 4.573
22.353 4.7259
22.745 4.8955
23.137 5.0878
23.529 5.2468
23.922 5.4634
24.314 5.6551
24.706 5.873
25.098 6.0997
25.49 6.2808
25.882 6.5325
26.275 6.7112
26.667 6.9573
27.059 7.1521
27.451 7.3695
27.843 7.6116
28.235 7.8147
28.627 8.0818
29.02 8.3067
29.412 8.575
29.804 8.7809
30.196 9.0705
30.588 9.3161
30.98 9.5282
31.373 9.8228
31.765 10.039
32.157 10.37
32.549 10.589
32.941 10.893
33.333 11.128
33.725 11.401
34.118 11.727
34.51 11.949
34.902 12.294
35.294 12.56
35.686 12.889
36.078 13.14
36.471 13.475
36.863 13.757
37.255 14.016
37.647 14.349
38.039 14.658
38.431 15.038
38.824 15.312
39.216 15.654
39.608 15.937
40 16.211
40.392 16.61
40.784 16.854
41.176 17.252
41.569 17.549
41.961 17.928
42.353 18.218
42.745 18.586
43.137 18.907
43.529 19.191
43.922 19.573
44.314 19.856
44.706 20.267
45.098 20.578
45.49 20.97
45.882 21.24
46.275 21.559
46.667 22.004
47.059 22.281
47.451 22.685
47.843 22.989
48.235 23.435
48.627 23.725
49.02 24.241
49.412 24.608
49.804 25.008
50.196 25.632
50.588 26.022
50.98 26.552
51.373 26.957
51.765 27.321
52.157 27.806
52.549 28.216
52.941 28.744
53.333 29.126
53.725 29.65
54.118 30.043
54.51 30.584
54.902 30.984
55.294 31.363
55.686 31.896
56.078 32.34
56.471 32.854
56.863 33.263
57.255 33.789
57.647 34.211
58.039 34.613
58.431 35.121
58.824 35.54
59.216 36.113
59.608 36.508
60 37.031
60.392 37.459
60.784 38.035
61.176 38.455
61.569 38.836
61.961 39.444
62.353 39.844
62.745 40.428
63.137 40.814
63.529 41.401
63.922 41.842
64.314 42.298
64.706 42.806
65.098 43.266
65.49 43.878
65.882 44.335
66.275 44.892
66.667 45.287
67.059 45.996
67.451 46.384
67.843 46.846
68.235 47.433
68.627 47.933
69.02 48.52
69.412 48.989
69.804 49.584
70.196 50.095
70.588 50.65
70.98 51.203
71.373 51.7
71.765 52.383
72.157 52.891
72.549 53.5
72.941 54.052
73.333 54.697
73.725 55.273
74.118 55.803
74.51 56.403
74.902 56.932
75.294 57.763
75.686 58.326
76.078 58.82
76.471 59.531
76.863 60.151
77.255 60.792
77.647 61.266
78.039 62.094
78.431 62.652
78.824 63.312
79.216 63.887
79.608 64.527
80 65.291
80.392 65.792
80.784 66.584
81.176 67.275
81.569 68.002
81.961 68.594
82.353 69.232
82.745 70.009
83.137 70.711
83.529 71.479
83.922 72.027
84.314 72.926
84.706 73.589
85.098 74.395
85.49 74.987
85.882 75.642
86.275 76.489
86.667 77.138
87.059 77.841
87.451 78.561
87.843 79.339
88.235 79.917
88.627 80.513
89.02 81.345
89.412 82.03
89.804 82.769
90.196 83.309
90.588 84.128
90.98 84.84
91.373 85.466
91.765 86.023
92.157 86.738
92.549 87.3
92.941 87.878
93.333 88.506
93.725 89.114
94.118 89.884
94.51 90.355
94.902 90.722
95.294 91.549
95.686 92.157
96.078 92.938
96.471 93.281
96.863 94.081
97.255 94.848
97.647 96.037
98.039 96.311
98.431 97.104
98.824 98.158
99.216 98.983
99.608 99.906
100 100

Gamma_Curve 0 2.2

Another funny thing that you will have homogeneous black or white screen when you forget to replace the , with . decimal marks. (I would expect an error message or may be a smarter behavior.) :)

Otherwise, I think I will keep away from this state. I have BANDING problems (yes, I lost some visual informations) with these settings.
G.I. Joe and Pocahontas won't lie: 01 OFF (http://img12.tar.hu/janos666/img/82410398.png) - 01 ON (http://img12.tar.hu/janos666/img/82410397.png) ; 02 OFF (http://img12.tar.hu/janos666/img/82410395.png) - 02 ON (http://img12.tar.hu/janos666/img/82410396.png) ; 03 OFF (http://img12.tar.hu/janos666/img/82410623.png) - 03 ON (http://img12.tar.hu/janos666/img/82410624.png)
And a little extra, just for you (I found it right now). Marine Hero is a shapeshifter reptilian (http://www.google.hu/search?client=opera&rls=en&q=shapeshifter+reptilian&sourceid=opera&ie=utf-8&oe=utf-8), not an real Avatar (http://img12.tar.hu/janos666/img/82411521.png). :p

[yesgrey]

You should change your example in the manual.txt. The empty lines counts as an IRE.

Another funny thing that you will have homogeneous black or white screen when you forget to replace the , with . decimal marks.
I will take care of it. Thanks for reporting.

I have BANDING problems (yes, I lost some visual informations) with these settings.
Where is the banding? None of the pictures show it... It seems you're calling banding to shadow detail loss, but that's not what banding is.

The losing of visual information is a result of your gamma curve changing. yCMS only is doing what you're asking.;)
A pure gamma curve with a gamma value of 2.2 is darker than a bt.709 gamma curve with the same gamma value (what you get when disabling 3DLUTs), hence the darkened image you get that even make you lose shadow detail... Try with "Gamma_Curve 0.0 1.96", that should give you a brightness similar to without 3DLUT. However, I would suggest you to also try with "Gamma_Curve 1.0 2.35", what I'm currently using.

[janos666]

Where is the banding? None of the pictures show it... It seems you're calling banding to shadow detail loss, but that's not what banding is.

Yes, may be I used this word in wrong syntaxes.
I mean, some of the different R=G=B=0..256 values will produce the same luminance level. The image will loose some visual informations. Two different IREs will produce the same perceived luminance. (Dark shades will be equally black, ect.)

The losing of visual information is a result of your gamma curve changing. yCMS only is doing what you're asking.;)

This is what I talked about earlier. (The weird off-topic conversation with madshi.) The dithering won't solve everything when you try to do some heavy image manipulations like this.
This is where I feel the need for 10+ bit output (henceforward combined with dithering).

I made an illustrative diagram from my measurements which I took from the Custom display mode (Unfortunately, I think it is not a "native" mode either. But this one works with the widest gamut. But it has a slightly lower contrast ratio and weird tonal response....) The green curve is the pure power gamma 2.2 with black offset: Click (http://img12.tar.hu/janos666/img/82447648.png)
The near-white scale is strange, yes. It can be a display fault or a measurement error as well. (There are some really smooth sections, and the spectrophotometers like the brighter shades, so I think it is an exaggeratedly looking display fault.)
I didn't hope too much. I was only curious what madVR would produce with a 3DLUT which is created for this display mode.
And we didn't apply a proper white balance correction yet! It will be able to cause even more detail losses.

I thought I will have more dithering noise on dark areas instead of homogeneous black swatches. So, I think this rounding error exists on the 3DLUTs because madVR won't create dithering noise to mask the homogeneous black swatches. So I won't have a chance with DeepColor either.

May be I will play with the sRGB mode next time. That one has a much more neutral tonal response curve around the pure power 2.2 (bit still not a real gamma 2.2 without a high precision calibration).
But this experiment with the above mentioned circumstances let me believe that a 10+ VGA LUT calibration will produce better results. The 3DLUT processed data is dithered back to 8 bit and it won't do a real white balance correction with the current yCMS version.

[madshi]

Yes, may be I used this word in wrong syntaxes.
"Banding" is if you see visible steps in a color/brightness gradiant.

I mean, some of the different R=G=B=0..256 values will produce the same luminance level.
No, they won't. Depending on the gamma curve/value you choose, they may end up with *nearly* the same luminance level, but not with the same.

This is where I feel the need for 10+ bit output (henceforward combined with dithering).
Depending on the gamma curve/value you choose, you may also end up with nearly the same luminance level when using 10+ bit output. The bitdepth has nothing to do with that. You still don't understand dithering...

I thought I will have more dithering noise on dark areas instead of homogeneous black swatches. So, I think this rounding error exists on the 3DLUTs because madVR won't create dithering noise to mask the homogeneous black swatches. So I won't have a chance with DeepColor either.
Those black swatches are a consequence of the source material combined with the gamma curve type/value you've chosen. You would see the very same effect with infinite bitdepth, too.

What you are seeing is crushed shadow detail. You need to change the gamma curve type/value to fix that. Throwing more bitdepth at it won't change a thing.

[yesgrey]

And we didn't apply a proper white balance correction yet! It will be able to cause even more detail losses.
You need to understand one thing: the only way of not losing any details contained in the source is by using a perfect display with perfect viewing conditions, and the closest to these conditions are the mastering studio display/room. Out of this, anyone will start losing some detail.

When correcting the display's flaws some detail loss is unavoidable in certain situations, like gamut correction, specially when the display gamut does not contain the entire desired gamut, but that's the price to pay for more accurate colors. In other situations, like gamma correction, we might be able to increase the detail when compared to no correction.

The ultimate goal of yCMS is to perform all its corrections with the minimum detail loss, but in some cases that's simple not possible, and it has nothing to do with bit depths, but with displays limitations.

[Fer]

I show you more measurements to complete that of the previous post.

The Gamma curve is shown as a "Camera Gamma" in HCFR for transfer function 1.0 (fit to a BT.609 function).

Gamma_Curve 1.0 2.20
http://a.imageshack.us/img829/9359/gammatf1g220.th.jpg (http://img829.imageshack.us/i/gammatf1g220.jpg/)
http://a.imageshack.us/img832/9021/gamuttf1g220.th.jpg (http://img832.imageshack.us/i/gamuttf1g220.jpg/)

Gamma_Curve 1.0 2.45
http://a.imageshack.us/img822/757/gammatf1g245.th.jpg (http://img822.imageshack.us/i/gammatf1g245.jpg/)
http://a.imageshack.us/img820/2714/gamuttf1g245.th.jpg (http://img820.imageshack.us/i/gamuttf1g245.jpg/)

Gamma_Curve 1.0 2.55
http://a.imageshack.us/img411/642/gammatf1g255.th.jpg (http://img411.imageshack.us/i/gammatf1g255.jpg/)
http://a.imageshack.us/img821/5986/gamuttf0g220.th.jpg (http://img821.imageshack.us/i/gamuttf0g220.jpg/)

If we fit this last result to a pure power transfer function ("Display gamma" option) we have

http://a.imageshack.us/img822/9196/gammatf1g255pp.th.jpg (http://img822.imageshack.us/i/gammatf1g255pp.jpg/)

so, the standard BT.609 with a gamma 2.55 is "similar" (average gamma) to a pure power function with a gamma of 2.20. It must be said that
this configuration (Gamma_Curve 0.0 2.20) is the default target that must be reached after calibration according to HCFR software
http://www.curtpalme.com/forum/viewtopic.php?t=10457
In fact, now that I have smooth playback with my new ATI4850 card, although I need more time to test watching movies, this is an option that like very much:
daylight scenes with great contrast and depth and dark scenes good enough detailled.

One more time it can be seen that Gamma_Curve make a great job since the measured average gamma is very near to the value of the Gamma_Curve command.
Also we see that the gamut correction is near perfect for gamma=2.20 (deltaE < 2.8 for all the colors) and that slowly deteriorate for higher gammas.
yesgrey, can you explain this fact?

For achieving a higher gamut correction accuracy when using a pure power function you would need to use it as the input transfer function, and then not use the
Gamma_Curve command. This would give you higher accuracy when using the test patterns, but if the videos were created using the bt.709 transfer function you
would have lower accuracy when watching them, even though the test patterns would look perfect.

Perfect test patterns measurements only guaranty the same accuracy when watching the videos WHEN the test patterns are mastered exactly the same way the videos are...

But we do not have the certainty which of both transfer functions is used in video mastering, so it would be interesting to adjust the gamut whatever will be the transfer
function and the gamma value choosen for your particular display and ambient conditions?
:thanks:

[janos666]

Funny. If I want to oversimplify the last two posts: One of you speaks about the fact that some detail loss is inevitable, while the other one keeps telling me that the detail loss is impossible.

@yesgrey
Yes, I know it very well. I have a given display with it's given characteristics and there is no way to perfectly reproduce any images expect those which I create on this display. And all the published contents were made with other displays (and those were not absolutely perfect either) in consideration of an assumed end-user display. Nothing will be ever absolutely perfect on Earth. Even the Earth itself is not a perfect orb, so I don't have this illusion about "the perfection".
I have some available shades with their luma and chroma values to play with them. But my display has higher bit depth than usual source materials. It leaves me a little playground (when I can use 10+ bit output) and there is the dithering to make this game more interesting.

But I think you agree that we seek the optimal state between the theoretical perfection and the practically achievable states.

So, consider this when I talk about errors and side effects. I am not talking about the literal non-perfection but non-optimal states when the sacrifice is bigger than the benefit. These are the avoidable states. And of course, these are only my personal opinions.
For example, in this situation I felt that black swatches are worse than any benefit which is achievable by any gamma correction, even when the native tonal response is very weird. Of course, this was only one test with the given softwares and one material, and one given display, ect...


@madshi
I thought we agreed that we won't push this hypothetical dispute.

There is no any infinite thing in our existence. You can't build any infinite thing from limited resources. And an usual consumer display works with very limited number of available shades and pixels.
May be you can say that a human eye or a human made instrument won't be able to indicate the difference between the dithered 8 and dithered 16 bit or between the dithered 8 bit and native 10000 bit, ect. I don't have the instruments to do it but I think the quality of your dithering can be defined by an effective bit depth value. And I think this characteristic value is not a terrifyingly huge number.
You can say that it creates the illusion of the infinite bit dept if you resist to use this "infinite" word. But you refused the "illusion" word. So, I can't find any solution which acceptable for both of us.
Let's leave it alone.


I am not sure about these black swatches. I chose this movie because it has the best quality. (Yes, the Avatar was a "Horrible IMAX3D Experience" with the polarized lens, but this Blu-Ray material is the best available Blu-Ray material now.)
If the current processing will offer me the illusion of the perfect image, then Cameron thought that I should see black swatches in his shiny 3D movie. I doubt that. I saw it on an IMAX screen, so I am sure he wanted to show me the nicely rendered hairs and backgrounds.

But here is my explanation:
MadVR worked very well with nice dithering but it used 3DLUTs with 16 bit integer values. yCMS had to do some rounding! And madVR had to render homogeneous black swatches because the 3DLUT had equal values for those shades. MadVR won't render nice hair when the 3DLUT tells him that those shades should be equally black, right!? If it does render something else, then we are in a big trouble or we found the missing part around this "dithering issue".
Or... may be... The 3DLUT contains some rounding errors AND the dithering is not magically infinite, so the combination of these things created a perceived black swath on the screen.
Or... we are totally wrong about the ideal end-user tonal response.



What do you think about the double precision shader processing (it should work with any DX11 cards) and huge 3DLUTs? It won't harm if you start to use that unrar.dll (but I would suggest a 7zip.dll :D) and you make it as an optional mode.
But I doubt it will help. May be it will help if I try to think with your head in the dithering question. But it won't help if I resist that we need to increase the native output bit depth first (even if it can't happen any time soon -> so there is no solution yet).

Otherwise, I tried to set my desktop to 6 bit mode and the quality loss was evident without the dithering but I am sure it is noticeable with dithering as well. So, I think it would be true for 10+ bit output as well...


I think I really said some mad things this time. So, don't hesitate to teach me. :D

[6233638]

Just scanned over this topic and it looks like theres some confusion over gamma.

A flat 2.35 gamma is the correct way to be viewing film sources. Not an something that averages 2.35, but a flat 2.35 power function. You cannot use black level compensation either.
This requires a display with a high native contrast ratio - most modern displays just don't cut it. CRT, D-ILA projectors, the Kuros and possibly the latest Panasonic PDPs are really the only displays capable of this, and you need to be viewing in a dark room.

On any other display type black levels will be crushed and the image will look too dark/murky. Most other flat panels are not even capable of a flat 2.22 gamma down to black without looking bad. I have a Panasonic LCD here and it can't even display 2.1 properly it's so low contrast.

An inverse BT.709 transfer is wrong, even an inverse BT.709 modified for a higher system gamma. It might be the best compromise on low contrast displays like LCD though.

[Fer]

Otherwise, I think I will keep away from this state. I have BANDING problems (yes, I lost some visual informations) with these settings.
G.I. Joe and Pocahontas won't lie: 01 OFF (http://img12.tar.hu/janos666/img/82410398.png) - 01 ON (http://img12.tar.hu/janos666/img/82410397.png) ; 02 OFF (http://img12.tar.hu/janos666/img/82410395.png) - 02 ON (http://img12.tar.hu/janos666/img/82410396.png) ; 03 OFF (http://img12.tar.hu/janos666/img/82410623.png) - 03 ON (http://img12.tar.hu/janos666/img/82410624.png)
And a little extra, just for you (I found it right now). Marine Hero is a shapeshifter reptilian (http://www.google.hu/search?client=opera&rls=en&q=shapeshifter+reptilian&sourceid=opera&ie=utf-8&oe=utf-8), not an real Avatar (http://img12.tar.hu/janos666/img/82411521.png). :p

I don't have your problem. With Gamma_Curve 0.0 2.20. I don't lose shadow detail:

http://a.imageshack.us/img835/4521/avatarft0g230.th.jpg (http://img835.imageshack.us/i/avatarft0g230.jpg/)

neither with the "similar" Gamma_Curve 1.0 2.55

http://a.imageshack.us/img835/5049/avatarft1g255.th.jpg (http://img835.imageshack.us/i/avatarft1g255.jpg/)

[janos666]

Did you check this (http://img12.tar.hu/janos666/img/82447648.png) graph? I used a preset mode with very strange tonal response because I wanted to test a situation with big changes. Of course, it won't harm when your display is close to the desired values.
But you can check it with my settings (copy-paste the full code, it has a lot of lines :D) to see it in action.

@yesgrey: Do you use black offset for this gamma correction? Or do you scale the available values into the ideal curve (which starts from zero luminance)?

And may be it was a bad idea to include the measures from all IREs. Spectrophotometers don't like the dark, but I used the adaptive mode with long measurement times and I made this graph to check if it looks erroneous, but it's not. My colorimeter shows similar level graphs from the near black shades. (It miss the WP only.) And I wanted the best approximation.

[madshi]

@madshi
I thought we agreed that we won't push this hypothetical dispute.
Well, I thought so, too. But you keep on posting technical nonsense, so I feel like having to correct you.

So, I can't find any solution which acceptable for both of us.
The funny thing is that you admit yourself that you don't fully understand dithering. But still you insist on having an opinion about it. That's a typical concept for disaster...

MadVR worked very well with nice dithering but it used 3DLUTs with 16 bit integer values. yCMS had to do some rounding!
Ok, so there is some rounding involved, but at 16bit integer, which does not have twice the number of steps available compared to 8bit, but actually 256 x the number of steps. 8bit integer has 256 steps. 16bit integer has 65536 steps. The steps offered by a 16bit 3dlut are so small that your eyes will not be able to see a difference between 2 neighbor steps. Which means that the rounding at 16bit integer level cannot possibly harm perceived image quality, unless you're Superman. I've read that humans can see differences up until about 11-12 bits. So the 16bit 3dlut still has 16x - 32x more output steps than your eyes can resolve. As both yesgrey and I told you, the problems with the image quality you're seeing are caused by the gamma curve/value/correction, and not by technical limitations like bitdepth issues etc...

BTW, posting screenshots that were made with a correction 3dlut does not make much sense, because the only way to watch these screenshots correctly is to watch them on the display they were rendered for!!! So it's impossible for us to judge how that screenshot would look like on your display.

Just as an example: If your home cinema display had a heavily pink cast, yCMS would have to heavily color correct the image. As a result, if you took a screenshot of such a heavily color corrected image and displayed it on a different display, which doesn't have that pink cast, the screenshot would look very wrong. The same applies to screenshots that were taken with a 3dlut which corrects gamma for your specific display.

[yesgrey]

I show you more measurements to complete that of the previous post.
Thanks for your hard work. It shows that Gamma_Curve is working almost as it should, only the >80 IRE values drift a little, but maybe that's due to you being using such a coarse set of measurements... could you try with 4 or 5 IRE sized steps above 80 IRE?

It must be said that this configuration (Gamma_Curve 0.0 2.20) is the default target that must be reached after calibration according to HCFR software
If you've followed our recent discussion in this thread about it, that's not true. No one can say that we should target a specific curve type or gamma value, because it would depend on several factors...

this is an option that like very much:
daylight scenes with great contrast and depth and dark scenes good enough detailled.
That's the only way of knowing for sure: looking at it. Just watch some movies and choose which looks more natural to you. There is no 100% rule, just start from some of the suggestions and fine-tune to your personal set and taste.

Also we see that the gamut correction is near perfect for gamma=2.20 (deltaE < 2.8 for all the colors) and that slowly deteriorate for higher gammas.
yesgrey, can you explain this fact?
Let me try...
The gamut correction must be performed on linear light. For that, we need to remove the gamma encoding. After correcting the gamut, the colors are all where they should be, but then we need to reapply the gamma encoding to feed it in our displays. If the encoding gamma is the exact inverse of the display response, we will get the same linear light as before, hence the exact colors. Now, this has a side effect: by using the exact inverse of the display, we nullify it, and then the display response is changed to be like the transfer function used when removing the gamma encoding.
Now, comes another problem... for image processing in linear light, the correct way of removing the gamma is by using the standard transfer function, as yCMS does by default, but for watching it's not, because our watching experience (our eyes) is influenced by the surrounding ambient conditions. So, in this case, I don't think there is much that could be done, except using a compromise between both... The good on all this is that the quality of the viewing is good within a certain range of gamma values, so we don't need to lose too much color accuracy.

As I've told in the other post, we could "cheat" the test patterns by using a different source transfer function and then get perfect gamut test results, but in reality the color gamut would still be wrong, because then the colors would not be corrected in linear light as it should, though in something very close... ;)

But we do not have the certainty which of both transfer functions is used in video mastering, so it would be interesting to adjust the gamut whatever will be the transfer function and the gamma value choosen for your particular display and ambient conditions?
Yes, you could do that.
If you want, you could try using:
Input_Transfer_Function 1.0 0.0 0.454545 0.0
instead of
Gamma_Curve 0.0 2.0

You should get the same gamma response, but the gamut correction might not be as accurate, even though with this you should get an exact color gamut with the test patterns... ;)

[yesgrey]

If I want to oversimplify the last two posts
That's the problem of oversimplifications: you lose completely the sense of each post. The detail loss I referred is not the same detail loss madshi referred, and that's not a subtle difference... ;)

If you really want to understand dithering, look at this thread (http://www.avsforum.com/avs-vb/showthread.php?t=1011359). See posts #9 to #13. It has some great images that will show you what is banding, and how dithering works. You will also understand what madshi meant with the "infinite bit depth". It shows a 10 bit image down converted to 2 bit, and the dithered image looks very near to the original one, except with lots of noise, but that's the downside of using dithering. However, don't worry, because dithering from 16 bit to 8 bit gives much less noise than the showed case... ;)

For example, in this situation I felt that black swatches are worse than any benefit which is achievable by any gamma correction, even when the native tonal response is very weird. Of course, this was only one test with the given softwares and one material, and one given display, ect...
As I've said, try with other curve types and with other gamma values.

[yesgrey]

I used a preset mode with very strange tonal response because I wanted to test a situation with big changes. Of course, it won't harm when your display is close to the desired values.
As I've told you before, the "harm" was caused by the gamma curve you selected, not by yCMS.

@yesgrey: Do you use black offset for this gamma correction?
yes, but only when the 0 IRE value is set.

And may be it was a bad idea to include the measures from all IREs. Spectrophotometers don't like the dark, but I used the adaptive mode with long measurement times and I made this graph to check if it looks erroneous, but it's not. My colorimeter shows similar level graphs from the near black shades. (It miss the WP only.) And I wanted the best approximation.
If your meter is accurate enough you should use it.

[yesgrey]

BTW, posting screenshots that were made with a correction 3dlut does not make much sense, because the only way to watch these screenshots correctly is to watch them on the display they were rendered for!!! So it's impossible for us to judge how that screenshot would look like on your display.
Furthermore, even on the display they were rendered for the brightness and contrast should be set correctly.

For example: the bug that caused the corruption of the blue at very low levels (recently fixed) was barely noticeable if the brightness level was set correctly...

[yesgrey]

A flat 2.35 gamma is the correct way to be viewing film sources. Not an something that averages 2.35, but a flat 2.35 power function. You cannot use black level compensation either.
Where is the source for this statement?

[janos666]

BTW, posting screenshots that were made with a correction 3dlut does not make much sense, because the only way to watch these screenshots correctly is to watch them on the display they were rendered for!!! So it's impossible for us to judge how that screenshot would look like on your display.

Just as an example: If your home cinema display had a heavily pink cast, yCMS would have to heavily color correct the image. As a result, if you took a screenshot of such a heavily color corrected image and displayed it on a different display, which doesn't have that pink cast, the screenshot would look very wrong. The same applies to screenshots that were taken with a 3dlut which corrects gamma for your specific display.

Yes, but those screenshots are not absolutely useless.
The colors may look differently on every displays, but the identical shades on the images should be identical on the given screen. The loss of the visual informations (like the shadow detail vs black swatches) should be noticeable with any kind of proper 8 bit displays (until your display produces black crush, ect).

I already checked it before I posted them. I made the corrections for the Custom display mode (wide gamut, strange tonal response) characteristics. After that, I checked both of the images (corrected and uncorrected) in Custom and sRGB (relatively nice factory-calibrated) display modes too, and I checked both of the images on my secondary display (S-PVA in "native" mode - maxed RGB Gains without any LUT calibration - relatively natural characteristics with moderate black crush) as well.

I chose the dark gray (low contrast) instead, because you rarely see real blacks on the real world either. You will rarely see movie scenes where the black should be real black because real black exists on a perfectly darkened room only. Most of the cinema movies never show you scenes where you should see real deep black (only when the camera was in a perfectly darkened room).

For example: the bug that caused the corruption of the blue at very low levels (recently fixed) was barely noticeable if the brightness level was set correctly...

Yes, it was barely noticeable but you could see it too because you corrected it. :cool:
Should I stop reporting the "barely noticeable" and "really small" errors? :rolleyes:

By the way, not Superman but Riddick. :) My eyes work well in the dark. I will read a newspaper where others start behave like a blind.
On the other hand, you will never hear me talking about problems with the near-white scale. I can barely distinguish the differences in that range with 120 cd/m^2 white luminance and gamma ~2.2 tonal response.
But I am very sensitive to the shadow details. That's why I keep myself away from the relatively cheap plasma screens with noisy dark tones and PVA LCDs with their black-crush effect.

Otherwise, this is a very god example that you could see those errors on the screenshots as well...

As I've told you before, the "harm" was caused by the gamma curve you selected, not by yCMS.

Ok, let's see it again (and in the last time, I hope/swear):

In theory first:
If yCMS with MadVR + dithering produces infinite bit depth (ÁÁÁ), and my ColorMunki (in Adaptive HighRes mode) is good enough to accurately measure the luminance values for every available gray steps on a display with ~800:1 contrast ratio (so, dark tones are not really too dark...), and the ideal display has a pure power gamma ~2.35 tonal response, and I can see black swatches, then Cameron wanted to show black swatches.
Right?

But no. There were no black swathes on the IMAX screen and here is an other user with a different display (which lays closer to the theoretically ideal and desired characteristics, I think) and he doesn't have black swatches after the corrections. (May be because he needs less corrections.)

So, there should be an "error" somewhere. It can be:
- my instrument [*1]
- my eyes [*2]
- our assumptions about the ideal display [*3]
- Or... maybe... the current softwares are unable to produce infinite bit depth on limited display hardwares and you can lose some visual informations when you apply some corrections. [*4]

[*1] But what about Fer's results then? May be he used less measures, so he got lighter corrections - but his final result is less accurate, so it is not fully authoritative then).

[*2] You saw the black swatches on the screenshots. (Of course, it is not a real proof alone, but one fact.)

[*3] I will delineate an other test later.

[*4]You can freely do any kind of gamma, gamut and white balance corrections with infinite bit depth. But I can't do relatively big corrections with the current softwares.

------------------ New test results ----------------

I played with the sRGB display mode today.

I measured the display characteristics before and after the VGA LUT calibration and I checked it subjectively as well. After that, I used the measured values from the uncalibrated state and I put them into the yCMS input file. (Primaries, white and 256 luminance values.)

The 10+ bit VGA LUT could push my display very close to my target values. The measured gamma curve is very close to the ideal gamma 2.2 curve (with black offset, of course) and the white balance is nicely corrected as well (only the near-black shades left uncorrected where you hardly notice it - confirmed by my eyes...).
Of course, it didn't correct the gamut but it is very close to the sRGB gamut (one of the reasons why I bought this display - but this is an internal emulation!).
I tried to clear and reload the VGA LUT. The gamma changed but it didn't produce anything like those black swatches.

The yCMS and madVR + dithering combination produced some black swatches again. It was not as hard as the last one in Custom mode. But yes, it was easily noticeable by the naked eye.

Don't forget, the VGA LUT was created with the same spectrophotometer and I used the same software for the calibration and the characterization.

I used the Gamma_Curve 0 2.20 command. So, yCMS theoretically did the same thing what the VGA LUT did, expect the white balance tuning across the full tonal scale.

What is your first opinion?

I think yCMS and/or madVR with 3DLUT processing is not perfect yet.

Unfortunately, I can't measure the full gray scale from a media player window for a more objective and accurate comparison.

----------------

Otherwise, I also tried the default gamma correction (an input file without the Gamma_curve command). It produces much less (barely noticeable) detail loss.
It feels like you have a display with higher contrast ratio and the colors are more "vivid".
So, it is worth considering.

But the color of the brighter shades change too much. And I don't feel them natural but too "vivid". This effect is much softer with Gamma_Curve 1 2.35 but it kills too much shadow details again.

So, I will keep the madVR PC settings + VGA LUT corrections for now.
But it is still an open question for me, so I will reconsider it again is the future. And I will tire my visitors with the "Look! What do you think?" questions. :cool:

[6233638]

Where is the source for this statement?Professional CRT monitors used for mastering content measure 2.35 gamma. EBU guidelines specify 2.35 gamma based on extensive testing and Barco monitors are set to 2.35 by default. (EBU/SMPTE compliant)

If you are only measuring an average of 2.35, there is a high chance that detail near black is being compromised, and that there are errors elsewhere. You have to measure 2.35 at every point on the curve for the image to be accurate. Averages just don't cut it.

Black level compensation is used to avoid crushing shadow detail on low contrast displays, not display an accurate image. It is suitable for content creation but is not how you should watch things. Again, most displays on the market are too low contrast to display 2.35 gamma accurately and look better with lower gamma values. If your display is too low contrast, it might be an acceptable compromise.

When I say gamma should be "flat" I am referring to this representation of the data:http://a.imageshack.us/img822/9196/gammatf1g255pp.th.jpg (http://img822.imageshack.us/i/gammatf1g255pp.jpg/)