Can't achieve a decent color correction with Madvr + yCMS 3dlut

[cbw.lava]

Is that profile correct? His description says:
sRGB primaries, Gamma 2.2, D50
But Rec709 defines a D65 WP. (But the ICC file should contain primaries relatively to D50, so it can mean anything here. It's only a guess.)

It's just for testing, and assuming not using absolute colorimetric (or Photoshop with Adobe Ace, which gives the same results when using abs. colorimetric as rel. colorimetric between matrix profiles), the results will be ok. Of course, if the whitepoint of the display profile is not D65, and you do an absolute colorimetric transform to the non-D65 display profile (in an application that actually supports absolute colorimetric between matrix profiles correctly), then the D50 whitepoint will get simulated, which is probably not what one wants in this case.
[ btw, the main reason I chose a D50 whitepoint is the following behavior in Photoshop: If you have a custom matrix profile, sRGB primaries (or rather Rec. 709 primaries as thats what sRGB uses), pure power gamma 2.2, and D65 whitepoint, whenever you use this profile, Photoshop will silently use the 'real' sRGB profile instead, which of course has a different TRC! You can prevent this by choosing a different whitepoint. ]

[cbw.lava]

Do you have a profile with Gamma 2,35 too?

You can easily build one in Photoshop. Menu "Edit" -> "Color Settings", click "More Options" if not yet done. Then select "Custom RGB..." under "Working Spaces: RGB", enter the desired gamma, choose HDTV primaries, give a name and click "OK". Then, again under "Working Spaces: RGB", select "Save RGB...", done.

[yesgrey]

@darkbasic,
It took me a little more than I thought to appear here...

Let's start with a simple question: How did you get your primaries and WP coordinates that you are using with yCMS?

[yesgrey]

To be honest, I think yCMS should be modified. The current behaviour of the "gamut_curve" command, when no gamma measurements are given, is not intuitive. At least it's different to what you expected. But that's something yesgrey and I will have to discuss.
I thought about it when I created the command, but then realized that " Gamut_Curve" might also be useful when not using the "Grayscale_Measurements".

Imagine the situation when the user has it's display with a perfect gamma curve (by calibration, or by using a crt display) and that he wants to try how would it look with a different gamma curve but he doesn't have a meter? With the current working mode he only needs to tell yCMS which is the transfer function of the display using "Output_Transfer_Function", and then tell the desired one using "Gamma_Curve".

I agree that the current working mode might be somewhat confusing, but I think it's preferable to keep yCMS with more flexibility...

[yesgrey]

Here is the psd file with 4 layers: http://darkbasic.homelinux.com/images/madvr_tests.psd

Sorry, but I can't open the file. Could you put all the images in different files (png or jpg)?

Thanks.

[darkbasic]

I'm sorry, I usually do not use proprietary formats but even tiff doesn't support layers (at least the 'open' tiff format).
I will post 8bit/channel png images as soon as possible this afternoon.

How did you get your primaries and WP coordinates that you are using with yCMS?

Dumped from the profile. I did not chromatic adapt them because I specified the white coordinates too.

In the meantime I found where is the problem: gamma correction.
If I (hardware) calibrate my monitor to gamma 2.35 and I use the same input and output transfer function ([Input,Output]_Transfer_Function 1.0 0.0 0.425531914893617 0) the resulting image is VERY good, like the MPC-HC or the photoshop one. But if the monitor is calibrated to gamma 2.2 and I assume the source as being gamma 2.35 the result is terrible. Nothing changes if I use Grayscale_Measurements instead of output transfer function: the result is the same one.

I will post some more screenshots this afternoon.

[yesgrey]

But if the monitor is calibrated to gamma 2.2 and I assume the source as being gamma 2.35 the result is terrible. Nothing changes if I use Grayscale_Measurements instead of output transfer function: the result is the same one.
if you use the wrong source gamma it would be hard to fix it by changing the output gamma...

The gamma is very important for an accurate gamut correction, as you have just realized.;)

I will post some more screenshots this afternoon.
Ok.

[darkbasic]

Please forgot what I told about gamma, I just didn't remember the tests I did.


I can't still achieve coherent results with yCMS.

Here is the psd file with 4 layers: http://darkbasic.homelinux.com/images/madvr_tests.psd


[A] madvr, 3dlut OFF, sRGB primaries w/ Gamma 2.2
The output renderer is madvr, 3dlut was disabled and I assigned cbw.lava's profile.

[ B] madvr, 0.45 input transfer function, grayscale measurements
I used 0,45 input transfer function because cbw.lava's profile has a 2.2 gamma. This is the config:
Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Grayscale_Measurements 2
30 6.9079 7.3366 8.1785
31 7.2734 7.7528 8.6289
32 7.9362 8.4506 9.4189
33 8.3416 8.8640 9.8735
34 9.0097 9.5840 10.674
35 9.5043 10.110 11.189
36 10.187 10.827 12.019
37 10.667 11.356 12.561
38 11.415 12.128 13.471
39 11.966 12.704 14.085
40 12.728 13.530 14.979
41 13.520 14.394 15.906
42 14.025 14.901 16.503
43 14.943 15.867 17.553
44 15.566 16.553 18.309
45 16.482 17.518 19.305
46 17.142 18.202 20.096
47 18.090 19.223 21.208
48 18.755 19.922 21.933
49 19.817 21.055 23.183
50 20.934 22.245 24.449
51 21.601 22.975 25.175
52 22.669 24.080 26.459
53 23.429 24.933 27.307
54 24.574 26.083 28.626
55 25.378 26.950 29.597
56 26.556 28.214 30.845
57 27.349 29.076 31.831
58 28.713 30.513 33.402
59 29.493 31.345 34.194
60 30.786 32.744 35.705
61 32.086 34.142 37.253
62 33.070 35.175 38.418
63 34.343 36.529 39.851
64 35.327 37.541 40.992
65 36.758 39.060 42.626
66 37.753 40.104 43.842
67 39.287 41.737 45.528
68 40.220 42.730 46.647
69 41.861 44.480 48.522
70 43.430 46.165 50.288
71 44.469 47.285 51.545
72 46.094 48.988 53.401
73 47.210 50.201 54.758
74 48.734 51.805 56.405
75 49.887 53.046 57.819
76 51.775 55.033 59.929
77 52.852 56.181 61.201
78 54.733 58.172 63.448
79 56.020 59.559 64.999
80 57.802 61.445 66.966
81 59.636 63.339 69.109
82 60.964 64.778 70.671
83 62.842 66.752 72.771
84 64.110 68.128 74.213
85 66.203 70.319 76.682
86 67.407 71.598 78.033
87 69.607 73.955 80.548
88 71.010 75.425 82.166
89 73.040 77.564 84.580
90 75.259 79.938 86.998
91 76.677 81.375 88.767
92 78.807 83.635 91.275
93 80.420 85.356 93.168
94 82.516 87.637 95.686
95 84.018 89.270 97.406
96 86.442 91.762 100.20
97 87.964 93.425 101.88
98 90.283 95.862 104.61
99 92.184 97.895 106.70
100 94.176 100.00 109.09

Input_Transfer_Function 1.0 0.0 0.45 0

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561

[C] madvr, 0.45 input/output transfer function
I used a 0,45 output tranfer function instead of the grayscale measurements. This is the config:
Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Input_Transfer_Function 1.0 0.0 0.45 0
Output_Transfer_Function 1.0 0.0 0.45 0

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561

[D] madvr, 0.425 input transfer func, 0,45 output transfer func
I used a 0,425 input transfer function (gamma 2,35). I put it just to see the differences because cbw.lava's profile has a 2.2 gamma and I don't have a profile with 2.35 gamma. This is the config:
Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Input_Transfer_Function 1.0 0.0 0.425531914893617 0
Output_Transfer_Function 1.0 0.0 0.45 0

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561



Considerations:
- I can't see any significant difference between B and C, I think it means my monitor's calibration is good.
- There are a lot of differences between C and A and I don't know how to explain it :confused:

Here is A, B, C, D in png:
http://darkbasic.homelinux.com/images/ycms/madvr_tests_A.png
http://darkbasic.homelinux.com/images/ycms/madvr_tests_B.png
http://darkbasic.homelinux.com/images/ycms/madvr_tests_C.png
http://darkbasic.homelinux.com/images/ycms/madvr_tests_D.png


After this test, cbw.lava told me how to create a 2.35 gamma sRGB profile, so I calibrated my monitor again, this time for maximum contrast and gamma 2.35.
[2A] madvr, 3dlut OFF, sRGB primaries w/ Gamma 2.35
The output renderer is madvr, 3dlut was disabled and I assigned sRGB gamma 2.35 profile.
http://darkbasic.homelinux.com/images/ycms/madvr_tests2_A.png

[C] madvr, 0.45 input/output transfer function
I used a 0,45 output tranfer function instead of the grayscale measurements. This is the config:
Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Input_Transfer_Function 1.0 0.0 0.425531914893617 0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561
http://darkbasic.homelinux.com/images/ycms/madvr_tests2_B.png

This time the results are coherent.
But why didn't I achieved coherent results with the monitor calibrated to gamma 2.2?

[yesgrey]

Sorry, but I think this is getting very confusing with all this profiles thing...

I would suggest to start simple to see if I can understand...
Would you be able to create these 3 pictures for me, please?

[A] madvr, 3dlut OFF, and no profile assigned.

[B] madvr, 3dlut OFF, and assign your monitor's profile, the one you have created using ArgyllCMS.

[C] madvr, 3dlut ON, and no profile assigned.
Use this script for creating the 3DLUT:

Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Grayscale_Measurements 2
30 6.9079 7.3366 8.1785
31 7.2734 7.7528 8.6289
32 7.9362 8.4506 9.4189
33 8.3416 8.8640 9.8735
34 9.0097 9.5840 10.674
35 9.5043 10.110 11.189
36 10.187 10.827 12.019
37 10.667 11.356 12.561
38 11.415 12.128 13.471
39 11.966 12.704 14.085
40 12.728 13.530 14.979
41 13.520 14.394 15.906
42 14.025 14.901 16.503
43 14.943 15.867 17.553
44 15.566 16.553 18.309
45 16.482 17.518 19.305
46 17.142 18.202 20.096
47 18.090 19.223 21.208
48 18.755 19.922 21.933
49 19.817 21.055 23.183
50 20.934 22.245 24.449
51 21.601 22.975 25.175
52 22.669 24.080 26.459
53 23.429 24.933 27.307
54 24.574 26.083 28.626
55 25.378 26.950 29.597
56 26.556 28.214 30.845
57 27.349 29.076 31.831
58 28.713 30.513 33.402
59 29.493 31.345 34.194
60 30.786 32.744 35.705
61 32.086 34.142 37.253
62 33.070 35.175 38.418
63 34.343 36.529 39.851
64 35.327 37.541 40.992
65 36.758 39.060 42.626
66 37.753 40.104 43.842
67 39.287 41.737 45.528
68 40.220 42.730 46.647
69 41.861 44.480 48.522
70 43.430 46.165 50.288
71 44.469 47.285 51.545
72 46.094 48.988 53.401
73 47.210 50.201 54.758
74 48.734 51.805 56.405
75 49.887 53.046 57.819
76 51.775 55.033 59.929
77 52.852 56.181 61.201
78 54.733 58.172 63.448
79 56.020 59.559 64.999
80 57.802 61.445 66.966
81 59.636 63.339 69.109
82 60.964 64.778 70.671
83 62.842 66.752 72.771
84 64.110 68.128 74.213
85 66.203 70.319 76.682
86 67.407 71.598 78.033
87 69.607 73.955 80.548
88 71.010 75.425 82.166
89 73.040 77.564 84.580
90 75.259 79.938 86.998
91 76.677 81.375 88.767
92 78.807 83.635 91.275
93 80.420 85.356 93.168
94 82.516 87.637 95.686
95 84.018 89.270 97.406
96 86.442 91.762 100.20
97 87.964 93.425 101.88
98 90.283 95.862 104.61
99 92.184 97.895 106.70
100 94.176 100.00 109.09

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561

[darkbasic]

[A] madvr, 3dlut OFF, and no profile assigned.

http://darkbasic.homelinux.com/images/ycms/madvr_tests3_A.png

[B] madvr, 3dlut OFF, and assign your monitor's profile, the one you have created using ArgyllCMS.

http://darkbasic.homelinux.com/images/ycms/madvr_tests3_B.png

[C] madvr, 3dlut ON, and no profile assigned.
Use this script for creating the 3DLUT:

Input_Format HD YCbCr 8

Output_Format HD RGB_PC 16

Grayscale_Measurements 2
30 6.9079 7.3366 8.1785
31 7.2734 7.7528 8.6289
32 7.9362 8.4506 9.4189
33 8.3416 8.8640 9.8735
34 9.0097 9.5840 10.674
35 9.5043 10.110 11.189
36 10.187 10.827 12.019
37 10.667 11.356 12.561
38 11.415 12.128 13.471
39 11.966 12.704 14.085
40 12.728 13.530 14.979
41 13.520 14.394 15.906
42 14.025 14.901 16.503
43 14.943 15.867 17.553
44 15.566 16.553 18.309
45 16.482 17.518 19.305
46 17.142 18.202 20.096
47 18.090 19.223 21.208
48 18.755 19.922 21.933
49 19.817 21.055 23.183
50 20.934 22.245 24.449
51 21.601 22.975 25.175
52 22.669 24.080 26.459
53 23.429 24.933 27.307
54 24.574 26.083 28.626
55 25.378 26.950 29.597
56 26.556 28.214 30.845
57 27.349 29.076 31.831
58 28.713 30.513 33.402
59 29.493 31.345 34.194
60 30.786 32.744 35.705
61 32.086 34.142 37.253
62 33.070 35.175 38.418
63 34.343 36.529 39.851
64 35.327 37.541 40.992
65 36.758 39.060 42.626
66 37.753 40.104 43.842
67 39.287 41.737 45.528
68 40.220 42.730 46.647
69 41.861 44.480 48.522
70 43.430 46.165 50.288
71 44.469 47.285 51.545
72 46.094 48.988 53.401
73 47.210 50.201 54.758
74 48.734 51.805 56.405
75 49.887 53.046 57.819
76 51.775 55.033 59.929
77 52.852 56.181 61.201
78 54.733 58.172 63.448
79 56.020 59.559 64.999
80 57.802 61.445 66.966
81 59.636 63.339 69.109
82 60.964 64.778 70.671
83 62.842 66.752 72.771
84 64.110 68.128 74.213
85 66.203 70.319 76.682
86 67.407 71.598 78.033
87 69.607 73.955 80.548
88 71.010 75.425 82.166
89 73.040 77.564 84.580
90 75.259 79.938 86.998
91 76.677 81.375 88.767
92 78.807 83.635 91.275
93 80.420 85.356 93.168
94 82.516 87.637 95.686
95 84.018 89.270 97.406
96 86.442 91.762 100.20
97 87.964 93.425 101.88
98 90.283 95.862 104.61
99 92.184 97.895 106.70
100 94.176 100.00 109.09

Gamut_Measurements 0.691113 0.309136 0.229617 0.690836 0.151459 0.056993 0.306425 0.324561


http://darkbasic.homelinux.com/images/ycms/madvr_tests3_C.png

[yesgrey]

[B] madvr, 3dlut OFF, and assign your monitor's profile, the one you have created using ArgyllCMS.
http://darkbasic.homelinux.com/images/ycms/madvr_tests3_B.png

Strange... The pictures A and B are almost the same. Did you assign the monitor's profile in photoshop like you used to do? Maybe I wasn't clear enough and you only had assigned the profile to the monitor...

If my idea of all this is correct, pics B and C should be similar, and not A and B.

Furthermore, could you also create another pic?
[D] mpc-hc with its color correction using your monitor's profile.

[darkbasic]

Strange... The pictures A and B are almost the same.

This is possible only if you own a monitor similar to mine. Mine has a gamut similar to AdobeRGB. Be aware that photoshop automatically assumes the "working profile" as source profile for images without an embedded profile; so if you use Photoshop and your "working profile" is AdobeRGB, than A will be similar to B.

Did you assign the monitor's profile in photoshop like you used to do? Maybe I wasn't clear enough and you only had assigned the profile to the monitor...

It was clear, as you can see looking for the embedded profile. I assigned my monitor's profile in photoshop.

If my idea of all this is correct, pics B and C should be similar, and not A and B.
I agree A and B shouldn't be similar if your monitor's gamut is different from mine (A and B aren't similar in my other 3 color aware PCs).
C with my monitor's profile assigned should be similar to D if yCMS color management is correct. But again, it isn't.
Now let's try to assign a profile with Rec.709 Primaries and a 2.35 pure gamma curve to A, and compare to D: they are identical. This means MPC-HC color correction does a very good job.


Furthermore, could you also create another pic?
[D] mpc-hc with its color correction using your monitor's profile.
http://darkbasic.homelinux.com/images/ycms/madvr_tests3_D.png

[yesgrey]

Be aware that photoshop automatically assumes the "working profile" as source profile for images without an embedded profile
This raises another question... How are you creating the images? Are you using PrtScreen and then pasting the images into photoshop? If you are, then it's not what I wanted you to do. I want A, C and D pasted to Paint, so the profile could not affect the images. Only B should be pasted to photoshop, to be affected by the profile.

[darkbasic]

This raises another question... How are you creating the images? Are you using PrtScreen and then pasting the images into photoshop? If you are, then it's not what I wanted you to do. I want A, C and D pasted to Paint, so the profile could not affect the images.

Photoshop assumes the "working profile" as source profile when doing monitor compensation, but it doesn't embed it into the image! A and C are not affected by any profile if you don't use photoshop to see them. If you don't trust me you can check yourself: you have my yCMS config and THIS (http://www.mediafire.com/download.php?gjau13ct5x8ry4g) is my test pattern.

Only B should be pasted to photoshop, to be affected by the profile.

Maybe you mean B and D:
[D] mpc-hc with its color correction using your monitor's profile.

[yesgrey]

If you don't trust me you can check yourself
I will check. It's not a question of trust, but of understanding. I need to understand exactly how this works, and the only way is by testing it myself. Can you point me to your monitor's profile file? I believe you already have posted it, but since you posted more than one I don't know exactly which...

[darkbasic]

Here is my monitor's profile:
http://darkbasic.homelinux.com/monitor_140_f2000.icm

Monitor is a 30" Nec Spectraview 3090, HW calibrated with an Eye-One Colormunky spectrophotometer (Gamma 2.2, D65 and 140 cd/m^2) and profiled with argyll 1.3.1 with a 2000 patches chart.

Here is a dump of the wtpt, rXYZ, gXYZ and bXYZ tags from my minotor's profile:
C:\Users\Niko\Documents\monitor>iccdump.exe -v3 -t wtpt -t rXYZ -t gXYZ -t bXYZ monitor_140_f2000.icm
XYZArray:
No. elements = 1
0: 0.944122, 1.000000, 1.136963 [Lab 100.000000, -3.494897, -22.575995]
XYZArray:
No. elements = 1
0: 0.595306, 0.266281, -0.000214 [Lab 58.628512, 104.081738, 101.486962]
XYZArray:
No. elements = 1
0: 0.225998, 0.679947, 0.078293 [Lab 86.003793, -131.388852, 84.638767]
XYZArray:
No. elements = 1
0: 0.142899, 0.053772, 0.746811 [Lab 27.783450, 75.878480, -117.989972]


And here are grayscale measurements I made with argyll:

Grayscale_Measurements 2
30 6.9079 7.3366 8.1785
31 7.2734 7.7528 8.6289
32 7.9362 8.4506 9.4189
33 8.3416 8.8640 9.8735
34 9.0097 9.5840 10.674
35 9.5043 10.110 11.189
36 10.187 10.827 12.019
37 10.667 11.356 12.561
38 11.415 12.128 13.471
39 11.966 12.704 14.085
40 12.728 13.530 14.979
41 13.520 14.394 15.906
42 14.025 14.901 16.503
43 14.943 15.867 17.553
44 15.566 16.553 18.309
45 16.482 17.518 19.305
46 17.142 18.202 20.096
47 18.090 19.223 21.208
48 18.755 19.922 21.933
49 19.817 21.055 23.183
50 20.934 22.245 24.449
51 21.601 22.975 25.175
52 22.669 24.080 26.459
53 23.429 24.933 27.307
54 24.574 26.083 28.626
55 25.378 26.950 29.597
56 26.556 28.214 30.845
57 27.349 29.076 31.831
58 28.713 30.513 33.402
59 29.493 31.345 34.194
60 30.786 32.744 35.705
61 32.086 34.142 37.253
62 33.070 35.175 38.418
63 34.343 36.529 39.851
64 35.327 37.541 40.992
65 36.758 39.060 42.626
66 37.753 40.104 43.842
67 39.287 41.737 45.528
68 40.220 42.730 46.647
69 41.861 44.480 48.522
70 43.430 46.165 50.288
71 44.469 47.285 51.545
72 46.094 48.988 53.401
73 47.210 50.201 54.758
74 48.734 51.805 56.405
75 49.887 53.046 57.819
76 51.775 55.033 59.929
77 52.852 56.181 61.201
78 54.733 58.172 63.448
79 56.020 59.559 64.999
80 57.802 61.445 66.966
81 59.636 63.339 69.109
82 60.964 64.778 70.671
83 62.842 66.752 72.771
84 64.110 68.128 74.213
85 66.203 70.319 76.682
86 67.407 71.598 78.033
87 69.607 73.955 80.548
88 71.010 75.425 82.166
89 73.040 77.564 84.580
90 75.259 79.938 86.998
91 76.677 81.375 88.767
92 78.807 83.635 91.275
93 80.420 85.356 93.168
94 82.516 87.637 95.686
95 84.018 89.270 97.406
96 86.442 91.762 100.20
97 87.964 93.425 101.88
98 90.283 95.862 104.61
99 92.184 97.895 106.70
100 94.176 100.00 109.09

[janos666]

Many things changed since this conversation ended. There were some yCMS and lcms2 updates and EVR-CP was improved too. So, I thought I should compare the results now.
I used MPC-HC 1.4.2825.0 (modified by JanWillem32 to work with 32-bit EVR-CP surface ; and also can output 10-bit/color but I can't capture and save that, so it worked with 8-bit output), madVR 0.36 and yCMS 1.9
I used the same measurement data (XYZ coordinates calculated from spectral sensor data) to construct the yCMS input file which I used to build the ICM file with ArgyllCMS/colprof, except that I included many more additional data (single channel and multidimensional test patches, ~2000 points at total) to construct the XYZ-LUT based ICM file.
[I was curious if multidimensional patches are really matter on a RGB displays or not and I wanted to use all available options in every softwares...]
I used PhotoShop CS5 (with Adobe ACE) as a reference. It can use black point compensation with the relative colorimetric rendering intent only but yCMS is closer to the absolute colorimetric intent when you insert XYZ values for the grayscale_measurement command. However it's not a big problem because the display was set to D65 WP and it is relatively coherent across the grayscale, so it won't make too much difference. The lcms in MPC-HC was set to use the absolute colorimetric intent.

###Notice: These images are meant to look good on my display only. So, you are supposed to see incorrect colors. But it's still good to show the relative differences between the different CMS solutions. I also set my display to sRGB emulation mode when I wrote this post.

Firs test - primary and secondary colors: yCMS (http://img12.tar.hu/janos666/img/102838654.png) ; lcms (http://img12.tar.hu/janos666/img/102838649.png) ; PS without BPC (http://img12.tar.hu/janos666/img/102838658.png) ; PS with BPC (http://img12.tar.hu/janos666/img/102838664.png)
-> I can't really see any noticeable differences on the secondary colors. It would let me believe that multidimensional test patches (and the XYZ-LUT based ICC color management in this case) are only waste of time with my display.

Second test - gray gradients: yCMS (http://img12.tar.hu/janos666/img/102842341.png) ; lcms (http://img12.tar.hu/janos666/img/102842340.png) ; PS without BPC (http://img12.tar.hu/janos666/img/102842342.png) ; PS with BPC (http://img12.tar.hu/janos666/img/102842343.png)
-> They all produce different gradients and the biggest differences are noticeable on the near-black shades (as it was obvious on the previous images too). yCMS is very far from lcms and PS lays between them. The "PS without BPC" is close to lcms. And "PS with BPC" differs from all of them because it aggressively increased the black luminance too. (This result is not acceptable for movies...)

Third test - color clipping: yCMS (http://img12.tar.hu/janos666/img/102848380.png) ; lcms (http://img12.tar.hu/janos666/img/102848387.png)
-> I am not sure how to judge this but I think lcms passed and yCMS failed badly!

Additional test - "real world" image: yCMS (http://img12.tar.hu/janos666/img/102816904.png) ; lcms (http://img12.tar.hu/janos666/img/102816896.png) ; PS without BPC (http://img12.tar.hu/janos666/img/102816914.png) ; PS with BPC (http://img12.tar.hu/janos666/img/102834722.png)
"PS with BPC" is unacceptable again. "PS without BPC" is very close to lcms. yCMS is far from anything else. And the most interesting thing is that yCMS not only produces lighter image but it also shows different colors. It's more obvious when I watch these images with native gamut + low gamma display mode (where they belong). But without a reference display, it's hard to tell if it's "a bug or a feature".

The lcms results are more convincing for the first look because the colors are "deeper" and it feels like I have a displays with higher native contrast ratio. But, of course, I also loose some shadow details. It's very unnatural for me if I see deep black (as much as my display can show me but I am used to it...) instead of dark grays in light rooms or outdoor scenes with heavy sunlight.
But I have to mention that this is not a mathematical black-clipping (rounding or other error). Every near-black tone between 16 and 22 look the same for me with lcms but I can distinguish all of them if I set the brightness to a much higher level and I play with the "Contrast" slider (digital processing with an LCD but helps to magnify the difference...).

Another "real world" image: yCMS (http://img12.tar.hu/janos666/img/102885823.png) ; lcms (http://img12.tar.hu/janos666/img/102885815.png)
These images show obvious differences in white balance and near-black shades.
Here you can see what I talked about. The man wears a black suit, yes. But is he supposed to be a "black swatch" in this very light environment? I guess not. But the lcms result is more like a black swatch when compared to the yCMS result. But this can also have a withdraw effect: In very dark scenes it feels like I have a color-night-vision with yCMS because I can see everything on the dark where I think I should see some very dark shades only.

It's very hard to decide. I guess the obvious solution would be a display with much higher contrast ratio (like a plasma HDTV).

About the white balance... The background looks better with yCMS but I like the skin tone better with lcms (but I can't really judge by the skin tone under this artificial lightning).


I guess I will keep with madVR+yCMS for mow.


What are your opinions? (I mean, if you make your own experiment with these two solutions and not only from my images...)


By the way... What is the recommended white luminance for cinema movies? My current display settings produce ~120 cd/m^2 and I use it in a "perfectly" dark room. (I shut down the artificial light for movies but I keep it for other tasks.)

[yesgrey]

So, I thought I should compare the results now.
Thanks for the tests.

Second test - gray gradients ...
-> They all produce different gradients and the biggest differences are noticeable on the near-black shades (as it was obvious on the previous images too). yCMS is very far from lcms and PS lays between them.
I can see a difference, but it's not very noticeable. Might be a limitation of my CRT monitor, though.

Third test - color clipping...
-> I am not sure how to judge this but I think lcms passed and yCMS failed badly!
As I have told you here (http://forum.doom9.org/showthread.php?p=1469265#post1469265) yCMS is working as it should be.

Since you're still not getting it, I will try to explain it again hoping that this time I will be more clear...

Lets start by looking at the test pattern:
The test was created for video levels, hence the primaries at level 235 flashing over a background at level 255. However, the test pattern is not an RGB video, but a YCbCr video. So, lets now look at the respective values for each primary and background in RGB video levels and YCbCr:
Red Primary - RGB: [235,16,16]; YCbCr: [63,102,240]
Red Background - RGB: [255, 0, 0]; YCbCr: [66,100,249]

Green Primary - RGB: [16,235,16]; YCbCr: [173,42,26]
Green Background - RGB: [ 0,255, 0]; YCbCr: [186,35,18]

Blue Primary - RGB: [16,16,235]; YCbCr: [32,240,118]
Blue Background - RGB: [ 0, 0,255]; YCbCr: [33,249,117]

Now lets look only to the YCbCr->RGB conversion:
Considering the YCbCr values above, when you perform the YCbCr->RGB conversion expanding to PC levels the primaries values naturally are converted to 255 as they should, but the background values are converted to 275.
Red Primary - RGB: [255, 0, 0]; YCbCr: [63,102,240]
Red Background - RGB: [275, 0, 0]; YCbCr: [66,100,249]

Green Primary - RGB: [ 0,255, 0]; YCbCr: [173,42,26]
Green Background - RGB: [ 0,275, 0]; YCbCr: [186,35,18]

Blue Primary - RGB: [ 0, 0,255]; YCbCr: [32,240,118]
Blue Background - RGB: [ 0, 0,275]; YCbCr: [33,249,117]
However, due to the highest valid value in 8 bit RGB being 255 the background values need to be clipped to 255, ending
the primaries and backgrounds as the same.

Finally, lets see what happens when a gamut conversion to a wider gamut is applied (your reported case).
As we have seen previously, the primary values naturally are converted to 255, and the background values are converted to 275. Now, if we apply the gamut conversion to the values before performing any clipping, the values would be:
Red Primary - RGB: [210,44,20]; YCbCr: [63,102,240]
Red Background - RGB: [231,48,21]; YCbCr: [66,100,249]

Green Primary - RGB: [125,248,34]; YCbCr: [173,42,26]
Green Background - RGB: [138,265,37]; YCbCr: [186,35,18]

Blue Primary - RGB: [ 1,20,251]; YCbCr: [32,240,118]
Blue Background - RGB: [17,28,267]; YCbCr: [33,249,117]

Now, the primaries and background values are different. Even though the Green and Blue backgrounds still need some clipping, the effect on the image would be less severe.
The difference you are seeing between yCMS and the other method simply shows the limitations of the ICM method. yCMS is fed with YCbCr data, so it can perform the color correction in the full original signal, clipping only right before sending to the display, while the ICM method is fed with RGB data that was already clipped to RGB valid values.

In real world usage you would not see any difference, because the standard YCbCr range is 16-235 for Y and 16-240 CbCr, so the 249 values contained in the test pattern should not happen, but this is a test pattern, created for testing the equipments and their processing chain, and from this test you can conclude that madVR/yCMS processing chain preserves the most of the original signal. However, if the source contains any data outside the standard range, the ICM method will band really bad, while madVR/yCMS will preserve better the original colors and show a smooth gradient.

Additional test - "real world" image...
"PS with BPC" is unacceptable again. "PS without BPC" is very close to lcms. yCMS is far from anything else. And the most interesting thing is that yCMS not only produces lighter image but it also shows different colors.
Sorry, but with my monitor I couldn't see any difference between the images. For me looked all the same.

Another "real world" image...
These images show obvious differences in white balance and near-black shades.
I can see the differences in white balance, but not the near-black shades ones.

By the way... What is the recommended white luminance for cinema movies? My current display settings produce ~120 cd/m^2 and I use it in a "perfectly" dark room.
I think it's ~14 Fl, which is ~50 cd/m^2, but some people prefer an image with more punch and go for a little more...

[darkbasic]

Hi! Nice to hear you again :)

I will do some more tests when I will find some free time.

By the way... What is the recommended white luminance for cinema movies? My current display settings produce ~120 cd/m^2 and I use it in a "perfectly" dark room. (I shut down the artificial light for movies but I keep it for other tasks.)

You should set max luminance because you want to achieve the max contrast ratio. As side effect you will also obtain better shadow details. I also calibrated at 2.35 gamma.

[janos666]

You should set max luminance because you want to achieve the max contrast ratio.

The back-light control has nothing to do with the native contrast ratio of an LCD panel. (May be it lets more space for dynamic contrast tricks but I don't like that. I disabled the dynamic contrast feature.)

As side effect you will also obtain better shadow details. I also calibrated at 2.35 gamma.

One half of this is true. Higher back-light luminance helps me to distinguish the dark shades. But:
1 - I don't want to burn out my eyes with too bright near-white shades. :)
2 - It raises the black luminance too, so blacks become grays. Not a good idea for anything but very bad for movies.

[janos666]

The test was created for video levels, hence the primaries at level 235 flashing over a background at level 255.
[...]
when you perform the YCbCr->RGB conversion expanding to PC levels the primaries values naturally are converted to 255 as they should, but the background values are converted to 275.
[...]
However, due to the highest valid value in 8 bit RGB being 255 the background values need to be clipped to 255, ending
the primaries and backgrounds as the same.
[...]
this is a test pattern, created for testing the equipments and their processing chain, and from this test you can conclude that madVR/yCMS processing chain preserves the most of the original signal.


Thanks. It made it clear for me. I didn't know that the background is 255. I thought it's 235 as this value should be the highest value in a Rec709 reference video.
But ok... it's a test, so it should work like this... it makes sense now.


the other method simply shows the limitations of the ICM method. yCMS is fed with YCbCr data, so it can perform the color correction in the full original signal, clipping only right before sending to the display, while the ICM method is fed with RGB data that was already clipped to RGB valid values.


lcms can work with YCC and do YCC->RGB conversion by itself. It's only a limitation in the current MPC-HC implementation. (But as you said, it shouldn't make any difference with standard Rec709 material, in practice.)


I can see the differences in white balance, but not the near-black shades ones.


Is the man in the back suit a black swatch or can you see some details (like the edge of the black jacket above the black trousers)?

[darkbasic]

The back-light control has nothing to do with the native contrast ratio of an LCD panel. (May be it lets more space for dynamic contrast tricks but I don't like that. I disabled the dynamic contrast feature.)

Partially true: such a tricks usually dimm the backlight on dark scenes to achieve deeper blacks, if you calibrate to the higher luminance and disable dynamic contrast you will still achieve the higher contrast ratio because the black point drift will be lower compared to the white point one.

With my monitor I can achieve a 623:1 contrast ratio with a 140cd/m^2 luminance (black is 0,23 cd/m^2), while with the highest luminance I can achieve a 930:1 contrast ratio with a 0,33 cd/m^2 black (which is a bit higher of course, but nothing compared to the luminance increase which is now 308 cd/m^2).


1 - I don't want to burn out my eyes with too bright near-white shades. :)

It's not so annoying with an appropriate viewing distance.

2 - It raises the black luminance too, so blacks become grays. Not a good idea for anything but very bad for movies.
It's true, but if the monitor is good it will raise just a bit and it will be difficult to tell the difference.

[yesgrey]

lcms can work with YCC and do YCC->RGB conversion by itself. It's only a limitation in the current MPC-HC implementation.
Yes, I was referring to the ICM when used by MPC-HC, which outputs as RGB. I did not want to diminish lcms in anyway. In fact, if yCMS was also capable of generating an ICM file for using with MPC-HC the result would be the same.;)

Is the man in the back suit a black swatch or can you see some details (like the edge of the black jacket above the black trousers)?
It's a black swatch. I can't see any details in any of the pictures. However, using colorpic I noticed that the yCMS image has higher values, so this should be due to my CRT monitor.

[cyberbeing]

yesgrey, since you are unable to see the differences on some of janos666's images, while I see a very noticeable difference on my CRT, it sounds like you are crushing/compressing your near-black tones.

Take a look at the following images without any ICM color management (no Photoshop or the like). Both should result in a smooth transition from white-black-white. If you see a black bar of any type at the transition without a nice evenly spaced progression, you are crushing blacks and killing shadow detail. For reference 0-0-0 Black is only one pixel in those images, with the following steps being two pixels wide on average.

Horizontal Gradient (http://img94.imageshack.us/img94/9176/gradhor01.png)
Vertical Gradient (http://img33.imageshack.us/img33/1339/gradvert01.png)

[yesgrey]

Take a look at the following images without any ICM color management (no Photoshop or the like).
I don't have any of that applied.

If you see a black bar of any type at the transition without a nice evenly spaced progression, you are crushing blacks and killing shadow detail. For reference 0-0-0 Black is only one pixel in those images, with the following steps being two pixels wide on average.
I see a black bar ~100 pixels wide in both pics.

Now I'm realizing that my monitor is very bad. :( I never saw smooth transitions on the grayscale images, but I thought they were exactly like that... :o Now that I think a little more about it, I remember than on my projector the transitions were smoother...

I will try to see if I can improve this, but I've already used my monitor's brightness and contrast controls and ~100 pixels width is the best I can get... I will see if it's a problem with any OS setting.

Thanks for the enlightmen.

[janos666]

Continuing a conversation started in the MPC-HC topic...

@cyberbeing

Output_Transfer_Function does describes the TRC of the output device.
-> If you already calibrated your display with a known curve, it's a much better choice than re-measuring the calibrated result and describe the device TRC with XYZ measurements (so they will be approximated and may be tried to be counter-corrected again with additional deviations and artifacts due to measurement deviation and slightly different scaling and offset policy, etc...)

The Input_Transfer_Function does describes the curve which you want to use as a decode curve. This has to match with the assumed perfect display. It should match with the curve which you believe that it was intended to be a decode curve for your content.
-> If you believe that you want to decode with a scaled Rec709 curve, you should touch this and describe that curve.


I currently calibrated my display to gamma 2.35 and I use an input file like this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145
It results in almost perfectly clipped primaries and almost perfect TRC. My and "my clients" (friends) subjective opinions are very good about the result of this kind of correction.
But of course, the VGA LUT calibration can be skipped and the device TRC can be described by the Gamut_Measuremetns command while keeping the Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0 -> Or whatever you want to use as a decode curve (according to you current belief/religion/etc)


I could never produce any good results with the Gamma_Curve function and I could never achieve any subjectively acceptable results when the Rec709 curve was involved.


But think about it... How Photoshop works? (Ok, it's ICC based CMS but yCMS can be described as a matrix+curve or matrix+shaper style ICC CMS - depending on the involved functions. And PS can work with these type of profiles too, so they are comparable...)

PS has a source profile and a device profile. And it simply converts between these two profiles. That's all. (Of course, you can apply any post-processing but that's another story...)

What is the device profile with yCMS? -> Of course, the description of the output device:
- The CIE xy coordinates of the primary colors and the white point
- The description of the device TRC -> Can be either a gamma curve (described by the Output_Transfer_Function or the Grayscale_Measurement command with Y values only) or a shaper (described by the Grayscale_Measurement command with 3D coordinates*)

What is the source profile with yCMS? -> Well, it's a very big question mark on the sky of the moon... but some things are known for sure:
- The CIE xy coordinates of the primary colors and the white point.
- The description of the source TRC -> It can be described by the Input_Transfer_Function command only.
As of the exact TRC... It's a question of your religion, but don't ask your rabbi or priest, he won't know. -> If you ask me now, it's a pure-power curve. If I ask you now, it's a scaled Rec709 curve.
But the above things are sure: they should be described this way. Anything else is only a "manual tweak", nothing is exact anymore if you start to play with the Gamma_Curve command.
That's some kind of post-processing. It will mess with the gamut emulation and you can see your own diagrams in the yCMS thread... it's only a "so-so iteration".

*This is currently malfunctioning. That's why I use 1D VGA LUT calibration first.

/// Of course, this is only my current opinion. I can be convinced by the opposite. But only with good reasons.

[yesgrey]

I think I should join this discussion...;)

Output_Transfer_Function does describes the TRC of the output device.
No. Output_Transfer_Function states the gamma encoding applied to the source after leaving the 3DLUT. Ideally it should match the display's gamma curve, hence why your description is not that wrong... ;)

The Input_Transfer_Function does describes the curve which you want to use as a decode curve. This has to match with the assumed perfect display. It should match with the curve which you believe that it was intended to be a decode curve for your content.
-> If you believe that you want to decode with a scaled Rec709 curve, you should touch this and describe that curve.
He might, but it's not needed anymore. With v1.9 the basic and calibration settings commands are all that is needed.

I could never produce any good results with the Gamma_Curve function
Have you tried with yCMS v1.9? I changed the Gamma_Curve algorithm, so now you should get the same results by using Gamma_Curve 0.0 2.35 instead of the Input_Transfer_Function command. However, if you don't want to use the grayscale command you still need to use the Output_Transfer_Function command to let yCMS encode to what your display is expecting...

- The description of the source TRC -> It can be described by the Input_Transfer_Function command only.
That's not true. It can be described by Input_Format or by Input_Transfer_Function. However, with the first you are stuck to the standard's function, while with the latter you can use whatever you want.

*This is currently malfunctioning. That's why I use 1D VGA LUT calibration first.
Yes, it seems it's not correct yet. I am investigating...

[cyberbeing]

As I said previously, this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145

Should give identical results to this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145

When you list the same value for Input_Transfer_Function and Output_Transfer_Function, it just means no gamma correction is performed. yCMS defaults to this, so those values are redundant and are doing absolutely nothing as you are using them.

It's also worth mentioning that yCMS only uses up to 6 decimal places, not 15. So 0.425531914893617 should just be rounded to 0.425532 for yCMS. yesgrey, did something change in that respect recently?

Output_Transfer_Function does describes the TRC of the output device.
-> If you already calibrated your display with a known curve, it's a much better choice than re-measuring the calibrated result and describe the device TRC with XYZ measurements (so they will be approximated and may be tried to be counter-corrected again with additional deviations and artifacts due to measurement deviation and slightly different scaling and offset policy, etc...)
Calculating gamma with a known curve rather than grayscale_measurements would be more accurate (you just lose out on grayscale color correction), but it doesn't change that how you are currently using the value does nothing.

The Input_Transfer_Function does describes the curve which you want to use as a decode curve. This has to match with the assumed perfect display. It should match with the curve which you believe that it was intended to be a decode curve for your content.
-> If you believe that you want to decode with a scaled Rec709 curve, you should touch this and describe that curve.
Input_Transfer_Function represents the encoding curve of the camera. For HD video, this should be nothing other than your standard Rec.709 curve, which yCMS defaults to.

Since you seem to think it should be set to a gamma of 2.35, I'm curious where where you believe the gamma is being adapted from Rec.709 to 2.35 gamma (which btw would make the video brighter then Rec.709)?

Also, same as above, unless you are using different values for input and output they do nothing.

Anyway, I was falling asleep while I was typing this, so don't expect any further response from me for 8 or more hours.

Edit: yesgrey, I do hope you plan to fix 'Gamma_Curve 1.0' because I could not get it to match the scaled Rec.709 curve I get with Argyll CMS as I mentioned in the yCMS thread (yCMS was turning my Rec.709 curve into nearly a power-curve). Since Argyll CMS is the only widely available program which can calibrate to Rec.709, it wouldn't be a bad idea for yCMS to match how it scales Rec.709 to Argyll. Once fixed, it may also help to specify a power function rather then a gamma with 'Gamma_Curve 1.0' since Argyll CMS always spits one out when you are calibrating to Rec.709 with Ambient Light. 'Gamma_Curve 0.0' on the other hand requires no fixing and works well. There is just something strange with how 'Gamma_Curve 1.0' is scaling Rec.709. ...OK now I really am sleeping...

[yesgrey]

As I said previously, this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145

Should give identical results to this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145
No, it shouldn't.

When you list the same value for Input_Transfer_Function and Output_Transfer_Function, it just means no gamma correction is performed.
That is correct, but you are forgetting the gamut correction...

The gamut correction has to be performed on linear light. So, yCMS removes the gamma using the input transfer function, applies the gamut correction, and then applies the output transfer function. When using different input transfer functions, the gamut correction gives different values, hence the different results at the end even when you're using the same functions for degamma/engamma.

What you said only is true if no gamut correction is applied.

It's also worth mentioning that yCMS only uses up to 6 decimal places, not 15. So 0.425531914893617 should just be rounded to 0.425532 for yCMS. yesgrey, did something change in that respect recently?
I don't know where did you get the 6 decimal places idea. yCMS works and has always worked using 64 bit fp (double in C), so it can use up to 15 decimal places.

Input_Transfer_Function represents the encoding curve of the camera. For HD video, this should be nothing other than your standard Rec.709 curve, which yCMS defaults to.
I once thought that too, but now I'm not so certain. It would be true if we see the footage directly from the camera without any manipulation, but instead we see video that passed over several steps and with several gamma corrections involved to adapt images between themselves. IMHO it would only remain true if the displays where the mastering is performed are calibrated with the BT.709 curve, and it's consensual that there is no certainty about that.:(
Personally, I also prefer to use a BT.709 curve, but I don't claim that's what it should be used. I think it might depend on how the mastering is performed...

Since you seem to think it should be set to a gamma of 2.35, I'm curious where where you believe the gamma is being adapted from Rec.709 to 2.35 gamma (which btw would make the video brighter then Rec.709)?
Read my previous point. The only place where that could happen is during the mastering process.

Edit: yesgrey, I do hope you plan to fix 'Gamma_Curve 1.0' because I could not get it to match the scaled Rec.709 curve I get with Argyll CMS as I mentioned in the yCMS thread (yCMS was turning my Rec.709 curve into nearly a power-curve)
First I need to know and understand what's wrong. I will take a look into it.

[cyberbeing]

No, it shouldn't.
That is correct, but you are forgetting the gamut correction...

The gamut correction has to be performed on linear light. So, yCMS removes the gamma using the input transfer function, applies the gamut correction, and then applies the output transfer function. When using different input transfer functions, the gamut correction gives different values, hence the different results at the end even when you're using the same functions for degamma/engamma.
What you said only is true if no gamut correction is applied.

I do agree that it shouldn't, but that's how yCMS has always behaved. Which would make it a yCMS bug if you now acknowledge it. Whenever I use the same values for Input_Transfer_Function and Output_Transfer_Function I get identical results no matter what values are used. When I brought this up to you during my gamma rantings during early yCMS (cr3dlut) revisions, you didn't seem to think anything was wrong, so I gave up. Hopefully it gets fixed now.

If you still believe it really is working correctly, the curve choice for the linearizing step must be extremely insignificant in affecting what the final gamut adaption turns out to be, since nothing appears to change at all when I flip between luts using different values.

I don't know where did you get the 6 decimal places idea. yCMS works and has always worked using 64 bit fp (double in C), so it can use up to 15 decimal places.
I must have gotten mixed up by something along the way. Probably how standardized color coordinates are usually not listed with more then 6 decimal places. Looking back it seems two years ago I actually asked you and you said cr3dlut gained no value with over 14 decimal places. Now you're saying 15, but isn't Double Precision FP actually 16 decimal places?


I once thought that too, but now I'm not so certain. It would be true if we see the footage directly from the camera without any manipulation, but instead we see video that passed over several steps and with several gamma corrections involved to adapt images between themselves. IMHO it would only remain true if the displays where the mastering is performed are calibrated with the BT.709 curve, and it's consensual that there is no certainty about that.:(
Personally, I also prefer to use a BT.709 curve, but I don't claim that's what it should be used. I think it might depend on how the mastering is performed...

Read my previous point. The only place where that could happen is during the mastering process.
Yes, it happening in mastering is the only correct answer, but I don't think it would. Just think about it logically. Video out of a HD camera begins with a native Rec.709 gamma. If this is displayed on a mastering monitor with 2.35 gamma, the video would appear darker then the camera sees it. Ever discussion I've seen on Rec.709 at least seems to agree that viewing Rec.709 on a display with a higher gamma is the correct way to view.

To make the video appear similar to the Rec.709 gamma on a 2.35 gamma monitor, the video gamma would need to be reduced (brightened) significantly making the video even more washed out and reducing contrast from it's previous ~1.94 gamma. In this case you are arguing that mastering is done on a 2.35 gamma monitor with a perceived native Rec.709 curve.

If instead the video was processed the opposite way by increasing (darkening) the gamma adapting the Rec.709 to an actually 2.35 gamma, it would only look correct on a monitor with a native Rec.709 gamma. On a 2.35 gamma monitor, the gamma of the monitor would compound with the gamma adjustment applied to the video and the result would be an even higher gamma then 2.35 when seen by your eyes. In this case you are arguing that mastering is done on a native Rec.709 curve monitor with a perceived 2.35 gamma.

My argument is that logically, neither adjustment would be done. The native Rec.709 video from the camera would be displayed on the mastering monitor as-is. At this point they would fine-tweak the gamma of the video so it looks good on the mastering monitor. Eventually this gets broadcast on HDTV or released on Blu-ray with the gamma tweaks the studio decided looked good when mastering. At this point no further gamma adjustment to the video is needed, unless your display has a different gamma then was used when mastering. If you assume mastering was done with a 2.35 gamma monitor, and you calibrate your monitor to 2.35, no further gamma adjustment is needed. If you assume mastering was done on a scaled Rec.709 curve monitor, and you calibrate your monitor to a scaled Rec.709 curve, no further gamma adjustment is needed. In other words, you never touch the gamma of the video, only attempt to match your display to the mastering monitor.

There is still a problem if you need to linearize the video though. If the gamma of the video was tweaked subjectively during mastering, there would be no way to know what strange curve the gamma was changed to. If they used different gamma tweaks for different scenes, you are really screwed if you desire perfect linerization for the entire movie. Yet, tweaks purposefully make things non-linear to improve visual effect, and since the Rec.709 should still be the the baseline for mastering tweaks, it is the safest choice to use as the assumed input transfer function.

IMHO it would only remain true if the displays where the mastering is performed are calibrated with the BT.709 curve, and it's consensual that there is no certainty about that
Going back to what you said, and taking what I said into account, I believe you have it reversed. Using a Rec.709 curve at input would only only remain true if the display where the mastering is performed are calibrated with ANY GAMMA CURVE, and there is no consensus as to what this mastering gamma curve actually is.

[yesgrey]

I do agree that it shouldn't, but that's how yCMS has always behaved. Which would make it a yCMS bug if you now acknowledge it. Whenever I use the same values for Input_Transfer_Function and Output_Transfer_Function I get identical results no matter what values are used. When I brought this up to you during my gamma rantings during early yCMS (cr3dlut) revisions, you didn't seem to think anything was wrong, so I gave up. Hopefully it gets fixed now.
I don't know what happened in all previous versions (and I won't lose time testing all of them), but I have yCMS working like that for some versions. I've confirmed with v1.9 and it's working as it should. The results are different, however, it might not be a day and night difference, but they are different.

If you still believe it really is working correctly, the curve choice for the linearizing step must be extremely insignificant in affecting what the final gamut adaption turns out to be, since nothing appears to change at all when I flip between luts using different values.
It's not extremely insignificant, but the effect is not that big. It would depend on the target gamut, and might be noticeable only on test patterns with the primaries colors. In real world images it might be difficult to see any difference, because usually they don't contain highly saturated colors.

I must have gotten mixed up by something along the way. Probably how standardized color coordinates are usually not listed with more then 6 decimal places. Looking back it seems two years ago I actually asked you and you said cr3dlut gained no value with over 14 decimal places. Now you're saying 15, but isn't Double Precision FP actually 16 decimal places?
Guaranteed are only 15, but in most situations we can consider it to be 16.

The native Rec.709 video from the camera would be displayed on the mastering monitor as-is. At this point they would fine-tweak the gamma of the video so it looks good on the mastering monitor. Eventually this gets broadcast on HDTV or released on Blu-ray with the gamma tweaks the studio decided looked good when mastering.
That's exactly my point. That gamma fine-tweak so the video would look good on the mastering monitor is a gamma change, to look good on the mastering monitor.
Ideally, for us to see the video at its best we should use a display like the mastering display and in the same viewing conditions. However, the gamma fine-tuning are such that we should get a good image even if our display's gamma is slightly different.

In other words, you never touch the gamma of the video, only attempt to match your display to the mastering monitor.
Yes, but for that you need to calibrate the display.

There is still a problem if you need to linearize the video though. If the gamma of the video was tweaked subjectively during mastering, there would be no way to know what strange curve the gamma was changed to. If they used different gamma tweaks for different scenes, you are really screwed if you desire perfect linerization for the entire movie.
Yes, that's what I was referring...

Yet, tweaks purposefully make things non-linear to improve visual effect, and since the Rec.709 should still be the the baseline for mastering tweaks, it is the safest choice to use as the assumed input transfer function.
I'm not so sure about that... but since we don't know the gamma of the mastering displays I think that it might be the safest option too. It's what I use with my displays.

Going back to what you said, and taking what I said into account, I believe you have it reversed. Using a Rec.709 curve at input would only only remain true if the display where the mastering is performed are calibrated with ANY GAMMA CURVE, and there is no consensus as to what this mastering gamma curve actually is.
If we by any chance are able to know which gamma is used by each studio, we could always set it accordingly, but in the end the difference would not be that significant, as we already saw.

[cyberbeing]

And Doom9 lives once again. When I was previewing this post days ago, Doom9 went down. Talk about poor timing.
It's not extremely insignificant, but the effect is not that big. It would depend on the target gamut, and might be noticeable only on test patterns with the primaries colors. In real world images it might be difficult to see any difference, because usually they don't contain highly saturated colors.
I did some measurements with Primary and Secondary colors, and it seems there is a very slight change.

It doesn't seem like a positive change though. All it resulted in doing was reducing the saturation of my primarys, and making my gamut even smaller.

Input Rec.709 / Output Rec.709
Green Primary matched nearly perfectly to Rec.709 Green.
Red Primary matched nearly perfectly to gamut triangle edge for Rec.709 Red.
Blue Primary left slightly outside gamut triangle edge Rec.709 Blue.

Input Gamma any gamma power-curve / Output any gamma power-curve
Green Primary shifted inside the gamut triangle for Rec.709 Green.
Red Primary shifted inside the gamut triangle for Rec.709 Red.
Blue Primary matched nearly perfectly to gamut triangle edge for Rec.709 Blue.

It seems there is a slight inaccuracy of yCMS's gamut correction for the blue primary. The good news it that it shouldn't be hard for you to fix it, because the blue primary is near perfect when using the a power curve for the input and output transfer function. If you can make the default blue gamut adjustment near-identical to the power curve blue gamut adjustment, everything would be perfect. Hopefully you can look into it and find the cause.

Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
45.070 24.592 2.3291
31.553 65.733 10.673
18.132 9.3445 94.922
95.069 100.00 108.94

In other words, you never touch the gamma of the video, only attempt to match your display to the mastering monitor.
Yes, but for that you need to calibrate the display.
Why wouldn't you be calibrating your display?

[yesgrey]

It seems there is a slight inaccuracy of yCMS's gamut correction for the blue primary.
To get accurate gamut correction you need to use the Grayscale_Measurements command and the Gamma_Curve command (in case you want to change gamma).

Why wouldn't you be calibrating your display?
I didn't say that. I was only remind you that yCMS does not directly calibrate the displays. It accomplishes the same effect by changing the video fed into the display.

[cyberbeing]

To get accurate gamut correction you need to use the Grayscale_Measurements command and the Gamma_Curve command (in case you want to change gamma)

How is that related to what were talking about though?

Recap:
I currently calibrated my display to gamma 2.35 and I use an input file like this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145
It results in almost perfectly clipped primaries and almost perfect TRC.
As I said previously, this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Input_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Output_Transfer_Function 1.0 0.0 0.425531914893617 0.0
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145

Should give identical results to this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
70.28325 31.64575 1.074425
24.9915 84.4635 11.7725
23.6385 8.894625 124.175
118.385 124.48 136.145
No, it shouldn't.
It's not extremely insignificant, but the effect is not that big. It would depend on the target gamut, and might be noticeable only on test patterns with the primaries colors. In real world images it might be difficult to see any difference, because usually they don't contain highly saturated colors.


This made me retest things with Input_Transfer_Function and Output_Transfer_Function. In my previous post, I tried something similar to janos666.

I compared this:
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
45.070 24.592 2.3291
31.553 65.733 10.673
18.132 9.3445 94.922
95.069 100.00 108.94
With this:
Input_Format HD YCbCr 8
Input_Transfer_Function 1.0 0.0 0.45 0.0
Output_Format HD RGB_PC 16
Output_Transfer_Function 1.0 0.0 0.45 0.0
Gamut_Measurements 2
45.070 24.592 2.3291
31.553 65.733 10.673
18.132 9.3445 94.922
95.069 100.00 108.94
With my display calibrated to a gamma of 2.222222. I also tried the same thing with 1.0 0.0 0.425531914893617 0.0 and my display calibrated to a 2.35 gamma. yCMS gamut adaption was similar with both. My gamut shrunk smaller then the Rec.709 target gamut. My blue primary improved compared to yCMS defaults. My red and green got worse compared to yCMS defaults.

Are you now saying that janos666 is not getting correct gamut mapping since he doesn't use Grayscale_Measurements? And not using Grayscale_Measurements also makes my only my blue primary gamut adaption slightly off, while my red and green are perfect when when using the default input/output Rec.709 transfer? It also explains why the blue primary becomes accurate with a input/output 2.22 or 2.35 gamma transfer function while the red and green become inaccurate?

Since Output_Transfer_Function also appears to override Grayscale_Measurements, I'm a bit confused about what you are trying to say now...

[cyberbeing]

Here are some images if you are having trouble visualizing what I'm saying:

Display calibrated to a scaled Rec.709 curve.
http://img826.imageshack.us/img826/4486/madvrshader.th.png (http://img826.imageshack.us/img826/4486/madvrshader.png)
madVR (no 3DLUT)
The natural gamut of my display.

http://img404.imageshack.us/img404/7761/madvrlutgamutonly.th.png (http://img404.imageshack.us/img404/7761/madvrlutgamutonly.png)
madVR w/ Gamut_Measurements only (3DLUT)
Green Primary is near-perfect.
Red Primary is near-perfect.
Blue Primary is slightly off.
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
45.3810 24.7930 2.43780
31.5980 65.8290 10.6740
18.4250 9.51970 95.8370
95.4140 100.000 109.570


http://img41.imageshack.us/img41/3619/madvrlutgrayscaleplusga.th.png (http://img41.imageshack.us/img41/3619/madvrlutgrayscaleplusga.png)
madVR w/ Gamut_Measurements & Grayscale_Measurements (3DLUT)
Green Primary is worse.
Red Primary is worse.
Blue Primary is near-perfect.
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Grayscale_Measurements
0 2 0.10906 0.10754 0.11061
2 2 0.17094 0.17641 0.20074
4 2 0.35221 0.37006 0.41660
6 2 0.52895 0.55748 0.61778
8 2 0.75904 0.80585 0.84971
10 2 1.00460 1.07580 1.12740
12 2 1.22820 1.29220 1.39620
14 2 1.54250 1.62140 1.74750
16 2 1.86260 1.96320 2.11350
18 2 2.27180 2.39630 2.57770
20 2 2.72680 2.87420 3.07660
22 2 3.13680 3.28990 3.63420
24 2 3.63030 3.82290 4.09240
26 2 4.20880 4.43360 4.72950
28 2 4.87120 5.11900 5.61990
30 2 5.74220 6.03650 6.60680
32 2 6.50620 6.83620 7.47740
34 2 7.28970 7.67120 8.38000
36 2 8.22180 8.64440 9.43350
38 2 9.13340 9.60470 10.4830
40 2 10.1540 10.6820 11.6470
42 2 11.4830 12.0820 13.1780
44 2 12.6370 13.2870 14.4930
46 2 13.8320 14.5600 15.8630
48 2 15.5190 16.3400 17.7820
50 2 17.2250 18.1360 19.7440
52 2 18.8490 19.9450 21.7910
54 2 20.5540 21.6420 23.5520
56 2 22.5600 23.7740 25.8920
58 2 24.2910 25.5830 27.8440
60 2 26.4670 27.8770 30.2970
62 2 28.2850 29.8230 32.4660
64 2 30.8040 32.4630 35.3340
66 2 33.0350 34.9240 38.1120
68 2 35.7200 37.6900 41.2460
70 2 39.0070 41.1910 45.1170
72 2 41.6630 43.8540 47.7320
74 2 44.4820 46.9340 50.3330
76 2 47.2110 49.6530 54.2580
78 2 50.4890 53.1770 58.1300
80 2 53.8440 56.7570 61.2240
82 2 57.4260 60.5120 65.2090
84 2 60.6250 63.5940 69.0740
86 2 64.0560 67.3280 73.5100
88 2 67.9940 71.5750 77.9210
90 2 72.9330 76.6310 83.4940
92 2 77.0860 81.0900 88.3690
94 2 81.0900 85.3150 93.0880
96 2 85.7530 89.9550 97.8520
98 2 90.5180 95.1400 103.980
100 2 95.4140 100.000 109.570
Gamut_Measurements 2
45.3810 24.7930 2.43780
31.5980 65.8290 10.6740
18.4250 9.51970 95.8370
95.4140 100.000 109.570

___________
Display calibrated to a 2.222222 power-curve.
http://img171.imageshack.us/img171/3852/72953635.th.png (http://img171.imageshack.us/img171/3852/72953635.png)
madVR (no 3DLUT)
The natural gamut of my display.

http://img35.imageshack.us/img35/9604/22709.th.png (http://img35.imageshack.us/img35/9604/22709.png)
madVR w/ Gamut_Measurements only (3DLUT)
Green Primary is near-perfect.
Red Primary is near-perfect.
Blue Primary is slightly off.
Input_Format HD YCbCr 8
Output_Format HD RGB_PC 16
Gamut_Measurements 2
45.070 24.592 2.3291
31.553 65.733 10.673
18.132 9.3445 94.922
95.069 100.00 108.94

http://img130.imageshack.us/img130/560/2222fp.th.png (http://img130.imageshack.us/img130/560/2222fp.png)
madVR w/ Gamut_Measurements, Input_Transfer_Function & Output_Transfer_Function 1.0 0.0 0.45 0.0 (3DLUT)
Green Primary is worse.
Red Primary is worse.
Blue Primary is near-perfect.
Input_Format HD YCbCr 8
Input_Transfer_Function 1.0 0.0 0.45 0.0
Output_Format HD RGB_PC 16
Output_Transfer_Function 1.0 0.0 0.45 0.0
Gamut_Measurements 2
45.070 24.592 2.3291
31.553 65.733 10.673
18.132 9.3445 94.922
95.069 100.00 108.94

[yesgrey]

How is that related to what were talking about though?
You were saying that you were not getting a good gamut correction with any of the other options, so I wanted to remind you how it's supposed to be.

With my display calibrated to a gamma of 2.222222. I also tried the same thing with 1.0 0.0 0.425531914893617 0.0 and my display calibrated to a 2.35 gamma. yCMS gamut adaption was similar with both.
OK, now I got it. You were using the Output_Transfer_Function considering that your display was calibrated to a perfect known curve. However, the calibration is never perfect, so the results might be not that accurate, hence why I always recommend using the Grayscale_Measurements command instead.

Are you now saying that janos666 is not getting correct gamut mapping since he doesn't use Grayscale_Measurements?
It would depend on how accurate is its display calibration to the curve he uses on output.

Here are some images if you are having trouble visualizing what I'm saying
...
The natural gamut of my display.
If that's the natural gamut of your display you can never get perfect red or blue. Your display lacks blue and red, so there's nothing you can do.

I can see now that you were saying that you got near-perfect red and blue when the measured points lay over the triangle lines, but if you compare them to the standard coordinates you can easily see that they're way off.

madVR w/ Gamut_Measurements & Grayscale_Measurements (3DLUT)
Green Primary is worse.
Red Primary is worse.
Blue Primary is near-perfect.

In fact, IMHO this was the best image of all. You seem to think the opposite because the green is more off than on the other graphs, but it might be the preferable, because due to your display lacking in red and blue the slightly reduction on green would give better results...

[cyberbeing]

In fact, IMHO this was the best image of all. You seem to think the opposite because the green is more off than on the other graphs, but it might be the preferable, because due to your display lacking in red and blue the slightly reduction on green would give better results...
You still seem to be missing what I've been trying to relay. yCMS seems to reduce my gamut for all three primaries at the same time. Whenever yCMS improves Blue, it makes Red and Green get worse. Using Grayscale_Measurements seems to give me the worse result of all. The Green has it's gamut reduced significantly and the Red has it's gamut reduced slightly which causes the Yellow secondary to become way off. Only the Blue is shifted where is should be, but the Red and Green aren't... Overall the results are indeed much worse. It's a 2-1 majority rules deal. When two out the three primaries (Red & Green) are doing the same thing, and the last (Blue) is doing something totally different, the odd man out (Blue) is seen as the one behaving incorrectly.

I seem to only have two options with yCMS.
Option 1: Red and Green clipped to gamut triangle edges. Blue left slightly outside gamut triangle, when it should be on the triangle edge.

Option 2: All primary colors have their gamut reduced. Red and Green are shifted inside the gamut triangle. Blue is shifted from slightly outside to the gamut triangle edge.

I can see now that you were saying that you got near-perfect red and blue when the measured points lay over the triangle lines, but if you compare them to the standard coordinates you can easily see that they're way off.

My red primary pretty much should be left untouched, as it already sits in the optimal location for my display gamut, clipped to the edge of the Rec.709 gamut triangle.

My green primary has a larger gamut then Rec.709, so it should be feasible to make it perfectly match Rec.709 green.

My blue primary should be clipped to the gamut triangle edge.

Since my Red & Blue gamuts are smaller than Rec.709 gamut, the optimal adaption is clipping to the Rec.709 gamut triangle edges. yCMS should be taking care of all other colors with something like Relative Colorimetric or Perceptual rendering.
__________

Can't you see the problem the Blue primary gamut adaption?

When the Red and Green primary are clipped to the gamut triangle edge, the Blue primary should also be clipped to the gamut triangle edge, but it's not. The Blue Primary is left outside the gamut triangle.

When the Red and Green primary have their gamut reduced and are shifted inside the gamut triangle, the Blue primary should also be shifted inside gamut triangle, but it's not. The Blue Primary is now being clipped to the gamut triangle edge.


yCMS Red Gamut adaption is accurate in relation to Green.
yCMS Green Gamut adaption is accurate in relation to Red.
yCMS Blue Gamut adaption is always slightly inaccurate in relationship to Red and Green.

Can't you analyze my settings and try to improve the yCMS Blue Gamut adaption at all? You would be hard to claim that the Blue Primary is being adapted optimally in my case.

Is it possible that it got messed up when you were fixing those yCMS bugs which were causing blocking of the Blue Primary?

I have no clue how you currently have yCMS gamut adaption coded, but it's feasible that if yCMS took Secondary color measurements into account, yCMS would be able to identify and correct problems such as this on it's own. After-all it's usually recommended to take hundreds to thousands of measurements into account in order to create a high quality 3D LUT. yCMS has a long way to go before it reaches that point.

[yesgrey]

Since my Red & Blue gamuts are smaller than Rec.709 gamut, the optimal adaption is clipping to the Rec.709 gamut triangle edges.
It's not as simple as that. A color gamut is not the triangle that we see in these graphs. These graphs only show a 2D projection of the color gamut over the x,y axis. The fact that your primaries seem to be right over the triangle edges does not mean that they are right, because the Y (which is not shown in the graphs) may not be correct.

I'm perfectly aware that yCMS has some limitations, and I want to overcome them on yCMS v2.x. However, I don't know yet when it will see the light...

[cyberbeing]

It's not as simple as that. A color gamut is not the triangle that we see in these graphs. These graphs only show a 2D projection of the color gamut over the x,y axis. The fact that your primaries seem to be right over the triangle edges does not mean that they are right, because the Y (which is not shown in the graphs) may not be correct.

Yes, but color correcting in a 3D space should not normally make some of the discrepancies I'm seeing in the 2D projection. Usually you would favor clipping to the target gamut edge over everything else. It's also usually preferable to have a 0.5% larger gamut then a 0.5% smaller gamut compared to the target gamut 2D projection, if you need to make a choice between the two. Lost colors from shrinking the gamut too small are lost forever.

yCMS seems capable of making a decent gamut adaption when given only color primaries and white point, but how it's handing other data may not optimal, as I saw.

Yet I'm still curious as to why yCMS always skews my blue primary slightly off target... Have you analyzed my measurements and found the cause? Is this really just a case of yCMS not having enough data available on how the display behaves and making a poor choice?

Also, speaking of 2D vs 3D, it's too bad we have no way to render the 3D Gamut produced by yCMS 3DLUTs and compare it to other gamuts like Rec.709 in a 3D space. It can't be easy to fine-tune and troubleshoot a 3D space without a visual representation of the results.

[darkbasic]

Hi, 3 years have passed and I did some more tests :)
With latest madvr/ycms versions and without grayscale measurements (my display is hardware calibrated to native gamut and gamma 2.4) the color management is simply perfect. I will not even upload a photoshop vs yCMS comparison image because they are so hard to distinguish that you probably will not notice any difference unless you have a professional monitor. Congratulations, such a great improvement :D