gamut conversions through Avisynth ?

[leeperry]

D65 is not required for that
movies are mastered in D65/2.2/SMPTE-C or EBU...but well, whatever floats your boat. D50 is yellowish and D75 blueish...it kinda defeats the whole point of gamut conversion, innit?

[yesgrey]

Thanks Yesgrey !!! :thanks:
You're welcome.:)
But you also should thank other people, like Charles Poynton, JohnAd, tritical and madshi (listed by chronological order), who helped me achieving the current state.
:thanks:

[yesgrey]

movies are mastered in D65/2.2/SMPTE-C or EBU...but well, whatever floats your boat. D50 is yellowish and D75 blueish...it kinda defeats the whole point of gamut conversion, innit?
It would be preferable D65, but the Chromatic_Adaptation is exactly for compensating that...

[pbmtp]

Hi yesgrey,

Did you find anything about my issue ? Is it something miss configured ?

[leeperry]

It would be preferable D65, but the Chromatic_Adaptation is exactly for compensating that...
ahhhhh your LUT can also compensate for D75>D65? cool stuff :devil:
miss configured
did she win any pageant :confused:

[yesgrey]

Did you find anything about my issue ?
Please send me by e-mail the .reg file. The copy paste thing is not working...

[yesgrey]

ahhhhh your LUT can also compensate for D75>D65? cool stuff :devil:
ddcc already did that. With cr3dlut I improved it slightly because not running in real time allowed me to use the full Bradford transform to perform the chromatic adaptation, instead of the linear version of it; but it would always be better to use a display calibrated for the same illuminant (white point) as the source.;)

[leeperry]

it would always be better to use a display calibrated for the same illuminant (white point) as the source.;)
less banding?

[yesgrey]

less banding?
More accurate colors.

[pbmtp]

yesgrey you've got mail :)

[leeperry]

More accurate colors.
compared to what? how to measure the colors "accuracy"? ColorHCFR will give the RGBW coordinates and saturations, that's it.

BTW, if you could somehow allow us to import the ColorHCFR saturations within cr3dlut, that'd be really awesome...that's the Achilles' heel of gamut conversion at this point IMHO.

http://img141.imageshack.us/img141/7501/satu.png

[darkbasic]

ColorHCFR will give the RGBW coordinates
Please explain me because I can't understand and so I don't know which coordinates I need. Let's make an example with AdobeRGB!

According to the documenation,the primaries are at:

Red x=0.6400, y=0.3300
Green x=0.2100, y=0.7100
Blue x=0.1500, y=0.0600
White x=0.3127, y=0.3290 (the reference white is D65)


If I dump the wtpt, rXYZ, gXYZ and bXYZ tags form the profile I obtain:

C:\Argyll_V1.0.4\bin>iccdump.exe -v3 -t wtpt -t rXYZ -t gXYZ -t bXYZ AdobeRGB1998.icc
XYZArray:
No. elements = 1
0: 0.950455, 1.000000, 1.089050 [Lab 100.000000, -2.387320, -19.404505]
XYZArray:
No. elements = 1
0: 0.609741, 0.311111, 0.019470 [Lab 62.601347, 90.371212, 78.149349]
XYZArray:
No. elements = 1
0: 0.205276, 0.625671, 0.060867 [Lab 83.214105, -129.089932, 87.172524]
XYZArray:
No. elements = 1
0: 0.149185, 0.063217, 0.744568 [Lab 30.210038, 69.243738, -113.612302]

The tristimulus values (XYZ) of the "white point" (0.950455, 1.000000, 1.089050) are in the "AdobeRGB white" source (so D65, If I had chosen my monitor profile instead of AdobeRGB would be ~ D65).

In fact if I convert it from XYZ to xyY I obtain 0,312701 0,329001 0,999999 which is the value from the documentation, of course.


Now let's see rXYZ, gXYZ, bXYZ.
They are the XYZ values of the R, G, B primaries and they form a 3x3 matrix which converts the normalized RBG values to XYZ.

If the white of the device is D50 they are exactly the colorimetric coordinates of the primaries. If not, the colorimetric coordinates had to be chromatic adaptated, in fact ICC specifications want relative colorimetry, so the matrix have to port the device white to the PCS white (D50).

For rXYZ we have 0.609741, 0.311111, 0.019470. If we do chromatic adaptation (from D50 to D65) and then we convert from XYZ to xyY we obtain 0,640014 0,330001 0,297363 which is the value from the documentation.

So, what are the RGBW coordinates I need? XYZ or xyY? Chromatic adaptated or not?
Can you make and example for AdobeRGB so I can use use the spectrophotometer (which is not comatible with ColorHCFR) instead of the Eye-One Display v2 (which is not so good with extreme gamuts)?

[yesgrey]

Please explain me because I can't understand and so I don't know which coordinates I need. Let's make an example with AdobeRGB!

According to the documenation,the primaries are at:

Red x=0.6400, y=0.3300
Green x=0.2100, y=0.7100
Blue x=0.1500, y=0.0600
White x=0.3127, y=0.3290 (the reference white is D65)

So, what are the RGBW coordinates I need? XYZ or xyY?
Use only xy from the xyY. The chromatic adaptation is not needed, you only need to put the coordinates of the white point, and cr3dlut will perform the chromatic adaptation.

The values you'll need are like the above values. If your display is AdobeRGB, you don't need to specify the RGBW colors, simply select option 4, cr3dlut already knows the standard colors. You only need to specify them when they are not accurate relative to any standard.

[yesgrey]

I used the following patterns in ts files which are adaptation of AVSHD for being use with a video player. Here is the link http://kvcd.net/downloads/MIRES_1080P_POUR_MPCHC_V3.rar The ones I used were 100 % Color (folder ColorHCFR Fields\100% Color).

That's the problem.
These files were not created with the BT.709 transfer function, but with an aproximation of it, hence the good results you get with the PS (which uses the same approximated formula) vs the 3DLUT file, which was created with the exact BT.709 transfer function.
Here is the file for cr3dlut that will give you the same results as the PS script:
# Settings for creating a 3D LUT for watching the following Video formats:
# Blu-ray, HD DVD, ATSC HD Broadcast, PAL HD Broadcast
# without any Display correction
# Includes YCbCr->RGB conversion using PC Levels (Black: 0 and White: 255)
# using advanced parameters instead of 'x_Video_Format'

# Do not mess up with these settings
Chromatic_Adaptation 3 # same as PS
Out_Of_Gamut_Clipping 0 # same as PS
Input_Bit_Depth 8
Input_YCbCr_Full_Range 0 # Y: 16-235; CbCr: 16-240
Input_Gamma 9 1.0 0.0 0.45 0.0 # same as PS
Output_Bit_Depth 8
Output_YCbCr_Matrix 0 # RGB
#Output_YCbCr_Full_Range 1 # Ignored
Output_RGB_Black_White 0 255 # RGB: 0-255

# You can change the following settings
Input_YCbCr_Matrix 1 # BT.709
Input_Primaries 0 # BT.709

Output_Primaries 9 0.675 0.324 0.289 0.711 0.142 0.052 0.313 0.332 # custom primaries for video proj
Output_Gamma 9 1.0 0.0 0.45 0.0 # same as PS

Remember that official video material should be mastered using the exact standard curves.

You can compare the results using a tool like colorpic, so you can avoid turning on your projector and all the calibration stuff.;)

[pbmtp]

Hi yesgrey,

thanks for taking time investigating the problem, i am glad everything is working as expected in cr3dlut, i will try your input file and compared it with pixel shader to validate when i have some time to launch "all the calibration stuff" :). And i will switch back to cr3dlut + t3dlut.

Do you know if mp4 version of AVSHD files (http://www.avsforum.com/avs-vb/showthread.php?t=948496 ) suffer from the same problem ? If so maybe you could report the issue to the AVS HD thread so that it can be improved.

[darkbasic]

According to the documenation,the primaries are at:

Red x=0.6400, y=0.3300
Green x=0.2100, y=0.7100
Blue x=0.1500, y=0.0600
White x=0.3127, y=0.3290 (the reference white is D65)

The values you'll need are like the above values.

This is a problem, because those values are referred to an observer adaptated to the device white, while the values stored in the ICC tags are referred to an observer adaptated to D50.

From the ICC tags I can see my monitor's Red is x=0,675968 y=0,313882 which is _ABOUT_ x=0,671004 y=0,312852 referred to an observer adaptated to the device white. I told _ABOUT_ because I found those values with this (http://www.brucelindbloom.com/index.html?ColorCalculator.html) calculator (click on "Calc" and then on "Chromatic Adaptation Calculator") which doesn't let me choose the temperature in Kelvin but only D50/65/... and my monitor temperature is not exactly D65, but is 6439,1K (as you can see from the white point coordinates x=0,313635 y=0,330961).

[yesgrey]

And i will switch back to cr3dlut + t3dlut.
Remember that some of the settings are only for you to be able to get the same results as the PS cript. After that, you should use the recomended settings.

If so maybe you could report the issue to the AVS HD thread so that it can be improved.
Yes, I will take a look into it. Apparently, in the issues list on the first post, they already noted that something is not correct...;)

[yesgrey]

which is _ABOUT_ x=0,671004 y=0,312852 referred to an observer adaptated to the device white.
... and my monitor temperature is not exactly D65, but is 6439,1K (as you can see from the white point coordinates x=0,313635 y=0,330961).

Maybe I can find some time to include a calculator into cr3dlut... Until then, try using those coordinates and D65 coordinates and see the results.

[darkbasic]

How can I check which is the color space of a movie (BT 709, EBU, SMPTE-C...)? Is there an utility to check it?

[leeperry]

oh that's the easy part :)

SD: BT.601
HD: BT.709

european/russian/brazilian movies = EBU
USA/ASIAN: SMPTE-C

it's all explained in the OP I think ;)

[yesgrey]

european/russian/brazilian movies = EBU
USA/ASIAN: SMPTE-C
That would be for SD. For HD it's supposed to be BT.709 primaries.

[leeperry]

That would be for SD. For HD it's supposed to be BT.709 primaries.
I couldn't more disagree...movies are not mastered w/ BT.709 primaries.

We've discussed it many times, it's been discussed many times on AVS too..if you check the OP, the french CEO of the ISF has a list on his website and no movie on bluray whatsoever is ever mastered w/ BT.709 primaries :o

the HDTV gamut is used for demos, but not for movies...well at least for mastering studios that still master their stuff on CRT(99% of them?)

the CARS hero will always remain dark orangey in the movie, way to go! :D

[yesgrey]

I couldn't less disagree...movies are not mastered w/ BT.709 primaries.
According to some people that work in the area they use their displays gamut corrected to BT.709. Nobody knows for sure, there is not a 100% agreement on that. I only stated what the standards say, but I accept perfectly that in some cases they do not gamut correct their displays.;)

[leeperry]

that's the aforementioned thread: Question : Only for those that are 6500K/D65/REC709 calibrated ... - AVS Forum (http://www.avsforum.com/avs-vb/showthread.php?t=1038602)

it matches exactly what the ISF and Joe Kane say, movies on bluray are mastered in SMPTE-C on CRT :/

they get their CRT recalibrated on a weekly basis, and it goes off as SMPTE-C encoded in BT.709..

[yesgrey]

These files were not created with the BT.709 transfer function, but with an aproximation of it, hence the good results you get with the PS (which uses the same approximated formula) vs the 3DLUT file, which was created with the exact BT.709 transfer function.
Do you know if mp4 version of AVSHD files (http://www.avsforum.com/avs-vb/showthread.php?t=948496 ) suffer from the same problem ? If so maybe you could report the issue to the AVS HD thread so that it can be improved.
pbmtp,
I have looked into the files and the way they were created and there is not anything wrong with them.
I've thought a little more about it, and I think that the problem is only at the output gamma setting.
The displays usually have a gamma function that mimics a pure power curve, and the BT.709 gamma curve is not exactly like that.
It seems to me that the bad results you are getting are due to using an output gamma curve that does not fit correctly your display's gamma curve.
So, I would suggest that you also run your tests with:
Input_Gamma 1 # BT.709 gamma encoding
Output_Gamma 9 1.0 0.0 0.45 0.0 # same as PS

Let me know if it worked...

[pbmtp]

Hi yesgrey,

I will do some test next week and tell you what happened.

[Kazuya]

movies are mastered in D65/2.2/SMPTE-C or EBU...but well, whatever floats your boat. D50 is yellowish and D75 blueish...it kinda defeats the whole point of gamut conversion, innit?

Sorry I was'nt notified !

I didn't say D65 is useless, I say it is not necessary to have a perfect gamut on RGB points.
D65 only puts YCM at the right place.

Pbmtp : in my opinion, you have a problem on your display.

[pbmtp]

kaz, what could it be as when using the pixel shader script the CIE of my Z3000 is very close to be spot on with BT.709 ?

anyway i will try what yesgrey suggested next week and see what happens.

[Kazuya]

I don't understand your question.

It would be preferable D65, but the Chromatic_Adaptation is exactly for compensating that...

Yes, I noticed it when I measured greyscale with KMP ! :D

Is there a way to avoid it if for some reason we don't want a temperature correction ?


Well, Yesgrey, I have an other question for your colorimetric skills ! :p

I made some color patterns at only 20 IRE.
And, I was wondering how should be the gamut with this colors.
I used a 20IRE white pattern too.

Should the gamut be exactly the same than the CIE ?
Or should it be equaly narrow ?
Or whatelse ?

This is the measure on my Z4 :

http://img694.imageshack.us/img694/4351/clip.png

But maybe the Eye one is not enough sensitive to measure correctly those dark patterns ?
(probably the better explanation for the yellow position)

[yesgrey]

Is there a way to avoid it if for some reason we don't want a temperature correction ?
If you want to avoid the chromatic adaptation, setting it to 0 will disable it.

I made some color patterns at only 20 IRE.
And, I was wondering how should be the gamut with this colors.
I used a 20IRE white pattern too.

Should the gamut be exactly the same than the CIE ?
Or should it be equaly narrow ?
Or whatelse ?
Yes, The gamut should be exactly the same.

But maybe the Eye one is not enough sensitive to measure correctly those dark patterns ?
That's a possibility, because the sensors are less accurate at the low IRE levels, but I already noticed that the gamut correction is not working good at very low levels. I'm currently working on it to solve the problem, but probably I will simple disable the gamut correction at low levels, because our sensitivity to colors is not very high at those levels, and the low bit depth of the source (8bit) is a problem when working at those levels...

[Kazuya]

Ok, thanks ! :)
I will do more tests at different levels.

[yesgrey]

Test also without any gamut correction, that way you can have an idea of the accuracy of the meter at lower levels...

[Kazuya]

Yes, I forgot to do it on my Z4 but I do it on my LCD display :

http://img204.imageshack.us/img204/3699/clip12.png

I think the gamut conversion is working properly, even at 20IRE.
I watched Knowing yesterday, it was awesome colorimetricaly speaking, and I watched tonight an episode of Gossip Girl (DVD) and it was totally incredible ! :o Even in dark scenes.

[yesgrey]

I think the gamut conversion is working properly, even at 20IRE.
The problem is not the gamut conversion not working properly at low levels, it's the handling of out of gamut values at low levels, that could cause severe banding. I notice it some times, at very low levels. I'm currently investigating it to find the cause and how to solve it.

I watched Knowing yesterday, it was awesome colorimetricaly speaking, and I watched tonight an episode of Gossip Girl (DVD) and it was totally incredible ! :o Even in dark scenes.
Yes, I know what you're talking. After we start watching at accurate colorimetry, it's very hard to go back again...;)

[Kazuya]

I didn't notice severe banding at all, maybe depends of display ?
I will take a better look.

Well, yesterday I make gamut correction on a VW60 Sony blackpearl, with 334hours lamps.
This is what I obtained with shaders on MPC-HC, all patterns in MPC, even greyscale :

http://img17.imageshack.us/img17/272/clip19.png

The most interesting part is the gamut with 20% colors : it is narrow, but it looks great ! :o
Can I conclude the eye one is not so bad in low level colors ?
And that there's big differences between displays, like I suspected ?

[yesgrey]

I didn't notice severe banding at all, maybe depends of display ? I will take a better look.
With the shaders method there is no problem, only with the 3DLUT method, and if you change the gamma.
A good place to notice the banding is the start of the movie "Batman: The Dark Knight". At the beginning, when the logo of DC Comics appears, it will show severe banding around it.

Can I conclude the eye one is not so bad in low level colors ?
And that there's big differences between displays, like I suspected ?
From the images you posted that seems to be the most likely...

[Kazuya]

The 3Dlut method but where ? In madVR ?

[yesgrey]

The 3Dlut method but where ? In madVR ?
In madVR or t3dlut, the results are the same, so it should be a problem in the 3DLUT. I'm working on it...

[leeperry]

SD: BT.601
HD: BT.709

european/russian/brazilian movies = EBU
USA/ASIAN: SMPTE-C
apparently you can add Australia to the EBU list: http://www.indietalk.com/showthread.php?t=4240

and China too I think?

[pbmtp]

Hi yesgrey,

I finally had some time to make the measurement you asked me, all configuration file and HCFR chc can be found at the following location:
http://pbmtp.free.fr/3dlut-z3000/
On my Z3000, the clear winner is HD_PC_Z3000_PS.txt (same configuration for cr3dlut/rgb3dlut as the pixel shader). The corresponding HCFR chc file has all the possible measurement possible done, you can see that CIE is perfect and saturation are really as good as they can be.

Damien

[yesgrey]

Hi,
Great!
Thanks for posting the results. It's good to see that our work give such good results.:)

I suggest you to use Chromatic_Adaptation 2 with the PS, it should give you more accurate colors. The difference you are getting between HD_PC_Z3000_PS and HD_PC_Z3000_2, should be due to the different input gamma curves.

[Kazuya]

What is the difference between chromatic adaptation 2 and 3 ?

[yesgrey]

What is the difference between chromatic adaptation 2 and 3 ?

chromatic adaptation 2 is the preferred because it causes less errors. I only added 3 to be able to get with cr3dlut the same correction like the pixel shader method. We should always use 2. 1 and 3 are only for testing purpose.

If you want a more detailed description, read this (http://infoscience.epfl.ch/record/34077/files/FinlaysonS00.pdf). It explains the three methods implemented in cr3dlut, and also describes a fourth that, maybe one day, I will also add...

[Kazuya]

Ah ok, I was in 3 for chromatic adaptation, and didn't see anything wrong.
But I will put it at 2 and will see !
Thanks.

[leeperry]

chromatic adaptation 2 is the preferred because it causes less errors. I only added 3 to be able to get with cr3dlut the same correction like the pixel shader method. We should always use 2. 1 and 3 are only for testing purpose.
but only "3" works if the destination gamut is smaller, otherwise the green gamma is burned to death..and most DLP's have a lack of green in the first place, so you get a very greenish picture basically.

I know you're working on it as there seems to be a glitch in some part of the code, but "3" is mandatory on my DLP...please don't drop it in the next version :)

[yesgrey]

Ah ok, I was in 3 for chromatic adaptation, and didn't see anything wrong.
When the difference between the source and the output white points coordinates is small you should not notice any difference, but since it will take exactly the same time when watching your movies, I suggest setting it to 2.
Remember that this is not a 100% sure thing, because if you have read the article I linked above, you could see that this is just the results of several tests, so it's possible that some people prefer other settings.

but only "3" works if the destination gamut is smaller, otherwise the green gamma is burned to death..and most DLP's have a lack of green in the first place, so you get a very greenish picture basically.
From what I have tested previously, I thought that your problem was more due to the out of gamut clipping, than to the chromatic adaptation... Also, since you're color blind, I don't think that you would fit in the profile of the people used for when the chromatic adaptation algorythms were tested... but it's good to have you onboard, so we can make this work to all people.:)
Also, don't worry, usually I don't drop any features.;)

[leeperry]

ahhh don't give me the colorblind stuff again :D

I'm sure you can easily see that we got a problem w/ the green gamma in the sixth screenshot, and some of that code is in cr3dlut: http://forum.doom9.org/showpost.php?p=1292804&postcount=444

but indeed, it's got to do w/ the OOG...but the only way we could get the exact same colors as the PS script was through "Chromatic_Adaptation=3".

anyway, we already spoke about all this...the average guy will recalibrate his display every few weeks, and we don't use highly accurate spectrophotometers...and projectors MH lamps have fluctuating R/G/B levels, so all that stuff is slighty overkill IMVHO.

when you recalibrate on a weekly basis w/ a Minolta CS200 sensor(that's factory recalibrated every few months)...then yes, we can talk about this level of accuracy...but quite frankly ddcc and the PS script were going different ways(ddcc being more accurate), but even on test patterns screenshots you'd have a hard time identifying them in a DBT and tell whichever one is better(especially w/ a 16-235 SMPTE-C source).

[FoLLgoTT]

@yesgrey3
The primaries are defined by three coordinates in color space (usually xyY). Sometimes it is not sufficient to correct only x and y. It would be very cool if cr3dlut would support the luminance Y in the future.

[yesgrey]

The primaries are defined by three coordinates in color space (usually xyY).
Yes, I've already read that in several other places. I am considering looking into that so I could try to add it in a future version...
Thanks for the tip anyway.:)

[FoLLgoTT]

Yes, I've already read that in several other places. I am considering looking into that so I could try to add it in a future version...
Thanks for the tip anyway.:)

Great news. :)

Btw. a complete CMS with independent controls for all primaries and secondaries in RGB color space would be a killer application for the HTPC. Only the $5000 Lumagen Radiance (http://www.lumagen.com/docs/Tip0002_GamutCalibration.pdf) has implemented such a CMS. But now I'm only dreaming a bit. It is great what you have achieved with cr3dlut! :)