Pixel Shaders for chromatic expansion

[nand chan]

I've developed this three HLSL shaders mainly for personal use, to replace the ChromaticScale() function that I previously used with .3dluts (in madVR).

[RGB] Gamma RGB to XYZ (BT709)
sampler s0;
#define M float3x3(0.4124564, 0.3575761, 0.1804375, 0.2126729, 0.7151522, 0.0721750, 0.0193339, 0.1191920, 0.9503041)

float4 main(float2 tex : TEXCOORD) : COLOR
{
float4 s1 = mul(M, pow(tex2D(s0, tex), 2.4)).xyzz;
s1.w = max(s1.x, max(s1.y, s1.z)); // determine the brightest component

if (s1.w > 1)
s1.xyz /= s1.w; // stretch all components to fit into the range
s1.w /= 2; // cut the brightest info in half so it fits as well

return s1;
}

[XYZ] Chromatic Expansion (D65)
#define amount 1.2

#define f(t) (t>pow(6/29.,3)?pow(t,1/3.):pow(29/6.,2)/3*t+4/29.)
#define f_(t) (t>6/29.?pow(t,3):pow(6/29.,2)*3*(t-4/29.))
sampler s0;

float4 main(float2 tex : TEXCOORD) : COLOR
{
float4 col = tex2D(s0, tex);
col.w *= 2;
if (col.w > 1) col.xyz *= col.w; // get our brightness back

col.xyz = float3(116*f(col.y)-16,500*(f(col.x/0.95047)-f(col.y)),200*(f(col.y)-f(col.z/1.08883)));
col.yz = float2(sqrt(col.y*col.y+col.z*col.z)*amount,atan2(col.z,col.y));
col.xyz = float3((col.x+16)/116,cos(col.z)*col.y,sin(col.z)*col.y);
col.yzx = float3(f_(col.x),1.08883*f_(col.x-col.z/200),0.95047*f_(col.x+col.y/500));

col.w = max(col.x, max(col.y, col.z));
if (col.w > 1) col.xyz /= col.w;
col.w /= 2;
return col;
}

[XYZ] XYZ to Linear RGB (BT709)
sampler s0;
#define M_ float3x3 (3.2404542, -1.5371385, -0.4985314, -0.9692660, 1.8760108, 0.0415560, 0.0556434, -0.2040259, 1.0572252)

float4 main(float2 tex : TEXCOORD) : COLOR
{
float4 s1 = tex2D(s0, tex);
s1.w *= 2; // get the brightest info back

if (s1.w > 1) // if anything exceeded 1 originally
s1.xyz *= s1.w; // get back our old (WTW) values

return mul(M_, s1).xyzz;
}

Recommended order:

Pre-resize:
1. [RGB] Gamma RGB to XYZ (BT709)

Post-resize:
1. [XYZ] Chromatic Expansion (D65)
2. [XYZ] XYZ to Linear RGB (BT709)

Usage notes:

To function correctly, right click -> Renderer Settings -> Presentation -> Check “Disable RGB Gamma Linearization”. To adjust the chromaticity scaling factor, change the “amount”. Values between 1.0 and 1.3 are recommended. Higher = more chromatic. 1.0 means disabled.

You can still use the other two scripts without the chromatic expander in order to scale in XYZ space (and thus linear light), which will produce better quality results to scaling in gamma-encoded space.

I do not recommend using these scripts with 8-bit surfaces. Scripts should be compiled with ps_3_0.

[alph@]

Usage notes:

To function correctly, right click -> Renderer Settings -> Presentation -> Check “Disable RGB Gamma Linearization”.
I want this option “Disable RGB Gamma Linearization” in Mpc-HC ,yes?

[nand chan]

Usage notes:

To function correctly, right click -> Renderer Settings -> Presentation -> Check “Disable RGB Gamma Linearization”.
I want this option “Disable RGB Gamma Linearization” in Mpc-HC ,yes?

Yeah, it's an option in MPC-HC. I don't know if the normal builds have it as well, but JanWillem32's tester builds definitely do (they also have options for using 16 bit floating point surfaces instead of 8-bit, which I highly recommend for using this script as you will get a lot of banding otherwise).

I don't exactly know how EVR-CP linearizes gamma or I would reverse the effect so you don't need that option. Unfortunately all of my attempts seem to produce incorrect results, so we'll just have to disable the option for now.

[alph@]

yes this option it's only present in the last build of Janwillem (1.5.3.3755) now i test your shader.nand chan,you use the color management-little cms in MPC-HC of Jan willem ?,i have a display calibrate with dipscal,is there an interest in using the color management option?

[nand chan]

yes this option it's only present in the last build of Janwillem (1.5.3.3755) now i test your shader.nand chan,you use the color management-little cms in MPC-HC of Jan willem ?,i have a display calibrate with dipscal,is there an interest in using the color management option?

Yes! It's a brilliant option and works perfectly, even replacing madVR + .3dluts. Just make sure that your display's ICC profile is loaded in Windows. The “Little CMS” option will calibrate against whatever ICC profile you have.

You can also change the “Lookup” option to “256³ points”, that will be the same as an 8-bit .3dlut (in madVR), or change it to “128³ points” which will be the same as a 7-bit .3dlut.

I also suggest changing the “Ambient Light” to 2.4 (Dark), and “Rendering Intent” to “Absolute Colorimetric”, if you're viewing in fullscreen (use relative colorimetric otherwise).

[janos666]

You can also change the “Lookup” option to “256³ points”, that will be the same as an 8-bit .3dlut (in madVR), or change it to “128³ points” which will be the same as a 7-bit .3dlut.

Not entirely, the 3DLUT in MPC-HC uses float16 or float32 values (according to your surface format settings) and madVR uses integer16 values.
Not if it would change anything. There should be zero noticeable differences between these three formats, the limiting factor is the precision of lcms or yCMS (which is also predestined by the quality of your profile / input data), and of course the final dithering step.


I remember that we talked about it with JanWillem if the whole rendering chain should be set to work with XYZ color values. In this case, the CMS code should be rewritten to work directly with the YXZ color values. It currently needs RGB input and it produces RGB output. XYZ input/output would be more optimal for lcms.

But Jan found better things to do and I entirely abandoned the renderer because my current plasma display can't be calibrated with 3DLUTs (the undebatable dynamic gamma and brightness adjustments of the panel driving scheme make the 3D corrections useless, so not only I wouldn't be able to enjoy the benefits of my own work but I wouldn't be able to test if it works correctly...).
And I wasn't happy with the strictly expanded-range RGB output of the mixer, so I am using madVR with limited range RGB output now.

[nand chan]

Not entirely, the 3DLUT in MPC-HC uses float16 or float32 values (according to your surface format settings) and madVR uses integer16 values.
Not if it would change anything. There should be zero noticeable differences between these three formats, the limiting factor is the precision of lcms or yCMS (which is also predestined by the quality of your profile / input data), and of course the final dithering step.

LittleCMS can work with 16/32/64 bit floating point values if I'm not mistaken. Speaking of which, how does this even work? I thought LittleCMS only operates on textures in the RAM? So does EVR-CP copy the texture to VRAM, process it, copy it back to main memory, color correct it with LittleCMS and then render it? Or can LittleCMS operate on textures in VRAM directly?

Either way, EVR-CP can also output 10 bit values directly which should limit the amount of dithering needed, compared with madVR and its .3dluts.

Whether the .3dlut itself uses float16 or int16 doesn't really make any difference because madVR interpolates them on the GPU hardware either way with floating point precision iirc.

[janos666]

MPC-HC uses lcms to construct a volume texture in the VRAM (it's currently a RGB->RGB 3DLUT, like the recent yCMS version) and a shader interpolates between the available values (the 3D grid position coordinates mean the input, the texture color values mean the output values, the Alpha values of the texture are zero.)

I am not sure (I will make some tests and edit this post) but I think I have some problems with the quality of the EVR output using the latest MPC-HC and AMD driver versions (I think it is an AMD driver bug or "behind-your-back feature").

EDIT: Oh god, I can't even use EVR-CP with my current AMD driver since I disabled the "image improvement" features in CCC (application crash with either DXVA or software decoders). :(

[nand chan]

MPC-HC uses lcms to construct a volume texture in the VRAM (it's currently a RGB->RGB 3DLUT, like the recent yCMS version) and a shader interpolates between the available values (the 3D grid position coordinates mean the input, the texture color values mean the output values, the Alpha values of the texture are zero.)

Oh okay, that explains it.

I am not sure (I will make some tests and edit this post) but I think I have some problems with the quality of the EVR output using the latest MPC-HC and AMD driver versions (I think it is an AMD driver bug or "behind-your-back feature").

Do report back, I'd be interested to see for myself. Is it related to chroma upscaling?

[janos666]

I updated to the latest MPC-HC build by JanWillem and I reseted and reapplied my MPC-HC and CCC settings. MPC-HC is unstable but it seems to works now (it often crashes and I have to fight with the GUI to get a correct picture size).

The obvious difference is that madVR preserves the BTB and WTW ranges. BTB means nothing for me (I won't search for BTB content in each movies with high brightness settings and individually calibrate my brightness setting for them...) and WTW isn't important either but it doesn't hurt if your configuration allows WTW shades (videos shouldn't use them but if they do and it's not a problem for your display to show them in these rare cases, then why not preserve them...?).

Other than that, the 8-bit rounded results (I disabled the dithering for both video renderers) in the 16-235 range seems to be identical and correct... for gray colors.
However, primary colors show a significant deviation. I am not sure if it caused by the different chroma resample algorithms or the PC->TV range conversion shader in MPC-HC. (I think I should check this pattern again by setting up both renderers to use bilinear resize.)

http://desmond.imageshack.us/Himg189/scaled.php?server=189&filename=madvrvsevr.png&res=medium (http://img189.imageshack.us/img189/4966/madvrvsevr.png)

http://desmond.imageshack.us/Himg580/scaled.php?server=580&filename=madvrvsevrcolor.png&res=medium (http://img580.imageshack.us/img580/3082/madvrvsevrcolor.png)

(The upper half was made with madVR, the lower with EVR. Click for full resolution.)


EDIT: Ok, here is the color clipping pattern with bilinear chroma resize. The differences are still there and the winner is madVR again.

I guess the problem is with Jan's PC->TV shader. I never understood why he does this conversion on the way he does. May be because his method is incorrect.

Or some unwanted picture processing is going on in the background (in the EVR mixer, caused by the VGA driver).

http://desmond.imageshack.us/Himg542/scaled.php?server=542&filename=madvrvsevrcolor2.png&res=medium (http://img542.imageshack.us/img542/611/madvrvsevrcolor2.png)

[nand chan]

I'm thinking it might be related to the CMS settings. Do you have black point compensation etc. turned on? If you're using LittleCMS and you don't, that might explain it. Either way, how are you getting those test patterns? I want to confirm it myself.

I did do some testing with some test patterns I have, and I got 1:1 the same color values as I get in madVR, and the gradients worked perfectly for me as well. In particular, when in sRGB mode in my monitor, the primary colors did in fact result in the correct 255/0/0 and so forth.

[janos666]

I'm thinking it might be related to the CMS settings. Do you have black point compensation etc. turned on? If you're using LittleCMS and you don't, that might explain it. Either way, how are you getting those test patterns? I want to confirm it myself.

I did do some testing with some test patterns I have, and I got 1:1 the same color values as I get in madVR, and the gradients worked perfectly for me as well. In particular, when in sRGB mode in my monitor, the primary colors did in fact result in the correct 255/0/0 and so forth.

I didn't use CMS (or any other custom filters) in either cases.

These are from the AVSHD709 test disc (Misc / Additional section).

If you have correct results with Full Range RGB output then it must be Jan's PC->TV shader.
I used TV levels because:
1: I wish to output TV levels anyway.
2: I disabled the dithering (for easier and more correct numerical comparison - color picker in PhotoShop) and the rounding errors are awful if you don't use dithering after a level expansion (TV->PC).

[nand chan]

I didn't use CMS (or any other custom filters) in either cases.

These are from the AVSHD709 test disc (Misc / Additional section).

If you have correct results with Full Range RGB output then it must be Jan's PC->TV shader.
I used TV levels because:
1: I wish to output TV levels anyway.
2: I disabled the dithering (for easier and more correct numerical comparison - color picker in PhotoShop) and the rounding errors are awful if you don't use dithering after a level expansion (TV->PC).

I must mention that I did notice some issues with WTW myself, not with TV levels but with XYZ mode scaling - whenever either of the X/Y/Z exceeds 1, it seems to get cut off by EVR-CP.

The problem here is apparently not the shader itself (I get full WTW/BTB inside the shader itself), but it seemingly gets lost in the intermediate step /between/ shaders. I'll experiment some bit and see if I can use the unused alpha channel to pass scaling information for WTW/BTB content for my XYZ shaders, so those get correct behavior.

A demonstration of what I'm talking about can be seen here: http://screenshotcomparison.com/comparison/87980

In the EVR-CP version, the values very close to absolute white are a bit more yellow than they're supposed to be because the Z channel gets cut off at 1, and for D65 white the Z channel goes above 1 normally.

However, if I turn off my XYZ shaders, I get the exact same result as madVR. This confirms that the issue is not EVR-CP itself, but EVR-CP's handling of TV levels. Have you tested the “convert to limited color ranges” option in EVR-CP? Maybe that gives you favorable results.

[janos666]

I did use the "Convert to Limited Color Ranges" option in EVR. I guess it's the same shader what we can find in Jan's pixel shader pack (which converts the colors back to YCC first).

Could you please upload your screenshots (either edited like mine or EVR and madVR separately ; in a lossless picture format)?
(Disable any CMS or custom filters and use TV levels and bilinear chroma resizers with both renderers.)


I think the clipping outside the [0;1] range is normal. (I think some problems can occur in some rendering stages if you don't clip.)


By the way... Did you actually confirmed if the 10-bit output with EVR-CP exclusive mode is real 10-bit or it's just a dummy dithering inside the VGA card back to 8-bit output?
My TV refuses to tell anything about the actual input bit depth, I could however measure 10-bit effective precision with confirmed 8-bit output through single link DVI (which means the VGA card is capable to do some dithering...).

The card can output 10-bit on DisplayPort for sure (I could confirm it with my old semi-professional LCD) but I have no clue about the actual formats on HDMI.

[nand chan]

I'll experiment some bit and see if I can use the unused alpha channel to pass scaling information for WTW/BTB content for my XYZ shaders, so those get correct behavior.

I've added this for my XYZ->RGB and RGB->XYZ shaders (in the original post) and it seems to work fine. I haven't yet implemented it for the XYZ chromatic expand yet though, going to do that next.

I still want to test a few things, because I have a theory about what may cause it.

Edit: Updated it for the chromatic expand script as well.
Edit 2: My theory is correct, apparently it's the Resizer step that strips WTW/BTB information, not the individual filters.

[janos666]

Edit 2: My theory is correct, apparently it's the Resizer step that strips WTW/BTB information, not the individual filters.

Which one, chroma sampling or full screen RGB resize?

The test video is 1080p (luma res) and it didn't really make a difference (at least in the single channel color values in the middle of the flashing bars) when I disabled the chroma resize in MPC-HC. The near-235 primary colors are too bright with EVR-CP.

http://desmond.imageshack.us/Himg163/scaled.php?server=163&filename=madvrvsevrcolor3.png&res=medium (http://img163.imageshack.us/img163/1737/madvrvsevrcolor3.png)

MadVR always gives 235, EVR gives something random between 236-240
I think it's clearly a fault of Jan's PC->TV shader. I think he did it intentionally but it's wrong!


The BTB and WTW clipping starts in the EVR mixer. It outputs PC levels and thus it's already clipped, there is no way to get that information back. And there are some rendering steps which clips the values outside the [0;1] range.

I don't expect the EVR renderer to show me BTB and WTW, I expect it to give me R=235 G=B=16 for the 100% stimulus Red shade (or +/-1 in case of rounding back to 8-bit, but not 240...)

[nand chan]

Could you please upload your screenshots (either edited like mine or EVR and madVR separately ; in a lossless picture format)?
(Disable any CMS or custom filters and use TV levels and bilinear chroma resizers with both renderers.)

Done. All CMS/filters disabled, TV levels selected for both, and all scalers set to bilinear. EVR-CP is using 16 bit floating point textures, and madVR is using its internal 16 bit processing. Dithering disabled on both:

http://screenshotcomparison.com/comparison/87994

The main differences I see is that madVR's blue pattern is a tiny bit different. I should also note that this was a 10-bit source, which means that one got dithered by LAV Video (for EVR-CP) and the other got accepted directly by madVR.

By the way, I noticed some significant issues when using the integer surfaces (either 8 bit or 10 bit) in EVR-CP.

I think the clipping outside the [0;1] range is normal. (I think some problems can occur in some rendering stages if you don't clip.)

What exactly is the problem then, again?

By the way... Did you actually confirmed if the 10-bit output with EVR-CP exclusive mode is real 10-bit or it's just a dummy dithering inside the VGA card back to 8-bit output?

No, I have not tested it yet because I cannot get D3D to work with 10 bit output yet. (Only OpenGL)

My TV refuses to tell anything about the actual input bit depth, I could however measure 10-bit effective precision with confirmed 8-bit output through single link DVI (which means the VGA card is capable to do some dithering...).

Well yeah, single link DVI is only capable of 8 bits.

The card can output 10-bit on DisplayPort for sure (I could confirm it with my old semi-professional LCD) but I have no clue about the actual formats on HDMI.

If you still have your old LCD why not use it to confirm?

The test video is 1080p (luma res) and it didn't really make a difference (at least in the single channel color values in the middle of the flashing bars) when I disabled the chroma resize in MPC-HC. The near-235 primary colors are too bright with EVR-CP.

I was talking about the step between “pre-resize shaders” and “post-resize shaders”.

If I put something like “return input * 10;” into pre-resize, and “return input / 10” into post-resize, I end up with most of the image lost. However, if I put those filters immediately next to each other (in the same shader group), it works fine. This is the issue that caused my XYZ greater-than-1 info to be lost.

[janos666]

http://screenshotcomparison.com/comparison/87994

The main differences I see is that madVR's blue pattern is a tiny bit different. I should also note that this was a 10-bit source, which means that one got dithered by LAV Video (for EVR-CP) and the other got accepted directly by madVR.

I would be happier with 8-bit sources. You can't use a color picker for numerical comparison on dithered images.
But thank you anyway, and never mind, I am sure that it's a problem with Jan's shader. I will talk with him.

What exactly is the problem then, again?

The primary colors have higher luminance than they should (aka "gamut luminance error").

If you still have your old LCD why not use it to confirm?

1: I sold it already.
2: It was only indicated with DisplayPort connection. My current TV doesn't have DP, but HDMI only.

[janos666]

This is the problem (the manually added text on the colored bars shows the actual values, the text on the white background shows the reference values):

http://desmond.imageshack.us/Himg853/scaled.php?server=853&filename=evrcolor4.png&res=medium (http://img853.imageshack.us/img853/7863/evrcolor4.png)

Jan has some kind of theory about this PC->TV conversion which was always strange for me, but now I am sure it's wrong.
For some reason he forces the primary colors to clip at 240 because the Cb and Cr range is [16;240] but here we have constant chromaticity with different luminance levels, and the luma range is [0,235].

His PC>TV conversion shader is incorrect.

This was the reason why I was unable to calibrate the CMS of my display when I loaded up the patterns with MPC-HC. I always ended up with huge gamut luminance errors at any stimulus levels except the one which I used for the calibration.

[JanWillem32]

This thread is certainly interesting.

I was already planning to adopt the XYZ color space for working surfaces (actually xyY, but that's a minor difference).
The XYZ color space has already been used for studio editing and distribution of digital video for quite a while. (Note that the industry uses the 2° standard observer, and I intend to do so, too.)
For operating on binary color data, xyY is preferred. The XYZ color space requires X+Y+Z = 1, except for black; X = Y = Z = 0. For xyY, the default channel order is more favorable for processing and doesn't require normalization to X+Y+Z = 1 or X = Y = Z = 0 after every filtering step. The conversion step does need a check to prevent division by zero.
About “Disable RGB Gamma Linearization”, the function merely activates a 2.4-power function on the input video and subtitles (and is inverted in one of the phases in the final pass section). Note that enabling this function grays out the chroma up-sampling fix function (the pixel shaders for this can be inserted manually, I do so, too). I can upload any part or all of the working code on request to have look at the functions.
I'll take a look at the HLSL code here. Once I have some time to spare, I'll read the assembly output from the compiler and analyze it.

About dithering from the display devices drivers, it's pretty limited as far as I know. Look up "D3DRS_DITHERENABLE" for reference. I can test if it's capable of dithering to formats better than R5G6B6, but I doubt it. Dithering operations on the front buffer would be entirely new to me. The hardware part that normally reads the front buffer is generally a pretty weak processor, compared to the shadercores that write to the buffers.

The [0, 1] clipping issue was caused by the nearest neighbor resizer (a driver "optimization" issue, I guess). I've put the nearest neighbor resizer in a pixel shader and it was solved (will be included for the next release). Note that when the video input resolution is the same as the output resolution, the renderer automatically uses nearest neighbor. There are similar rules for resizing only the width or the height, too.

The "Convert to Limited Color Ranges" item is a tricky thing. A function like this can either convert to [16, 235], [16, 240], [16, 240] in a Y'CbCr matrixed form or do the same as the old integrated pixel shader that converts to [16, 235], [16, 235], [16, 235] R'G'B'. The latter will generally produce a red/green and green/blue color shift when the display device's processor expects limited range Y'CbCr input levels. (My projector has such a mode, I don't use it, though.)
janos666, how does the old integrated pixel shader fare with 8-bit surfaces in your case? I can add a mode that does the same, but it seems odd to me that any device would need levels that were re-compressed in a R'G'B' stage.

[nand chan]

This thread is certainly interesting.

I was already planning to adopt the XYZ color space for working surfaces (actually xyY, but that's a minor difference).
The XYZ color space has already been used for studio editing and distribution of digital video for quite a while. (Note that the industry uses the 2° standard observer, and I intend to do so, too.)

I used this color space because of ease of conversion to/from L*Ch space.

For operating on binary color data, xyY is preferred. The XYZ color space requires X+Y+Z = 1, except for black; X = Y = Z = 0. For xyY, the default channel order is more favorable for processing and doesn't require normalization to X+Y+Z = 1 or X = Y = Z = 0 after every filtering step. The conversion step does need a check to prevent division by zero.

This is wrong, the XYZ color space does not require X+Y+Z to be 1. This requirement is only needed for the spectral primaries of a color space. However, when processing actual color information, this requirement is neither met nor needed (eg. a dark gray will have X+Y+Z ≪ 1). The practical result of having X+Y+Z>1 would simply be WTW information at some points, which may or may not be lost during color management. LittleCMS, for example, will automatically de-saturate all out of gamut colors to lie on the gamut border, same for WTW/BTB information.

I will eventually have to write pixel shaders for this desaturation process, to make sure colors outside the gamut boundaries are desaturated instead of clamped (which would shift the hue).

About “Disable RGB Gamma Linearization”, the function merely activates a 2.4-power function on the input video and subtitles (and is inverted in one of the phases in the final pass section). Note that enabling this function grays out the chroma up-sampling fix function (the pixel shaders for this can be inserted manually, I do so, too). I can upload any part or all of the working code on request to have look at the functions.
I'll take a look at the HLSL code here. Once I have some time to spare, I'll read the assembly output from the compiler and analyze it.

Right, so the gamma linearization option applies a 2.4 PPC to the input it gets, post-resize, correct? It just seems strange then, that applying a 1/2.4 PPC to my info in the post-resize shader (and leaving the option unchecked) produced a different result from removing the 1/2.4 PPC and enabling the option.

The [0, 1] clipping issue was caused by the nearest neighbor resizer (a driver "optimization" issue, I guess). I've put the nearest neighbor resizer in a pixel shader and it was solved (will be included for the next release). Note that when the video input resolution is the same as the output resolution, the renderer automatically uses nearest neighbor. There are similar rules for resizing only the width or the height, too.

Okay, that will solve this issue then.