out of gamut fix

[kolak]

Is it possible to use avisynth as out of gamut filter?
Some YUV signals produce illegal RGB values once converted over Rec.601/Rec.709 inverse matrixes. Could avisynth help?

More about the problem:

http://www.vidcheck.com/support/white-papers.php

[colours]

Slightly inefficient, but one way you can check for YUV exceeding the gamut is to first reduce the range then convert to RGB as usual, after which we can apply some sort of fancy soft limiting for out-of-gamut values. Maybe something like this (entirely untested):

source
Dither_lut8(expr="x 128 - 3 / 128 + 256 *",yexpr="x 125.5 - 3 / 125.5 + 256 *",u=3,v=3)
# replace 128 and 125.5 with 127.5 if working with a full-range source
Dither_convert_yuv_to_rgb(matrix="709",lsb_in=true,output="rgb48y")
r = SelectEvery(3,0)
g = SelectEvery(3,1)
b = SelectEvery(3,2)
undershoot = cl_exprxyz(r,g,b,"21760 x y z min min -",lsb=true).SelectEvery(1,0,0,0)
overshoot = cl_exprxyz(r,g,b,"x y z max max 43520 -",lsb=true).SelectEvery(1,0,0,0)
rgb_fixed = cl_exprxyz(last,undershoot,overshoot,"x 21760 y - - 21760 y z + + / 65280 *",lsb=true)
# the above is not exactly "fancy" soft limiting, but derp.
rgb_fixed.DitherPost(mode=6)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))


Edit: now actually tested; forgot the lsb_in=true earlier.

[kolak]

Will try, thx.

So whole difficultly is to have fancy soft clipping method?

I assume first step (or maybe not necessarily 1st) is to limit luma and chroma to YUV values, which is also part of the whole legalisation process.

Another thing, which seams to be easier is to report out of gamut frames. This has some threasholds, described in EBU R103 standard. This has details:

https://www.google.co.uk/url?sa=t&source=web&rct=j&ei=KtjuVJ6hLY7fPYiKgNAE&url=http://dpp-assets.s3.amazonaws.com/wp-content/uploads/2014/03/DPP_Quality_Control_Requirements_V1.0.pdf&ved=0CDMQFjAG&usg=AFQjCNElNljJe1yacCG5RsggTMGEJGdqHw&sig2=LAgZ5tJeKyKP5DGXFzNgFA

They mention low pass filtering, before checking, is this possible?

[kolak]

Slightly inefficient, but one way you can check for YUV exceeding the gamut is to first reduce the range then convert to RGB as usual, after which we can apply some sort of fancy soft limiting for out-of-gamut values. Maybe something like this (entirely untested):

source
Dither_lut8(expr="x 128 - 3 / 128 + 256 *",yexpr="x 125.5 - 3 / 125.5 + 256 *",u=3,v=3)
# replace 128 and 125.5 with 127.5 if working with a full-range source
Dither_convert_yuv_to_rgb(matrix="709",output="rgb48y")
r = SelectEvery(3,0)
g = SelectEvery(3,1)
b = SelectEvery(3,2)
undershoot = cl_exprxyz(r,g,b,"21760 x y z min min -",lsb=true).SelectEvery(1,0,0,0)
overshoot = cl_exprxyz(r,g,b,"x y z max max 43520 -",lsb=true).SelectEvery(1,0,0,0)
rgb_fixed = cl_exprxyz(last,undershoot,overshoot,"x 21760 y - - 21760 y z + + / 65280 *",lsb=true)
# the above is not exactly "fancy" soft limiting, but derp.
rgb_fixed.DitherPost(mode=6)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))



Have an error that mt_lut supports only planar color spaces :( - fixed

but...

cl_exprxyz fails due to my card being old/not supported probably. Is there work around?

[hanfrunz]

a very brutal method is to use converttorgb32() and then converttoyuy2(), but if you need broadcast safe colors better use a plugin or a hardware legalizer (eyeheight is a good brand)

[StainlessS]

I believe Wilbert and Gavino might have been involved with something along those lines a few years ago.

[kolak]

a very brutal method is to use converttorgb32() and then converttoyuy2(), but if you need broadcast safe colors better use a plugin or a hardware legalizer (eyeheight is a good brand)

Yes, eyeheight is good, but they have only NLE plugins.

I want something which I can chain, script using pipes etc.
I believe it's way easier to do than some other avisynth plugins, scripts, just not needed by many people.

[Gavino]

I believe Wilbert and Gavino might have been involved with something along those lines a few years ago.
You're probably thinking of the thread U and V ranges for valid RGB.
In post #14 of that thread, I gave a function ShowBadRGB() that shows the areas of a YUV clip that contain 'invalid' RGB values.
(Note: it takes a long time to load because of the mt_lutxyz function.)

It doesn't correct the problem, but it does show whether the problem exists and which areas need attention.

[StainlessS]

Yes Gavino, that was the thread I was thinking of although was wrong about Wilbert being involved (I think that was another thread).
I mainly remembered the animation which I still find fascinating.

[kolak]

Can anything be developed in this area- maybe sponsored (but probably rather for vapourysnth)?

[raffriff42]

Limiter (http://avisynth.nl/index.php/Limiter) is the gamut filter you are looking for (I think).
[EDIT wrong, see next post]

Also, subtracting Limiter's output from the source shows the out-of-range pixels.
Something like this:
https://www.dropbox.com/s/0pv7zq6q0nfbfx7/YUV-Gamut-test~4.jpg?raw=1
On the right-hand "Diff" image, luma < 16 shows as black, white > 235 shows as white, etc
(note source contrast is exaggerated to show the effect)Assert(IsYUV(Last), "YUV source required")

## for demo purposes, exaggerate contrast
#ColorYUV(levels="TV->PC")

return Last.ShowOutOfGamut()

##################################
### Show out-of-gamut pixels.
##
## @ diff_scale - exaggerate the diff signal (range 1..128, default 32)
##
function ShowOutOfGamut(clip C, int "diff_scale")
{
diff_scale = Min(Max(1, Default(diff_scale, 32)), 128)

C ## Last==C
Diff=Subtract(Last, Limiter())
\ .ColorYUV(
\ off_y=2*diff_scale,
\ cont_y=f2cuv(diff_scale),
\ cont_u=f2cuv(diff_scale),
\ cont_v=f2cuv(diff_scale))

return StackHorizontal(
\ Last,
\ Diff.Subtitle("Diff x "+String(Round(diff_scale)), size=Last.Height/8))
\ .BilinearResize(Last.Width, Last.Height)
}

## scale "normal" float arguments to suit ColorYUV
function f2cuv(float f) { return Round((f - 1.0) * 256.0) }

[colours]

Limiter only limits the YUV values to [16,235] or [16,240], which is not enough to guarantee that the resulting RGB values are in [0,255].

I fixed the script I posted earlier, and it seems to work exactly as expected (http://diff.pics/tOuEecjmoLCs) now. The test image is a U/V gradient with Y set to 168. The Mach bands are annoying, but I can't think of any obvious way to deal with them at the moment.

It shouldn't be too difficult to port this directly to a C/C++ filter for Avisynth or VapourSynth, though it might not be useful as is.

[kolak]

Thanks a lot. It doesn't look very bad. I think it lacks some soft roll-off.
Do you know what are the requirements for cl_expr plugin in terms of GPU?
Any CPU replacement for it?

I've checked and it looks quite good- in gamut are is practically untouched. Here is a difference (boosted):

http://s10.postimg.org/qtehbo3a1/diff.png


I assume we just don't want this hard borders?
How/can it be "smoothed"?

[colours]

I think it lacks some soft roll-off.

Probably because it's actually still hard limiting the colours, albeit in a way different from limiting the red/green/blue channels individually. What I'm doing preserves hue while sacrificing luma and saturation accuracy.

Do you know what are the requirements for cl_expr plugin in terms of GPU?

Not sure, but it works with my laptop's integrated graphics (Intel).

I've checked and it looks quite good- in gamut are is practically untouched. [snip] I assume we just don't want this hard borders?
How/can it be "smoothed"?

The problem is, to implement soft limiting, it's necessary to sacrifice accuracy even for in-gamut YCbCr values. I have a rough idea on how to do this, but CLExpr likes to error out when I use it too many times in a single script (tp7 says this is probably because Intel's OpenCL drivers are bad) so I'm not particularly interested in implementing it at the moment.

[kolak]

I may have more interesting news tomorrow, once I test your image with pro solution. It may end up that your script is actually very good:)

As far as I know you have to change correct values also in order to have some soft clipping. I think this is exactly the case for pro solutions, so please carry on with your ideas :)

[kolak]

No idea which one is correct, but pro plugin gives basically no change for 0-100% RGB.
To show some adjustment I've adjusted it to 90% clip (85% knee) settings and than it gives this:

http://s15.postimg.org/b1aqglcpn/image.png

By the logic this is correct. I'm confused now, as looks like avs script is wrong.

[colours]

No idea which one is correct, but pro plugin gives basically no change for 0-100% RGB.
To show some adjustment I've adjusted it to 90% clip (85% knee) settings and than it gives this:

By the logic this is correct. I'm confused now, as looks like avs script is wrong.

1: You shouldn't be feeding an RGB image, because, well, the levels are already crushed. I generated my test image with mt_lutspa; something like blankclip(width=256,height=256,pixel_type="yv24").mt_lutspa(mode="absolute",yexpr="168",uexpr="x",vexpr="y",u=3,v=3).pointresize(768,768).

2: Just to be clear, the mouseout pic in the comparison I linked before is Dither_convert_yuv_to_rgb, while the mouseover pic (with the obvious diagonal Mach banding) is the script I wrote.

[kolak]

I'm lost...

I need image which is wrong, so I can compare against pro solutions.

If I generate one as per your command and than save it as yuv uncompressed files will this a "bad source" with areas out of gamut?
(looks like it's way out)

[colours]

I'm lost...

I need image which is wrong, so I can compare against pro solutions.

Arguably, anything with out-of-gamut/out-of-range YCbCr values is already wrong, and clamping the converted RGB values to [0,255] is PSNR-optimal.

I uploaded my sample clip (http://demo.ovh.eu/download/80e4bb5e52762d71b2c9717f1b72772b/uvgradient.mkv) to a temporary file hosting site; the luma value changes by frame, while each frame is a 0–255 chroma gradient.

[kolak]

What is the point of whole legalisation in digital world? It will be clipped anyway at some point.
I assume it had more sense for analog transmission.

[Asmodian]

My understanding is that this "Intelligent Auto Correction" legalization is only useful if you want to keep the out of gamut details visible. Of course, this requires you to change in-gamut values as well or you are simply clipping.

It is the difference between perceptual gamut mapping and absolute gamut mapping. IMO small differences in gamut should use absolute (clipped) while large differences in gamut need a perceptual method or you get banding. There are really a lot of possible perceptual methods developed for the graphic design field that are applicable to video, I do not know why Vidcheck thinks they are special.

It did have more sense for analog but when using perceptual methods it isn't clipped even in the digital world. I suppose legalization could also be important if dealing with a bad/non-standard YCbCr to RGB converter that was willing to output RGB values below 0 or above 255.

[kolak]

Well, yes. I assume it's for more advaced than simple clipping conversion, so actual look of the frame changes as little as possible.
Do they think they are special? Hmmm...maybe because their algorythm tries to preserve 'the look' as you mentioned. For example if you have some pure black frames (16) and auto correction happens than this may be raised to eg 20. They say this is desired, some place reject such a files, where 'pure' black is not at 16:)

[Soulnight]

Slightly inefficient, but one way you can check for YUV exceeding the gamut is to first reduce the range then convert to RGB as usual, after which we can apply some sort of fancy soft limiting for out-of-gamut values. Maybe something like this (entirely untested):

source
Dither_lut8(expr="x 128 - 3 / 128 + 256 *",yexpr="x 125.5 - 3 / 125.5 + 256 *",u=3,v=3)
# replace 128 and 125.5 with 127.5 if working with a full-range source
Dither_convert_yuv_to_rgb(matrix="709",lsb_in=true,output="rgb48y")
r = SelectEvery(3,0)
g = SelectEvery(3,1)
b = SelectEvery(3,2)
undershoot = cl_exprxyz(r,g,b,"21760 x y z min min -",lsb=true).SelectEvery(1,0,0,0)
overshoot = cl_exprxyz(r,g,b,"x y z max max 43520 -",lsb=true).SelectEvery(1,0,0,0)
rgb_fixed = cl_exprxyz(last,undershoot,overshoot,"x 21760 y - - 21760 y z + + / 65280 *",lsb=true)
# the above is not exactly "fancy" soft limiting, but derp.
rgb_fixed.DitherPost(mode=6)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))


Edit: now actually tested; forgot the lsb_in=true earlier.

Hi,

interesting post. I understand only partially the code what you wrote but would like to do something similar.

My target is to analyse 4k HDR REC2020 clips to see what pixel lies outside REC709.

For this, If I understood correctly,

I need to:
1) Convert YUV to RGB
2) Convert non linear RGB to linear RGB
3) Convert REC2020 RGB to REC709 RGB

I have created the matrix for the step 3, and everything which is outside REC709 should then get illegal RGB triplet.

And I would like to give those illegal RGB triplet a certain value, like (0,0,0) to paint them black.

Could you help me?


My current code:

loadplugin("C:\Users\flole\Downloads\ManualColorMatrix\ManualColorMatrix_26.dll")
loadplugin("C:\Users\flole\Videos\AviSynth\HDRTools\x86\Release_W7\HDRTools.dll")
Directshowsource("F:\Kodi\Demo 4k HDR\The World In HDR 4K Demo.mkv")
ConvertYUVtoLinearRGB(Color=1)
#isRGB()
ManualColorMatrix(1, 1.66049, -0.58764, -0.07285, -0.12455, 1.13290, -0.00835, -0.01815, -0.10058, 1.11873)

Thank you.
:thanks:

Florian