U and V ranges for valid RGB

[Gavino]

Following a conversation with WorBry about the U and V ranges which correspond to valid RGB, I have put together this script which shows, for each value of Y, the valid ranges of U and V (in itself an interesting exercise in the use of MaskTools expressions :)).
# Demonstration of valid YUV values, showing (for both TV ranges and PC ranges):
# - ranges of U and V for each possible value of Y;
# - ranges of Y and V for each possible value of U;
# - ranges of Y and U for each possible value of V.
# Requires MaskTools v2

# Support function.
# Given MaskTools expressions for Y, U, and V,
# returns expression which yields 255 for valid RGB and 0 for invalid.
function ValidRGB(string y, string u, string v, bool "pcRange") {
pcRange = Default(pcRange, false)

# Normalise Y to [0,1] and U, V to [-1,+1]:
y = pcRange ? y+" 255 /" : y+" 16 - 219 /"
u = pcRange ? u+" 128 / 1 -" : u+" 16 - 112 / 1 -"
v = pcRange ? v+" 128 / 1 -" : v+" 16 - 112 / 1 -"

# Rec 601 coefficients:
Kr = " 0.299" Kg = " 0.587" Kb = " 0.114"
Kr1 = " 1"+Kr+" -" Kb1 = " 1"+Kb+" -"
# From http://avisynth.org/mediawiki/Color_conversions:
# R = Y + V*(1-Kr)
# G = Y - U*(1-Kb)*Kb/Kg - V*(1-Kr)*Kr/Kg
# B = Y + U*(1-Kb)
r = y + v + Kr1 + " * +"
g = y + u + Kb1 + Kb + Kg + " / * * -" + v + Kr1 + Kr + Kg + " / * * -"
b = y + u + Kb1 + " * +"
lwb = " 0 >="
upb = " 1 <="
and = " &"
good = r+lwb+r+upb+and+g+lwb+and+g+upb+and+b+lwb+and+b+upb+and
return good+" 255 0 ?"
}

function PlotYUV(clip c, string y, string u, string v, bool "pcRange") {
plot = c.mt_lutspa(yexpr=y, uexpr=u, vexpr=v, U=3, V=3)
mask = c.mt_lutspa(expr=ValidRGB(y, u, v, pcRange), U=3, V=3)
c.mt_merge(plot, mask, U=3, V=3)
}

function PlotUV(clip c, int yIn, bool "pcRange") {
# Plot valid U and V for given Y
pcRange = Default(pcRange, false)
title = "UV plot V ^-> U "+(pcRange?"PC":"TV")+" Range\nY = "+string(yIn)

y = " " + string(yIn)
u = " x 256 *"
v = " 1 y - 256 *"

c.PlotYUV(y, u, v, pcRange)
pcRange ? NOP : LetterBox(16, 16, 16, 16, color_gray)
Histogram("levels")
StackVertical(last, BlankClip(last, height=4, color=color_gray10),
\ BlankClip(last, height=44).Subtitle(title, lsp=0))
}

function PlotYU(clip c, int vIn, bool "pcRange") {
# Plot valid Y and U for given V
pcRange = Default(pcRange, false)
title = "YU plot Y ^-> U "+(pcRange?"PC":"TV")+" Range\nV = "+string(vIn)

y = " 1 y - 255 *"
u = " x 256 *"
v = " " + string(vIn)

c.PlotYUV(y, u, v, pcRange)
pcRange ? NOP : LetterBox(20, 16, 16, 16, color_gray)
Histogram("levels")
StackVertical(last, BlankClip(last, height=4, color=color_gray10),
\ BlankClip(last, height=44).Subtitle(title, lsp=0))
}

function PlotYV(clip c, int uIn, bool "pcRange") {
# Plot valid Y and V for given U
pcRange = Default(pcRange, false)
title = "YV plot Y ^-> V "+(pcRange?"PC":"TV")+" Range\nU = "+string(uIn)

y = " 1 y - 255 *"
u = " " + string(uIn)
v = " x 256 *"

c.PlotYUV(y, u, v, pcRange)
pcRange ? NOP : LetterBox(20, 16, 16, 16, color_gray)
Histogram("levels")
StackVertical(last, BlankClip(last, height=4, color=color_gray10),
\ BlankClip(last, height=44).Subtitle(title, lsp=0))
}

b = BlankClip(256, 256, 256, pixel_type="YV12")
b1 = b.Trim(16, 235)
b2 = b.Trim(16, 240)
bpc = b.ColorYUV(levels="TV->PC")
tv = Animate(b1, 0, 219, "PlotUV", 16, 235)
\ + Animate(b2, 0, 224, "PlotYU", 16, 240)
\ + Animate(b2, 0, 224, "PlotYV", 16, 240)
pc = Animate(bpc, 0, 255, "PlotUV", 0, true, 255, true)
\ + Animate(bpc, 0, 255, "PlotYU", 0, true, 255, true)
\ + Animate(bpc, 0, 255, "PlotYV", 0, true, 255, true)
return tv.ConvertToRGB32() + pc.ConvertToRGB32("PC.601")

The top-left 256x256 area shows the UV-plane, with invalid values replaced by black.
It shows strikingly that most of the YUV space is in fact unused (ie does not correspond to a valid RGB color), especially for low and high values of Y, and that for no value of Y is the entire range of U or V valid.
For Y=0 or Y=255 of course, it shrinks to a single point (black or white) with U=V=128.

None of this is a problem, of course - it's just the way it works, but it might surprise some people. And it's something to be aware of when generating YUV values from scratch, such as when creating YUV color gradients, the topic of WorBry's post. An 'invalid' YUV value will produce clipping to 0 or 255 on one or more colors when converted to RGB.

Edit: Script extended to add YU and YV plots, and to show plots for both TV range and PC range.

Some selected examples (orig script):

[WorBry]

Thanks for that. Very interesting......and revealing.


It shows strikingly that most of the YUV space is in fact unused (ie does not correspond to a valid RGB color), especially for low and high values of Y, and that for no value of Y is the entire range of U or V valid.For Y=0 or Y=255 of course, it shrinks to a single point (black or white) with U=V=128.

I'd reached a similar conclusion looking at U and V spectra with a superimposed greyscale gradient i.e. (using my 'round-about' script):

colorbars().converttoyv12().mt_lutspa(relative=true,expr="x 255 *").greyscale()
GSG = colorbars().converttoyv12().mt_lutspa(relative=true,expr="x 256 * ").Greyscale().LanczosResize(640,640)
GSG2 = GSG.Reduceby2()
GSGL = GSG.TurnLeft()
NGry2 = GSG.mt_lut("x 128 ", y=3).Reduceby2()
UspecG = YtoUV(GSG2, NGry2, GSGL)
VSpecG = YtoUV(NGry2, GSG2, GSGL)
Return Interleave(USpecG, VSpecG)

Assuming you can see the 'clipping':
U Spectrum with greyscale gradient
http://www.mediafire.com/file/gjnhzqnntn4/USpec_Grad.png
V Spectrum with greyscale gradient
http://www.mediafire.com/file/dq1jnzqnuna/VSpec_Grad.png

.......But your function, with the 'invalid RGB value' mask, really nails it.

.......And it's something to be aware of when generating YUV values from scratch, such as when creating YUV color gradients, the topic of WorBry's post. An 'invalid' YUV value will produce clipping to 0 or 255 on one or more colors when converted to RGB.

With that, it would be interesting to see a composite picture for the entire (0 - 255) luma scale i.e. like the 'UV spectra-greyscale gradient' thing I did, but with the 'invalid value' mask applied.

In fact, what would be really useful is if the function could be developed to apply the analysis to 'real' YV12 image frames and map (maybe with separate color coding for U and V) those areas where the chroma values are rendered 'RGB invalid' and clipped. That would be of great practical value. I wonder how many folks, like myself, apply all manner of luma and chroma pixel transformations to their YUV videos, blissfully unaware that the resulting colors may not be faithfully reproduced in RGB display i.e. what they are seeing on the screen may not be the 'true' YUV colors. Pretty significant I think.

[Dark Shikari]

http://web.archive.org/web/20140315154448/http://img245.imageshack.us/img245/7905/coloryuv.gif

Animated GIF version.

[WorBry]

Way-Hey !!! Where's the pause button ? :p

[pbristow]

Way-Hey !!! Where's the pause button ? :p

It's built into VirtualDub. :)

(I've never tried loading an animated GIF straight into Vdub before. Works like a charm! :) )

[2Bdecided]

Note to anyone who hasn't actually read the script:

It defaults to PC range, with TV range part commented out. However, valid YUV is always in TV range, so the default behaviour is misleading (to say the least)!

Very cool script though. Nice use of masktools!

Cheers,
David.

[WorBry]

It's built into VirtualDub. :)

(I've never tried loading an animated GIF straight into Vdub before. Works like a charm! :) )

I was joking. The function script itself is animated.

[Gavino]

It defaults to PC range, with TV range part commented out. However, valid YUV is always in TV range, so the default behaviour is misleading (to say the least)!
Yes, the values displayed in the histograms and the subtitle are PC range, 0-255 for both luma and chroma. I left it that way because WorBry was using PC range in his work.

However, the calculation for valid/invalid RGB takes that into account, being done on normalised/scaled values in the range [0, 1] for luma and [-1, +1] for chroma. And the colors are converted with matrix PC.601 for output. So the general behaviour is exactly the same for TV range YUV, if you take the UV plot to represent the entire available range (16-240 for TV) and the displayed Y value in brackets to be the fraction of 'full-range'.
it would be interesting to see a composite picture for the entire (0 - 255) luma scale i.e. like the 'UV spectra-greyscale gradient' thing I did, but with the 'invalid value' mask applied.

In fact, what would be really useful is if the function could be developed to apply the analysis to 'real' YV12 image frames and map (maybe with separate color coding for U and V) those areas where the chroma values are rendered 'RGB invalid' and clipped.
I'll have a go at both those. Watch this space. :)

[Gavino]

I have now extended the script in the first post to add plots of YU (for each valid V) and YV (for each valid U), and to show plots for both TV range and PC range. The TV range plots have a grey border marking the smaller overall ranges available (16-235 for luma, 16-240 for chroma).

Next job will be to create a function (as requested by Worbry) to analyse arbitrary YV12 images and map the areas with 'RGB invalid' chroma.

Examples from new script:

[WorBry]

Once again, a very enlightening demonstration, not only of the 'RGB-invalid chroma effect', but also the psycho-visual basis of YUV color perception. I've made it my new screen-saver :p

Thanks for your work - looking forward to the 'applied' function

[WorBry]

While Gavino is working on the 'applied' function, I have been thinking (ahead) about possible strategies to avoid/minimize/resolve RGB-invalid YUV values.

Came across this earlier thread on the subject

http://forum.doom9.org/showthread.php?t=135248&highlight=camcorder+black+overshoot

The cited (fairly dated) BBC paper on 'Limiting of YUV digital video signals', is interesting reading, although I cant pretend to understand all of it

http://downloads.bbc.co.uk/rd/pubs/reports/1987-22.pdf

As a general (software) strategy, the author advocates a 'constant hue' approach, whereby the luma is not touched and the U and V values are scaled back (limited) to the maximum 'valid' values having the same hue as the invalid U and V values before limiting.

Seems reasonable, although I cant see it being the optimum solution in all cases, for example, if one is dealing with an already under-saturated source (as was the case in the aforementioned thread). Also, if 'invalid' values are a result of deliberate alteration of the luma (for example, through creation of a contrast luma curve) a more logical approach might be to settle on a trade-off between desired luma effect and chroma (de)saturation. Maybe the selective saturation and hue (SelSah) function that I put together (allows saturation and hue control to be applied to a defined luma range) will be helpful in this regard (see the thread referred to in Gavino's first post) - we'll see.

Either way, I'm anticipating that the 'applied' function will prove to be a useful visualization tool for guiding such corrective measures. If it does, maybe some auto-corrective option could be incorporated in the future. I'm jumping the gun, of course. No pressure. :p

[Die*wrek*show]

woooo! I love this.

http://web.archive.org/web/20140315154448/http://img245.imageshack.us/img245/7905/coloryuv.gif

:thanks: for making that, D.S.

[WorBry]

Also came across this, more recent, article, proposing an improved approach to the resolution of out-of-gamut YUV values:

http://sas-origin.onstreammedia.com/origin/smpte0109/New%20Folder%201/Chan.pdf

I wonder if his 'in-range chroma reconstruction' model is something than could be tested with AVISynth (MaskTools maybe), although, in my minds-eye, both methods (Spill and Proportion) seem tantamount to chroma blur.

[Gavino]

Here are two functions which can be used to test any YV12 clip for 'invalid' RGB values. See the comments for details.
# Returns a mask with 255 where YV12 clip has valid RGB and 0 elsewhere
function RGBMask(clip c, bool "pcRange") {
c2 = c.BilinearResize(2*c.width, 2*c.height)
return c.mt_lutxyz(c2.UToY(), c2.VToY(), ValidRGB(" x", " y", " z", pcRange))
}

# Shows the areas of a YV12 clip which contain 'invalid RGB';
# good pixels are replaced by 'color', default black.
function ShowBadRGB(clip c, bool "pcRange", int "color") {
mask = c.RGBMask(pcRange)
c.mt_merge(BlankClip(c, color=color), mask, U=3, V=3, luma=true)
}

Both require the ValidRGB() function from the first post.
Script loading is very slow because of the mt_lutxyz function (which has to create a lookup table with 2^24 entries).

Below I have shown the results of applying ShowBadRGB to (the equivalent of) your USpec and VSpec clips, using the following code:
BlankClip(2, 320, 320, pixel_type="YV12")
USpec = mt_lutspa(yexpr="1 y - 255 *", uexpr="x 255 *", U=3, V=-128)
VSpec = mt_lutspa(yexpr="1 y - 255 *", vexpr="x 255 *", U=-128, V=3)
Interleave(USpec, VSpec)
ShowBadRGB(pcRange=true)
return ConvertToRGB("PC.601")
In effect, the result is basically the converse of the 'valid' area plots shown earlier.

The papers you referenced look interesting, although I haven't had time to study them in detail.
The BBC one gives a fairly good explanation of the shape of the plots shown earlier.

Results from ShowBadRGB():

[WorBry]

Excellent. As you say, very slow to load, but it gets there. Thanks so much for doing this. Now to set about doing some testing.

[WorBry]

For testing, I'm using clips from a variety of sources, including some AVCHD camcorders (Panasonic HMC-41 and HMC-151). For compatability with the test system, they need to be converted from Rec.709 to Rec.601 and scaled for 'TV' and 'PC' ranges. Trouble is that, like many consumer/prosumer camcorders, the luma scale is effectively 16 - 255 i.e. allows 235-overshoot (super-white). Here's an example:

http://www.mediafire.com/file/qm5ytiz4djy/Test%20Original.png

(Probably not the best example, in terms of black-point, but it serves to illustrate the 235-overshoot)

The following offers a solution:

AVCSource("C:\...........Test clip.dga")
clp=last
PCY = clp.YLevels(16, 1.0, 255, 0, 255)
UV601 = clp.ColorMatrix(mode="Rec.709->Rec.601")
PCUV = UV601.ColorYUV(levels="TV->PC")
mt_lutxy(PCY, PCUV, y=2, u=4, v=4)
#LanczosResize(1024,576) #Resized for Histogram
#Histogram(mode="levels")

Result:
http://www.mediafire.com/file/yiqdmmz3ym5/Test%20Y%2016to0%20UV%20709to601toPC.png
ShowBadRGB analysis:
http://www.mediafire.com/file/xiwowtmvrjy/Test%20Y%2016to0%20UV%20709to601toPC%20BadRGB%20PCRange%20PC601.png

But, it seems a little convoluted and I am wondering if there is a simpler, more direct solution.

Similar considerations apply for 'TV range'.
A simple ColorMatrix(mode="Rec.709->Rec.601") conversion clips the overshot luma to 235:

Result:
http://www.mediafire.com/file/dymdt4jdqqm/Test%20Rec709to601.png
ShowBadRGB analysis:
http://www.mediafire.com/file/uzinjxz3ygi/Test%20Rec709to601%20BadRGB%20TVRange%20Rec601.png

So, again, would it be valid (for these tests) to scale down the luma from 255 and 235 and take the chroma from Rec.709->Rec.601

AVCSource("C:\...........Test clip.dga")
clp=last
TVY = clp.YLevels(16, 1.0, 255, 16, 235)
TVUV = clp.ColorMatrix(mode="Rec.709->Rec.601")
mt_lutxy(TVY, TVUV, y=2, u=4, v=4)
#LanczosResize(1024,576) #Resized for Histogram
#Histogram(mode="levels")

Result:
http://www.mediafire.com/file/nxmnjbzknzn/Test%20Y%20255to235%20UV%20709to601.png
ShowBadRGB analysis:
http://www.mediafire.com/file/mywkfcmzjtw/Test%20Y%20255to235%20UV%20709to601%20BadRGB%20TVrange%20Rec601.png

Would appreciate advice.

Edit: Also added the results of the 'ShowBadRGB' analysis for each of the above. Interesting that the 'PC Range' and 'TV Range' (using the 255->235 downscaled clip) analyses give very similar patterns - RGB-invalid values associated with the shiny glare on the knights armour (presumably the source of the overshot luma). The straight 'Rec.709->Rec.601 converted' clip however shows more extensive 'bad-RGB', presumably due to the 235 clipping. I've started to do some tests with applied luma contrast curves. Predictably, invalid values then begin to appear on the more saturated colors e.g. the bright reds on the knights tunics, and to a lesser extent the greens.......Edit: not surprising since the rec.601 YUV values for (true) red are stated to be 81, 90, 240

Is it just me, or are the knights getting shorter? :rolleyes:

[Gavino]

I'm not sure your approach is valid as you seem to be mixing Rec.709 luma with Rec.601 chroma, which does not seem correct.

Assuming the input chroma is TV range (am I right to assume this?), then for TV range output, I think you want:
YLevels(16, 1.0, 255, 16, 235)
ColorMatrix(mode="Rec.709->Rec.601")

and for PC range, simply extend this with:
ColorYUV(levels="TV->PC")

Another possibility is to extend my function to work with Rec.709 and avoid doing any colorimetry conversions at all.

A reminder in case you need it: for viewing the results of all these things, or producing screenshots, be sure to add ConvertToRGB(matrix=whatever) to your script unless you are viewing them in something that knows it is not dealing with TV-range Rec.601 input.

PS. Once a knight, always a knight; ... ;)

[WorBry]

The following offers a solution:

AVCSource("C:\...........Test clip.dga")
clp=last
PCY = clp.YLevels(16, 1.0, 255, 0, 255)
UV601 = clp.ColorMatrix(mode="Rec.709->Rec.601")
PCUV = UV601.ColorYUV(levels="TV->PC")
mt_lutxy(PCY, PCUV, y=2, u=4, v=4)
#LanczosResize(1024,576) #Resized for Histogram
#Histogram(mode="levels")

.......But, it seems a little convoluted and I am wondering if there is a simpler, more direct solution.


I guess an alternative would be to convert the Rec.601 chroma to luma and use YLevels to upscale from 16-240 -> 0-255 and then back to chroma, but again, that's rather convoluted (more rounding errors etc). I dunno, maybe I'm over-complicating matters. What is a/the 'correct' way to re-scale YUV chroma 'directly' with MaskTools ?

[Gavino]

Not sure if your last post was in response to mine or if you missed it in passing.

[WorBry]

Just missed it in passing.

I'm not sure your approach is valid as you seem to be mixing Rec.709 luma with Rec.601 chroma, which does not seem correct.

I was wondering about that.

Assuming the input chroma is TV range (am I right to assume this?), then for TV range output, I think you want:
YLevels(16, 1.0, 255, 16, 235)
ColorMatrix(mode="Rec.709->Rec.601")

and for PC range, simply extend this with:
ColorYUV(levels="TV->PC")

Ah yes, that way the luma overshoot is 'preserved' (i.e. not clipped, and then crushed in the upscale to 255). AFAIK, the ('TV') chroma range (16-240) is the same in Rec.709 and Rec.601, and I assume that these AVCHD camcorders are Rec.709 compliant.

Another possibility is to extend my function to work with Rec.709 and avoid doing any colorimetry conversions at all.

True, especially if folks are post-processing their AVCHD videos in Rec.709 with a view to displaying on HDTV.

A reminder in case you need it: for viewing the results of all these things, or producing screenshots, be sure to add ConvertToRGB(matrix=whatever) to your script unless you are viewing them in something that knows it is not dealing with TV-range Rec.601 input.

True, I didn't do that with the clip + histogram frame-shots. Thanks for the advice.

PS. Once a knight, always a knight; ... ;)

Yes, but the downside to medieval pageantry is having to work the knights on weekends. :p

[WorBry]

Regarding a Rec.709 version of ShowBadRGB; looks like it's simply a matter of changing the coefficients from Rec.601 to Rec.709 in the ValidRGB support function. Correct?

Edit:......and of course adding ConvertToRGB("Rec.709") or ConvertToRGB("PC.709"), as appropriate, after ShowBadRGB

Edit2: Is it certain that ColorYUV can be applied to Rec.709 YV12 sources, in the same manner as Rec.601, for 'TV->PC' range scaling that is? Since the ColorYUV documentation makes no such distinction, and the luma and chroma ranges are the same for both, I'm assuming yes, but thought it best to clarify.

[Gavino]

Regarding a Rec.709 version of ShowBadRGB; looks like it's simply a matter of changing the coefficients from Rec.601 to Rec.709 in the ValidRGB support function. Correct?
Yes, at its simplest it's a one-line change.
But I suggest making it subject to an extra boolean parameter, "Rec709" say, default false, and add this parameter also to RGBMask and ShowBadRGB.
Is it certain that ColorYUV can be applied to Rec.709 YV12 sources, in the same manner as Rec.601, for 'TV->PC' range scaling that is? Since the ColorYUV documentation makes no such distinction, and the luma and chroma ranges are the same for both, I'm assuming yes, but thought it best to clarify.
That was my reasoning too. I've just checked the source of ColorYUV and it should work OK.

[hanfrunz]

Interesting reads about all that gammut stuff:

Tektronix: Understanding Color and Gammut (http://www.tek.com/zh/education/PDF/25W_15618_3.pdf)
http://tech.ebu.ch/docs/techreview/trev_301-gierlinger.pdf
EBU Technical Recommendation R103-2000 Tolerances on "Illegal" colours in television (http://tech.ebu.ch/webdav/site/tech/shared/r/r103.pdf)
Eyeheight: Video Legalizer (product) (http://www.eyeheight.com/FileDownload.asp?filename=tech_sheets/compliance/LE-2U_multi-rate_dual_link_444_legaliser_module_tech_sheet.pdf)
The famous Color FAQ by Charles Poynton (http://www.poynton.com/PDFs/ColorFAQ.pdf)
my allcolors-filters to play with all of them (http://forum.doom9.org/showthread.php?t=104846) :)

[WorBry]

It appears that Photoshop CS4 has a similar function for mapping 'out-of-gamut' colors (when converting RGB to CMYK) :

http://www.thelightsright.com/TameOutOfGamutColors

What a coincidence :eek: . Great minds think alike :cool:

As per the cited tutorial, the recommended tool for 'taming' invalid colours is Vibrance control. Used in conjunction with Saturation control, the Vibrance tool (in negative mode) can be applied to bias desaturation toward the more saturated colours.

Of course this prompted me to delve into possibility of concocting a MaskTools-based 'Vibrance' equivalent, only to find that our dear friend Tweak already provides this functionality (without the trendy name) through it's minSat and maxSat range parameters, added in Tweak v2.58.

With that, I've updated the SelSah function to incorporate this feature, along with hue range control.

http://forum.doom9.org/showthread.php?p=1405481#post1405481

Still experimenting, but initial indications are that it can be applied to reign-in RGB-invalid YUV colours quite effectively and selectively.

[WorBry]

Yes, at its simplest it's a one-line change.
But I suggest making it subject to an extra boolean parameter, "Rec709" say, default false, and add this parameter also to RGBMask and ShowBadRGB.

Tried modifying as suggested, but I`m getting an access violation. Something related to the added bool Rec709 parameter, but I`m not sure what.

# SupportFunctions:
# Given MaskTools expressions for Y, U, and V,
# returns expression which yields 255 for valid RGB and 0 for invalid.
function ValidRGB(string y, string u, string v, bool "pcRange", bool "Rec709") {

pcRange = Default(pcRange, false)
Rec709 = Default(Rec709, false)

# Normalise Y to [0,1] and U, V to [-1,+1]:
y = pcRange ? y+" 255 /" : y+" 16 - 219 /"
u = pcRange ? u+" 128 / 1 -" : u+" 16 - 112 / 1 -"
v = pcRange ? v+" 128 / 1 -" : v+" 16 - 112 / 1 -"

# Rec601 and Rec709 coefficients:
Kr = Rec709 ? " 0.2125" : " 0.299"
Kg = Rec709 ? " 0.7154" : " 0.587"
Kb = Rec709 ? " 0.0721" : " 0.114"
Kr1 = " 1"+Kr+" -" Kb1 = " 1"+Kb+" -"
# From http://avisynth.org/mediawiki/Color_conversions:
# R = Y + V*(1-Kr)
# G = Y - U*(1-Kb)*Kb/Kg - V*(1-Kr)*Kr/Kg
# B = Y + U*(1-Kb)
r = y + v + Kr1 + " * +"
g = y + u + Kb1 + Kb + Kg + " / * * -" + v + Kr1 + Kr + Kg + " / * * -"
b = y + u + Kb1 + " * +"
lwb = " 0 >="
upb = " 1 <="
and = " &"
good = r+lwb+r+upb+and+g+lwb+and+g+upb+and+b+lwb+and+b+upb+and
return good+" 255 0 ?"
}

# Returns a mask with 255 where YV12 clip has valid RGB and 0 elsewhere
function RGBMask(clip c, bool "pcRange", bool "Rec709") {
c2 = c.BilinearResize(2*c.width, 2*c.height)
return c.mt_lutxyz(c2.UToY(), c2.VToY(), ValidRGB(" x", " y", " z", pcRange, rec709))
}

# Shows the areas of a YV12 clip which contain 'invalid RGB';
# good pixels are replaced by 'color', default black.
function ShowBadRGB(clip c, bool "pcRange", bool "Rec709", int "color") {
mask = c.RGBMask(pcRange, Rec709)
c3 = c.mt_merge(BlankClip(c, color=color), mask, U=3, V=3, luma=true)
Mtx = (Rec709 && pcRange) ? "PC.709"
\ : (Rec709 && pcrange == false) ? "Rec709"
\ : (Rec709 == false && pcRange) ? "PC.601"
\ : "Rec601"
return c3.ConverttoRGB(matrix=Mtx)
}

Note, also included the final ConverttoRGB in ShowBadRGB, with auto-selection of the appropriate color matrix. Untested due to the access violation.

[Gavino]

It appears that Photoshop CS4 has a similar function for mapping 'out-of-gamut' colors (when converting RGB to CMYK).
What a coincidence :eek: . Great minds think alike :cool:
Interesting. I wasn't aware of that at all.
I prefer my version which shows not only where the bad colors are, but what they are. :cool:
Tried modifying as suggested, but I`m getting an access violation. Something related to the added bool Rec709 parameter, but I`m not sure what.
It's a problem with the form of the expressions. Rather than tediously put in a space (required by MaskTools) between each string concatenation, I chose to put a space on the front of all the constituent string literals. (It would work equally well to put a space on the end instead, as long as it's done consistently.) Perhaps I should have explained that earlier.

Therefore the coefficients need to be set up as:
# Rec601 and Rec709 coefficients:
Kr = Rec709 ? " 0.2125" : " 0.299"
Kg = Rec709 ? " 0.7154" : " 0.587"
Kb = Rec709 ? " 0.0721" : " 0.114"

[WorBry]

I prefer my version which shows not only where the bad colors are, but what they are. :cool:

Absolutely.

Therefore the coefficients need to be set up as:
# Rec601 and Rec709 coefficients:
Kr = Rec709 ? " 0.2125" : " 0.299"
Kg = Rec709 ? " 0.7154" : " 0.587"
Kb = Rec709 ? " 0.0721" : " 0.114"

Yep, that resolves it. Corrected above script accordingly.

[Yellow_]

Does xv colour have a place in this discussion?

I'm looking to eek out as much colour range as possible from Canon h264 DSLR video, it's full range BT601, 709 primaries but the camera like a few others, Sony, Panasonic, supports xv colour and so I believe h264AVC can support it based on an extension to the specification?

I'm currently using yesgreys yCMS 3D LUT tool + tritical's t3dlut script and getting good results.

BUT currently yCMS doesn't handle xv colour, it's on yesgreys todo list and I'm wondering whether doing the conversion via an xv colour to a wide RGB gamut like AdobeRGB via the 3D LUT would yield more colour values, re munsell colours. This is with regard for intermediate work, colour grading etc before reducing to delivery, sRGB or Rec709.

I believe this is similar to how Adobe CS5 works internally with video sources, quote from Adobe:

Yes, H.264 is definitely not considered a finishing codec, but to be clear, Premiere Pro does not use it in that way. The H.264 is read natively by Premiere and once it is decoded into the app. it “resides” internally in a 32 bit float extended color space that is unmatched for color fidelity and dynamic range.

Adobe CS5 reads the H.264 files natively into Premiere Pro and After Effects at the highest possible quality. Our color gamut and dynamic range for tonal detail from shadow to highlight is unsurpassed. There is even support for over-brights beyond 100% in After Effects. i.e. in plain English, we squeeze more out of these files than anything else out there!

The magic comes from the use of proprietary interpretation algorithms and I might also mention that we bypass QuickTime for this process, which avoids the whole gamma conundrum. Once the file is living inside our apps on the timeline or project, we deal with the image information at the 32 bit float level.

Most of what he mentions is standard stuff?, where he mentions 32bit extended colour space he's referring to RGB? and where he mentions 'interpretation algorithms' I assume he means 'interpolation' ie bilinear, bicubic etc, that area is interesting too, Cineform does similar I believe.

Does any of this fit with your experiments, although from quick scan over your thread you are looking at selectively choosing what to pass through and what to discard for the final RGB image?

[jmac698]

I don't think that's refering to interpolation. I think it refers to gamma correct resizing, gamut mapping, something along those lines.
And the mapping from invalid yuv to rgb is something we (IanB and I) had to do to add correct colorbars to Avisynth 2.56. We chose to preserve tint; this is correct since the use of -I on a vectorscope is too check an angle. We had to sacrifice luma to do so (IIRC).
I'd be interested in other ways of doing this, maybe there's some kind of HDR tone mapping type algoirthm that lets you see more of the invalid yuv.

[Yellow_]

I don't think that's refering to interpolation. I think it refers to gamma correct resizing, gamut mapping, something along those lines.

Ok, I see, I'd assumed as triticals t3dlut script gave choice of duplicate, bilinear and bicubic that was what he was talking about, I need to read up on gamma resizing, sounds like a linear space operation from another current thread?.

I'd be interested in other ways of doing this, maybe there's some kind of HDR tone mapping type algoirthm that lets you see more of the invalid yuv.

That's an interesting idea to pursue. Cheers.

Not directly applicable but a start, Red are working with HDR here:

http://reduser.net/forum/showthread.php?t=50670

[jmac698]

I can't say further, I'm not knowledgable enough on gamut conversions, just my sense of the passage you quoted.
I looked at tritical's stuff but didn't know what bicubic could refer to without reading the whole manual.
Maybe it's interpolation of the calibration measurements?

[pandy]

Evaluate case when Y=0 (ie 1 or 16 accordingly to the CCIR)

[Yellow_]

Any chance of a quick example of using ShowBadRGB and ValidRGB with a simple test clip.

I've copied WorBry's script to plugins folder as an avsi, have masktools version 2.a48 from Manaos site but Vdub just freezes when testing:

c= ffmpegsource2("test.m2t")

ShowBadRGB(c,pcRange=false, Rec709=true, color=1)

Also trying it with version Masktools 1.5.8 same freezing. Virtualdub 1.9.10 :-(

[Gavino]

Any chance of a quick example of using ShowBadRGB and ValidRGB with a simple test clip.

I've copied WorBry's script to plugins folder as an avsi, have masktools version 2.a48 from Manaos site but Vdub just freezes when testing:
Be patient. :)
As I explained in post #14, the script takes a long time to load, because of the mt_lutxyz function.
Once loaded, frame serving is relatively quick.

That post also has a simple test script and demonstration which generates a varying range of valid and invalid colors.

[Yellow_]

Gavino thanks, not kidding really is quite slow, would it be too involved to make use of yesgreys yCMS to generate the 3D LUTs in advance?

Not sure if I'm using it correctly but if I use pcRange=true does ShowbadRGB show black just for the invalid RGB outside of gamut (assume this is sRGB), when validating 0 255 YCbCr otherwise it shows black for any invalid RGB (sRGB) outside of on 16 235 and 240?

And the Rec709=false would be for video sources that use a BT601 color matrix but BT709 primaries and transfer?

One source video I tested, h264AVC, containing a shot of a lot of cheap gold jewelery gives me an image containing about 80000 more unique colors outside of 16 235 conversion.

With regard to NLEs is it typical to see them squeeze the full range into 16 235 to process and does that give access to those 80000 more unique colors or a fraction of. Or is a xvYCC workflow and some sort of gamut remapping without clipping a better option?

[Gavino]

Not sure if I'm using it correctly but if I use pcRange=true does ShowbadRGB show black just for the invalid RGB outside of gamut (assume this is sRGB), when validating 0 255 YCbCr otherwise it shows black for any invalid RGB (sRGB) outside of on 16 235 and 240?
It shows black (or the specified color) for pixels with valid colors - invalid ones are left alone.
The idea is to show you where the 'bad' pixels are (and what color they are after conversion to RGB).

An invalid pixel is one that produces a value of R, G, or B outside the range 0-255 (and hence clipped to 0 or 255) on conversion to RGB. The pcRange and Rec709 parameters indicate how the conversion to RGB is to be done, determining the equivalent matrix parameter of ConvertToRGB as follows:
pcRange: false false true true
Rec709: false true false true
matrix: Rec601 Rec709 PC.601 PC.709
I'm afraid I don't know enough about yCMS or xvYCC to answer your other questions.

[Yellow_]

It shows black (or the specified color) for pixels with valid colors - invalid ones are left alone.
The idea is to show you where the 'bad' pixels are (and what color they are after conversion to RGB).

I totally worded that wrong. I meant to say as you describe, certainly is what I'm seeing, black as non bad. :-)

An invalid pixel is one that produces a value of R, G, or B outside the range 0-255 (and hence clipped to 0 or 255) on conversion to RGB.

My understanding, which could be way off, please feel free to correct me is the following. :-):

What equates to 0 - 255 RGB from the YCbCr is dependant on your following comment the pcRange=true or false? ie

Considering so called 'legal' 8bit digital video, which when converted to RGB is to fit within the sRGB gamut (due to colour primaries) and I believe the YCbCr range used is 16 - 235 & 240 for U & V? Mapped to 0 - 255 in RGB to comply? So that would be pcRange=false. Colors that show using ShowBad are those that fall outside sRGB gamut, with regard for Rec709, would be clipped. But those values showing as 'bad' still stand a chance of being valid in a wider RGB gamut ie xvYCC, an extension of BT601/709 with suggested sRGB x 1.8 munsell colors useable?

Whether or not the original video source has sufficient colour info to saturate sRGB gamut anyway, certainly the cheap gold video shot I use to test has a lot of yellow 'bad' pixels (assume out of sRGB gamut) according to ShowBad and much still visible using pcRange=true.

If I use pcRange=true then the full 0 - 255 YCbCr 8bit range would be mapped directly to 0 - 255 RGB but this is still sRGB due to colour primaries? Clipping is still possible due to sRGB gamut not wide enough to contain all values generated?

xvYCC uses 1 to 254 range (pcRange) and I think maps 16 to 0 RGB and 235/240 to 255 RGB to maintain compatibility with BT601 BT709 Rec. (sRGB) but maps the 'out of range' YCbCr to below 0 and above 255 RGB values.

However a greater gamut than sRGB is required and greater than 8bit float precision to handle negative RGB values and values over 255 ie (0 and 1).

The pcRange and Rec709 parameters indicate how the conversion to RGB is to be done, determining the equivalent matrix parameter of ConvertToRGB as follows:
pcRange: false false true true
Rec709: false true false true
matrix: Rec601 Rec709 PC.601 PC.709

So for me to try to understand then using pcRange=false will show all 'bad' colors outside of BT601 / BT709 Rec. ie only 16 - 235/240 range is legal?

pcRange=true will use all YCbCr 0 - 255 range and map that to fit within 0 - 255 RGB which is still sRGB gamut and therefore still possibility to clip, could still see colors in the black due to clipped sRGB gamut?

I think I may have things in a muddle, complicating things with RGB gamut in considerations and the above is not meant as fact, it's me trying to understand what's going on. :-)

I'm afraid I don't know enough about yCMS or xvYCC to answer your other questions.

No problem, thank you for your time.

[Gavino]

So for me to try to understand then using pcRange=false will show all 'bad' colors outside of BT601 / BT709 Rec. ie only 16 - 235/240 range is legal?
Yes, with pcRange=false, everything outside 16-235/240 will be marked as 'bad'.
And of course, some (indeed, most) YCbCr combinations inside 16-235/240 will also be 'bad', as shown by the demonstration at the start of this thread.
pcRange=true will use all YCbCr 0 - 255 range and map that to fit within 0 - 255 RGB which is still sRGB gamut and therefore still possibility to clip, could still see colors in the black due to clipped sRGB gamut?
pcRange=true maps the full YCbCr 0-255 to RGB 0-255; those combinations that produce an RGB value outside 0-255 are marked as 'bad'.
But those values showing as 'bad' still stand a chance of being valid in a wider RGB gamut ie xvYCC, an extension of BT601/709 with suggested sRGB x 1.8 munsell colors useable?
Yes - the function only works with the four standard Avisynth YUV->RGB conversions and doesn't consider the possibility of a wider RGB gamut.

[Yellow_]

Yes, with pcRange=false, everything outside 16-235/240 will be marked as 'bad'.
And of course, some (indeed, most) YCbCr combinations inside 16-235/240 will also be 'bad', as shown by the demonstration at the start of this thread.

Will reread the thread again for that. :-)

Yes - the function only works with the four standard Avisynth YUV->RGB conversions and doesn't consider the possibility of a wider RGB gamut.

Excellent, that's helpful to know and is why I've been using yCMS 3D LUT system to go 0 - 255 YCbCr (xvYCC) to AdobeRGB in order to get more 'valid' RGB into the image export.

Going a little OT and will start a new thread with a sample source file for testing but:

I'm finding that something appears to be preventing the additional values out to images.

Virtualdub author says Vdub can't write out wider than sRGB, which is fine, good to know. So I've recently tried AVSPMod but no better output.

Even using Wilberts Imagemagick plugin doesn't appear to get me AdobeRGB gamut range from 0 - 255 YCbCr so wonder if Avisynth is presenting the problem.

Sorry OT and will start new thread.

[2Bdecided]

Did anyone get any further with creating a function to change the "bad" U and V values into "good" ones - i.e. to desaturate each pixel individually until it generates an in-range RGB value?

It's totally beyond me, and I've hit another clip where I need it.

EDIT: I've been looking+hoping for a long time...!
http://forum.doom9.org/showthread.php?t=135248

Cheers,
David.

[Mounir]

I concur we need such a tool to auto correct mathematically bad U/V values

[kolak]

Is this difficult to do? Anyone?

[gwendolien]

YES!!! That would be highly appreciated by me!

[raffriff42]

VirtualDub ColorTools (http://trevlac.us/colorCorrection/colorTools.html) plugin:

image 1 (https://www.dropbox.com/s/qlm08cutm0v53u1/colortools%201%20view.jpg?dl=0) view=Video(RGB)
view=Hot Pixel Scan
* image 2 (https://www.dropbox.com/s/z5n0fo6qpts1iry/colortools%202%20black.jpg?dl=0) change=To Black: set all illegal pixels to black (flag for manual intervention)
* image 3 (https://www.dropbox.com/s/lch66dm5ditf80f/colortools%203%20inten.jpg?dl=0) change=intensity: decrease "intensity" as required to force pixels within range
* image 4 (https://www.dropbox.com/s/huayl66wossm86o/colortools%204%20sat.jpg?dl=0) change=saturation: decrease saturation as required to force pixels within range

Change=saturation looks better to me.

Can ColorTools be made to work in Avisynth with vdub_filters.avsi (http://avisynth.nl/index.php/VirtualDub_I)...? Have just begun to look at this.

[Reel.Deel]

Can ColorTools be made to work in Avisynth with vdub_filters.avsi (http://avisynth.nl/index.php/VirtualDub_I)...?

Have a look at this thread (http://forum.doom9.org/showthread.php?t=72634).

[raffriff42]

Hey, great, @Reel.Deal!
## uses VD_clrtools() - see http://forum.doom9.org/showthread.php?t=72634
## (RGB mode source required)

## set illegal pixels to black (flag for manual intervention)
#return VD_clrtools(NTSC=true, rng255=false, stdDef=false, mode=1, hotFixMode=0)

## decrease intensity as required to force pixels within range
#return VD_clrtools(NTSC=true, rng255=false, stdDef=false, mode=1, hotFixMode=1)

## decrease saturation as required to force pixels within range
return VD_clrtools(NTSC=true, rng255=false, stdDef=false, mode=1, hotFixMode=2)
Set NTSC, rng255, and stdDef as appropriate for the source.
Does a good job it seems to me; it corrects invisibly as long as the original isn't too far out of legality.

EDIT - I'm not saying this filter is necessarily "correct" as defined earlier in this discussion. Further experimentation is needed.