U and V ranges for valid RGB

[WorBry]

Regarding a Rec.709 version of ShowBadRGB; looks like it's simply a matter of changing the coefficients from Rec.601 to Rec.709 in the ValidRGB support function. Correct?

Edit:......and of course adding ConvertToRGB("Rec.709") or ConvertToRGB("PC.709"), as appropriate, after ShowBadRGB

Edit2: Is it certain that ColorYUV can be applied to Rec.709 YV12 sources, in the same manner as Rec.601, for 'TV->PC' range scaling that is? Since the ColorYUV documentation makes no such distinction, and the luma and chroma ranges are the same for both, I'm assuming yes, but thought it best to clarify.

[Gavino]

Originally Posted by WorBry

Regarding a Rec.709 version of ShowBadRGB; looks like it's simply a matter of changing the coefficients from Rec.601 to Rec.709 in the ValidRGB support function. Correct?

Yes, at its simplest it's a one-line change.
But I suggest making it subject to an extra boolean parameter, "Rec709" say, default false, and add this parameter also to RGBMask and ShowBadRGB.

Is it certain that ColorYUV can be applied to Rec.709 YV12 sources, in the same manner as Rec.601, for 'TV->PC' range scaling that is? Since the ColorYUV documentation makes no such distinction, and the luma and chroma ranges are the same for both, I'm assuming yes, but thought it best to clarify.

That was my reasoning too. I've just checked the source of ColorYUV and it should work OK.

[hanfrunz]

Interesting reads about all that gammut stuff:

Tektronix: Understanding Color and Gammut
http://tech.ebu.ch/docs/techreview/t...gierlinger.pdf
EBU Technical Recommendation R103-2000 Tolerances on "Illegal" colours in television
Eyeheight: Video Legalizer (product)
The famous Color FAQ by Charles Poynton
my allcolors-filters to play with all of them

[WorBry]

It appears that Photoshop CS4 has a similar function for mapping 'out-of-gamut' colors (when converting RGB to CMYK) :

http://www.thelightsright.com/TameOutOfGamutColors

What a coincidence . Great minds think alike

As per the cited tutorial, the recommended tool for 'taming' invalid colours is Vibrance control. Used in conjunction with Saturation control, the Vibrance tool (in negative mode) can be applied to bias desaturation toward the more saturated colours.

Of course this prompted me to delve into possibility of concocting a MaskTools-based 'Vibrance' equivalent, only to find that our dear friend Tweak already provides this functionality (without the trendy name) through it's minSat and maxSat range parameters, added in Tweak v2.58.

With that, I've updated the SelSah function to incorporate this feature, along with hue range control.

http://forum.doom9.org/showthread.ph...81#post1405481

Still experimenting, but initial indications are that it can be applied to reign-in RGB-invalid YUV colours quite effectively and selectively.

[WorBry]

Originally Posted by Gavino

Yes, at its simplest it's a one-line change.
But I suggest making it subject to an extra boolean parameter, "Rec709" say, default false, and add this parameter also to RGBMask and ShowBadRGB.

Tried modifying as suggested, but I`m getting an access violation. Something related to the added bool Rec709 parameter, but I`m not sure what.

# SupportFunctions:
# Given MaskTools expressions for Y, U, and V,
# returns expression which yields 255 for valid RGB and 0 for invalid.
function ValidRGB(string y, string u, string v, bool "pcRange", bool "Rec709") {
   
  pcRange = Default(pcRange, false)
  Rec709  = Default(Rec709, false)
  
  # Normalise Y to [0,1] and U, V to [-1,+1]:
  y = pcRange ? y+" 255 /" : y+" 16 - 219 /"
  u = pcRange ? u+" 128 / 1 -" : u+" 16 - 112 / 1 -"
  v = pcRange ? v+" 128 / 1 -" : v+" 16 - 112 / 1 -"

  # Rec601 and Rec709 coefficients:
  Kr  = Rec709 ? " 0.2125" : " 0.299"  
  Kg  = Rec709 ? " 0.7154" : " 0.587" 
  Kb  = Rec709 ? " 0.0721" : " 0.114" 
  Kr1 = " 1"+Kr+" -"  Kb1 = " 1"+Kb+" -"
  # From http://avisynth.org/mediawiki/Color_conversions:
  # R = Y + V*(1-Kr)
  # G = Y - U*(1-Kb)*Kb/Kg - V*(1-Kr)*Kr/Kg
  # B = Y + U*(1-Kb)
  r = y + v + Kr1 + " * +"
  g = y + u + Kb1 + Kb + Kg + " / * * -" + v + Kr1 + Kr + Kg + " / * * -"
  b = y + u + Kb1 + " * +"
  lwb = " 0 >="
  upb = " 1 <="
  and = " &"
  good = r+lwb+r+upb+and+g+lwb+and+g+upb+and+b+lwb+and+b+upb+and
  return good+" 255 0 ?"
}

# Returns a mask with 255 where YV12 clip has valid RGB and 0 elsewhere
function RGBMask(clip c, bool "pcRange", bool "Rec709") {
  c2 = c.BilinearResize(2*c.width, 2*c.height)
  return c.mt_lutxyz(c2.UToY(), c2.VToY(), ValidRGB(" x", " y", " z", pcRange, rec709))
}

# Shows the areas of a YV12 clip which contain 'invalid RGB';
# good pixels are replaced by 'color', default black.
function ShowBadRGB(clip c, bool "pcRange", bool "Rec709", int "color") {
  mask =  c.RGBMask(pcRange, Rec709)
  c3    =  c.mt_merge(BlankClip(c, color=color), mask, U=3, V=3, luma=true)
  Mtx   =  (Rec709 && pcRange) ? "PC.709" 
\        : (Rec709 && pcrange == false) ? "Rec709"
\        : (Rec709 == false && pcRange) ? "PC.601"
\        : "Rec601"
  return c3.ConverttoRGB(matrix=Mtx)
}

Note, also included the final ConverttoRGB in ShowBadRGB, with auto-selection of the appropriate color matrix. Untested due to the access violation.

[Gavino]

Originally Posted by WorBry

It appears that Photoshop CS4 has a similar function for mapping 'out-of-gamut' colors (when converting RGB to CMYK).
What a coincidence . Great minds think alike

Interesting. I wasn't aware of that at all.
I prefer my version which shows not only where the bad colors are, but what they are.

Originally Posted by WorBry

Tried modifying as suggested, but I`m getting an access violation. Something related to the added bool Rec709 parameter, but I`m not sure what.

It's a problem with the form of the expressions. Rather than tediously put in a space (required by MaskTools) between each string concatenation, I chose to put a space on the front of all the constituent string literals. (It would work equally well to put a space on the end instead, as long as it's done consistently.) Perhaps I should have explained that earlier.

Therefore the coefficients need to be set up as:

  # Rec601 and Rec709 coefficients:
  Kr  = Rec709 ? " 0.2125" : " 0.299"  
  Kg  = Rec709 ? " 0.7154" : " 0.587" 
  Kb  = Rec709 ? " 0.0721" : " 0.114"

[WorBry]

Originally Posted by Gavino

I prefer my version which shows not only where the bad colors are, but what they are.

Absolutely.

Therefore the coefficients need to be set up as:

  # Rec601 and Rec709 coefficients:
  Kr  = Rec709 ? " 0.2125" : " 0.299"  
  Kg  = Rec709 ? " 0.7154" : " 0.587" 
  Kb  = Rec709 ? " 0.0721" : " 0.114"

Yep, that resolves it. Corrected above script accordingly.

[Yellow_]

Does xv colour have a place in this discussion?

I'm looking to eek out as much colour range as possible from Canon h264 DSLR video, it's full range BT601, 709 primaries but the camera like a few others, Sony, Panasonic, supports xv colour and so I believe h264AVC can support it based on an extension to the specification?

I'm currently using yesgreys yCMS 3D LUT tool + tritical's t3dlut script and getting good results.

BUT currently yCMS doesn't handle xv colour, it's on yesgreys todo list and I'm wondering whether doing the conversion via an xv colour to a wide RGB gamut like AdobeRGB via the 3D LUT would yield more colour values, re munsell colours. This is with regard for intermediate work, colour grading etc before reducing to delivery, sRGB or Rec709.

I believe this is similar to how Adobe CS5 works internally with video sources, quote from Adobe:

Yes, H.264 is definitely not considered a finishing codec, but to be clear, Premiere Pro does not use it in that way. The H.264 is read natively by Premiere and once it is decoded into the app. it “resides” internally in a 32 bit float extended color space that is unmatched for color fidelity and dynamic range.

Adobe CS5 reads the H.264 files natively into Premiere Pro and After Effects at the highest possible quality. Our color gamut and dynamic range for tonal detail from shadow to highlight is unsurpassed. There is even support for over-brights beyond 100% in After Effects. i.e. in plain English, we squeeze more out of these files than anything else out there!

The magic comes from the use of proprietary interpretation algorithms and I might also mention that we bypass QuickTime for this process, which avoids the whole gamma conundrum. Once the file is living inside our apps on the timeline or project, we deal with the image information at the 32 bit float level.

Most of what he mentions is standard stuff?, where he mentions 32bit extended colour space he's referring to RGB? and where he mentions 'interpretation algorithms' I assume he means 'interpolation' ie bilinear, bicubic etc, that area is interesting too, Cineform does similar I believe.

Does any of this fit with your experiments, although from quick scan over your thread you are looking at selectively choosing what to pass through and what to discard for the final RGB image?

[jmac698]

I don't think that's refering to interpolation. I think it refers to gamma correct resizing, gamut mapping, something along those lines.
And the mapping from invalid yuv to rgb is something we (IanB and I) had to do to add correct colorbars to Avisynth 2.56. We chose to preserve tint; this is correct since the use of -I on a vectorscope is too check an angle. We had to sacrifice luma to do so (IIRC).
I'd be interested in other ways of doing this, maybe there's some kind of HDR tone mapping type algoirthm that lets you see more of the invalid yuv.

[Yellow_]

Originally Posted by jmac698

I don't think that's refering to interpolation. I think it refers to gamma correct resizing, gamut mapping, something along those lines.

Ok, I see, I'd assumed as triticals t3dlut script gave choice of duplicate, bilinear and bicubic that was what he was talking about, I need to read up on gamma resizing, sounds like a linear space operation from another current thread?.

I'd be interested in other ways of doing this, maybe there's some kind of HDR tone mapping type algoirthm that lets you see more of the invalid yuv.

That's an interesting idea to pursue. Cheers.

Not directly applicable but a start, Red are working with HDR here:

http://reduser.net/forum/showthread.php?t=50670

[jmac698]

I can't say further, I'm not knowledgable enough on gamut conversions, just my sense of the passage you quoted.
I looked at tritical's stuff but didn't know what bicubic could refer to without reading the whole manual.
Maybe it's interpolation of the calibration measurements?

[pandy]

Evaluate case when Y=0 (ie 1 or 16 accordingly to the CCIR)

[Yellow_]

Any chance of a quick example of using ShowBadRGB and ValidRGB with a simple test clip.

I've copied WorBry's script to plugins folder as an avsi, have masktools version 2.a48 from Manaos site but Vdub just freezes when testing:

c= ffmpegsource2("test.m2t")

ShowBadRGB(c,pcRange=false, Rec709=true, color=1)

Also trying it with version Masktools 1.5.8 same freezing. Virtualdub 1.9.10 :-(

[Gavino]

Originally Posted by Yellow_

Any chance of a quick example of using ShowBadRGB and ValidRGB with a simple test clip.

I've copied WorBry's script to plugins folder as an avsi, have masktools version 2.a48 from Manaos site but Vdub just freezes when testing:

Be patient.
As I explained in post #14, the script takes a long time to load, because of the mt_lutxyz function.
Once loaded, frame serving is relatively quick.

That post also has a simple test script and demonstration which generates a varying range of valid and invalid colors.

[Yellow_]

Gavino thanks, not kidding really is quite slow, would it be too involved to make use of yesgreys yCMS to generate the 3D LUTs in advance?

Not sure if I'm using it correctly but if I use pcRange=true does ShowbadRGB show black just for the invalid RGB outside of gamut (assume this is sRGB), when validating 0 255 YCbCr otherwise it shows black for any invalid RGB (sRGB) outside of on 16 235 and 240?

And the Rec709=false would be for video sources that use a BT601 color matrix but BT709 primaries and transfer?

One source video I tested, h264AVC, containing a shot of a lot of cheap gold jewelery gives me an image containing about 80000 more unique colors outside of 16 235 conversion.

With regard to NLEs is it typical to see them squeeze the full range into 16 235 to process and does that give access to those 80000 more unique colors or a fraction of. Or is a xvYCC workflow and some sort of gamut remapping without clipping a better option?

[Gavino]

Originally Posted by Yellow_

Not sure if I'm using it correctly but if I use pcRange=true does ShowbadRGB show black just for the invalid RGB outside of gamut (assume this is sRGB), when validating 0 255 YCbCr otherwise it shows black for any invalid RGB (sRGB) outside of on 16 235 and 240?

It shows black (or the specified color) for pixels with valid colors - invalid ones are left alone.
The idea is to show you where the 'bad' pixels are (and what color they are after conversion to RGB).

An invalid pixel is one that produces a value of R, G, or B outside the range 0-255 (and hence clipped to 0 or 255) on conversion to RGB. The pcRange and Rec709 parameters indicate how the conversion to RGB is to be done, determining the equivalent matrix parameter of ConvertToRGB as follows:

pcRange: false  false  true   true
Rec709:  false  true   false  true
matrix:  Rec601 Rec709 PC.601 PC.709

I'm afraid I don't know enough about yCMS or xvYCC to answer your other questions.

[Yellow_]

Originally Posted by Gavino

It shows black (or the specified color) for pixels with valid colors - invalid ones are left alone.
The idea is to show you where the 'bad' pixels are (and what color they are after conversion to RGB).

I totally worded that wrong. I meant to say as you describe, certainly is what I'm seeing, black as non bad. :-)

An invalid pixel is one that produces a value of R, G, or B outside the range 0-255 (and hence clipped to 0 or 255) on conversion to RGB.

My understanding, which could be way off, please feel free to correct me is the following. :-):

What equates to 0 - 255 RGB from the YCbCr is dependant on your following comment the pcRange=true or false? ie

Considering so called 'legal' 8bit digital video, which when converted to RGB is to fit within the sRGB gamut (due to colour primaries) and I believe the YCbCr range used is 16 - 235 & 240 for U & V? Mapped to 0 - 255 in RGB to comply? So that would be pcRange=false. Colors that show using ShowBad are those that fall outside sRGB gamut, with regard for Rec709, would be clipped. But those values showing as 'bad' still stand a chance of being valid in a wider RGB gamut ie xvYCC, an extension of BT601/709 with suggested sRGB x 1.8 munsell colors useable?

Whether or not the original video source has sufficient colour info to saturate sRGB gamut anyway, certainly the cheap gold video shot I use to test has a lot of yellow 'bad' pixels (assume out of sRGB gamut) according to ShowBad and much still visible using pcRange=true.

If I use pcRange=true then the full 0 - 255 YCbCr 8bit range would be mapped directly to 0 - 255 RGB but this is still sRGB due to colour primaries? Clipping is still possible due to sRGB gamut not wide enough to contain all values generated?

xvYCC uses 1 to 254 range (pcRange) and I think maps 16 to 0 RGB and 235/240 to 255 RGB to maintain compatibility with BT601 BT709 Rec. (sRGB) but maps the 'out of range' YCbCr to below 0 and above 255 RGB values.

However a greater gamut than sRGB is required and greater than 8bit float precision to handle negative RGB values and values over 255 ie (0 and 1).

The pcRange and Rec709 parameters indicate how the conversion to RGB is to be done, determining the equivalent matrix parameter of ConvertToRGB as follows:

pcRange: false  false  true   true
Rec709:  false  true   false  true
matrix:  Rec601 Rec709 PC.601 PC.709

So for me to try to understand then using pcRange=false will show all 'bad' colors outside of BT601 / BT709 Rec. ie only 16 - 235/240 range is legal?

pcRange=true will use all YCbCr 0 - 255 range and map that to fit within 0 - 255 RGB which is still sRGB gamut and therefore still possibility to clip, could still see colors in the black due to clipped sRGB gamut?

I think I may have things in a muddle, complicating things with RGB gamut in considerations and the above is not meant as fact, it's me trying to understand what's going on. :-)

I'm afraid I don't know enough about yCMS or xvYCC to answer your other questions.

No problem, thank you for your time.

[Gavino]

Originally Posted by Yellow_

So for me to try to understand then using pcRange=false will show all 'bad' colors outside of BT601 / BT709 Rec. ie only 16 - 235/240 range is legal?

Yes, with pcRange=false, everything outside 16-235/240 will be marked as 'bad'.
And of course, some (indeed, most) YCbCr combinations inside 16-235/240 will also be 'bad', as shown by the demonstration at the start of this thread.

pcRange=true will use all YCbCr 0 - 255 range and map that to fit within 0 - 255 RGB which is still sRGB gamut and therefore still possibility to clip, could still see colors in the black due to clipped sRGB gamut?

pcRange=true maps the full YCbCr 0-255 to RGB 0-255; those combinations that produce an RGB value outside 0-255 are marked as 'bad'.

But those values showing as 'bad' still stand a chance of being valid in a wider RGB gamut ie xvYCC, an extension of BT601/709 with suggested sRGB x 1.8 munsell colors useable?

Yes - the function only works with the four standard Avisynth YUV->RGB conversions and doesn't consider the possibility of a wider RGB gamut.

[Yellow_]

Originally Posted by Gavino

Yes, with pcRange=false, everything outside 16-235/240 will be marked as 'bad'.
And of course, some (indeed, most) YCbCr combinations inside 16-235/240 will also be 'bad', as shown by the demonstration at the start of this thread.

Will reread the thread again for that. :-)

Yes - the function only works with the four standard Avisynth YUV->RGB conversions and doesn't consider the possibility of a wider RGB gamut.

Excellent, that's helpful to know and is why I've been using yCMS 3D LUT system to go 0 - 255 YCbCr (xvYCC) to AdobeRGB in order to get more 'valid' RGB into the image export.

Going a little OT and will start a new thread with a sample source file for testing but:

I'm finding that something appears to be preventing the additional values out to images.

Virtualdub author says Vdub can't write out wider than sRGB, which is fine, good to know. So I've recently tried AVSPMod but no better output.

Even using Wilberts Imagemagick plugin doesn't appear to get me AdobeRGB gamut range from 0 - 255 YCbCr so wonder if Avisynth is presenting the problem.

Sorry OT and will start new thread.

[2Bdecided]

Did anyone get any further with creating a function to change the "bad" U and V values into "good" ones - i.e. to desaturate each pixel individually until it generates an in-range RGB value?

It's totally beyond me, and I've hit another clip where I need it.

EDIT: I've been looking+hoping for a long time...!
http://forum.doom9.org/showthread.php?t=135248

Cheers,
David.