xvYCC to Wide Gamut RGB Help Please. :-)

[Yellow_]

I thought I'd better start a new thread rather than go off topic on the "U & V ranges for valid RGB." or "gamut conversion through AVISynth"

http://forum.doom9.org/showthread.php?t=154731&page=2

What I'm trying to get to is suggested in Post's #35 & #39

and this thread:

gamut conversions through AVISynth

http://forum.doom9.org/showthread.php?t=139389&page=33

Post #644

In a nutshell it's a query about extracting as much data out of a 'full range' video source from a typical HD DSLR video camera shooting h264AVC. xvYCC into a RGB image with a wider gamut than sRGB.

My reference to 0 & 1 normalisation is the 'unity cube' and retaining / using / processing negative RGB values and positive RGB values above 1 and then gamut mapping via tonemapping or 3D LUT back to smaller gamuts as/if required.

I'm using FFmpegSource2, Avisynth 2.5.8, yesgreys yCMS 3D LUT for xvYCC to AdobeRGB, Triticals t3dlut plugin and AVSPMod 2.0.7 and Wilberts Imagemagick plugin, tried Virtualdub but author informs me Vdub is not suitable for extracting xvYCC to RGB. I've also used CoreAVC Pro and mkvmerge to matroska for Haali as alternative decompression methods incase something is happening right at the start of the process.

I'm using The Gimp and ImageJ to check unique color count and histogram values. As some means other than my eyes to quantify, 'quality'. But this is on single frames not motion.

I assume 32bit float precision is required to handle negative values.

xvYCC explained:

http://www.sony.net/SonyInfo/technol.../xvycc_01.html


Source video of mine toy gold jewelery to test, deliberately shot full range, badly white balanced under tungsten lighting to give yellow cast and generate a good helping of so called 'out of gamut' colors (for sRGB/Rec601/709 that is) but far less for xvYCC.

http://www.yellowspace.webspace.virg....com/Gold.aMOV

Rename the .amov extension .MOV ;-)

Does anyone have any suggestions as to why I seem to be struggling getting substantial more colour values that would suggest successful xvYCC to RGB conversion.

Could this be as a result of normalisation in the chain, just simply not handling negative values? Maybe happening with yv12 to yuy2, or at the t3dlut plugin or is it just happening straight away when processing with Avisynth?

I'm not a video tech guy so could well be talking total rubbish, not unusual. :-)

Is using ConvertToRGB(matrix="PC.709") all I need to handle xvYCC?

I'm getting far more (approx 30%) than a typical Rec709 YCbCr to sRGB conversion which is great but not what I feel is handling xvYCC correctly.

Sorry a lot to digest, but I'm sure there's members here who are in the know. Any help, advice or own testing would be great. Also if anyone interested in my plight has access to apps like Nuke, Adobe Premiere CS5 etc that probably handle this query no problem, it would also be very interesting to see a frame from the source (say frame 25 for consistency) to compare interpolation, unique colors against what I'm getting so far.

Also slightly separate query but I'll ask :-), regarding interpolation generally in YCbCr to RGB conversions, I've see various interpolation methods used but wonder whether with regard to further RGB image processing, grading etc whether there is good reason to use a basic interpolation in the conversion, than say Bicubic or Lanczos as first hit, the idea of edge refinement later in the chain before encoding, does it make sense to do that or go for a good balance between edge refinement like bicubic against haloing and ringing from something like Lanczos straight off at the conversion, obviously depends on heavy handedness of settings used, not trying to suggest it's the case in general.

[poisondeathray]

Isn't this impossible if you are using the native recording format ? I think you would need to access upstream

e.g. if you use a canon dslr, the Rec601 values are "baked in" . The sensor sampling in RGB uses Rec601 to convert to Y'CbCr when recording the MOV . The HDMI feed of canon DSLR's is foobar, but there might be some tweaks you can do with Magic Lantern

For other cameras, most use Rec709 , but the HDMI or HD-SDI feed still might be compromised . You have to do some tests to see if the data is valid, and not converted by Rec709 .

Some more expensive cameras like the Sony F3 have user definable LUTS and hypergammas. I think that is what you want. This avoids the baked in Rec601/709 issues and you have access to more data. I think you need a dual link HD-SDI recorder. The cheapest one is ~$10K (cinedeck). By the time Rec601/709 has been applied, it's too late. An bad analogy would be "raw" in the photo world; essentially you're accessing data father upstream that has not been processed as much

[Yellow_]

Originally Posted by poisondeathray

Isn't this impossible if you are using the native recording format ? I think you would need to access upstream

I believe that h264AVC-5 by specification can contain xvYCC. The specification appears to be the same as BT601/709 but the wider gamut is achieved by using the 1 - 15 & 240 - 254 to hold the additional data but as the primaries and matrix are the same as BT601/709 but transfer differs first incarnation was BT1361 later BT1966 I think keeps the source back compatible with BT601/709.

However when mapping the 16 - 235 range to 0 - 255 RGB the under and overshoot live in the negative RGB range and over 1 ie 255.

yesgrey has added xvYCC to RGB or AdobeRGB (which extends gamut by defining wider color primaries) to yCMS.

e.g. if you use a canon dslr, the Rec601 values are "baked in" . The sensor sampling in RGB uses Rec601 to convert to Y'CbCr when recording the MOV . The HDMI feed of canon DSLR's is foobar, but there might be some tweaks you can do with Magic Lantern

The tethered HDMI via Blackmagic Intensity or similar is a route but due to problems with that I've looked to what is potentially captured by default with many 'consumer' vidcams, even HDV and DV capture beyond the 16 - 235 range.

Some more expensive cameras like the Sony F3 have user definable LUTS and hypergammas. I think that is what you want. This avoids the baked in Rec601/709 issues and you have access to more data. I think you need a dual link HD-SDI recorder. The cheapest one is ~$10K (cinedeck). By the time Rec601/709 has been applied, it's too late. An bad analogy would be "raw" in the photo world; essentially you're accessing data father upstream that has not been processed as much

Yes, the Canon DSLR offers Picture Profiles for movie mode as well as images and there are things like Marvels Panalog profiles getting supposedly close to log capture. And there may be something to be gained there but as xvYCC are values beyond the sRGB gamut boundary I assume that muting colour is going to have the opposite effect and not generate usable xvYCC data.

Also the profiles are all based on some NLE workflow or other, it's then questionable whether that is a full range workflow, a squeezed into 16 - 235 workflow etc, are they making assumptions on what they see after the NLE's had it's way with the data.? No disrespect to what they are doing but left it out of the equation for the time being and concentrated on what the xvYCC spec offers.

Thanks for the response.

[poisondeathray]

I believe that h264AVC-5 by specification can contain xvYCC

That doesn't mean Canon uses it. I think once Rec601/709 has been applied, such as in your h264 MOV - it's game over. Yes you can access full range values (but a lot of the data isn't usable in the first place, there's data there but <15 is often black, no color). Also , that's not the same thing as converting full sensor RGB to xvYCC. You've already lost all that information. You can't recover it if it hasn't been recorded in the first place. I think you need a better recording format. I don't think you can convert Y'CbCr to xvYCC that has been converted by Rec601 .

Marvels Panalog profiles getting supposedly close to log capture.

This is what I was getting into - LogC and user definable LUTS . I don't think any of the DSLRs are close to what Alexa or F3 can record, especially not the h264 native recorded format. It's 8-bit . LogC in the Alexa requires at least 10-bit 4:2:2 , and I think the F3 requires 10-bit 4:4:4 plus a firmware update for it's LogC equivalent

For example, you can work in a wider workspace like AdobeRGB, or wide gamut RGB in after effects, but this doesn't affect your actual asset which is Y'CbCr. It's been recorded that way.

yesgrey has added xvYCC to RGB or AdobeRGB (which extends gamut by defining wider color primaries) to yCMS.

So what does that mean ? If this works, then you can take whatever that export is and use AdobeRGB workspace in after effects

I'm using The Gimp and ImageJ to check unique color count and histogram values.

I'm getting far more (approx 30%) than a typical Rec709 YCbCr to sRGB conversion which is great but not what I feel is handling xvYCC correctly.

I think you're getting what you should, because of PC.709 matrix

If you're using that gimp analyzer, remember you're only using 8-bit values with the native recorded format. The maximum you can get is 2^8 * 2^8 *2^8 = 16.7M colors (that's if you use full range, standard Rec709 range is less) . If you use 10bit you get trillons of colors 2^10 * 2^10 *2^10

Working in a wider gamut space doesn't mean your actual asset has more colors. If you put honda civic engine in a ferrari body kit, it's still a honda civic engine.

[Yellow_]

Originally Posted by poisondeathray

That doesn't mean Canon uses it. I think once Rec601/709 has been applied, such as in your h264 MOV - it's game over. Yes you can access full range values

Now this is where it gets confusing for me, if Rec601/709 has been 'applied' that would suggest the data is clipped outside of the 16 - 235/240 range? So where would the full range values come from?

Could it be the case that the sensor data gets clipped to 16 - 235 and then encoded to h264 full range? I know that the image off the sensor is less than 1920, 1700 and a bit and resized before encoding.

When using something like Avisynths histogram 'classic' mode the waveform shows full 0 - 255 data and that waveform looks well formed, smooth undulations, no gaps or compressed areas and so does the histogram, using matrix PC.601 over Rec 601 appears to provide additional data in the conversion to RGB.

And when using Rec601 from a transcode to say Huffy, the histogram compresses and spikes so too the waveform using the same 8bit processing chain as deriving full range RGB but adjusting to suit restricted range. The spiky histogram and compressed waveform suggest the data has been degraded.

(but a lot of the data isn't usable in the first place, there's data there but <15 is often black, no color). Also , that's not the same thing as converting full sensor RGB to xvYCC.

Yes with BT601/709 but is that the case when processing xvYCC data, again out of my knowledge but could the differing transfer apply here? The xvYCC gamma curve is the same for BT709 but extends below 0 and above 255.

You've already lost all that information. You can't recover it if it hasn't been recorded in the first place. I think you need a better recording format. I don't think you can convert Y'CbCr to xvYCC that has been converted by Rec601 .

What you have said above is exactly what i'm trying to ascertain. :-) To take first assume that xvYCC data is there, then looking at the processing chain and ask "in theory does or could this operation prevent the correct handling of xvYCC" after ruling out the processing chain then if data is still not there then I can assume xvYCC isn't there but having proved the chain then look to picture profiles etc to eek a bit more out. :-)

This is what I was getting into - LogC and user definable LUTS . I don't think any of the DSLRs are close to what Alexa or F3 can record, especially not the h264 native recorded format. It's 8-bit . LogC in the Alexa requires at least 10-bit 4:2:2 , and I think the F3 requires 10-bit 4:4:4 plus a firmware update for it's LogC equivalent

Absolutely not, you're correct, DSLR's are no where near but that is part of the 'game', trying to eek out what is there rather than some default handling and not gaining what could be there.

For example, you can work in a wider workspace like AdobeRGB, or wide gamut RGB in after effects, but this doesn't affect your actual asset which is Y'CbCr. It's been recorded that way.

Of coarse not. :-) The reason I'm trying AdobeRGB is because if I understand correctly as YCbCr uses sRGB colour primaries and the 16 - 235/240 range is the 'valid' range, that is what 'fits' into the sRGB gamut, in order to gain what is described in the Sony link is that the xvYCC gamut is potentially 1.8x wider and therefore as I'm trying to avoid restricting the conversion one solution is to go to AdobeRGB, yCMS offers that as a 3D LUT so it's there to be utilised.

So what does that mean ? If this works, then you can take whatever that export is and use AdobeRGB workspace in after effects

Well not really, sure if that becomes a reality someone may find it useful. :-) If yCMS offered other wide RGB gamuts then it could be one of those.

There's some nice 'RAW' apps with 32bit wide gamut processing and tonemapping algorithms like in Oloneo's PhotoEngine that might be beneficial even for LDR formats.

I think you're getting what you should, because of PC.709 matrix

If you're using that gimp analyzer, remember you're only using 8-bit values with the native recorded format. The maximum you can get is 2^8 * 2^8 *2^8 = 16.7M colors (that's if you use full range, standard Rec709 range is less) . If you use 10bit you get trillons of colors 2^10 * 2^10 *2^10

Yes, considered that, however ImageJ works at 8, 16 & 32bit and the unique count in ImageJ is identical to Gimp on all the images I've tested.

Working in a wider gamut space doesn't mean your actual asset has more colors. If you put honda civic engine in a ferrari body kit, it's still a honda civic engine.

Sure, as explained above, Rec709 into sRGB, xvYCC 1.8x more potentially. And I'm not expecting to fill a wide gamut, I feel there is more than sRGB gamut available. Re U & V Ranges and Valid RGB thread, Gavino suggests that even the Rec601/709 restricted range can generate 'invalid' colors when going to sRGB.

This is very a useful discussion for me, cheers.

[poisondeathray]

Originally Posted by Yellow_

Now this is where it gets confusing for me, if Rec601/709 has been 'applied' that would suggest the data is clipped outside of the 16 - 235/240 range? So where would the full range values come from?

This is a good point, The 7d does record a bit of usable data on the high end Y'>235 (I don't know about CbCr). But there are differences between decoders (see below), and recover it using PC matrix. You're asking if there is more, and I don't think there is.

You say h.264 supports xvYCC in the specs, but h.264 supports 4:4:4 and 16bit as well. It doesn't mean you have used it.

Could it be the case that the sensor data gets clipped to 16 - 235 and then encoded to h264 full range? I know that the image off the sensor is less than 1920, 1700 and a bit and resized before encoding.

The sensor is much larger than that, there is some processing either line skipping or pixel binning that allows for the small 1920x1080 image. (large part of the reason for the aliasing). I don't know at what stage clipping occurs, but the recorded format is Y'CbCr , and the sensor is working in RGB. Somewhere along the line a lot of data is discarded.

When using something like Avisynths histogram 'classic' mode the waveform shows full 0 - 255 waveform with data in it and that waveform looks well formed, smooth undulations, no gaps or compressed areas and so does the histogram, using matrix PC.601 over Rec 601 appears to provide additional data in the conversion to RGB.

This depends on the decoder used as well. Quicktime doesn't decode the Y'CbCr to full range, but ffmpeg does. I agree PC.601 provides additional (usable) data over Rec601, even with Quicktime decoder, but I don't think there is any more than using PC matrix with the native recorded MOV

Do you see what's happening here? When you use libavcodec/ffmpeg, it decodes using full range values, so you're forced to use a PC matrix. If you decode through another decoder e.g. through quicktime API , black is scaled to 16 and white to 235 . (ffmpeg also decodes 1920x1088 uncropped) .

Yes with BT601/709 but is that the case when processing xvYCC data, again out of my knowledge but could the differing transfer apply here?

I don't know; You said yesgrey added it, so how is that testing going ?

I think it would help if you had an actual verified sample that conforms to xvYCC

Yes, considered that, however ImageJ works at 8, 16 & 32bit and the unique count in ImageJ is identical to Gimp on all the images I've tested.

But did you check actual 16 & 32 bit content? 8-bit content from a canon DSLR is still 8-bit content. If you convert it to v210, that's a 10bit format, but the actual data is still 8-bit

This bit depth is different concept than the color working space or gamut, but I only bring it up because you're using it to check "unique colors"

in order to gain what is described in the Sony link is that the xvYCC gamut is potentially 1.8x wider and therefore as I'm trying to avoid restricting the conversion one solution is to go to AdobeRGB, yCMS offers that as a 3D LUT so it's there to be utilised.

But I think that sony link refers to RGB converted to xvYCC (or xvYCC converting to RGB). You're starting with Y'CbCr that had already been converted from RGB sensor data upstream using Rec601

HDMI specs supposedly support xvYCC ; so if you theoretically had a capture card that supports xvYCC, and a camera that outputs xvYCC through HDMI, and a recording format that supports xvYCC (Maybe some subset of h.264), then maybe it would work ?

[Yellow_]

Originally Posted by poisondeathray

This is a good point, The 7d does record a bit of usable data on the high end Y'>235 (I don't know about CbCr).

I get about 30% more unique colours exposing to full range, using Magic Lantern zebras set 1 - 254 for control. Neutral picture style and a bit more with Marvels Panalog, high or low contrast version depending on scene. For example an outdoor scene is easy to get 254. :-) harder to get down below about 10. Where as indoor 0 to 254 are easy. Whether that's usable range I haven't investigated very far and also need to consider the perceptual encoding and where those low values are coming off the sensor, signal to noise ratio and all that jaz. :-) Aware of it though.

But there are differences between decoders (see below), and recover it using PC matrix. You're asking if there is more, and I don't think there is.

You say h.264 supports xvYCC in the specs, but h.264 supports 4:4:4 and 16bit as well. It doesn't mean you have used it.

The sensor is much larger than that, there is some processing either line skipping or pixel binning that allows for the small 1920x1080 image. (large part of the reason for the aliasing).

Yes, that's how it's so easy to get the trendy shallow DOF. :-)

I don't know at what stage clipping occurs, but the recorded format is Y'CbCr , and the sensor is working in RGB. Somewhere along the line a lot of data is discarded.

No doubt. However Canon support tell me that the 550D/7D/5DmkII all capture 0 - 255. Pinch of salt though. :-)

This depends on the decoder used as well. Quicktime doesn't decode the Y'CbCr to full range, but ffmpeg does. I agree PC.601 provides additional (usable) data over Rec601, even with Quicktime decoder, but I don't think there is any more than using PC matrix with the native recorded MOV

Yes, aware of the differences with codecs, mentioned in first post that I'd also tried CoreAVC Pro which gives options to use full or restricted range.

As you say, QT converts to YUY2 and crops. FFmpeg leaves it 4:2:0 and full range. The source is 4:2:0 and it is 1920x1088 so which route appears to leave the source alone? :-)

Do you see what's happening here? When you use libavcodec/ffmpeg, it decodes using full range values, so you're forced to use a PC matrix. If you decode through another decoder e.g. through quicktime API , black is scaled to 16 and white to 235 . (ffmpeg also decodes 1920x1088 uncropped) .

Ok, but nothing suggests to me that the source is restricted range expanded to full by ffmpeg, if anything I'd say QT is messing with the source. :-)

Again for testing I'm assuming full range, but take your point about possibility of in camera levels change negating xvYCC.

I've uploaded two images, same frame one decoded with QTSource and the other FFmpeg. Used histogram classic, I know the exposure hard clips at 255 and used it to illustrate which waveform looks the least affected? No conversion to RGB has been done to the source to affect the waveform.

http://www.yellowspace.webspace.virg...a.com/boat.zip

The same thing happens with transcoding DSLR to intermediates like Huffy, by default ffmpeg will scale the DSLR source into 16 - 235 of huffy's YUY2. Not lossless. :-)

I don't know; You said yesgrey added it, so how is that testing going ?

Very well. I like it. :-) Interpolation methods I think is the only other query with the conversion to RGB.

I think it would help if you had an actual verified sample that conforms to xvYCC

Certainly would, will have a look, some of the Canon vidcams have a xvYCC mode and maybe that mode, not sure what that does, maybe not in the mode the source is 16 235/240. :-)

But I think that sony link refers to RGB converted to xvYCC (or xvYCC converting to RGB). You're starting with Y'CbCr that had already been converted from RGB sensor data upstream using Rec601

Am I though? Also the header file using mediainfo provides a BT1361 along with BT709 for transfer? Not proof but it's there. :-)

It's the rest of the Avisynth chain now I'd like to ascertain can pass xvYCC through and out to RGB and maybe compare against other apps too.

[poisondeathray]

Originally Posted by Yellow_

Ok, but nothing suggests to me that the source is restricted range expanded to full by ffmpeg, if anything I'd say QT is messing with the source. :-)

The cropping is definitely messing with the raw avc stream.

But if you look at how you measured exposure for the shot and the decoded image, which looks more similar to how you shot it?

It's really the same thing. You can expand the range of the QT decoded image or compress the range of the ffmpeg decoded image.

I know that boat picture was just to show the Y' waveform, but that accompanying screenshot used Rec601 for the RGB conversion. You can tell because of less detail in the superbrights.

But if you shot slightly more overexposed, which decoding method might clip more data, and which would leave you more "headroom" so you could recover the data ?

The same thing happens with transcoding DSLR to intermediates like Huffy, by default ffmpeg will scale the DSLR source into 16 - 235 of huffy's YUY2. Not lossless. :-)

Directly with native file or though avs ? this doesn't happen to me with v210 with ffmpeg or encoded to huffyuv/lagarith/ut video etc... through vdub

If you have Y' 0-255 (e.g. decoded with ffms2) it will pass through Y' 0-255 untouched with those methods

If this is true, this would suggest ffmpeg is decoding it differently that ffms2 or libavcodec in other decoders, or you're incurring a Rec601/709 RGB conversion somewhere in your workflow

Certainly would, will have a look, some of the Canon vidcams have a xvYCC mode and maybe that mode, not sure what that does, maybe not in the mode the source is 16 235/240. :-)

This would be interesting to test; see if you can get samples.

If you have a verified sample, then you should be able to use yesgrey's yCMS to go xvYCC <=> RGB , provided you have a format that is able to record xvYCC

Am I though? Also the header file using mediainfo provides a BT1361 along with BT709 for transfer? Not proof but it's there. :-)

You're right, you can't be sure, but I'm fairly positive xvYCC wasn't used for the conversion in camera

[Yellow_]

Originally Posted by poisondeathray

The cropping is definitely messing with the raw avc stream.

But if you look at how you measured exposure for the shot and the decoded image, which looks more similar to how you shot it?

It's really the same thing. You can expand the range of the QT decoded image or compress the range of the ffmpeg decoded image.

Well that's difficult because I didn't use a calibrated field monitor :-), just the LCD with a brightness control, not that I judge exposure with that, it was done before Magic Lantern so I shot a couple of exposures and checked the camera's histogram and blinkies, I know the boat's white hood blew out. Also shoot RAW+jpeg of each scene for the EXIF data and the .thm file (renamed jpg) is useful too, they show the same hard clip blinkies and luma histogram.

I know that boat picture was just to show the Y' waveform, but that accompanying screenshot used Rec601 for the RGB conversion. You can tell because of less detail in the superbrights.

Yes, just vdub's 'extract output to clipboard' and default RGB conversion. Quick and does the job for the histogram. Thanks for the tip on BT601 luma.

But if you shot slightly more overexposed, which decoding method might clip more data, and which would leave you more "headroom" so you could recover the data ?

I think personally I like to expose to the full range and get the sort of waveform distribution of the ffmpeg boat image and have a suitable workflow for that rather than additional image processing to stretch out the levels after they've been squeezed by the decoding codec with the QT route.

As it's not just the luma to consider, for example in the Gold mov white balance is way off so there's a lot of yellow and that is going to be well out of 'legal' range. According to Gavino's ShowBadRGB script something like 80000 unique colors are out of range based on 16 - 235/240 as legal (that is the sRGB gamut) and 21000 of those are out of range based on 0 - 255 as legal (beyond sRGB gamut), mostly the yellow/orange I'd think. Maybe I'm just using the script wrong.

I wonder whether I'm already getting close to xvYCC but topping out because I can't get decode into a wider RGB gamut via Avisynth to see the extent of the additional data. Which makes me question the chain.

Which would be the better way to try to correct that white balance? Working on evenly distributed full range colours via ffmpeg where there is undoubtedly a finer gradient or on a squeezed QT decode where it's all got to be teased out or expanded otherwise working on what I'd consider compressed yellow/orange gradient with less unique colours in fewer bins?

It's a test I was going to try out, have some very blue/cyan badly white balanced source too which again will surely out of sRGB/Rec709 gamut.

Directly with native file or though avs ? this doesn't happen to me with v210 with ffmpeg or encoded to huffyuv/lagarith/ut video etc... through vdub

]If you have Y' 0-255 (e.g. decoded with ffms2) it will pass through Y' 0-255 untouched with those methods

If this is true, this would suggest ffmpeg is decoding it differently that ffms2 or libavcodec in other decoders, or you're incurring a Rec601/709 RGB conversion somewhere in your workflow

It was on the CLI both on Linux and Windows. It's a workflow seen a lot using ffmpeg to batch transcode on the CLI with a shell script. But it was to huffy which can be RGB so I'd better double check, although I'd assume RGB is not the default color model for a h264 to huffy transcode :-) Didn't include any swscale flags.

This would be interesting to test; see if you can get samples.

If you have a verified sample, then you should be able to use yesgrey's yCMS to go xvYCC <=> RGB , provided you have a format that is able to record xvYCC

Something's bound to come up on a forum somewhere.

You're right, you can't be sure, but I'm fairly positive xvYCC wasn't used for the conversion in camera

Canon should be back to me soon with a bit more info hopefully.

But for now I'd really like to establish whether the rest of the tool chain supports xvYCC to RGB.

[poisondeathray]

Originally Posted by Yellow_

As it's not just the luma to consider, for example in the Gold mov white balance is way off so there's a lot of yellow and that is going to be well out of 'legal' range. According to Gavino's ShowBadRGB script something like 80000 unique colors are out of range based on 16 - 235/240 as legal (that is the sRGB gamut) and 21000 of those are out of range based on 0 - 255 as legal (beyond sRGB gamut), mostly the yellow/orange I'd think. Maybe I'm just using the script wrong.

I haven't had a chance to take a good look at that showbadrgb script yet, but some cameras clip color before Y'. So if you are monitoring exposure through luminance (like a waveform monitor or zebras), you might think you're safe, but chroma can be clipped earlier . Canon's typically clip red channel early. Your gold jewellry clip demonstrates that too

I wonder whether I'm already getting close to xvYCC but topping out because I can't get decode into a wider RGB gamut via Avisynth to see the extent of the additional data. Which makes me question the chain.

I don't know. Some verified samples would help give a better idea, otherwise everything else is a guess. If you look at the curve in the sony link, I don't think these results are close at all. I think the problem is it has already been converted to Y'CbCr with something-other-than xvYCC . I don't think anything is salvagable beyond using PC matrix - and that's more than most programs are able to do.

I know your original question has to do with avisynth, but part of the problem might be the software you're using. For example with AE, you have color management and can assign color profiles and color workspaces and "see" more of the data. You can import avs directly as well. But it doesn't have xvYCC profile , so I don' t know how it compares to Wide Gamut RGB. You can get negative histogram values with some combinations, but the differences don't seem as large as suggested that sony link . You do get more "unique colors" according to gimp - whatever that means - But if you adjusted values in Y'CbCr before RGB conversion, I bet you could increase "unique colors" as well. All you are doing is making sure no channel is clipped (or CbCr aren't too extreme). Tell me if you want the samples uploaded.

Which would be the better way to try to correct that white balance? Working on evenly distributed full range colours via ffmpeg where there is undoubtedly a finer gradient or on a squeezed QT decode where it's all got to be teased out or expanded otherwise working on what I'd consider compressed yellow/orange gradient with less unique colours in fewer bins?

Do you mean after it's shot in post , or in camera settings / picture profiles ?

If you're going by the "fewer bin" analogy, I think it's better to use to a higher bit depth intermediate. Then you have even more inbetween "bins" (even though they are interpolated) , and the histogram is even smoother when making adjustments. If you're using RGB, then I would use full range. If using Y'CbCr intermediate then you either have to correct for levels in Y'CbCr space before RGB conversion, or make sure the program you're using doesn't do a Rec601/709 conversion and clip data . You could also use smoothtweak to adjust CbCr into more legal ranges before going to either intermediate format