YUV<->RGB lossless conversions (using higher bit depths) - requirement calculations?

[wonkey_monkey]

So we all know that YUV<->RGB conversions will involve losses if the bit depth remains the same. But I'm wondering about using higher bit depths, and whether there are calculations you can do to work out whether it will be lossless or not.

I also want to allow for clipping by scaling the RGB values to remain in range (it's for editing purposes). I'd do that by calculating the minimum and maximum out-of-range for each channel, and scaling the values back into range.

Is there any way to work out conclusively - without checking every single possible value - whether, say, 12-bit or 14-bit YUV can be converted to 16-bit RGB, and back again, without loss?

I can see in an RGB->YUV BT.2020 matrix that the lowest coefficient comes out to -0.16455, which to me suggests that you need at least 3 extra bits to ensure that each G value differs from its neighbours by at least 1. But that doesn't mean you won't need more.

I'm also wondering if there is a geometrical argument that can be made. Like, thinking of the RGB and YUV values as blocks in a 3D grid, and the requirement is that, after rotation and scaling, the centers of each block are always each others' closest neighbours?

[Z2697]

The rule of thumb is +2 bits (including BT.2020 NCL Limited Range)
Depending on the matrix and color range you can get around with +1 bit, like YCgCo Full Range.
Of source the rounding during conversion also plays a role.

[huhn]

8 bit is lossy with 32 bit math so i would give up.

i'm ignoring illegal colors here.

[wonkey_monkey]

What I mean by lossless it that you can convert back to the original colourspace and bit depth and get back identical original values.

[huhn]

yeah you can do that if you do one of them wrong but is that a conversation?

technically there are only 8^3 values.

if you do it correctly doing 32 bit still has a loss. if you not dither that it may fall in place but that doesn't make it lossless because that only works if notmath has been used on any intermediate. as soon as you do that the rounded results would be wrong and dithering or insane high bit is needed and if nothing is done to it why do it in the first place?

[wonkey_monkey]

Originally Posted by huhn

yeah you can do that if you do one of them wrong but is that a conversation?

Huh? Do what wrong? I tested it on YUV 10-bit to RGB 16-bit and back again, without doing anything "wrong", and it worked. The original values were restored.

The idea is to convert 8- to 12-bit YUV to 16-bit RGB that can be painted on and converted back without any loss to the unchanged areas. I probably have to rule out 14-bit because Photoshop only really does 15-bit RGB.

[huhn]

doing it without dithering is mathematically wrong it is just that the numbers fall in place. the math is still wrong.

if you now do something to the image cause you have converted it to RGB for a reason the results are plain wrong yes not touched parts fall back in place but the changed parts need dithering.

there are some rare exception like NN scaling but i ignore that here.

[Z2697]

Dithering is just an additional process not really necessary and did no one tell him that his english is hard to understand or is that just because I'm not a native english speaker but why the hell does he not use punctuations

[huhn]

Originally Posted by Z2697

Dithering is just an additional process not really necessary and did no one tell him that his english is hard to understand or is that just because I'm not a native english speaker but why the hell does he not use punctuations

what's that?

you always need dithering.

just encode it in 8 bit without dithering:
https://www.bealecorner.org/red/test...ient-16bit.png

[Z2697]

You need dithering not because it's incorrect without dithering but because dithering (pseudo-)randomizes the "incorrectness" (formally quantization error) through out the image yes dithering looks better and you should dither if you are doing bitdepth reduction but no dithering does not magically make things "correct" looking better does not equal to mathematically correct but looking better is important but it's not what we are takling about here.

You see, the quantization errors are technically still there, they are just randomized.

[huhn]

no dithering is needed for accurate representation.

it's to spread the error you would get with rounding to lower the perceived error. it there to fake a higher bit deep.

[Z2697]

You just agreed on what I said, then why did you think without dithering, it's mathematically wrong?
Well second thought, you are not wrong.
Without dithering, it's wrong.
But with dithering, it's also wrong. (You didn't literally say it's correct, to be fair)
It's wrong regardless of dithering or not.

This dithering debate should end now.

[wonkey_monkey]

After some more experimentation:

With Rec. 2020 or Rec. 709 coefficients, the maximum error going from 14-bit to 16-bit and back again seems to be about 0.25 (meaning you can definitely round back to the original number).

And as you might guess, the maximum error goes down by half for every additional bit of difference (e.g. max error of about 0.07 for 12-bit to 16-bit).

Choosing nicer "round" coefficients Kr = 1/4 and Kb = 1/16 reduces the errors by more than half, but only when there's more than 1-bit difference. When there is a 1-bit difference, Rec. 709 and Rec. 2020 both seem to keep just below 0.5 as a maximum error, while the simplified coefficients reach 0.5. That's for full range, though; limited range probably pushes both over, and takes away the "nice" coefficients general advantage.