Quote:
Originally Posted by feisty2
think that should be 2^n-0.5 not +0.5 (255 = 2^8 - 1 not 2^8), who's bad at arithmetic now?  |
You and me alike, I guess. (Fixed.)
Quote:
Originally Posted by feisty2
I think that's just fairly wrong, midpoint goes (255-1)/2=127 by that definition and 32767 at uint16_t, and 127*257=32639 and that's pretty far from 32767..
|
The midpoint is still 128 for the AVS Convert* convention, and the scaling factor would be 32767/127 ≈ 258.01 if we were to assume a nominal chroma range of 1 to 65535 for 16-bpc YCbCr.
The nonlinear mapping you're suggesting is bad for the reason that it does not mesh with any of the full-range conventions in use,
and, being non-linear, necessarily requires a bit more computation than a linear mapping. Changing bit depth should be an essentially free operation, not one that requires branching on every other pixel.
The mappings involved for the different chroma full-range conventions are thus:
Code:
# naive full-range ([0,255] to [0,65535])
x *= 257
# Dither_convert_* ([0.5,255.5] to [0.5⋅256,255.5⋅256])
x *= 256
# H.264-like ([0.5,255.5] to [0.5,65535.5])
# note: needs to be clamped to [0,65535]
x = 257 * x - 128
# AVS Convert*-like ([1,255] to [1,65535])
# note: needs to be clamped to [0,65535]
x = (32767 * x - 32640) / 127