10-bit safe colors

[leoenc]

I've posted this in the AutoAdjust forum with no replies so it might be more of a general AVIsynth topic:

Is it possible to use AVIsynth to limit the chroma and black levels of 10-bit video?
According to bt.709 specs we need to keep the Y within 64-940 and UV at 64-960, and black has to be limited at 64.

[feisty2]

here's the math
(1023-a)/(a-0)=(940-x)/(x-64)
#simplified version: (1023-0)/(a-0)=(940-64)/(x-64)
a is the value in full range (0-1023)
x is the target limited range value (64-940)
now all you have to do is convert the expression above to "x = ???? a" form and wrap it in Dither_lut16 ()
I don't wanna do it, cuz I might be the laziest guy on this planet :P
EDIT:
fine, here
Dither_lut16 ("x 876 * 1023 / 64 +", y=3, u=2, v=2)
Dither_lut16 ("x 896 * 1023 / 64 +", y=2, u=3, v=3)

[colours]

What feisty2 posted is correct if your source is full-range 10-bit and is zero-padded in the MSB instead of LSB.

If all you want is to clip the Y/Cb/Cr values for a source that is already nominally limited-range 10-bit (with peaks/troughs that might exceed the 16/235/240 limits because of compression artifacts or whatever), that's easily done with Dither_lut16 too.

Dither_lut16(expr="x 4 / 16 max 240 min 4 *", yexpr="x 4 / 16 max 235 min 4 *", u=3, v=3) # zero padding in LSB
# OR (don't use both)
Dither_lut16(expr="x 256 / 16 max 240 min 256 *", yexpr="x 256 / 16 max 235 min 256 *", u=3, v=3) # zero padding in MSB

But of course we don't know that for sure, since you've not provided any information about your source. We're not psychic!

[leoenc]

Thanks.
How can I tell if my source is full-range 10-bit and is zero padded in the MSB instead of LSB?
All I know about it is that its a ProRes high profile.
So far I was using FFvideoSource with the 10-bit hack to decode it, followed by f3kdb_dither, but I'm not sure I'm doing it right.

[colours]

The builds of FFMS that have the enable10bithack option return the result with zero padding in the least significant bits. Not every source filter does the same thing, which is why confirming this is pretty important.

If you want to confirm whether your source is limited-range or full-range, put a Histogram() (http://avisynth.nl/index.php/Histogram) after the f3kdb_dither and look out for particularly dark or bright scenes. If you see white spots extending well into the brown regions, that means your source is full-range. If you don't, it could be either full-range or limited-range, and you should check other dark/bright scenes. (This is not an exact science and necessarily involves some guesswork.)

If it turns out that the source is limited-range (which I'm guessing it is, because full-range >8-bit is kind of rare), you can use either the Dither_lut16 line I posted before or the keep_tv_range option in f3kdb_dither.

[hanfrunz]

To get legal and valid colors it's not enough to bring Y'CbCr values into the needed boundaries. You also have to limit in R'G'B' color space.

Some very interesting documents about this topic:
http://www.tek.com/document/application-note/preventing-illegal-colors
http://www.poynton.com/notes/video/Confusion.html

regards
hanfrunz