How to correctly process limited color range?

[tormento]

I have never looked into this topic but now that I am trying to use 10 bit encoding on HEVC, some doubts arose.

Example: the Godfather. Looking into the M2TS I have:

Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Should I encode with the Limited flag in x265 and let the player and the TV manage it? Is it important to pass the limited flag and the color range to the encoders or they can work ok without having knowledge of it?

I know that televisions traditionally have a 16-235 range, while computer monitors work with 0-255. Capabilities of TVs have increased in the last years. Should I stretch the color range of the movie prior to apply any filter and before encoding? How?

[FranceBB]

Originally Posted by tormento

Should I encode with the Limited flag in x265?

Yes.

Originally Posted by tormento

Is it important to pass the limited flag and the color range to the encoders or they can work ok without having knowledge of it?

It is important! If the source is in Limited TV Range, your encode should also have the flag Limited TV Range. This is because the TV has to know how to convert from YUV to RGB to display it properly. For instance, if the source is 16-235 Limited TV Range, the TV will take the black as 16 and convert accordingly 'till the peak white of 235, which in SDR, corresponds to 100 nits. If the flag is not right, it will convert things incorrectly, so you might end up with washed out levels (i.e grey black etc). If you don't specify it, on the other hand, the TV will assume it and by default it's gonna assume Limited TV Range 'cause that's the most widespread one and the one officially supported by almost any standard.

Originally Posted by tormento

I know that televisions traditionally have a 16-235 range, while computer monitors work with 0-255. Capabilities of TVs have increased in the last years.

That's correct, the number of TVs that can correctly display Full PC Range contents increased with the years, however standard broadcast and media are always gonna be in Limited TV Range 16-235 for 8bit and 64-940 for 10bit planar.

Originally Posted by tormento

Should I stretch the color range of the movie prior to apply any filter and before encoding? How?

Oh God no!
If it's Limited TV Range, leave it as Limited TV Range, that's how it went on air and how it has been graded and how it's supposed to look like. In theory you could use Levels() to convert from Limited TV Range to Full PC Range and then flag it as Full PC Range, but what's the purpose of that? You will get potentially less compatibility with hardware decoders (like TVs, consoles, BD players etc).


Personal Note:
as a broadcast encoder, working in broadcasting, we make sure that everything is in Limited TV Range all the time. I've seen plenty of out of range footages, especially in the highlights (so over 235). The problem that comes with out of range footage is that anything over 235 is NEVER going to be displayed by the TV that receives it, so we always apply a soft highlight rollback, bringing down the peaks to legal values. This is extremely important and everything that is NOT in Limited TV Range won't pass the QC and the guys in the Media Management Area will send the file back to us in the Encoding Department to re-encode it with luma and chroma in the right values. About chroma, it's not 16-235, it's 16-240. During live events, this is a bit trickier as you make sure that the levels are right but you have no control over the environment most of the time (sports) and there might be sudden changes in the subject etc. To prevent overshooting, hard clipping is performed via hardware to avoid to transmit out of range stuff and potentially get fines etc. (as a very side note, if you use a waveform monitor, you'll find out that a certain private Italian TV which starts with the "M" airs on DTT pretty much everything out of range as it seems they just don't care...)



Reference: this is a 10bit clip which starts with the black at 64 (so 0.00V) and with the white that goes way over 940 (so way over 0.7V) and it's out of range, i.e illegal and should NOT be aired:




the user at the other end of the TV won't see the sky and it will instead see just a clipped out white sky if highlights rollback isn't applied to bring it back to limited tv range.

One more sample taken from another user here on Doom9 on a separate topic: Out of range - Limited TV Range

[tormento]

Originally Posted by FranceBB

Yes.

Wow! Thank you so much for the exaustive reply and examples.

I thought that BDs, above all the most recent ones from digital cameras, had full color range.

Not knowing what to do, I did not specify the range and according to your precious explanation, it seems I did the right thing

[feisty2]

Should I stretch the color range of the movie prior to apply any filter and before encoding?

convert whatever you have to fp32 (0.0 ~ 1.0, there's no "range" thing for floating point samples) before applying any arithmetic filter (e.g. convolution). when you're all done with filters, you may optionally convert the filtered clip to something like limited range int10 or int12 before encoding.

[feisty2]

Oh God no!
If it's Limited TV Range, leave it as Limited TV Range, that's how it went on air and how it has been graded and how it's supposed to look like. In theory you could use Levels() to convert from Limited TV Range to Full PC Range and then flag it as Full PC Range, but what's the purpose of that? You will get potentially less compatibility with hardware decoders (like TVs, consoles, BD players etc).

wrong, he was asking about applying filters, need I remind you how error propagation works since you have repeatedly claimed that you love math?

[StainlessS]

I hate math and anybody who professes to luv it, the're all mad.

[lansing]

I just tested encoding a commercial blu-ray to 10 bit x265 without adding the color range flag in the argument, the video still came out with the limited range flag.

[tormento]

Originally Posted by lansing

I just tested encoding a commercial blu-ray to 10 bit x265 without adding the color range flag in the argument, the video still came out with the limited range flag.

Did you use StaxRip?

[Boulder]

x265 uses "undefined" as the default and it seems that most if not all applications assume limited in that case. I've never set that flag, the only ones I do are the color primaries, transfer and matrix flags. No issues during playback on many different devices.

[lansing]

Originally Posted by tormento

Did you use StaxRip?

I use Simple X265 Launcher.

[FranceBB]

Originally Posted by feisty2

wrong, he was asking about applying filters, need I remind you how error propagation works since you have repeatedly claimed that you love math?

That doesn't mean that he should filter everything with 32bit float precision all the time. I mean, of course it's gonna be more precise and accurate and he can later let x265 dither it down to whatever bit depth he wants to encode in (8/10/12bit) however that might be an overkill in most cases. I personally encode most stuff with 16bit (either planar, stacked or interleaved) precision and then dither it down to 10bit. Even when I made the XYZ 4:4:4 Motion JPEG2000 12bit DCP for the cinema when I had to filter out issues I didn't go above 16bit. I mean, don't get me wrong, it's a very good thing that we have 32bit float in Avisynth, but many filters don't yet support it and it will definitely be a valuable asset in the future but for me we're not there yet. As to limited Vs full, once he brings something to another bit depth, values are gonna be scaled accordingly, so limited 8bit would be scaled to limited 16bit and so on. The same would happen with float afaik, so the values wouldn't span the whole 32bit float as the source is limited and it will still be limited as he dithers down to his target bit depth from 32bit. I know that you insist that there's no limited or full range in 32bit, but as I said when we were talking about Safe Color Limiter, technically there is, it's just not standard but you will get values from a limited source mapped (scaled) to limited values in float, it's just the way it is. (It's a bit like limited range RGB, there's no such a thing per specs but according to how you're mapping a limited source you might eventually get it, although it's "wrong" and you should need special equipment to support it to be properly decoded, but I don't wanna get into this whole thing once again, we're both right in what we're saying, just like we were in the Safe Color Limiter topic back then )

P.s "love" is a strong word eheheh. I definitely like and enjoy Linear Algebra, but there are things in math I don't particularly like as well, for instance statistics (which I found utterly boring) or topology (which is actually interesting but taught in such a way that makes the entertainment fade away most of the time) etc.

P.p.s I have an exam coming up at the end of the month about coding in AMPL, so minimisation/maximisation problems, simplex method etc wish me good luck 'cause I'm gonna need it.

Originally Posted by Boulder

x265 uses "undefined" as the default and it seems that most if not all applications assume limited in that case. I've never set that flag. No issues during playback on many different devices.

Yep it uses "undefined" and the vast majority of decoders assumes limited TV range and since pretty much everything is in limited TV range it's rare to have issues, but as a broadcast guy I always specify it as it doesn't hurt. Anyway most people are like "unless I have a full range file I don't have to specify it" and they're indeed gonna be ok with their limited TV range unspecified encode.

[tormento]

Originally Posted by FranceBB

wish me good luck 'cause I'm gonna need it.

Good luck!

[StainlessS]

Heres all the luck you will need :- https://www.youtube.com/watch?v=7EUytEX_XkE

[FranceBB]

Originally Posted by tormento

Good luck!

Thanks.

Originally Posted by StainlessS

Heres all the luck you will need :- https://www.youtube.com/watch?v=7EUytEX_XkE

hahahaha and you wonder why we call you "old chap" xD
Google (and its Recommendation Algorithm) is probably confused right now and is wondering why I'm listening to Gracie Fields when it was expecting me to play something like Shawn Mendes, Ed Sheeran, Anne Marie, Jason Derulo, Ellie Goulding, Becky Hill, Sigala, Ariana Grande, Taylor Swift, Ava Max, Marshmello etc xD

Anyway, thanks.

[StainlessS]

Yeah you liked that.
Its enough to make you turn vegan, "Whale Meat Again" :- https://www.youtube.com/watch?v=8Nzy1cfnKh4
And another fishy story, something about a Marlin :- https://www.youtube.com/watch?v=ZSMuTm649Hk

EDIT: To below:

I was sad when I read the news of his death last summer (1917 - June 2020).

Yeah, great lady, crystal clear voice, was a sad day indeed.

[FranceBB]

Originally Posted by StainlessS

Yeah you liked that.
Its enough to make you turn vegan, "Whale Meat Again" :- https://www.youtube.com/watch?v=8Nzy1cfnKh4
And another fishy story, something about a Marlin :- https://www.youtube.com/watch?v=ZSMuTm649Hk

Ah, Vera Lynn... I didn't know her before she came back during the lockdown with "We'll Meet Again", in fact I generally listen to Capital and I was surprised when they began playing it in the Big Top 40, but I guess people were listening to it, comparing this disease that affects us all to the second world war. I was happy when I realized she was still alive back then and I thought "wow, what a life she had, she went through so many events that made history", but then I was sad when I read the news of her death last summer (1917 - June 2020).

[Emulgator]

Whale Meat Again ROFL ! And the lilly Marlin...

[DTL]

Originally Posted by FranceBB

Personal Note:
as a broadcast encoder, working in broadcasting, we make sure that everything is in Limited TV Range all the time. I've seen plenty of out of range footages, especially in the highlights (so over 235). The problem that comes with out of range footage is that anything over 235 is NEVER going to be displayed by the TV that receives it, so we always apply a soft highlight rollback, bringing down the peaks to legal values. This is extremely important and everything that is NOT in Limited TV Range won't pass the QC and the guys in the Media Management Area will send the file back to us in the Encoding Department to re-encode it with luma and chroma in the right values.

The use of 'super-white' levels above 235 in 8bit is not very simple like treating as error and no-good for QC. It depends on the company's politic.
Good TVs/monitors do display super-white levels because it is normal and physically correct. 235 is nominal system white but not highest legal video peak white level (that is 254, -1 under 255 system sync code).
Good quality sources may compress super-whites (usually small enough highlights) in the 235+ levels range to make transition to 'sharp-limiting' softer. So better trained QC will analyse footage to test 'do real nominal white is placed around 235 code level ? ' also 'do not real highlights and actual signal samples like overshoots do not hard clipped to 235 ?' etc.
Hardclipping all code values to 235 is bad idea because it will cause blown-out hard clipped highlights and possibly a bit less sharpness with possibly more ringing because of non-linear signal distortion.
Broadcast-quality video cams must have dynamic range like 600% over nominal system white and can compress it to code values in 235..254 range. Depends on 'knee' family of controls settings. The camera setup engineer can decide to compress all available range before actual hardware hardclipping to be in 235- or in up to 254 code levels. Also there is 'auto knee' function for lazy people and it also good to see how it works. Yes - the camera setup engineer may be instructed to adjust 'knee' control to fit all real levels under 235 code level, but it will cause dimmed image with grey nominal whites or hardclipping of highlights like on cheap consumer-level cameras.
It is about real video coding levels (of brightness). But the 'aperture correction' processing may also create code levels above 235 if even processing up to nominal white levels. On level transitions. Like 'sharpening processing'. So hardclipping super-white levels to 235 will cause less signal rise time (less visual sharpness) and may end with ringing because of damaged 'conditioning for channel' by this non-linear operation.
Same applied to clipping below 16 code level but it reserved only for handling signal undershoots because negative brightness is physically illegal.

Actual auto-QC algorithm for 'super-whites' levels ls closer to calculate weight in % of super-white samples values above 235 code level to all other code values and if it > some threshold so treat is as 'no good'. Or put this frame or sequence of frames for manual QC operator analysys to make final decision if producer do not put enough efforts to adjust levels in better way for this current frame/scene content. Sharpness processing overshoots above 235 code level usually very small % of all frame samples so it pass this auto-QC test with appropriate selection of threshold.

[feisty2]

just use fp32 and be done with it.
0.0 = perceived blackness
1.0 = perceived whiteness
it is capable of faithfully representing the radiance of a supernova explosion (brightness >> 1.0) or the event horizon of a black hole (brightness < 0), if you have such cosmic entities in your video. what else could you possibly want?

[DTL]

Also about levels for color video systems: Most of data is defined in https://www.itu.int/dms_pubrec/itu-r...3-I!!PDF-E.pdf :
2.5.3 Quantization
In this Recommendation, levels 0.00d and 255.75d are reserved for synchronization data, while levels
1.00d to 254.75d are available for video.

But also with 2.5.5 Limiting of Y, CR , CB signals
Digital coding in the form of Y, CR, CB signals can represent a substantially greater gamut of signal
values than can be supported by the corresponding ranges of R, G, B signals. Because of this it is
possible, as a result of electronic picture generation or signal processing, to produce Y, CR, CB signals
which, although valid individually, would result in out-of-range values when converted to R, G, B. It
is both more convenient and more effective to prevent this by applying limiting to the Y, CR, CB
signals than to wait until the signals are in R, G, B form. Also, limiting can be applied in a way that
maintains the luminance and hue values, minimizing the subjective impairment by sacrificing only
saturation.

And Table 3:
8) Correspondence between
video signal levels and
quantization levels:
– scale
– luminance signal
– each colour-difference
signal

220 (8-bit) or 877 (10-bit) quantization levels with the black level
corresponding to level 16.00d and the peak white level corresponding
to level 235.00d. The signal level may occasionally excurse beyond
level 235.00d or below level 16.00d.
225 (8-bit) or 897 (10-bit) quantization levels in the centre part of the
quantization scale with zero signal corresponding to level 128.00d. The
signal level may occasionally excurse beyond level 240.00d or below
level 16.00d.

9) Code-word usage
Code words corresponding to quantization levels 0.00d and 255.75d are
used exclusively for synchronization. Levels 1.00d to 254.75d are
available for video

So is the resulted findings:
1. If using (rarely) RGB limited range we can freely use all valid code ranges from 1 to 254. Understanding using of sub-16 code values.
2. If using (usually) YUV limited range: Care is recommended to check if used YUV combinations, being decoded to RGB, do not cause 'out of gamut' RGB values. If so - additional limiting is recommended in the way of mostly distorted saturation if possible as the least significant image quality parameter.
But producers of syntetic YUV content must pay attention of not producing out of (RGB) gamut colors (like over-bright 100% saturated titles).
Also it depends on actual monitor decoder to RGB how it will treat 'out of gamut' YUV - may be it possible to shows at least some combinations without significant error (on WCG/HDR etc monitors). It looks for safety and old cheap and slow 8bit integer processing monitors/tv limited to 0..255 RGB values it is recommended to take care about resulted RGB gamut do not travels outside 0..255 range.
Also the 'out of gamut' situation is definetly applicable only to real video objects data encoding, not for small over/under shoots due to 'sharpness' processing.