range limited required?

[jriker1]

I have formulated what I hope to be a nice setting for 4k content, however recently noticed some individuals having range=limited in their configuration. Is this a good idea to set? I know my output shows a color range of Limited but didn't physically set that. The source is Limited as well.

[benwaggoner]

Originally Posted by jriker1

I have formulated what I hope to be a nice setting for 4k content, however recently noticed some individuals having range=limited in their configuration. Is this a good idea to set? I know my output shows a color range of Limited but didn't physically set that. The source is Limited as well.

Limited Range is, sadly, the industry default. While lots of players handle full range correctly, some will not, so Limited is safest for content to play on arbitrary players.

There's also no benefit for encoding in Full from Limited source unless you're coming from a higher bit depth. So doing 8-bit full from 10-bit limited makes sense (64-960 to 0-255 instead of 16-235 preserves more details with less dithering or banding risk). But if the source is already 64-960, there's not really a good way to turn it into 0-1023 that results in better quality.

And in the HEVC world, 10-bit decode is pretty universal today, with the exceptions mainly being old mobile devices.

[jriker1]

Thanks Ben. Do we know if ffmpeg defaults to the source value when encoding? So as part of my workflow should I check the source in MediaInfo and if Color range comes back with Full then add range=full and if it comes back Limited add range=limited? Not sure if it's required to set either way.

[Asmodian]

It does not default to the same as the source, it simply does not set that flag at all.

Players will assume limited if nothing is set, but you must set range=full if the range is full.

[jriker1]

Thanks. Trying to find it out there. When you are defining this in an x265-params block do you set range=0 if the video is Limited and range=1 if Full? Or do you spell it out?

[benwaggoner]

Originally Posted by jriker1

Thanks. Trying to find it out there. When you are defining this in an x265-params block do you set range=0 if the video is Limited and range=1 if Full? Or do you spell it out?

I spell it out, per the documentation. 0/1 normally applies to a feature that can be on/off. Whether full or limited would be 0 or 1 seems ambiguous to me.

[jriker1]

Originally Posted by benwaggoner

I spell it out, per the documentation. 0/1 normally applies to a feature that can be on/off. Whether full or limited would be 0 or 1 seems ambiguous to me.

Yeah I've seen a few examples where it shows the features that were used when you interrogate the video and it shows these / marks I think between them and some examples out there had / range=0 / ... for the output.

[asarian]

Originally Posted by jriker1

I have formulated what I hope to be a nice setting for 4k content, however recently noticed some individuals having range=limited in their configuration. Is this a good idea to set? I know my output shows a color range of Limited but didn't physically set that. The source is Limited as well.

Yours is a good question. I too have been surprised to learn all my HDR blu-rays so far (all original, of course) have color range set to Limited. I would have thought HDR would, as first order of business, go to Full range directly.

[FranceBB]

I personally always set --range limited to make sure the result is flagged as Limited TV Range and I also make sure that it indeed IS limited tv range when I encode.
Some people also turn on the hard-clipping feature in x265 like so: --min-luma 64 --max-luma 940 for 10bit planar Limited TV Range output.
I found that to be a pretty cool feature, 'cause most of the times I output a Limited TV Range 16bit from Avisynth and I checked most of the source, so it should be limited, but just to be sure, I turn on 10bit hard-clipping in x265 as well.
I wish x264 had it as well, it would be useful, really useful.

Originally Posted by jriker1

Is this a good idea to set?

Yes.
It is a good idea to set it, 'cause if you don't, the decoder (so the player) will have to "guess" and it might guess wrong.
Keep in mind that all displays work in RGB and RGB is always Full Range ***, therefore when you specify Limited TV Range, you're telling the decoder how to convert to RGB.
For instance, if your source is 64-940 Limited TV Range and you set --range Limited, then the decoder is gonna "expand" 64 to 0 and 940 to 1023 and then it's gonna display it to the screen. On the other hand, if your source is Full Range 0-1023, it will convert it to RGB without changing levels.
If you don't set the flag, the player will guess and most of them will just guess that it's Limited TV Range, so nothing bad is gonna happen if your source is limited, but something bad is gonna happen if your source is full.

Not just that: people should check using a waveform monitor regardless of the flag when re-encoding.
The reason is that I've found plenty of Limited TV Range files whose luma starts right at 64 (as it should) but the highlights exceed 940 by a lot and hence are considered out of range. What happens in this situation? Easy: if you re-encode with the Limited Flag and you send it to a TV, the TV will convert it to RGB and will expand 64 to 0 and 940 to 1023.
Fair enough, but what about the values over 940? Well, they're gonna be lost (i.e "clipped out") hence "clipping" occurs. What should be done in that case is a soft highlights roll-back so that those are gonna be brought down to 940.

This is a classic example of a 10bit video ending up out of range:



over the 0.7V (770mV) and hence over the 940 as you can see from the brown bar.
See the wall that looks white?
Well, it ain't white, there's still something there, but the end user won't be able to see it, sadly...



Few notes:

- Broadcast
In broadcast studios it is mandatory to have everything in Limited TV Range without it going out of range 'cause not only people are not gonna see it but it *might* result in fines according to which watchdog is watching, how much is out of range and the country you live in, which is why not only I make sure that everything is limited by manually adjusting the waveform, but I also add hard-clipping at the very end so that in things like Live Events like in Sports I don't get surprises.

- Overshooting
Some light overshoots are still allowed, though. The reason is that, even though you might get the waveform perfectly fine in Avisynth, once you encode with a lossy codec, the approximations that are gonna occur might lead to overshooting, so slightly out of range values. This was very much true for MPEG-2, but way less pronounced for H.264 and H.265, but it still occurs, which is why nobody is gonna tell you anything if you get some light compression overshooting every now and then.


- ***RGB is always full range
Well, I had to put * here 'cause Studio RGB exists and it's very much a thing. I recently received files from Warner Bros in MJPEG 2000 4:4:4 Studio RGB 12bit planar, which means Limited TV Range 12bit in BT2100 PQ, so not only Studio RGB exists but what I found out is that plenty of the LUTs that are professionally made and agreed between parties work in Studio RGB 16bit planar, which is why in another occasion I had to employ avsresize telling it to convert without changing levels back and forth between RGB and YUV.

[Boulder]

What should be done in that case is a soft highlights roll-back so that those are gonna be brought down to 940.

Interesting, somehow I had never thought of that. Is there any trick to do that in Avisynth while processing in 16 bits? I'd expect that filtering or resampling to a different resolution might also cause exceeds.

[Sharc]

Originally Posted by FranceBB

This is a classic example of a 10bit video ending up out of range:

.... over the 0.7V (770mV) and hence over the 940 as you can see from the brown bar.
See the wall that looks white?
Well, it ain't white, there's still something there, but the end user won't be able to see it, sadly...

Correct me if I'm wrong, but even though one might tweak the Y (luma) to be within the limited range (64....960 for 10bit, 16 to 235 for 8bit) it would not warrant that the YUV will result in legal RGB values, hence there might still be some information (details) lost on the RGB monitor after conversion to RGB.

[FranceBB]

Originally Posted by Sharc

Correct me if I'm wrong, but even though one might tweak the Y (luma) to be within the limited range it would not warrant that the YUV will result in legal RGB values

That is correct: although luma goes to 16-235 for 8bit and chroma goes to 16-240 for 8bit and you use Limiter(), once you get to RGB it's not assured that you're gonna be within broadcast safe levels for chroma as broadcast safe is referenced in the vectorscope as Bars 75%, therefore you would also need to desaturate to achieve the right values, which is exactly the issue arose when I made SafeColorLimiter() which currently DOES NOT take care of that...

Originally Posted by Boulder

Interesting, somehow I had never thought of that. Is there any trick to do that in Avisynth while processing in 16 bits?

When I'm at that point I'm generally already working on a file in our MAM with Davinci Resolve or AVID Media Composer, but Avisynth-wise that would be done with a custom made LUT which you can apply in RGBP16 with Cube().

[benwaggoner]

Limited Range is the new interlaced

[Sharc]

Originally Posted by benwaggoner

Limited Range is the new interlaced

Is 'limited range' really legacy only? Isn't it a a measure to prevent signal clipping which would produce harmonics and intermodulation, broadening the analog spectrum which may cause crosstalk and interference between adjacent broadcast (TV) channels? Even in digital systems the transmitted signal is still an analog waveform (e.g. modem on last mile copper wires).

[benwaggoner]

Originally Posted by Sharc

Is 'limited range' really legacy only? Isn't it a a measure to prevent signal clipping which would produce harmonics and intermodulation, broadening the analog spectrum which may cause crosstalk and interference between adjacent broadcast (TV) channels? Even in digital systems the transmitted signal is still an analog waveform (e.g. modem on last mile copper wires).

IIRC, assigning black to 16 and peak white to 235 was made for the D1 tape format to allow for minor fluctuations in video levels without those ancient processing chips rolling over. 16 - 10 is 6, but 0 - 6 wrapped around to 249. As in many areas of video technology, a long-term problem created to solve a short-term problem

While RF may require a limited range in the signal domain, today RF is just delivering a digital payload, video levels and RF levels are entirely decoupled. And it's been a long time since we've had to worry about 0 - 6 <> 0.

I advocated for us to use PQ in full range, because a new EOTF needed to be implemented anyway, and with PQ there cannot be anything below the minimum value because you can't have negative light, and being able to code >10,000 nits would be asking for trouble. And getting the extra 128 steps back could help reduce banding a bit. But someone had already shipped something that needed limited range for HDR, and the opportunity was lost.

And thus a lot of time was lost instead in messed-up range conversions for RGB or XYZ to YUV PQ, which took months to iron out.

[Sharc]

Originally Posted by benwaggoner

IIRC, assigning black to 16 and peak white to 235 was made for the D1 tape format to allow for minor fluctuations in video levels without those ancient processing chips rolling over. 16 - 10 is 6, but 0 - 6 wrapped around to 249. As in many areas of video technology, a long-term problem created to solve a short-term problem

While RF may require a limited range in the signal domain, today RF is just delivering a digital payload, video levels and RF levels are entirely decoupled. And it's been a long time since we've had to worry about 0 - 6 <> 0.

I advocated for us to use PQ in full range, because a new EOTF needed to be implemented anyway, and with PQ there cannot be anything below the minimum value because you can't have negative light, and being able to code >10,000 nits would be asking for trouble. And getting the extra 128 steps back could help reduce banding a bit. But someone had already shipped something that needed limited range for HDR, and the opportunity was lost.

And thus a lot of time was lost instead in messed-up range conversions for RGB or XYZ to YUV PQ, which took months to iron out.

Thanks for the explanations. I remember the 'levels wrap around' issue leading to ugly color distortions. Must be long time ago....

[benwaggoner]

Originally Posted by Sharc

Thanks for the explanations. I remember the 'levels wrap around' issue leading to ugly color distortions. Must be long time ago....

The 1980s! https://en.wikipedia.org/wiki/D-1_(Sony)

I don't think I've seen an actual wraparound problem in video this millennium. Although there are so many kinds of video glitches it can be hard to know what's the cause of any given one.

[Sharc]

Originally Posted by benwaggoner

The 1980s! https://en.wikipedia.org/wiki/D-1_(Sony)

I don't think I've seen an actual wraparound problem in video this millennium. Although there are so many kinds of video glitches it can be hard to know what's the cause of any given one.

Oh well, time flies .....
While full range or limited range and Studio Encoding acc. ITU-R BT.601-7 etc. may indeed be confusing, I found it often more difficult to ensure that the encoded YUV (YCbCr) triplets - although all valid within the specified range - result in valid RGB values after conversion to RGB (for viewing on a monitor) without compromising too much on 'vibrance' and saturation.

[benwaggoner]

Originally Posted by Sharc

Oh well, time flies .....
While full range or limited range and Studio Encoding acc. ITU-R BT.601-7 etc. may indeed be confusing, I found it often more difficult to ensure that the encoded YUV (YCbCr) triplets - although all valid within the specified range - result in valid RGB values after conversion to RGB (for viewing on a monitor) without compromising too much on 'vibrance' and saturation.

Ugh, yeah. The same color volume, but rotated 45 degrees.

In Y'CbCr one can define a very red-saturated black mathematically, even though that can't be expressed in RGB (or via light for that matter).

Half of my contributions in video pipeline optimization over the decades was asking "do we REALLY need to convert into RGB to do that? Hand me the dry erase marker..." I've seen Y'CbCr -> RGB -> HSL to figure out luminance too many times.

In my ideal world it'd all be Bayer -> ACES -> encoder. Or, ideally ACES -> ACES profile of VVC -> ACES display controller -> native panel color volume.

[FranceBB]

Originally Posted by benwaggoner

Limited Range is the new interlaced

Well, since we're talking about annoying things from the past, what about overscan/safe area?
To this very day, in 2021, some TVs still crop the image... and for what? Only 'cause back in the days timecodes and teletex subtitles were not muxed but rather displayed in the inactive lines of the screen which were not meant to be displayed to the user... Nowadays, in 2021, everything is muxed, it would be madness to put the timecode or subtitles there when you can easily just mux them in the container, but TVs still crop all around the screen, getting rid of what is actually a part of the image that should be displayed and they do that "just to play safe"...