I Made a PQ to HLG Converter

[wswartzendruber]

Oh no, you don't want to hard-clip it!
As you're working with curves, just knee it down to the desired level.
I think that applying a knee is a far better approach than brutally clipping values above 1000 nits
It looks like I need to apply the Rec.2020 coefficients to the RGB tuple to get that pixel's luminosity, map that to a lower luminosity, get the adjustment ratio, and then adjust each of the RGB channels by that ratio...

[wswartzendruber]

Tomorrow's agenda:

1. Sober up.
2. Implement the BT.2390 tone mapping algorithm.
3. Derive unit tests for said algorithm.
4. Rule the world.

[wswartzendruber]

Here's the latest command line usage for pq2hlg:

PQ2HLG 0.1.0
Converts from HDR-PQ (HDR10) to HDR-HLG

USAGE:
pq2hlg [OPTIONS] <INPUT-FILE> <OUTPUT-FILE> --height <PIXELS> --width <PIXELS>

FLAGS:
--help Prints help information
-V, --version Prints version information

OPTIONS:
-h, --height <PIXELS> Height of the input video stream
-m, --max-cll <NITS> Brightness of the input video stream's brightest pixel [default: 10000]
-r, --ref-white <NITS> Brightness of the input video stream's reference white level [default: 203]
-w, --width <PIXELS> Width of the input video stream

ARGS:
<INPUT-FILE> Raw input video stream (PQ, BT.2020, RGB48); use - for STDIN
<OUTPUT-FILE> Raw output video stream (HLG, BT.2020, RGB48); use - for STDOUT

This utility follows the BT.2390 method for converting PQ to HLG. If a reference white value is supplied, then the input
will first be linearly scaled to bring that level to 203 nits. If a MaxCLL value has been provided, then it will be
internally adjusted by this factor. If the internal MaxCLL value exceeds 1,000 nits, then BT.2390 tone mapping will be
applied.
As you can all see, tone mapping support has now been added. It maps each color channel independently and not overall luminance.

Next up is to get DV/HDR10+ dynamic tone mapping in place.

[FranceBB]

Good! :)
Well done.
Since your project continues to grow and you're basically accepting an input file and giving an output file to the user, have you ever thought of accepting also a stream as an input and outputting a stream as an output?
You probably figured out yourself what I mean by that, but I'm gonna say it anyway: introducing Avisynth+ support. :D
There are still alternatives, but, you know, the more tools we have, the better.

[wswartzendruber]

Good! :)
Well done.
Since your project continues to grow and you're basically accepting an input file and giving an output file to the user, have you ever thought of accepting also a stream as an input and outputting a stream as an output?
You probably figured out yourself what I mean by that, but I'm gonna say it anyway: introducing Avisynth+ support. :D
There are still alternatives, but, you know, the more tools we have, the better.
Your encouragement and feedback are appreciated.

I have very little experience with Avisynth+. As it stands now, you can pass "-" as the input and output files, causing pq2hlg to use STDIN and STDOUT instead of files.

Also, this thing is nowhere near ready for prime time. While you have access to professional grade equipment, I do not. This means that I am relying solely on mathematical correctness. In other words, I need more unit tests and lots of them.

So while I am very interested in Avisynth+ support, that is not the part of the roadmap that I am focusing on right now.

pq2hlg is also really slow (at the moment).

[videoh]

OK, then, I have to ask. Given the existence of several good Avisynth filters that convert PQ to HLG, and given that processing by an EXE reading and writing raw files is unlikely to be easily usable in typical workflows, what is the reason for this project?

[wswartzendruber]

OK, then, I have to ask. Given the existence of several good Avisynth filters that convert PQ to HLG, and given that processing by an EXE reading and writing raw files is unlikely to be easily usable in typical workflows, what is the reason for this project?

I do all of my transcoding in Linux. Piping RGB48 between processes is trivial.

ffmpeg -> pq2hlg -> ffmpeg

How good are the other solutions at dealing with arbitrary MaxCLL values?

This project is principally for my own consumption, but others are free to use and share it.

[wswartzendruber]

@FranceBB

The BBC says that the HLG OOTF needs to operate on luminance and scale R, G, and B by a fixed ratio. BT.2390 doesn't seem to care one way or the other.

Can I get away with having the HLG OOTF transform each channel independently? I could make this thing overwhelmingly faster if I can.

EDIT: Rats. I don't think I can.

[wswartzendruber]

I'm now at the part of this journey where it's time to use dynamic metadata to tone map on a scene-by-scene basis.

Is there already tooling for parsing Dolby Vision out into a JSON file (or something), or do I have to create that as well? I'm assuming it's stored in HEVC extension fields throughout the stream.

[FranceBB]

Did something happen? I was pretty sure to have answered the question before and I also spent a fair amount of time on it (which I'm not gonna spend now 'cause it's midnight)... Uhm... maybe there was a connection error, anyway I'm sorry you didn't get my answer. :(

@FranceBB

The BBC says that the HLG OOTF needs to operate on luminance and scale R, G, and B by a fixed ratio. BT.2390 doesn't seem to care one way or the other.

Can I get away with having the HLG OOTF transform each channel independently? I could make this thing overwhelmingly faster if I can.

You really shouldn't. Technically you can try and see the result, but you would miss the correlation between them and I would encourage you not to do it.

I'm now at the part of this journey where it's time to use dynamic metadata to tone map on a scene-by-scene basis.

Is there already tooling for parsing Dolby Vision out into a JSON file (or something), or do I have to create that as well? I'm assuming it's stored in HEVC extension fields throughout the stream.

Ah... I've seen people talking about it in the H.265 section of the forum; unfortunately I haven't been dealing with Dolby Vision contents that much so I don't know but there must be, 'cause other people have done it.
I mean, they have extracted the metadata from the Dolby Vision, parsed it into a json and used it in x265 to re-encode, so yes, that can be done, with which tool, though, I don't know/don't remember.
I hope someone from the H.265 section will pop here to give you a hand.

[wswartzendruber]

I'm getting to the point where I'm happy with how this is working. Also, I'm turning it into a LUT generator. My vision is for HLG content to become more accessible, but converting HDR10 movies is rather difficult. This utility could be used to generate movie-specific LUTs that are tuned for the movies they will transcode. Anyone could then download the LUT for their movie (clarified by UPC code) and do the conversion themselves.

I have some more testing to do and some open source compliance issues to work out, then I'll post the 1.0.0 binary.

[FranceBB]

I'm getting to the point where I'm happy with how this is working. Also, I'm turning it into a LUT generator.

Great, the more the merrier. :)
And since we're here: Merry Christmas (late) and Happy New Year (soon).

[wswartzendruber]

Great, the more the merrier. :)
And since we're here: Merry Christmas (late) and Happy New Year (soon).
By the way, how long should a LUT be on each side? The current default for this thing is 32x32x32. Also, it generates .cube files. And do LUTs need titles?

[FranceBB]

There are two kinds of LUTs, 1D and 3D.
The 1D ones are 1024, 4096 and 16384.
Now, in your case we're talking about 3D LUTs.
For 3D, sizes are 17x17x17 (this is just barely acceptable, but I would not recommend it), 33x33x33 (average quality, generally good and widely supported by lots of programs, even the home-user oriented ones). Lastly there's 65x65x65 which is broadcast quality and it's supported by programs like AVID Media Composer and surprisingly also by the Avisynth plugin "cube". As to FFMpeg, it didn't support them when I tried years ago, but users reported on my thread that it now does so it seems that they recently introduced support (where "recently" means 2 years ago) but I didn't have time to test them on FFMpeg yet. As to the title, I do put the title in my matrices and I also put a comment inside that states what it does and who made it (i.e me). As to the ".cube" extension, you're gonna be absolutely fine as it's the most widely supported one. There are also.ilut, .olut, .lut, .3dl, .spi3d, .3dl but again, I would go with a generic .cube as it's the most supported one.

[wswartzendruber]

I've gotten FFMPEG to work with a 65x65x65 LUT, but setting the interpolation to "tetrahedral" results in everything having a really green tint.

The Cube specification from Adobe states that some software works faster with LUTs that are sized according to an even power of two. Would it be rational to host two sizes, then: 32x32x32 and 64x64x64?

[FranceBB]

Well, since I use Avisynth and AVID most of the times, I didn't really test FFMpeg as I said. If you want, cross check the result in Avisynth to see whether it's FFMpeg misinterpreting the matrix or something wrong in the cube, but I bet it's FFMpeg.

[wswartzendruber]

Aha! It's a bug with FFMPEG when dealing YUV444P10LE. When forcing the signal down to YUV420P10LE, it looks correct.

[FranceBB]

Aha! It's a bug with FFMPEG when dealing YUV444P10LE. When forcing the signal down to YUV420P10LE, it looks correct.

Ah...
Then it has to be reported in the bug tracker so they can fix it, I guess...

[wswartzendruber]

Ah...
Then it has to be reported in the bug tracker so they can fix it, I guess...

I should. I'm just dragging my feet on it. I can't believe something like this doesn't have unit tests.

[FranceBB]

Pro and cons of open source, I guess... sometimes bug are there and nobody tested things :(

[wswartzendruber]

Holy cow, Man of Steel drives one of the RGB channels all the way to the max. And reference white seems to be at 100 nits. Maybe because it's an early title?

This will be an interesting stress test for the tone mapping system. It's going to be doubling linear luminosity to bring reference white up to 203 nits. That's going to put MaxCLL over 10,000 (which, as float-64 is used internally, won't cause clipping).

[FranceBB]

Considering that it has been released in 2013, it can be.
I don't have the m2ts of that title, but what did they write in the metadata? Besides, how come it maxes out RGB? I mean, reference white at 100 nits and RGB maxed out makes me lean towards BT2020 SDR.

[wswartzendruber]

I'll get the metadata and parsed stats when I'm off work. In the meantime, here's what MPV shows for BT.709. Putting reference white all the way down to 100 nits still makes it seem a bit dark.

https://imgur.com/gallery/KRSkpFj

[wswartzendruber]

Here's what mediainfo says:


Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 7770 cd/m2
Maximum Frame-Average Light Level : 1468 cd/m2
Original source medium : Blu-ray


The main thing that stands out to me is that MaxCLL actually exceeds the Mastering Display Luminance. In other words, they graded it higher than their reference monitor could show them?

Manually parsing every pixel in the stream with a utility I wrote reveals a maximum luminance of 5,560. This is calculated by first applying the PQ EOTF to each color channel (R', G', B') and then multiplying each one by its appropriate BT.2020 coefficient, and then adding the three channels together.

EDIT: Why is color range limited? I thought a bit value of 0 represents 0 nits, while a bit value of 1023 represents 10,000 nits.

[FranceBB]

Why is color range limited? I thought a bit value of 0 represents 0 nits, while a bit value of 1023 represents 10,000 nits.

Color range limited is perfectly valid and covers the 64-940 range of values in which the color curve spreads, so you just have to re-scale the calculations. For instance, on HDR Monitors, when you turn on the waveform in the settings, it lets you choose Full or Limited, along with the curve. From my experience, PQ and HLG target end users, so they're always limited tv range, while Slog, Clog, LogC, Zlog, Flog etc target studios, so they're almost always full pc range.


The main thing that stands out to me is that MaxCLL actually exceeds the Mastering Display Luminance. In other words, they graded it higher than their reference monitor could show them?


Possibly, but since it's a movie about GFX, they probably pushed things high with spikes on graphic effects as they're fake anyway. Heck, it's hard to believe that there is a camera with enough stops to be able to go to 7770 nits now, let alone in 2013, so it's definitely about graphic effects that have been pushed high beyond their limits. I think it's pretty lame to grade something like this. Sure, the standard was just taking shape and it was very new and they probably wanted to make it future-proof, but honestly they made it look impossible to reproduce for any TV at the time as every single TV had to apply an highlights roll off and/or re-scale everything to its maximum displayable amount of nits. For me it's a "no". The HDR Documentary I made in 2019 had 400 nits 'cause it was the maximum I could go with the Canon C300 from CLog3 to HLG without "inventing" anything. For them, just because they have computer-made graphic effects doesn't mean they can go as high as they want, 'cause there are still gonna be real scenes captured with a real camera and if you push things higher than they should, not only you're literally inventing things, but you "stretch" the values so much that you end up getting lots of noise in the dark scenes, so much so that you're forced to "crush" the blacks to make it look ok and average out all the other values. For me, they got cocky with the GFX and ended up getting it wrong...

[FranceBB]

I actually checked what they used. They claim they recorded with Arri cameras in Log-C with 14 stops. If you compare it with PQ, you can easily see how the first (Log-C) is similar, although it starts slightly higher in the blacks and then it goes all the way up to exhaust itself at 14 stops, while PQ still has lots to give as it goes up way slower (and therefore it takes longer to get to the maximum value, so it has more stops, hence more nits). So, in a nutshell, they recorded in Log-C, they were able to go all the way up to 2000 nits from Log-C to PQ and then they used graphic effects to get to the "fake" 7770 nits on a 4000 nits monitor:

https://i.imgur.com/MObOuY5.png

(this is plotted with x representing stops and y representing 10bit values scaled to 100%)

[wswartzendruber]

Hold on a sec, I need to go back and redo the LUT generator to put minimum luminance at 64 and maximum luminance at 940, for each curve?

[FranceBB]

Hold on a sec, I need to go back and redo the LUT generator to put minimum luminance at 64 and maximum luminance at 940, for each curve?

I'm afraid so, but don't delete the ones you've already made, rather offer the option to the users to choose whether it's Full or Limited range.

[wswartzendruber]

Blargh.

It looks like "legal" range RGB uses 64-940 (as you said) while "extended" range uses 4-1019.

Why is this even a thing? Who said, "Oh, you know, we have a completely defined signal space with definite boundaries, but we're going to artificially not use all of it."

EDIT: I wonder if I'm ever truly going to finish this thing, or if I'm going to take another left hook.

EDIT: I'm tempted to just have this thing document that you need to expand any limited RGB range before using it.

EDIT: I think I got it. I'll follow up with unit tests tomorrow.

[wswartzendruber]

When converting to RGB48LE, FFMPEG expands limited Y values into full ones. At least, according to this hex editor I'm looking at. I see Y vales of 64 (little-endian) for the raw YUV420P10LE stream. But when I convert to RGB48LE, they become 0. Converting back to YUV420P10LE from RGB48LE brings it back to limited range.

Where's that bottle of sake... I need to drink... I'll try to wrap my head around this in the morning. Or maybe use it to drive nails into the siding of my house.

[FranceBB]

Where's that bottle of sake... I need to drink...

Kanpai!! :D xD


When converting to RGB48LE, FFMPEG expands limited Y values into full ones. At least, according to this hex editor I'm looking at. I see Y vales of 64 (little-endian) for the raw YUV420P10LE stream. But when I convert to RGB48LE, they become 0. Converting back to YUV420P10LE from RGB48LE brings it back to limited range.

Right... So, you've been doing this correctly all the time as you made calculations and analysis in RGB and also applied the LUTs in RGB before converting back to YUV, so, even if you assumed that it was full range, since FFMpeg scaled to full range while bringing things to RGB, the stats were correct. I think that's because there isn't anything such as Limited Range RGB per se (I mean, by definition) outside intermediate broadcast use, so, while it would be logical to assume that a simple "regular mapping" of values would occur when going from YUV to RGB (so 64 to 64 etc) to get a "Limited" range RGB, the FFMpeg devs probably thought that, since there's no such a thing as a flag to display Limited RGB properly, it was a sensible thing to scale those values going from limited to full and then back from full to limited.
When I was making SafeColorLimiter this discussion popped up:

there's no limited range RGB.

and

There is such a thing as "limited range RGB" . It's also known as "studio range RGB" . In 8bit RGB , 16-235 is black to white (sRGB or "computer RGB" is 0-255 black to white in 8bit, it's far more common) . Limited range RGB is used in r103 compliance checks for broadcast, and some NLE's like vegas use studio range RGB .


but also


a) no television system transmits video in RGB space, so it makes no sense to preserve some "headroom" from the 8-bit range for some metadata that does not exist in the first place.
b) YUV<->RGB conversions are ALWAYS defined on full range RGB, this limited range RGB could not be properly displayed (without some non-standard scaling applied first), or properly converted to YUV by any standard.

and

these standards exist, you should be aware of them, and we have to live with it because it's still used today.




So, in a nutshell, this is what I think led to the misunderstanding.

1) End users receive a Limited TV Range 10bit 64-940 file with a color curve applied, either HLG or PQ

2) Broadcast studios can also use Full PC Range YUV values for those curves, however it's much more likely to have Full Range only for curves like Slog, Clog, LogC, etc

3) You're checking values and doing your calculations in RGB (assuming that the content is in Full Range) as LUTs must be applied in RGB

but here's the thing: when you have a Limited TV Range YUV file that is delivered to a TV, the TV can read the flag and convert to RGB correctly (since the display is always in RGB), but if you try to deliver an RGB whose black start at 64 and white peaks at 940, there's no way to flag it as "Limited", therefore it would almost definitely be displayed incorrectly unless a YUV conversion is applied or there's some system that is aware of this. This is probably the reason why the FFMpeg guys thought that, when you convert a Limited TV Range content to RGB, you don't want to "work with it", but you want to "deliver it" and, since it wouldn't be displayed correctly if it's linearly mapped, it will expand it! So, in the end, you were doing all correctly in the first place as your calculations assumed Full PC Range, the content was in Limited, but FFMpeg took care of it during the YUV to RGB conversion!

So... I think I may have made you lose time on something that has already been addressed by a conversion you were doing, but hey, we both learned something new today, which is that FFMpeg expands Limited to Full in YUV -> RGB. Who would have thought that...

I'll leave you to the Sakè:

https://www.japan-guide.com/g9/2037_sake_02.jpg

(fun fact I: the first time I tried sakè was in 2009 with my friends in an allegedly Japanese restaurant which was probably run by Chinese pretending to be Japanese; anyway I didn't like it that much.)

(fun fact II: there are three glass in the picture I grabbed from a website 'cause one is for you, one is for me and the other is from whoever wants to grab it. I think StainlessS is more of a Beer guy, so let's see... xD)

[wswartzendruber]

I just learned how to revert a range of commits in Git.

Since you're handing out drinks, I'll let it go.:D

Hell, I have a Ryzen 3950X. I can do two movies at a time and make up for lost time.

[wswartzendruber]

I believe that as of this very moment, my tooling is complete. This includes a pgsmod utility I have written to crop and tonemap SUP subtitles.

The one thing I don't understand is why my pqstat utility returns higher MaxCLL values than what's in the metadata. Usually it's marginally higher.

[wswartzendruber]

pq2hlg and pqstat have now been consolidated under this single repository: https://github.com/wswartzendruber/hlg-tools. The original pq2hlg and pqstat repositories have been archived.

There will soon be a comprehensive HOWTO regarding converting 4K UltraHD Blu-rays to HLG for universal viewing.

[Blue_MiSfit]

Great work on this. Thank you for all your effort and making this open source!

[wswartzendruber]

Great work on this. Thank you for all your effort and making this open source!

You're welcome. Here's the release thread: https://forum.doom9.org/showthread.php?t=182499

That's the best place I could think of to put it.

[wyup]

The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range
This is very interesting. Despite HLG being the 'de facto' standard for HDR television today, I don't really understand the reason it's been chosen over PQ.


Backwards compatibility, the main reason of HLG is completely defeated as no SDR tv can play 10-bit depth and Rec.2020 colorspace. 99% SDR tvs will show an error when trying to play or select a HLG broadcast. The first HDR tvs in 2016/17 were already PQ compatible just before HLG came on.
PQ doesn't need metadata either, so it is not an excuse for live broadcast. PQ was designed from the ground up to provide perceptual brightness without banding. HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.
HLG is a compromise buiding on top of SDR. It can't be as good as PQ. HDR10 has been the standard in 4k blu-rays for years. HLG hasn't even cought-up on tv yet, apart from 2022 World Cup, the 2024 Euro and the Olympic Games. Roland Garros and Wimbledon are also in HLG. There are no native public HLG broadcasts in Europe that I know of.
Scene referred HLG is no better in practice vs PQ: Today TVs calibrate HLG at their peak brightness. 1,000 nits is the standard range in Rec.2100 HLG streams. Scaled SDR is 100 nits and HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display, and even dimmer on 500-nit ones, since reference white luminosity is lowered from default 200 nits at 1,000 to less. Being scene refered doesn't really help watch better on mosts screens. How do you brighten up a 200-nit broadcast on a 500-nit tv? The only way to bring up low HDR luminosity is to enable a brightness normalisation function on tvs. In the end, it doesn't matter that PQ is meant to watch in the dark. Home tvs should show PQ brighter on convenience aswell. Just turn up peak brightness and set a knee point. Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed.

In my opinion it would be a mistake if global HDR tv stablished on a non-existent backwards compatible and compromised HLG format. It would take years to transition to PQ, as this should be the right choice from the start.

[wswartzendruber]

This is very interesting. Despite HLG being the 'de facto' standard for HDR television today, I don't really understand the reason it's been chosen over PQ.
Well, let's go over it, then.

Backwards compatibility, the main reason of HLG is completely defeated as no SDR tv can play 10-bit depth and Rec.2020 colorspace. 99% SDR tvs will show an error when trying to play or select a HLG broadcast. The first HDR tvs in 2016/17 were already PQ compatible just before HLG came on.
In the United States, at least, these devices aren't going to be able to even tune to such a thing in the first place. They don't have ATSCv3 tuners. So...how is an error message going to appear? I guess you could try to open a file on a thumb drive or some such, but a PQ stream would also show an error here.

What HLG does facilitate, though, is easy conversion to SDR with a static lookup table. Here's one I generated:

https://wswartzendruber.net/uploads/bt2020to709-2.cube

This deterministically converts BT.2020 (both HLG and SDR) to BT.709 SDR. It's a use-everywhere solution. I generated it using OKLCH. FranceBB also has some lookup tables that he provides in his thread.

A set-top converter box connected to a BT.709 television can do the same, but only with HLG. Downconverting PQ would complicate things, although it could still be done if you had metadata and the reference white level.

PQ doesn't need metadata either, so it is not an excuse for live broadcast. PQ was designed from the ground up to provide perceptual brightness without banding. HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.
PQ doesn't need metadata? That's weird. My Man of Steel disc surpasses 7,000 nits. Just how aggressively should my TV tone map a scene that reaches 900 nits?

And how does PQ "help tvs adjust brightness"?

HLG is a compromise buiding on top of SDR. It can't be as good as PQ. HDR10 has been the standard in 4k blu-rays for years. HLG hasn't even cought-up on tv yet, apart from 2022 World Cup, the 2024 Euro and the Olympic Games. Roland Garros and Wimbledon are also in HLG. There are no native public HLG broadcasts in Europe that I know of.
In this thread, we learn the basics of how mathematics works. For consumer distribution, both PQ and HLG use 10-bit sampling. So there are however many values that uses given a limited-range signal. HLG distributes these points across 0-1,000 nits. But PQ distributes them across 0-10,000 nits. This means that although HLG has limited dynamic range compared to PQ, it has more bits to work with across that limited range than PQ does. This lets HLG increase the brightness to accommodate a well-lit environment with less banding.

And UltraHD discs probably use PQ because that market is for cinemaphiles who typically have controlled viewing environments. PQ is arguably superior in such cases.

Furthermore, HLG is not built on top of SDR. One casual glance at its transfer function should show you that it is an SDR curve that blends into an HDR one as the value increases.

Ask FranceBB about HLG broadcasts in Europe. He knows the practices of that region much better than I do. We're already going live with it in the United States and South Korea. And probably Japan (given NHK).

Scene referred HLG is no better in practice vs PQ: Today TVs calibrate HLG at their peak brightness. 1,000 nits is the standard range in Rec.2100 HLG streams. Scaled SDR is 100 nits and HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display, and even dimmer on 500-nit ones, since reference white luminosity is lowered from default 200 nits at 1,000 to less. Being scene refered doesn't really help watch better on mosts screens. How do you brighten up a 200-nit broadcast on a 500-nit tv? The only way to bring up low HDR luminosity is to enable a brightness normalisation function on tvs. In the end, it doesn't matter that PQ is meant to watch in the dark. Home tvs should show PQ brighter on convenience aswell. Just turn up peak brightness and set a knee point. Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed.
The last revision of BT.2100 I looked at didn't say anything about 1,000 nits. That was in BT.2408.

As for the rest of this paragraph, I don't understand any of it. I see words and terms I'm familiar with, but I do not comprehend whatever it is you are attempting to convey.

Actually, I do understand this:

"Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed."

And this is a lot of why HLG was created.

In my opinion it would be a mistake if global HDR tv stablished on a non-existent backwards compatible and compromised HLG format. It would take years to transition to PQ, as this should be the right choice from the start.
The BBC and NHK disagreed with you to the point that they created and entirely new system and then successfully persuaded the electronics industry to support it.

[FranceBB]

HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.

Dynamically changing metadata on a broadcast feed isn't exactly the easiest thing to implement in a chain. Where is that info gonna come from? Think about live events, like Sports. You can shoot in BT.2020 HLG directly with the various cameras and then have all the graphics etc in BT.2020 HLG in your gallery, however you can't shoot PQ. You would have to shoot log, then convert to PQ in real time with a PlanarRGB 16bit roundtrip and then overlay things like the score, the statistics etc, which also need to be in PQ and ideally at a no-so-much-different peak brightness compared to the main sport event. Can it be done? Potentially, you could set all the cameras to record Slog3 and then bring everything to, let's say, 800 nits PQ and then also have the various graphics etc at 400 nits and signal that the whole stream is 800 nits, but even then it adds a lot of complexity and it still doesn't make use of the PQ curve the way it should be used. Then think about things like commercials and other things that have probably been upscaled from a FULL HD BT709 source. What do you bring them up to? Keep in mind that most of the time a single TX Ready file is created, so you would only do one such a conversion once and then use it every time, but different live events could have different cameras with different stops and therefore different nits and all of this can get really messy really quickly.


A set-top converter box connected to a BT.709 television can do the same, but only with HLG.

Yep. That's exactly it. To answer wyup's question, essentially, if you have a BT2020 SDR TV, then it would interpret the colormatrix, but it wouldn't interpret the transfer, which means that it would show you the image being dimmer (as it would peak at 75% instead of 100%) but still ok. Using the same concept, a set top box can easily convert such a signal from BT.2020 to BT.709 SDR. Once again, we would treat it as if it was a BT.2020 SDR stream so that it would still peak at 75% rather than 100%, so the white would look dimmer, but the overall image would be displayed in an acceptable manner. This is something accepted by the industry and something that is actually being done already today. On the other hand, converting the transfer from a totally logarithmic curve like PQ to SDR 100 nits to get a BT709 output not only requires a much more complex approach with some kind of dynamic tonemapping (like what MPV does, for instance), but it would never be accepted by any studio / content provider / supplier.


This means that although HLG has limited dynamic range compared to PQ, it has more bits to work with across that limited range than PQ does.


Yeah. One could make the case about Dolby Vision with the normal 10bit standard layer and the other metadata (enhancement layer) so that a compatible decoder could output 12bit, but that only works for bluray disks. For a linear channel, this is basically impossible and of course going to 12bit as a standard source isn't the way as it may not be decoded at all. I thought we were gonna see the switch when H.266 VVC arrived and the new 8K specs were supposed to be announced, but they ultimately settled on 10bit again as a standard.


Ask FranceBB about HLG broadcasts in Europe.

Pretty much every linear UHD channel in Europe is BT2020 HLG, be it for Sports or Movies / TV Series, it's all HLG all the time.

[FranceBB]

HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display

Some Premier League matches are 180 nits. I watch them all the time as I'm a Chelsea supporter. This is how they would look like on a BT2020 SDR display by just interpreting the colormatrix and ignoring the transfer. It's still very watchable. Not great, but very watchable. On my SDR TV I generally just turn off the light in the room.

https://i.imgur.com/ur0SisZ.png

[wswartzendruber]

We have new specs for 8K? Are we staying with BT.2020 color and BT.2100 transfer functions or is there something new?

I would think they'd take the opportunity to do 12-bit... HLG could be cranked to well over 4,000 nits with that. PQ could be cranked...four times higher than with 10-bit, I suppose? I don't know as much about its limitations.

EDIT: Now that I check BT.2020 and BT.2100 again, I see support for 8K (and 12-bit) clearly enumerated. And up to 120 Hz.