I Made a PQ to HLG Converter

[wswartzendruber]

I wrote a small utility for converting raw video streams from PQ to HLG. It's available on GitHub. (https://github.com/wswartzendruber/pq2hlg)


USAGE: pq2hlg [format] [nits] [width] [height] [input] [output]
format - The raw pixel format to use. Supported options are:
rgb48
nits - Luminance value of the brightest pixel in the input.
width - Width of the video stream in pixels.
height - Height of the video stream in pixels.
input - Raw RGB48 input file; use - for STDIN.
output - Raw RGB48 output file; use - for STDOUT.


My motivation was that I wanted to rip my 4K UltraHD library, but wanted to do so in a viewing format that is as universal as possible, even if that entails some amount of compromise. I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I learned that ffmpeg supports the use of lookup tables, but couldn't find any such thing for cheaper than a couple hundred dollars. Others were available from the BBC under license.

All in all, I got quite fed up with the situation and started researching PQ, HLG, metadata, gamma curves, etc. until I got nauseated and started work. Then I studied some more and then went back to coding. Wash, rinse, and repeat a few times. I'm at the point where I'm comfortable with releasing my work to the public. Principally, it uses, to the best of my understanding, the algorithm published by BBC Research & Development (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf).

Future improvements may include passing dynamic metadata in for variable PQ->HLG tone mapping. I definitely want to add YUV10 support as soon as I can figure out how the U and V components are encoded.

This is also the first thing I've written in Rust, so it's probably not going to be as idiomatic or as proper as it could be.

EDIT: Attached a screenshot from Alita: Battle Angel using this utility, taken from a nightly build of VLC 3.0.9 on a SDR monitor. The video codec in use is VP9 Profile 2.

EDIT: I also forgot to mention: This isn't really suitable for any brightness value aside from 1,000 nits. If a PQ video has a peak brightness of less than 1,000 nits, pass 1,000 nits in as the second argument anyway. If any pixel ever exceeds 1,000 nits, then external tone mapping really needs to be applied to the PQ source before passing it into this utility. A peak brightness of other than 1,000 nits can be specified, but this will cause reference white to be pushed up or down. This addendum is in accordance with ITU-R BT.2390-8 Section 7.2.

[Blue_MiSfit]

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.

However, I don't understand how (without some other component) it can be backwards compatible from a color gamut standpoint. If you're doing HLG with Rec2020 or P3, from what I understand that will NOT look correct on a non wide-gamut display. Gamut conversion is not trivial.

So... how does that work?

Your screenshot looks at least mostly correct to me, so I'm wondering how you / VLC converted the gamut from Rec2020 to Rec709.

[wswartzendruber]

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.

However, I don't understand how (without some other component) it can be backwards compatible from a color gamut standpoint. If you're doing HLG with Rec2020 or P3, from what I understand that will NOT look correct on a non wide-gamut display. Gamut conversion is not trivial.

So... how does that work?

Your screenshot looks at least mostly correct to me, so I'm wondering how you / VLC converted the gamut from Rec2020 to Rec709.
It's entirely on the player to do that, as you have surmised. VLC does an outstanding job of it while Kodi doesn't even seem to try (everything looks washed out).

My assumption with VLC is that it is using some kind of logarithmic roll-off to compress the color dynamics. However they do it, the results are outstanding.

EDIT: I have to take that back. I put Alita through Kodi and Rec.2020->Rec.709 mapping was indeed happening. That other clip I played must not have had the right colorspace signaled.

[scharfis_brain]

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.


Interestingly enough, I do it the other way around:
Watching most SDR content with HLG.
This works quite well for most movie-like content
and b0rks most of the time with video-like content.

It seems that gradual luminance saturation (and chrominance desaturation) of the SDR film-curve in bright areas of the image fits the HLG-curve quite well.

On the other hand SDR with video-content peaking colors will look terrible with HLG applied.

My Sony OLED-TV allows to force the use of HLG on any type of input signal.

[wswartzendruber]

I've attached some more screenshots for demonstration. All in all, the HLG-on-SDR picture is somewhat dim, but I find that this leads to a more pleasant viewing experience, especially with the lights turned out. Reference white comes in on my monitor somewhere around 100 nits and bright whites come in around 200. So the specular highlights are noticeable as the director intends. In contrast, the SDR grading of the film looks washed out and loud.

[FranceBB]

My motivation was that I wanted to rip my 4K UltraHD library, but wanted to do so in a viewing format that is as universal as possible, even if that entails some amount of compromise. I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I learned that ffmpeg supports the use of lookup tables, but couldn't find any such thing for cheaper than a couple hundred dollars. Others were available from the BBC under license.


Oh, really...? *coff coff* Link *coff coff* (https://forum.doom9.org/showthread.php?t=176091)

By the way, the fact that people were charging a lot of money for those matrices of linear transformation is exactly what made me want to share them for free...
Of course, as I always said, the ones made by AVID, BlackMagic, BBC and so on have had a lot of engineers working on them, while I made mine together with my boss in a studio room and they have been tested against a single professional monitor by Sony and not a wide range of them, but still...


All in all, I got quite fed up with the situation and started researching PQ, HLG, metadata, gamma curves, etc. until I got nauseated and started work. Then I studied some more and then went back to coding. Wash, rinse, and repeat a few times. I'm at the point where I'm comfortable with releasing my work to the public. Principally, it uses, to the best of my understanding, the algorithm published by BBC Research & Development (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf).


Yep, starting with the BBC Whitepaper is always a good starting point as it basically explains how the conversion should be done.



This is also the first thing I've written in Rust, so it's probably not going to be as idiomatic or as proper as it could be.


I don't actually know rust as programming language as I never studied it, but still hats off for researching and implementing your own converter.

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.


Correct.


However, I don't understand how (without some other component) it can be backwards compatible from a color gamut standpoint.


In fact, it isn't.


If you're doing HLG with Rec2020 or P3, from what I understand that will NOT look correct on a non wide-gamut display. Gamut conversion is not trivial.

So... how does that work?

It's not converted. Let me clarify that.
The whole reason why we want to air with HLG, as broadcasters, is that we wanna have both HDR and SDR on the same channel, BUT (and here's the catch) most people tend to think that when we say "SDR" we're actually referring to Linear BT709 100 nits, but we are not.

When we air a 4K UHD stream via satellite it has two things mainly: the colormatrix and the color curve.
If you have a new 4K UHD TV, it's gonna read the stream, understand the BT2020nc, understand the HLG curve and display the color curve correctly, thus showing you the right amount of nits (up to 1000) that the content has. You're definitely watching an HDR content.
If you have an old 4K UHD TV, however, when it's gonna read the stream, it's gonna correctly interpret the BT2020nc, but it's just gonna ignore any information relative to the color curve, thus displaying the content as BT2020nc SDR 100 nits. The more a content is graded towards HDR, the worse it's gonna look on BT2020nc SDR panels, the more is graded towards SDR, the less you're gonna have dynamic range on HDR monitors, hence the compromise and the 1000 nits limit.
For quite some time I had to grade live sports events to 400 nits maximum 'cause many people didn't have HDR panels at the very beginning.

For the records, the reason why you can't do that with something like PQ is that if you ignore the color curve information on a PQ stream, then everything is gonna look very dull, as PQ is a proper logarithmic curve, while HLG is indeed Hybrid and compromises details in the shadows to make it SDR compatible but still has room for the peak brightness to be way higher than 100 nits.

Out of curiosity, this is how you're gonna see the very same HDR HLG signal on two different monitors: one BT2020nc SDR which ignores the color curve and the other that reads and interprets the color curve:

https://i.imgur.com/S8pJItb.png
https://i.imgur.com/jQHYj7A.png

As you can clearly see, it's a compromise and it's never gonna look exactly as we intended it to look, which is something that kinda pisses us all off (broadcast encoders), but hey, we barely have the bandwidth to have 25 Mbit/s H.265 on all our 4K channels, so it would be madness to have two 4K channels for everything...

[videoh]

I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I cough like FranceBB too (no, not the virus!):

http://rationalqm.us/hdr/DGPQtoHLG_1.0.rar

It's an Avisynth filter that supports both SW mode and CUDA mode.

[wswartzendruber]

Yep, starting with the BBC Whitepaper is always a good starting point as it basically explains how the conversion should be done.
Am I crazy for wanting to add scene-specific tone mapping to compress anything over 1,000 nits via dynamic PQ metadata? This was a goal from the beginning, otherwise, I would have written this thing to just output a static LUT.

And what's this color curve you speak of? I don't remember reading anything about that. I know that I have to weigh R, G, and B by differing factors to determine the overall luminance of a PQ pixel, but that's about it. Everything else seems to treat R, G, and B independently of each other.

Oh, and here are some more screenshots.

[wswartzendruber]

I've made some modifications. Here's the new help screen:

PQ2HLG 0.1.0
Converts from HDR-PQ (HDR10) to HDR-HLG

USAGE:
pq2hlg [OPTIONS] <INPUT-FILE> <OUTPUT-FILE> --height <PIXELS> --width <PIXELS>

FLAGS:
--help Prints help information
-V, --version Prints version information

OPTIONS:
-h, --height <PIXELS> Height of the input video stream
-r, --ref-white <NITS> Brightness of the input video stream's reference white level [default: 203]
-w, --width <PIXELS> Width of the input video stream

ARGS:
<INPUT-FILE> Raw input video stream (PQ, Rec.2020, RGB48)
<OUTPUT-FILE> Raw output video stream (HLG, Rec.2020, RGB48)

This utility follows the BBC R&D method for converting PQ to HLG. Any pixel brighter than 1,000 nits will be clipped and
no tone mapping will be applied. If a reference white value is supplied, then the input signal will first be linearly
scaled so that the reference white level becomes 203 nits.

I would appreciate someone letting me know if linear scaling for reference white adjustment is the way to go. When it comes to Alita: Battle Angel, I've noticed that the darker areas of the picture are marginally darker than with the native Rec.709 grading from the 1080p Blu-ray. Again, this is when comparing in VLC on a SDR monitor without WCG.

As is the usual case, there are a few more screenshots. These were taken from an HLG encode that uses the new reference white scaling. I sampled the credits and determined that this movie's PQ grading uses 157 nits. So that was linearly scaled up to 203 across all three color channels.

In the future, I intend to add support for Kim-Kautz tone mapping.

EDIT: Imgur Album of Latest Screenshots (https://imgur.com/a/IN63ytr)

[FranceBB]

I'd suggest you to use an image host like imgur 'cause attached screenshot have to be approved first and that can take time (and it also uses space from the Doom9 server).

[wswartzendruber]

Done. :-)

[VictorLS]

wswartzendruber
Once you're strong in HLG may be you can help us (SAT-amateurs) in writing good working (giving near real colors) HLGtoSDR shader https://forum.doom9.org/showthread.php?t=176909 ?

[Blue_MiSfit]

Following up on some posts from above, it sounds like the plan is to have a single 4K BT. 2020 HLG stream that will serve two groups of customers:

1) Folks with modern 4K TVs with HLG and 2020 support who will see the extended dynamic range that HLG brings in addition to the wide 2020 color gamut

2) Folks with first generation 4K TVs that are SDR, but still support 2020/WCG. They get the wider gamut, but and will still be able to watch the content acceptably even being totally naive to HLG. I imagine folks with modern 4K TVs that have HDR10 and DoVi, but still lack HLG for some reason (like my older LG B6) would fall into this category as well.

This leaves everyone else, but you'll still uplink a standard HD SDR 709 feed for them?

So... the industry is going through the hassle and compromises of HLG just to help out that small group of 4K early adopters with SDR panels that support BT. 2020? Seems a little silly to me, honestly! How big is that user-base, and how much of even the P3-D65 gamut will they even see (forget about 2020)?

There must be some other reason for using HLG. Is it just the momentum around HLG in production workflows (cameras, etc)?

[wswartzendruber]

Following up on some posts from above, it sounds like the plan is to have a single 4K BT. 2020 HLG stream that will serve two groups of customers:

1) Folks with modern 4K TVs with HLG and 2020 support who will see the extended dynamic range that HLG brings in addition to the wide 2020 color gamut

2) Folks with first generation 4K TVs that are SDR, but still support 2020/WCG. They get the wider gamut, but and will still be able to watch the content acceptably even being totally naive to HLG. I imagine folks with modern 4K TVs that have HDR10 and DoVi, but still lack HLG for some reason (like my older LG B6) would fall into this category as well.

This leaves everyone else, but you'll still uplink a standard HD SDR 709 feed for them?

So... the industry is going through the hassle and compromises of HLG just to help out that small group of 4K early adopters with SDR panels that support BT. 2020? Seems a little silly to me, honestly! How big is that user-base, and how much of even the P3-D65 gamut will they even see (forget about 2020)?

There must be some other reason for using HLG. Is it just the momentum around HLG in production workflows (cameras, etc)?
If you are speaking to me, then you misunderstand. This isn't about the industry or what they want, this is about me and what I want. With that said, I am also a very big believer in open source software and in sharing work. At least a few other people will likely find this useful as well. Eventually, that is.

I want a process to rip a 4K disc such that I have a single encode that can play back on any device with a competent player. Right now, I have Alita on my desktop (Rec.709), my phone (Rec.709), and it will also be on my HTPC when I get that, which will be Rec.709 first and then Rec.2020 later on. I want a single encode for all of these devices.

And this is because I find PQ to be an absolute joke for content distribution, and a bad one at that. I am utterly convinced that the reason Dolby persuaded everyone to go along with it was because it created an artificial need for Dolby Vision, which they would in turn profit from.

My goal is to create a single workflow for PQ content that allows me to adjust and transcode anything I have into a consistent HLG stream. This means, for example, that each transcode will have a reference white level of 75% signal strength (or as best as I can manage this). VLC and Kodi already handle 2020->709 amazingly well as they are, so I don't have a colorspace issue. And I don't have a luminosity problem, either, because I've determined that VLC clips HLG at precisely the 75% mark, which is exactly what I want it to do.

Something I'm really interested in at the moment is scene-specific tone mapping via dynamic metadata, since I have to stay within 1,000 nits.

But regardless of how I get to HLG, I will never need metadata during playback. What a farce. I seriously cannot believe everyone went along with this.

EDIT: Well, I guess the BBC and NHK didn't go along with it.

[Blue_MiSfit]

Oh no, I wasn't directing that at you :) I was directing it at FranceBB's description of their transmission process :)

To be honest though, as someone who's implemented PQ based workflows (HDR10 and Dolby Vision) for streaming services, it all works fine. For HDR10, Most of the displays just ignore the metadata and do whatever they want.

The Dolby Vision stuff is great, and works really well for preserving high luminance details and getting consistent color reproduction, especially on cheaper devices (TCL TVs etc). It also kicks in special rendering pipelines on the displays that do less bad things, and can result in sane output settings. People do wacky things to their TVs ;)

[FranceBB]

Derek was referring to this post (https://forum.doom9.org/showthread.php?p=1908011#post1908011)

As to the reason, believe it or not, that's one of the main reasons. What happened is that some companies reacted quicker than others in a period in which there was a huge technological switch and that made things complicated. The first thing is of course about manufacturers that released TV that were able to see BT2020 only without the HLG color curve as I mentioned above. In that period for instance (I believe 2015) we rolled out UHD in an H.265 stream with BT2020 SDR 100 nits. That worked like a charm. Many companies did the same but when we had to release it across the group as well like in Italy and other countries two years later, there were HLG and PQ already and we wanted to have a process as consistent as possible by launching HDR but also keeping old users with the old UHD BT2020 SDR option to be able to see contents. Being like "you can't watch UHD anymore unless you buy a new TV from today" after as little as two years wasn't really ideal for users... So we ended up with the HLG compromise as we didn't really want to air two separate 4K streams and waste twice the bandwidth, especially 'cause it's extremely costly on HotBird as it's really overcrowded. Sure, people may say that internet based decoders will be the future, but not everyone has a good connection and you're also moving the cost from a satellite to a CDN which isn't exactly cheap (although it's way cheaper than the satellite), which is why we're inviting customers to subscribe to our fiber internet offer.
Anyway, I'm digressing.
The thing is that this is our situation and many companies did the same. You also mentioned the cameras and that's true, for live sporting events they can shoot directly in BT2020nc HLG 10bit which is a big added bonus 'cause you can easily connect them all via SDI, route them to your beloved video server, adjust them and you're fine. You can also live downscale and tonemap the signal to FULL HD Linear BT709 SDR 100 nits and since it's a 50p stream it can be easily divided in fields to get a 25i all live with specialized hardware so that it can be aired in FULL HD as well (and in 720x576 25i Linear BT601 SD from the FULL HD stream for what matters). So it's a win win. If you wanna go to PQ instead, it becomes a bit trickier. Cameras don't generally shoot in PQ, they either use Linear BT709, HLG or Log (where Log can be SLog, CLog, LogC, FLog, VLog etc) depending on the manufacturers to get as many stops/nits as possible. Of course you can't air in Log (although I've had much fun laughing at other companies that did this wrong and ended up with things like highlights in Log on air by mistake xD) so you have to apply a matrix of Linear Transformation to go from Log to PQ and that has to be manually written every single time, for every event and possibly in the same even, adjusted for every camera. So you still have everything connected through SDI which goes to your hardware which performs the linear transformation and delivers it to the video server where you can adjust everything and go on air. Now, what about the ones with a TV that ignores the PQ colour curve? They're gonna have to watch the FULL HD Linear BT709 SDR 100 nits 25i stream... Right, speaking of which, how are you going to produce it? The same way as before? Oh no, that would be lame, you shouldn't start from an already linearly transformed signal (the PQ one) and tonemap it to Linear BT709 'cause your original signal is in Log, so you should start from the Log. Therefore, you're gonna need a second tech ops, encoding and ingest department that is gonna do exactly the same as the 1st one but instead of linearly transforming to PQ, they linearly transform to Linear BT709 and divide the 50p in fields to get the 25i. This is a mess, you have redundant people doing the same thing and you're gonna have to put extra stress on your departments and have two separate production teams... And for what, since many TV and cameras were not able to go past 780 nits leaving alone 1000 nits 'till not so long ago? If you take all these things into account, then HLG suddenly looks like a feasible option...


Side note: I think that BT Sport actually tried a similar workflow but I'm not sure 'cause I don't work there but apparently some former Sky employees are working there and these were the rumors, go figure...


Side note II: this is what I learned from my very personal experience, things might be different in other countries and other companies, but from what I saw I'm pretty confident that we're all in the same shoes more or less.

[wswartzendruber]

Wow, and here I am liking HLG simply because it's relative and not absolute.

[videoh]

Wow, and here I am liking HLG simply because it's relative and not absolute. I'm with you, bro. PQ requires a reference background brightness that diverse environments cannot all satisfy.

[wswartzendruber]

Avengers is encoding now. I pegged reference white to the "MARVEL" text at the start. Hopefully it turns out good.

To be honest though, as someone who's implemented PQ based workflows (HDR10 and Dolby Vision) for streaming services, it all works fine.
I genuinely want to know why streaming these days is not done in HLG.

[VictorLS]

I genuinely want to know why streaming these days is not done in HLG.
I believe due to not all players (devices like TVsets etc) support HLG as well as SDR but in nearest future HLG must becomes standard de facto because declared backward compatibility with SDR.

[wswartzendruber]

I believe due to not all players (devices like TVsets etc) support HLG as well as SDR but in nearest future HLG must becomes standard de facto because declared backward compatibility with SDR.
I've concluded that HLG was never intended to be viewed on SDR devices unprocessed. The compatibility that has been mentioned refers to the production environment. Specifically, the BBC states that HLG can be viewed on existing devices for "non-critical monitoring."

From my experimentation, HLG can look really good on SDR (with 709 color) as long as:

1. The player is HLG-aware and tone maps accordingly.
2. The reference white level for the picture is at 75% signal strength.

The Avengers came out looking really good after using the "MARVEL" text at the beginning to set reference white. I'll post an Imgur link when I'm off work.

Screenshots: https://imgur.com/a/9nEZhlR

[VictorLS]

I've concluded that HLG was never intended to be viewed on SDR devices unprocessed.
Seems you're right.
Screenshots
Great colors.
Can you upload some PQ video and therefore HLG video after your PQ to HLG Converter?

[Blue_MiSfit]

I find it interesting that such a significant technical decision (PQ vs HLG) appears to have been made solely based on concern for such a small group of 4K early adopters with UHD SDR BT2020 displays.

If you are okay with giving that small cohort FHD SDR 709 along with all the other people with legacy HDTVs, then you can just have that legacy feed and a pure HDR feed.

You could make it HLG, sure, but there's a lot of very good HDR TVs that don't support HLG. That seems like a much more important cohort than the tiny number of UHD SDR TVs. Understood that production and acquisition must be done in LOG or HLG, fine, but what you COULD do is distribute in PQ that's compatible with Dolby Vision Profile 8.1, HDR10+, and HDR10.

This is compatible with every single HDR TV ever manufactured.

This is all doable today for VOD, and I think soon for live (tho not sure live HDR10+ is even a thing? Doesn't seem possible to me). Even if it's just HDR10, that works literally everywhere AFAIK. Show me a single HDR TV without support for HDR10.

For live via traditional satellite distribution where transponder capacity is so expensive, this seems like the ideal HDR WCG solution to me.

I guess, it comes down to -- who do you screw over? Someone with an old UHD SDR display, or someone with an earlier UHD HDR display that is excellent other than its lack of support for HLG? For example, I bought an LG B6 a few years ago. It's a great TV still to this day, but it only supports HDR10 and Dolby Vision. Do you screw me over, or the guy with a super expensive first generation UHD TV who, let's face it, has probably already upgraded because he wants HDR!

[FranceBB]

Understood that production and acquisition must be done in LOG or HLG, fine, but what you COULD do is distribute in PQ that's compatible with Dolby Vision Profile 8.1, HDR10+, and HDR10.


Technically yes, but that would require testing from scratch the whole playout ports behavior in all different scenarios and most importantly metadata... Yeah... metadata, which should change according to the content and would have to be included and aired as well and that could create issues as the whole thing would have to be tested again. Just think about ads in between, you would have a movie at a certain amount of nits with a reference display and then an ad with a totally different amount of nits and reference, with its own metadata... It's not exactly easy to do and test, while in HLG almost everybody actually don't air metadata other than the fact that it's HLG and BT2020nc and leave the TV to deal with the incoming feed, but that shouldn't really happen with PQ, which brings yet one added nag.




This is all doable today for VOD


Yep, 'cause VOD is a completely different thing, but who would ever encode the same movie or TV Series twice once it has been encoded in HLG already? No one (just like no one would do it in PQ only as playout channels are still in HLG). In other words, it can be done, but no TV Studio does it 'cause it would require two encodes, two mezzanine files and two QCs unfortunately... (but services that are VOD-only and not owned by a TV Company could potentially do it and are doing it already I think). Derek, I don't know if I can ask you directly out of curiosity, but you work for big D, is your + offer in PQ with HDR10? HDR10+? Dolby Vision? HLG? SDR only? Are you dragged behind by linear channels as well?



and I think soon for live (tho not sure live HDR10+ is even a thing? Doesn't seem possible to me). Even if it's just HDR10, that works literally everywhere AFAIK.


Well, HLG is not here to last in the very long run, but I think we'll have to live with it for quite some time...


I guess, it comes down to -- who do you screw over? Do you screw me over, or the guy with a super expensive first generation UHD TV who, let's face it, has probably already upgraded because he wants HDR!

lol your TV would still recognize the feed as BT2020nc SDR and you would see the same "just barely watchable" feed someone with an old BT2020 SDR UHD TV would see, so none is screwed (or both of you are, depending of the point of view hahahahahaha)

[wswartzendruber]

I find it interesting that such a significant technical decision (PQ vs HLG) appears to have been made solely based on concern for such a small group of 4K early adopters with UHD SDR BT2020 displays.
The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range

[Blue_MiSfit]

Technically yes, but that would require testing from scratch the whole playout ports behavior in all different scenarios and most importantly metadata... Yeah... metadata, which should change according to the content and would have to be included and aired as well and that could create issues as the whole thing would have to be tested again.

It's pretty interesting to see how many displays totally ignore the ST2086 metadata and just do their own thing :D


Yep, 'cause VOD is a completely different thing, but who would ever encode the same movie or TV Series twice once it has been encoded in HLG already? No one (just like no one would do it in PQ only as playout channels are still in HLG). In other words, it can be done, but no TV Studio does it 'cause it would require two encodes, two mezzanine files and two QCs unfortunately
This is always required. If you deliver both 2020 and 709 you've got two files to QC. If your 2020 file happens to use HLG or PQ that doesn't change this.


Derek, I don't know if I can ask you directly out of curiosity, but you work for big D, is your + offer in PQ with HDR10? HDR10+? Dolby Vision? HLG? SDR only? Are you dragged behind by linear channels as well?

I'll take a moment here to clearly state for the record that anything I have ever said or will say here on doom9 is NOT as a representative of Disney, but is from me as an individual :)

I don't work on D+, but I know the service offers SDR, HDR10, and Dolby Vision. There's not currently any HLG or HDR10+ support.

I DO work on Movies Anywhere, which supports the same formats.

[Blue_MiSfit]

The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?

[wswartzendruber]

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?
FUD? You mean Fear, Unbelief, and Doubt?

My background is that I didn't get into HDR at all until HDR10+ was already a thing. So I'm very late to the game. Research and study (various docs from the ITU and BBC) revealed that for some reason, the industry had chosen to begin distributing content in a system that used absolute brightness coupled with metadata. Distribution to end users inherently entails differing viewing environments, which metadata doesn't address at all.

If you are going to point at HLG and call its rationale FUD, then I want you to defend distributing content in PQ instead. What advantages do you receive?

[FranceBB]

I'll take a moment here to clearly state for the record that anything I have ever said or will say here on doom9 is NOT as a representative of Disney, but is from me as an individual :)


Yeah, just like mine represent my own point of view and have nothing to do with the company I work for as they're my personal thoughts.

(Since you specified it, I specified it as well; better safe than sorry xD)


I don't work on D+, but I know the service offers SDR, HDR10, and Dolby Vision. There's not currently any HLG or HDR10+ support.

I DO work on Movies Anywhere, which supports the same formats.

Oh, ok, I see.

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?

As long as nobody brings up the joke about broadcast guys screaming when they hear the word "metadata", I'm gonna keep posting in the thread xD
Jokes aside, non-static metadata on a linear channels with contents played by a playout port, encoded live and aired can be a pain in the... *forbidden word :P * so I understand why some decision were taken in the past (along with all the other reasons I stated before).

[Blue_MiSfit]

FUD? You mean Fear, Unbelief, and Doubt?

My background is that I didn't get into HDR at all until HDR10+ was already a thing. So I'm very late to the game. Research and study (various docs from the ITU and BBC) revealed that for some reason, the industry had chosen to begin distributing content in a system that used absolute brightness coupled with metadata. Distribution to end users inherently entails differing viewing environments, which metadata doesn't address at all.

If you are going to point at HLG and call its rationale FUD, then I want you to defend distributing content in PQ instead. What advantages do you receive?

FUD = fear, uncertainty, and doubt ;)

To be honest, I was just trolling a little :D I don't have any hard evidence whatsoever that HLG is inferior to PQ, or that using PQ offers some advantage over HLG --- aside from my unchallenged point that PQ offers wider device compatibility in the rec2020 case, especially on early HDR TVs.

Thankfully, almost everyone seems to support HLG on modern devices now from what I can tell. The notable exception is Roku set-top boxes (distinct from TVs with Roku's software like TCL). This is a ___HUGE___ market in the US, actually, and is the single biggest cohort for 10 foot viewing here, so that's a major factor.

Does anyone know of any other new devices that support a PQ format but not HLG?

The creatives I've spoken with with prefer PQ, and are in general fans of having an absolute signal that is display referenced. Their standpoint is that with relative luminance, for example, HLG can misrepresent contrast.

However, I absolutely do feel the pain of needing to worry about a 1000 nit grade or a 2000 nit grade, and worrying about how mapping these to a 4000 nit display is not defined, even if there are a lot of pretty good solutions to map either of those examples to a 600 nit display, for example. I can see the draw of sticking to a relative, scene-referenced signal, and having a reasonable degree of faith that it will display nicely on a broad swath of HDR WCG displays without needing to futz with metadata.

My point was that, to be totally pragmatic, many displays totally ignore metadata and just do their own thing. So like, I wouldn't be surprised if everything would mostly be fine if we just had no metadata at all (other than the transfer, primaries, and differencing flags to identify content as PQ or HLG, and 709 or 2020, of course!) hence the "metadata FUD" troll ;)

Personally I still prefer Dolby Vision Profile 5 for a number of reasons, but that's getting OT pretty fast!

[wswartzendruber]

The creatives I've spoken with with prefer PQ, and are in general fans of having an absolute signal that is display referenced. Their standpoint is that with relative luminance, for example, HLG can misrepresent contrast.
Compliant HLG displays are supposed to have an EOTF modifier that adjusts the gamma curve depending on ambient light, if that is what you are referring to. With that said, I've never actually gotten my grubby paws on a HLG set. My workflow has been:

1. Study spec sheets.
2. Write code.
3. Write tests for code (checking math values).
4. How does it look on SDR through a HLG-aware (tone mapping) player?

Nailing #4 is all about figuring out a good reference white value (in nits) for the PQ source material. That's everything. It'll look too dim if that's set too high and it'll look completely clipped if it's set too low. I think I've said this before, but I am going to say it again and again because of how important it is.

Nailing #4 is a massive step in having HLG content that can play back on 709 devices. Assuming of course, you have a competent color+tone mapping player, which even the open source community has access to.

My point was that, to be totally pragmatic, many displays totally ignore metadata and just do their own thing. So like, I wouldn't be surprised if everything would mostly be fine if we just had no metadata at all (other than the transfer, primaries, and differencing flags to identify content as PQ or HLG, and 709 or 2020, of course!) hence the "metadata FUD" troll ;)
BT.2408 makes an attempt at this, which puts reference white at 203 nits. Notice how we're taking something relative (reference white) and pinning it to something absolute (203 nits). This, of course, depends on movie studios following these conventions. And I can tell you with absolute certainty that this is not the case.

FOR FUN: Here's an attempt at Rogue One on VLC (SDR/709 tone mapping) with reference white set too low. https://imgur.com/gallery/AyQW7Pg

EDIT: I need to watch Rogue One again just to pick out a good reference white scene. That movie is really throwing me around.

Alita: easy
Avengers: easy
Age of Ultron: slightly difficult
Rogue One: "Ohhhhhhh no! You are NOT converting me to HLG!!!"

EDIT: I wonder if I could build a sort of predictive model that analyzes every pixel of every frame and then computes an approximation for where reference white should be...

[Blue_MiSfit]

Theoretically, if they were present and accurate wouldn't the MaxCLL / MaxFALL values in the ST2086 metadata be helpful here?

[wswartzendruber]

Theoretically, if they were present and accurate wouldn't the MaxCLL / MaxFALL values in the ST2086 metadata be helpful here?
Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.

In any event, I can compute this stuff myself. And that's what I'm already doing with pqstat (https://github.com/wswartzendruber/pqstat). I could have it compute MaxFALL as well, if I wanted.

But what I'm really after is reference white and I'm not seeing how MaxCLL and MaxFALL will help with that. By reference white, I mean the border between SDR and HDR brightness levels.

[FranceBB]

Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.


Really?! O_o Leaving aside the 1000 nits, even if they're lower, they should all have metadata and it's actually very important in HDR10 PQ. I would understand HLG .ts files, but not UHD-BD disk with PQ... Are you ripping the original m2ts file? What does ffprobe say? Perhaps it's just MediaInfo not showing them for some reason? I mean, it would be weird for an HDR10 official disk not to have metadata.

[Blue_MiSfit]

Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.

In any event, I can compute this stuff myself. And that's what I'm already doing with pqstat (https://github.com/wswartzendruber/pqstat). I could have it compute MaxFALL as well, if I wanted.

But what I'm really after is reference white and I'm not seeing how MaxCLL and MaxFALL will help with that. By reference white, I mean the border between SDR and HDR brightness levels.

I see, yeah that's the real trick. I wonder how companies like Colorfront do this in real-time with a pleasing result...

[wswartzendruber]

Can you upload some PQ video and therefore HLG video after your PQ to HLG Converter?
What you have asked for no doubt seems simple, but I do not know how to accurately do that. I suppose I could start by compressing the Rec.2020 colorspace into the Rec.709 triangle.

Really?! O_o Leaving aside the 1000 nits, even if they're lower, they should all have metadata and it's actually very important in HDR10 PQ. I would understand HLG .ts files, but not UHD-BD disk with PQ... Are you ripping the original m2ts file? What does ffprobe say? Perhaps it's just MediaInfo not showing them for some reason? I mean, it would be weird for an HDR10 official disk not to have metadata.
VLC has a metadata display. It's probably just not working. I haven't tried ffprobe. I've actually been using the pqstat program just to be sure.

I use MakeMKV to remux whatever streams I pick from the disc into Matroska.

I see, yeah that's the real trick. I wonder how companies like Colorfront do this in real-time with a pleasing result...
I have no idea, but the deeper I've gotten into this, the less it's become about math and the more it's become about eyeballing it. Which is what I did with Rogue One finally to put reference white at 180 nits. I'll link to screenshots of that when I'm off work. So maybe they have some fancy machine learning algorithm.

Speaking of which, my friend bought a HDR-capable phone and had no idea. So we sampled Rogue One (my latest attempt) on both our sets side-by-side. And while I didn't notice much difference in luminousity, the color on his was absolutely gorgeous compared to mine.

EDIT: Rogue One HLG Screenshots (https://imgur.com/gallery/T9PvLtT)

[FranceBB]

VLC has a metadata display. It's probably just not working. I haven't tried ffprobe. I've actually been using the pqstat program just to be sure.

I use MakeMKV to remux whatever streams I pick from the disc into Matroska.

Well, the remux shouldn't really get rid of the metadata as they're in the video essence itself, so if they put them there, they're still gonna be there after the remux. Anyway, VLC is probably one of the worst players for HDR contents, in fact my choice falls on MPV, especially if you have an SDR monitor. By the way, I actually found the MPV tone-mapping for PQ contents with metadata to be pretty much fine, while it's kinda sloppy for HLG materials without metadata...

[videoh]

VLC is probably one of the worst players for HDR contents You should provide evidence when making baseless claims that impugn others' work. And 'probably'? Is that only on random days or do you roll dice every day to decide?

...while it's kinda sloppy for HLG materials ... What does "kinda sloppy" mean here? Can you show it with screenshots?

When someone resorts to rhetoric we have to question their confidence in their technical arguments.

Don't mean to be harsh. ;)

[FranceBB]

You should provide evidence when making baseless claims that impugn others' work.


Well, when I tried it years ago it wasn't doing any kind of mapping, it was just wrongly displaying the data on SDR displays, outputting both BT2100 PQ and BT2020nc HLG without taking into account their color curve nor their color matrix (i.e grey-ish).

VLC: https://imgur.com/JktOMOt
MPV: https://imgur.com/ky2Kg6k

You can clearly see that VLC got it wrong and that it shouldn't really be displayed that way on SDR displays...
I gotta say that I don't have it installed anymore, so I don't know whether things improved or not, but I can say that MPV definitely applies the tone-mapping when it has to display HDR on an SDR monitor (and it did it years ago as well, while VLC didn't).


What does "kinda sloppy" mean here? Can you show it with screenshots?


Well, it's kinda hard to do it with screenshots 'cause they would look ok. What I mean by "sloppy" is that when I try it with 1000 nits HLG files without metadata (other than the fact that they're HLG and BT2020nc 10bit) that have frequent transitions between a scene with high lights and low lights, it struggles. When I was watching Riviera, for instance, there was a fairly dark room with sharp lights every few meters. When Cherry was walking and was not under the light, the picture was going to dim while, as she walked under the light, the picture was going all the way up to the maximum brightness allowed by SDR. This is fine, however imagine a person walking with the monitor going bright, dark, bright, dark, bright, dark all the time in a matter of seconds with the whole scene (not just the main character) changing (including the background). I never noticed such a thing with PQ with metadata in which a compromise between being "under the light" and "not under the light" was chosen and it was much "easier" to watch and way less stroboscopic. The absence of scene-referred metadata might play a very big role in some contests, especially with SDR monitors watching an HDR content which is supposed to be viewed on an HDR display. This of course doesn't really mean that the HLG tone-mapping of MPV is bad nor that the developers didn't do a good job; heck, the tone-mapping it's good and I don't even know whether it can be improved much further given that HLG doesn't have metadata, but I just pointed out that in my personal experience MPV generally behaves better with PQ files with metadata than it does with HLG ones without metadata and with very high spikes in terms of nits on some scenes.



When someone resorts to rhetoric we have to question their confidence in their technical arguments.

Don't mean to be harsh. ;)

lol, no problem at all, when I say things they're always based on my own personal usage and on what I found out in my examples, not as "absolute statement" or "dogma". They're definitely open to discussion. So... even if I don't specify anything, always take them as my personal point of view ;)

[wswartzendruber]

I've noticed that VLC tone maps HLG differently depending on OS, which is strange.

Anyway, I also remember the release notes for 3.0.0 saying that they corrected HLG tone mapping.

[FranceBB]

I've noticed that VLC tone maps HLG differently depending on OS, which is strange.

Anyway, I also remember the release notes for 3.0.0 saying that they corrected HLG tone mapping.

Since you have the latest version of VLC, can you try to compare its HLG tonemapping to the one done by MPV? It would be interesting to see how they look like and whether the VLC guys actually really improved it compared to years ago.

[butterw2]

VLC uses different video output APIs vs OS.
Default on Win10 is Dx11. If you check the changelog you will see there have been a number of HDR updates for this mode in the last year.

there is also an OpenGL for windows mode.

Dx9 doesn't do tonemapping without using a shader.

[wswartzendruber]

Since you have the latest version of VLC, can you try to compare its HLG tonemapping to the one done by MPV? It would be interesting to see how they look like and whether the VLC guys actually really improved it compared to years ago.

Here you go; VLC is on the top in each one while MPV is on the bottom: https://imgur.com/a/2JeNJvT

I was blown away by this at first. MPV seems to handle 2020->709 conversion with dramatic superiority along with having much more pleasing gamma. In fact with MPV, I can't even tell it wasn't graded natively in 709.

[FranceBB]

Here you go; VLC is on the top in each one while MPV is on the bottom: https://imgur.com/a/2JeNJvT


Thanks.


I was blown away by this at first. MPV seems to handle 2020->709 conversion with dramatic superiority along with having much more pleasing gamma. In fact with MPV, I can't even tell it wasn't graded natively in 709.

Yeah, that's exactly what I was talking about when I said that I switched to MPV years ago. Looks like VLC did improve, but it still lacks behind MPV when it comes to tone-mapping HDR contents.

[wswartzendruber]

Two things:

1. MPV on Linux is using luminance mapping. Bright specular areas will dim over a couple of seconds, and it's not part of the movie.

2. VLC on Windows doesn't exhibit this and looks much better than VLC on Linux.

[wswartzendruber]

I finally got to watch my encodes on a HLG display, a Samsung from either this year or last. It was absolutely abysmal. The TV correctly detected the gamma curve as HLG, and I disabled the TV's enhancements for everything, but it just looked bland and washed out.

Luminance

I could see the TV continually adjusting the brightness level depending on the average luminance of the frame being shown. I'm not understanding why it wouldn't just apply tone mapping similar to VLC and MPV, although much less aggressively. The tone mapping used would depend on what the TV's brightness is set to. The higher the brightness setting, the more quickly specular detail gets compressed. This...shouldn't be hard.

Color

I don't know how to describe this other than "underwhelming."

HEVC Decode

On my PC, the HEVC decoding is exceptional. For movies where I instructed x265 to preserve grain, that comes through nicely. Whereas with this television, each frame seemed more like a matte painting. I wasn't quite sure what I was even looking at or how messed up the decode filtering would have to be to produce this result.

This isn't the result I was looking for. My converter only does linear brightness adjustment followed by gamma conversion. That's it.

[wswartzendruber]

I did The Matrix.

https://imgur.com/gallery/SL3NYCe

What surprises me is how overly bright and washed out the 2018 Blu-ray release is.

[FranceBB]

For me, the worst thing about Matrix is the green-ish tint they spread everywhere on the SDR BD.
I mean, I know it's an artistic choice, but still...
Thankfully, the HDR BD isn't affected, as your MPV preview shows.
Your version looks good too, by the way.

[wswartzendruber]

Is there a defacto standard tonemapping algorithm for compressing +1,000 nit brightness levels down to within 1,000 nits?

HDR TVs are doing something to apply metadata...

Right now, pq2hlg is just hard-clipping.

[FranceBB]

Oh no, you don't want to hard-clip it!
As you're working with curves, just knee it down to the desired level.
I think that applying a knee is a far better approach than brutally clipping values above 1000 nits

[wswartzendruber]

Oh no, you don't want to hard-clip it!
As you're working with curves, just knee it down to the desired level.
I think that applying a knee is a far better approach than brutally clipping values above 1000 nits
It looks like I need to apply the Rec.2020 coefficients to the RGB tuple to get that pixel's luminosity, map that to a lower luminosity, get the adjustment ratio, and then adjust each of the RGB channels by that ratio...

[wswartzendruber]

Tomorrow's agenda:

1. Sober up.
2. Implement the BT.2390 tone mapping algorithm.
3. Derive unit tests for said algorithm.
4. Rule the world.

[wswartzendruber]

Here's the latest command line usage for pq2hlg:

PQ2HLG 0.1.0
Converts from HDR-PQ (HDR10) to HDR-HLG

USAGE:
pq2hlg [OPTIONS] <INPUT-FILE> <OUTPUT-FILE> --height <PIXELS> --width <PIXELS>

FLAGS:
--help Prints help information
-V, --version Prints version information

OPTIONS:
-h, --height <PIXELS> Height of the input video stream
-m, --max-cll <NITS> Brightness of the input video stream's brightest pixel [default: 10000]
-r, --ref-white <NITS> Brightness of the input video stream's reference white level [default: 203]
-w, --width <PIXELS> Width of the input video stream

ARGS:
<INPUT-FILE> Raw input video stream (PQ, BT.2020, RGB48); use - for STDIN
<OUTPUT-FILE> Raw output video stream (HLG, BT.2020, RGB48); use - for STDOUT

This utility follows the BT.2390 method for converting PQ to HLG. If a reference white value is supplied, then the input
will first be linearly scaled to bring that level to 203 nits. If a MaxCLL value has been provided, then it will be
internally adjusted by this factor. If the internal MaxCLL value exceeds 1,000 nits, then BT.2390 tone mapping will be
applied.
As you can all see, tone mapping support has now been added. It maps each color channel independently and not overall luminance.

Next up is to get DV/HDR10+ dynamic tone mapping in place.

[FranceBB]

Good! :)
Well done.
Since your project continues to grow and you're basically accepting an input file and giving an output file to the user, have you ever thought of accepting also a stream as an input and outputting a stream as an output?
You probably figured out yourself what I mean by that, but I'm gonna say it anyway: introducing Avisynth+ support. :D
There are still alternatives, but, you know, the more tools we have, the better.

[wswartzendruber]

Good! :)
Well done.
Since your project continues to grow and you're basically accepting an input file and giving an output file to the user, have you ever thought of accepting also a stream as an input and outputting a stream as an output?
You probably figured out yourself what I mean by that, but I'm gonna say it anyway: introducing Avisynth+ support. :D
There are still alternatives, but, you know, the more tools we have, the better.
Your encouragement and feedback are appreciated.

I have very little experience with Avisynth+. As it stands now, you can pass "-" as the input and output files, causing pq2hlg to use STDIN and STDOUT instead of files.

Also, this thing is nowhere near ready for prime time. While you have access to professional grade equipment, I do not. This means that I am relying solely on mathematical correctness. In other words, I need more unit tests and lots of them.

So while I am very interested in Avisynth+ support, that is not the part of the roadmap that I am focusing on right now.

pq2hlg is also really slow (at the moment).

[videoh]

OK, then, I have to ask. Given the existence of several good Avisynth filters that convert PQ to HLG, and given that processing by an EXE reading and writing raw files is unlikely to be easily usable in typical workflows, what is the reason for this project?

[wswartzendruber]

OK, then, I have to ask. Given the existence of several good Avisynth filters that convert PQ to HLG, and given that processing by an EXE reading and writing raw files is unlikely to be easily usable in typical workflows, what is the reason for this project?

I do all of my transcoding in Linux. Piping RGB48 between processes is trivial.

ffmpeg -> pq2hlg -> ffmpeg

How good are the other solutions at dealing with arbitrary MaxCLL values?

This project is principally for my own consumption, but others are free to use and share it.

[wswartzendruber]

@FranceBB

The BBC says that the HLG OOTF needs to operate on luminance and scale R, G, and B by a fixed ratio. BT.2390 doesn't seem to care one way or the other.

Can I get away with having the HLG OOTF transform each channel independently? I could make this thing overwhelmingly faster if I can.

EDIT: Rats. I don't think I can.

[wswartzendruber]

I'm now at the part of this journey where it's time to use dynamic metadata to tone map on a scene-by-scene basis.

Is there already tooling for parsing Dolby Vision out into a JSON file (or something), or do I have to create that as well? I'm assuming it's stored in HEVC extension fields throughout the stream.

[FranceBB]

Did something happen? I was pretty sure to have answered the question before and I also spent a fair amount of time on it (which I'm not gonna spend now 'cause it's midnight)... Uhm... maybe there was a connection error, anyway I'm sorry you didn't get my answer. :(

@FranceBB

The BBC says that the HLG OOTF needs to operate on luminance and scale R, G, and B by a fixed ratio. BT.2390 doesn't seem to care one way or the other.

Can I get away with having the HLG OOTF transform each channel independently? I could make this thing overwhelmingly faster if I can.

You really shouldn't. Technically you can try and see the result, but you would miss the correlation between them and I would encourage you not to do it.

I'm now at the part of this journey where it's time to use dynamic metadata to tone map on a scene-by-scene basis.

Is there already tooling for parsing Dolby Vision out into a JSON file (or something), or do I have to create that as well? I'm assuming it's stored in HEVC extension fields throughout the stream.

Ah... I've seen people talking about it in the H.265 section of the forum; unfortunately I haven't been dealing with Dolby Vision contents that much so I don't know but there must be, 'cause other people have done it.
I mean, they have extracted the metadata from the Dolby Vision, parsed it into a json and used it in x265 to re-encode, so yes, that can be done, with which tool, though, I don't know/don't remember.
I hope someone from the H.265 section will pop here to give you a hand.

[wswartzendruber]

I'm getting to the point where I'm happy with how this is working. Also, I'm turning it into a LUT generator. My vision is for HLG content to become more accessible, but converting HDR10 movies is rather difficult. This utility could be used to generate movie-specific LUTs that are tuned for the movies they will transcode. Anyone could then download the LUT for their movie (clarified by UPC code) and do the conversion themselves.

I have some more testing to do and some open source compliance issues to work out, then I'll post the 1.0.0 binary.

[FranceBB]

I'm getting to the point where I'm happy with how this is working. Also, I'm turning it into a LUT generator.

Great, the more the merrier. :)
And since we're here: Merry Christmas (late) and Happy New Year (soon).

[wswartzendruber]

Great, the more the merrier. :)
And since we're here: Merry Christmas (late) and Happy New Year (soon).
By the way, how long should a LUT be on each side? The current default for this thing is 32x32x32. Also, it generates .cube files. And do LUTs need titles?

[FranceBB]

There are two kinds of LUTs, 1D and 3D.
The 1D ones are 1024, 4096 and 16384.
Now, in your case we're talking about 3D LUTs.
For 3D, sizes are 17x17x17 (this is just barely acceptable, but I would not recommend it), 33x33x33 (average quality, generally good and widely supported by lots of programs, even the home-user oriented ones). Lastly there's 65x65x65 which is broadcast quality and it's supported by programs like AVID Media Composer and surprisingly also by the Avisynth plugin "cube". As to FFMpeg, it didn't support them when I tried years ago, but users reported on my thread that it now does so it seems that they recently introduced support (where "recently" means 2 years ago) but I didn't have time to test them on FFMpeg yet. As to the title, I do put the title in my matrices and I also put a comment inside that states what it does and who made it (i.e me). As to the ".cube" extension, you're gonna be absolutely fine as it's the most widely supported one. There are also.ilut, .olut, .lut, .3dl, .spi3d, .3dl but again, I would go with a generic .cube as it's the most supported one.

[wswartzendruber]

I've gotten FFMPEG to work with a 65x65x65 LUT, but setting the interpolation to "tetrahedral" results in everything having a really green tint.

The Cube specification from Adobe states that some software works faster with LUTs that are sized according to an even power of two. Would it be rational to host two sizes, then: 32x32x32 and 64x64x64?

[FranceBB]

Well, since I use Avisynth and AVID most of the times, I didn't really test FFMpeg as I said. If you want, cross check the result in Avisynth to see whether it's FFMpeg misinterpreting the matrix or something wrong in the cube, but I bet it's FFMpeg.

[wswartzendruber]

Aha! It's a bug with FFMPEG when dealing YUV444P10LE. When forcing the signal down to YUV420P10LE, it looks correct.

[FranceBB]

Aha! It's a bug with FFMPEG when dealing YUV444P10LE. When forcing the signal down to YUV420P10LE, it looks correct.

Ah...
Then it has to be reported in the bug tracker so they can fix it, I guess...

[wswartzendruber]

Ah...
Then it has to be reported in the bug tracker so they can fix it, I guess...

I should. I'm just dragging my feet on it. I can't believe something like this doesn't have unit tests.

[FranceBB]

Pro and cons of open source, I guess... sometimes bug are there and nobody tested things :(

[wswartzendruber]

Holy cow, Man of Steel drives one of the RGB channels all the way to the max. And reference white seems to be at 100 nits. Maybe because it's an early title?

This will be an interesting stress test for the tone mapping system. It's going to be doubling linear luminosity to bring reference white up to 203 nits. That's going to put MaxCLL over 10,000 (which, as float-64 is used internally, won't cause clipping).

[FranceBB]

Considering that it has been released in 2013, it can be.
I don't have the m2ts of that title, but what did they write in the metadata? Besides, how come it maxes out RGB? I mean, reference white at 100 nits and RGB maxed out makes me lean towards BT2020 SDR.

[wswartzendruber]

I'll get the metadata and parsed stats when I'm off work. In the meantime, here's what MPV shows for BT.709. Putting reference white all the way down to 100 nits still makes it seem a bit dark.

https://imgur.com/gallery/KRSkpFj

[wswartzendruber]

Here's what mediainfo says:


Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 7770 cd/m2
Maximum Frame-Average Light Level : 1468 cd/m2
Original source medium : Blu-ray


The main thing that stands out to me is that MaxCLL actually exceeds the Mastering Display Luminance. In other words, they graded it higher than their reference monitor could show them?

Manually parsing every pixel in the stream with a utility I wrote reveals a maximum luminance of 5,560. This is calculated by first applying the PQ EOTF to each color channel (R', G', B') and then multiplying each one by its appropriate BT.2020 coefficient, and then adding the three channels together.

EDIT: Why is color range limited? I thought a bit value of 0 represents 0 nits, while a bit value of 1023 represents 10,000 nits.

[FranceBB]

Why is color range limited? I thought a bit value of 0 represents 0 nits, while a bit value of 1023 represents 10,000 nits.

Color range limited is perfectly valid and covers the 64-940 range of values in which the color curve spreads, so you just have to re-scale the calculations. For instance, on HDR Monitors, when you turn on the waveform in the settings, it lets you choose Full or Limited, along with the curve. From my experience, PQ and HLG target end users, so they're always limited tv range, while Slog, Clog, LogC, Zlog, Flog etc target studios, so they're almost always full pc range.


The main thing that stands out to me is that MaxCLL actually exceeds the Mastering Display Luminance. In other words, they graded it higher than their reference monitor could show them?


Possibly, but since it's a movie about GFX, they probably pushed things high with spikes on graphic effects as they're fake anyway. Heck, it's hard to believe that there is a camera with enough stops to be able to go to 7770 nits now, let alone in 2013, so it's definitely about graphic effects that have been pushed high beyond their limits. I think it's pretty lame to grade something like this. Sure, the standard was just taking shape and it was very new and they probably wanted to make it future-proof, but honestly they made it look impossible to reproduce for any TV at the time as every single TV had to apply an highlights roll off and/or re-scale everything to its maximum displayable amount of nits. For me it's a "no". The HDR Documentary I made in 2019 had 400 nits 'cause it was the maximum I could go with the Canon C300 from CLog3 to HLG without "inventing" anything. For them, just because they have computer-made graphic effects doesn't mean they can go as high as they want, 'cause there are still gonna be real scenes captured with a real camera and if you push things higher than they should, not only you're literally inventing things, but you "stretch" the values so much that you end up getting lots of noise in the dark scenes, so much so that you're forced to "crush" the blacks to make it look ok and average out all the other values. For me, they got cocky with the GFX and ended up getting it wrong...

[FranceBB]

I actually checked what they used. They claim they recorded with Arri cameras in Log-C with 14 stops. If you compare it with PQ, you can easily see how the first (Log-C) is similar, although it starts slightly higher in the blacks and then it goes all the way up to exhaust itself at 14 stops, while PQ still has lots to give as it goes up way slower (and therefore it takes longer to get to the maximum value, so it has more stops, hence more nits). So, in a nutshell, they recorded in Log-C, they were able to go all the way up to 2000 nits from Log-C to PQ and then they used graphic effects to get to the "fake" 7770 nits on a 4000 nits monitor:

https://i.imgur.com/MObOuY5.png

(this is plotted with x representing stops and y representing 10bit values scaled to 100%)

[wswartzendruber]

Hold on a sec, I need to go back and redo the LUT generator to put minimum luminance at 64 and maximum luminance at 940, for each curve?

[FranceBB]

Hold on a sec, I need to go back and redo the LUT generator to put minimum luminance at 64 and maximum luminance at 940, for each curve?

I'm afraid so, but don't delete the ones you've already made, rather offer the option to the users to choose whether it's Full or Limited range.

[wswartzendruber]

Blargh.

It looks like "legal" range RGB uses 64-940 (as you said) while "extended" range uses 4-1019.

Why is this even a thing? Who said, "Oh, you know, we have a completely defined signal space with definite boundaries, but we're going to artificially not use all of it."

EDIT: I wonder if I'm ever truly going to finish this thing, or if I'm going to take another left hook.

EDIT: I'm tempted to just have this thing document that you need to expand any limited RGB range before using it.

EDIT: I think I got it. I'll follow up with unit tests tomorrow.

[wswartzendruber]

When converting to RGB48LE, FFMPEG expands limited Y values into full ones. At least, according to this hex editor I'm looking at. I see Y vales of 64 (little-endian) for the raw YUV420P10LE stream. But when I convert to RGB48LE, they become 0. Converting back to YUV420P10LE from RGB48LE brings it back to limited range.

Where's that bottle of sake... I need to drink... I'll try to wrap my head around this in the morning. Or maybe use it to drive nails into the siding of my house.

[FranceBB]

Where's that bottle of sake... I need to drink...

Kanpai!! :D xD


When converting to RGB48LE, FFMPEG expands limited Y values into full ones. At least, according to this hex editor I'm looking at. I see Y vales of 64 (little-endian) for the raw YUV420P10LE stream. But when I convert to RGB48LE, they become 0. Converting back to YUV420P10LE from RGB48LE brings it back to limited range.

Right... So, you've been doing this correctly all the time as you made calculations and analysis in RGB and also applied the LUTs in RGB before converting back to YUV, so, even if you assumed that it was full range, since FFMpeg scaled to full range while bringing things to RGB, the stats were correct. I think that's because there isn't anything such as Limited Range RGB per se (I mean, by definition) outside intermediate broadcast use, so, while it would be logical to assume that a simple "regular mapping" of values would occur when going from YUV to RGB (so 64 to 64 etc) to get a "Limited" range RGB, the FFMpeg devs probably thought that, since there's no such a thing as a flag to display Limited RGB properly, it was a sensible thing to scale those values going from limited to full and then back from full to limited.
When I was making SafeColorLimiter this discussion popped up:

there's no limited range RGB.

and

There is such a thing as "limited range RGB" . It's also known as "studio range RGB" . In 8bit RGB , 16-235 is black to white (sRGB or "computer RGB" is 0-255 black to white in 8bit, it's far more common) . Limited range RGB is used in r103 compliance checks for broadcast, and some NLE's like vegas use studio range RGB .


but also


a) no television system transmits video in RGB space, so it makes no sense to preserve some "headroom" from the 8-bit range for some metadata that does not exist in the first place.
b) YUV<->RGB conversions are ALWAYS defined on full range RGB, this limited range RGB could not be properly displayed (without some non-standard scaling applied first), or properly converted to YUV by any standard.

and

these standards exist, you should be aware of them, and we have to live with it because it's still used today.




So, in a nutshell, this is what I think led to the misunderstanding.

1) End users receive a Limited TV Range 10bit 64-940 file with a color curve applied, either HLG or PQ

2) Broadcast studios can also use Full PC Range YUV values for those curves, however it's much more likely to have Full Range only for curves like Slog, Clog, LogC, etc

3) You're checking values and doing your calculations in RGB (assuming that the content is in Full Range) as LUTs must be applied in RGB

but here's the thing: when you have a Limited TV Range YUV file that is delivered to a TV, the TV can read the flag and convert to RGB correctly (since the display is always in RGB), but if you try to deliver an RGB whose black start at 64 and white peaks at 940, there's no way to flag it as "Limited", therefore it would almost definitely be displayed incorrectly unless a YUV conversion is applied or there's some system that is aware of this. This is probably the reason why the FFMpeg guys thought that, when you convert a Limited TV Range content to RGB, you don't want to "work with it", but you want to "deliver it" and, since it wouldn't be displayed correctly if it's linearly mapped, it will expand it! So, in the end, you were doing all correctly in the first place as your calculations assumed Full PC Range, the content was in Limited, but FFMpeg took care of it during the YUV to RGB conversion!

So... I think I may have made you lose time on something that has already been addressed by a conversion you were doing, but hey, we both learned something new today, which is that FFMpeg expands Limited to Full in YUV -> RGB. Who would have thought that...

I'll leave you to the Sakè:

https://www.japan-guide.com/g9/2037_sake_02.jpg

(fun fact I: the first time I tried sakè was in 2009 with my friends in an allegedly Japanese restaurant which was probably run by Chinese pretending to be Japanese; anyway I didn't like it that much.)

(fun fact II: there are three glass in the picture I grabbed from a website 'cause one is for you, one is for me and the other is from whoever wants to grab it. I think StainlessS is more of a Beer guy, so let's see... xD)

[wswartzendruber]

I just learned how to revert a range of commits in Git.

Since you're handing out drinks, I'll let it go.:D

Hell, I have a Ryzen 3950X. I can do two movies at a time and make up for lost time.

[wswartzendruber]

I believe that as of this very moment, my tooling is complete. This includes a pgsmod utility I have written to crop and tonemap SUP subtitles.

The one thing I don't understand is why my pqstat utility returns higher MaxCLL values than what's in the metadata. Usually it's marginally higher.

[wswartzendruber]

pq2hlg and pqstat have now been consolidated under this single repository: https://github.com/wswartzendruber/hlg-tools. The original pq2hlg and pqstat repositories have been archived.

There will soon be a comprehensive HOWTO regarding converting 4K UltraHD Blu-rays to HLG for universal viewing.

[Blue_MiSfit]

Great work on this. Thank you for all your effort and making this open source!

[wswartzendruber]

Great work on this. Thank you for all your effort and making this open source!

You're welcome. Here's the release thread: https://forum.doom9.org/showthread.php?t=182499

That's the best place I could think of to put it.

[wyup]

The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range
This is very interesting. Despite HLG being the 'de facto' standard for HDR television today, I don't really understand the reason it's been chosen over PQ.


Backwards compatibility, the main reason of HLG is completely defeated as no SDR tv can play 10-bit depth and Rec.2020 colorspace. 99% SDR tvs will show an error when trying to play or select a HLG broadcast. The first HDR tvs in 2016/17 were already PQ compatible just before HLG came on.
PQ doesn't need metadata either, so it is not an excuse for live broadcast. PQ was designed from the ground up to provide perceptual brightness without banding. HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.
HLG is a compromise buiding on top of SDR. It can't be as good as PQ. HDR10 has been the standard in 4k blu-rays for years. HLG hasn't even cought-up on tv yet, apart from 2022 World Cup, the 2024 Euro and the Olympic Games. Roland Garros and Wimbledon are also in HLG. There are no native public HLG broadcasts in Europe that I know of.
Scene referred HLG is no better in practice vs PQ: Today TVs calibrate HLG at their peak brightness. 1,000 nits is the standard range in Rec.2100 HLG streams. Scaled SDR is 100 nits and HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display, and even dimmer on 500-nit ones, since reference white luminosity is lowered from default 200 nits at 1,000 to less. Being scene refered doesn't really help watch better on mosts screens. How do you brighten up a 200-nit broadcast on a 500-nit tv? The only way to bring up low HDR luminosity is to enable a brightness normalisation function on tvs. In the end, it doesn't matter that PQ is meant to watch in the dark. Home tvs should show PQ brighter on convenience aswell. Just turn up peak brightness and set a knee point. Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed.

In my opinion it would be a mistake if global HDR tv stablished on a non-existent backwards compatible and compromised HLG format. It would take years to transition to PQ, as this should be the right choice from the start.

[wswartzendruber]

This is very interesting. Despite HLG being the 'de facto' standard for HDR television today, I don't really understand the reason it's been chosen over PQ.
Well, let's go over it, then.

Backwards compatibility, the main reason of HLG is completely defeated as no SDR tv can play 10-bit depth and Rec.2020 colorspace. 99% SDR tvs will show an error when trying to play or select a HLG broadcast. The first HDR tvs in 2016/17 were already PQ compatible just before HLG came on.
In the United States, at least, these devices aren't going to be able to even tune to such a thing in the first place. They don't have ATSCv3 tuners. So...how is an error message going to appear? I guess you could try to open a file on a thumb drive or some such, but a PQ stream would also show an error here.

What HLG does facilitate, though, is easy conversion to SDR with a static lookup table. Here's one I generated:

https://wswartzendruber.net/uploads/bt2020to709-2.cube

This deterministically converts BT.2020 (both HLG and SDR) to BT.709 SDR. It's a use-everywhere solution. I generated it using OKLCH. FranceBB also has some lookup tables that he provides in his thread.

A set-top converter box connected to a BT.709 television can do the same, but only with HLG. Downconverting PQ would complicate things, although it could still be done if you had metadata and the reference white level.

PQ doesn't need metadata either, so it is not an excuse for live broadcast. PQ was designed from the ground up to provide perceptual brightness without banding. HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.
PQ doesn't need metadata? That's weird. My Man of Steel disc surpasses 7,000 nits. Just how aggressively should my TV tone map a scene that reaches 900 nits?

And how does PQ "help tvs adjust brightness"?

HLG is a compromise buiding on top of SDR. It can't be as good as PQ. HDR10 has been the standard in 4k blu-rays for years. HLG hasn't even cought-up on tv yet, apart from 2022 World Cup, the 2024 Euro and the Olympic Games. Roland Garros and Wimbledon are also in HLG. There are no native public HLG broadcasts in Europe that I know of.
In this thread, we learn the basics of how mathematics works. For consumer distribution, both PQ and HLG use 10-bit sampling. So there are however many values that uses given a limited-range signal. HLG distributes these points across 0-1,000 nits. But PQ distributes them across 0-10,000 nits. This means that although HLG has limited dynamic range compared to PQ, it has more bits to work with across that limited range than PQ does. This lets HLG increase the brightness to accommodate a well-lit environment with less banding.

And UltraHD discs probably use PQ because that market is for cinemaphiles who typically have controlled viewing environments. PQ is arguably superior in such cases.

Furthermore, HLG is not built on top of SDR. One casual glance at its transfer function should show you that it is an SDR curve that blends into an HDR one as the value increases.

Ask FranceBB about HLG broadcasts in Europe. He knows the practices of that region much better than I do. We're already going live with it in the United States and South Korea. And probably Japan (given NHK).

Scene referred HLG is no better in practice vs PQ: Today TVs calibrate HLG at their peak brightness. 1,000 nits is the standard range in Rec.2100 HLG streams. Scaled SDR is 100 nits and HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display, and even dimmer on 500-nit ones, since reference white luminosity is lowered from default 200 nits at 1,000 to less. Being scene refered doesn't really help watch better on mosts screens. How do you brighten up a 200-nit broadcast on a 500-nit tv? The only way to bring up low HDR luminosity is to enable a brightness normalisation function on tvs. In the end, it doesn't matter that PQ is meant to watch in the dark. Home tvs should show PQ brighter on convenience aswell. Just turn up peak brightness and set a knee point. Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed.
The last revision of BT.2100 I looked at didn't say anything about 1,000 nits. That was in BT.2408.

As for the rest of this paragraph, I don't understand any of it. I see words and terms I'm familiar with, but I do not comprehend whatever it is you are attempting to convey.

Actually, I do understand this:

"Household tvs are not meant to be a lab, a tv must adapt to environment brightness and scale as needed."

And this is a lot of why HLG was created.

In my opinion it would be a mistake if global HDR tv stablished on a non-existent backwards compatible and compromised HLG format. It would take years to transition to PQ, as this should be the right choice from the start.
The BBC and NHK disagreed with you to the point that they created and entirely new system and then successfully persuaded the electronics industry to support it.

[FranceBB]

HDR10 Metadata is not a bad thing either. There is no reason for a broadcast signal not to include it to help tvs adjust brightness.

Dynamically changing metadata on a broadcast feed isn't exactly the easiest thing to implement in a chain. Where is that info gonna come from? Think about live events, like Sports. You can shoot in BT.2020 HLG directly with the various cameras and then have all the graphics etc in BT.2020 HLG in your gallery, however you can't shoot PQ. You would have to shoot log, then convert to PQ in real time with a PlanarRGB 16bit roundtrip and then overlay things like the score, the statistics etc, which also need to be in PQ and ideally at a no-so-much-different peak brightness compared to the main sport event. Can it be done? Potentially, you could set all the cameras to record Slog3 and then bring everything to, let's say, 800 nits PQ and then also have the various graphics etc at 400 nits and signal that the whole stream is 800 nits, but even then it adds a lot of complexity and it still doesn't make use of the PQ curve the way it should be used. Then think about things like commercials and other things that have probably been upscaled from a FULL HD BT709 source. What do you bring them up to? Keep in mind that most of the time a single TX Ready file is created, so you would only do one such a conversion once and then use it every time, but different live events could have different cameras with different stops and therefore different nits and all of this can get really messy really quickly.


A set-top converter box connected to a BT.709 television can do the same, but only with HLG.

Yep. That's exactly it. To answer wyup's question, essentially, if you have a BT2020 SDR TV, then it would interpret the colormatrix, but it wouldn't interpret the transfer, which means that it would show you the image being dimmer (as it would peak at 75% instead of 100%) but still ok. Using the same concept, a set top box can easily convert such a signal from BT.2020 to BT.709 SDR. Once again, we would treat it as if it was a BT.2020 SDR stream so that it would still peak at 75% rather than 100%, so the white would look dimmer, but the overall image would be displayed in an acceptable manner. This is something accepted by the industry and something that is actually being done already today. On the other hand, converting the transfer from a totally logarithmic curve like PQ to SDR 100 nits to get a BT709 output not only requires a much more complex approach with some kind of dynamic tonemapping (like what MPV does, for instance), but it would never be accepted by any studio / content provider / supplier.


This means that although HLG has limited dynamic range compared to PQ, it has more bits to work with across that limited range than PQ does.


Yeah. One could make the case about Dolby Vision with the normal 10bit standard layer and the other metadata (enhancement layer) so that a compatible decoder could output 12bit, but that only works for bluray disks. For a linear channel, this is basically impossible and of course going to 12bit as a standard source isn't the way as it may not be decoded at all. I thought we were gonna see the switch when H.266 VVC arrived and the new 8K specs were supposed to be announced, but they ultimately settled on 10bit again as a standard.


Ask FranceBB about HLG broadcasts in Europe.

Pretty much every linear UHD channel in Europe is BT2020 HLG, be it for Sports or Movies / TV Series, it's all HLG all the time.

[FranceBB]

HLG native content for now is just conservatively above 100 nits, let's say 200. Both will look dim and low contrast on any display

Some Premier League matches are 180 nits. I watch them all the time as I'm a Chelsea supporter. This is how they would look like on a BT2020 SDR display by just interpreting the colormatrix and ignoring the transfer. It's still very watchable. Not great, but very watchable. On my SDR TV I generally just turn off the light in the room.

https://i.imgur.com/ur0SisZ.png

[wswartzendruber]

We have new specs for 8K? Are we staying with BT.2020 color and BT.2100 transfer functions or is there something new?

I would think they'd take the opportunity to do 12-bit... HLG could be cranked to well over 4,000 nits with that. PQ could be cranked...four times higher than with 10-bit, I suppose? I don't know as much about its limitations.

EDIT: Now that I check BT.2020 and BT.2100 again, I see support for 8K (and 12-bit) clearly enumerated. And up to 120 Hz.