I Made a PQ to HLG Converter [Archive]

wswartzendruber

13th April 2020, 05:44

I wrote a small utility for converting raw video streams from PQ to HLG. It's available on GitHub. (https://github.com/wswartzendruber/pq2hlg)

USAGE: pq2hlg [format] [nits] [width] [height] [input] [output]
format - The raw pixel format to use. Supported options are:
rgb48
nits - Luminance value of the brightest pixel in the input.
width - Width of the video stream in pixels.
height - Height of the video stream in pixels.
input - Raw RGB48 input file; use - for STDIN.
output - Raw RGB48 output file; use - for STDOUT.

My motivation was that I wanted to rip my 4K UltraHD library, but wanted to do so in a viewing format that is as universal as possible, even if that entails some amount of compromise. I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I learned that ffmpeg supports the use of lookup tables, but couldn't find any such thing for cheaper than a couple hundred dollars. Others were available from the BBC under license.

All in all, I got quite fed up with the situation and started researching PQ, HLG, metadata, gamma curves, etc. until I got nauseated and started work. Then I studied some more and then went back to coding. Wash, rinse, and repeat a few times. I'm at the point where I'm comfortable with releasing my work to the public. Principally, it uses, to the best of my understanding, the algorithm published by BBC Research & Development (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf).

Future improvements may include passing dynamic metadata in for variable PQ->HLG tone mapping. I definitely want to add YUV10 support as soon as I can figure out how the U and V components are encoded.

This is also the first thing I've written in Rust, so it's probably not going to be as idiomatic or as proper as it could be.

EDIT: Attached a screenshot from Alita: Battle Angel using this utility, taken from a nightly build of VLC 3.0.9 on a SDR monitor. The video codec in use is VP9 Profile 2.

EDIT: I also forgot to mention: This isn't really suitable for any brightness value aside from 1,000 nits. If a PQ video has a peak brightness of less than 1,000 nits, pass 1,000 nits in as the second argument anyway. If any pixel ever exceeds 1,000 nits, then external tone mapping really needs to be applied to the PQ source before passing it into this utility. A peak brightness of other than 1,000 nits can be specified, but this will cause reference white to be pushed up or down. This addendum is in accordance with ITU-R BT.2390-8 Section 7.2.

Blue_MiSfit

13th April 2020, 20:17

wswartzendruber

13th April 2020, 21:36

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.

However, I don't understand how (without some other component) it can be backwards compatible from a color gamut standpoint. If you're doing HLG with Rec2020 or P3, from what I understand that will NOT look correct on a non wide-gamut display. Gamut conversion is not trivial.

So... how does that work?

Your screenshot looks at least mostly correct to me, so I'm wondering how you / VLC converted the gamut from Rec2020 to Rec709.
It's entirely on the player to do that, as you have surmised. VLC does an outstanding job of it while Kodi doesn't even seem to try (everything looks washed out).

My assumption with VLC is that it is using some kind of logarithmic roll-off to compress the color dynamics. However they do it, the results are outstanding.

EDIT: I have to take that back. I put Alita through Kodi and Rec.2020->Rec.709 mapping was indeed happening. That other clip I played must not have had the right colorspace signaled.

scharfis_brain

15th April 2020, 20:40

wswartzendruber

17th April 2020, 05:33

I've attached some more screenshots for demonstration. All in all, the HLG-on-SDR picture is somewhat dim, but I find that this leads to a more pleasant viewing experience, especially with the lights turned out. Reference white comes in on my monitor somewhere around 100 nits and bright whites come in around 200. So the specular highlights are noticeable as the director intends. In contrast, the SDR grading of the film looks washed out and loud.

FranceBB

17th April 2020, 13:23

My motivation was that I wanted to rip my 4K UltraHD library, but wanted to do so in a viewing format that is as universal as possible, even if that entails some amount of compromise. I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I learned that ffmpeg supports the use of lookup tables, but couldn't find any such thing for cheaper than a couple hundred dollars. Others were available from the BBC under license.

Oh, really...? *coff coff* Link *coff coff* (https://forum.doom9.org/showthread.php?t=176091)

By the way, the fact that people were charging a lot of money for those matrices of linear transformation is exactly what made me want to share them for free...
Of course, as I always said, the ones made by AVID, BlackMagic, BBC and so on have had a lot of engineers working on them, while I made mine together with my boss in a studio room and they have been tested against a single professional monitor by Sony and not a wide range of them, but still...

All in all, I got quite fed up with the situation and started researching PQ, HLG, metadata, gamma curves, etc. until I got nauseated and started work. Then I studied some more and then went back to coding. Wash, rinse, and repeat a few times. I'm at the point where I'm comfortable with releasing my work to the public. Principally, it uses, to the best of my understanding, the algorithm published by BBC Research & Development (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf).

Yep, starting with the BBC Whitepaper is always a good starting point as it basically explains how the conversion should be done.

This is also the first thing I've written in Rust, so it's probably not going to be as idiomatic or as proper as it could be.

I don't actually know rust as programming language as I never studied it, but still hats off for researching and implementing your own converter.

Fun project!

Here's one thing I've been trying to wrap my mind around regarding HLG: I get that one of the big appeals is "backwards compatibility", and I totally see how, from an EOTF standpoint it's somewhat backwards compatible with SDR. In other words, you could feed an HLG signal to a dumb SDR display and the luminance would look mostly correct.

Correct.

However, I don't understand how (without some other component) it can be backwards compatible from a color gamut standpoint.

In fact, it isn't.

If you're doing HLG with Rec2020 or P3, from what I understand that will NOT look correct on a non wide-gamut display. Gamut conversion is not trivial.

So... how does that work?

It's not converted. Let me clarify that.
The whole reason why we want to air with HLG, as broadcasters, is that we wanna have both HDR and SDR on the same channel, BUT (and here's the catch) most people tend to think that when we say "SDR" we're actually referring to Linear BT709 100 nits, but we are not.

When we air a 4K UHD stream via satellite it has two things mainly: the colormatrix and the color curve.
If you have a new 4K UHD TV, it's gonna read the stream, understand the BT2020nc, understand the HLG curve and display the color curve correctly, thus showing you the right amount of nits (up to 1000) that the content has. You're definitely watching an HDR content.
If you have an old 4K UHD TV, however, when it's gonna read the stream, it's gonna correctly interpret the BT2020nc, but it's just gonna ignore any information relative to the color curve, thus displaying the content as BT2020nc SDR 100 nits. The more a content is graded towards HDR, the worse it's gonna look on BT2020nc SDR panels, the more is graded towards SDR, the less you're gonna have dynamic range on HDR monitors, hence the compromise and the 1000 nits limit.
For quite some time I had to grade live sports events to 400 nits maximum 'cause many people didn't have HDR panels at the very beginning.

For the records, the reason why you can't do that with something like PQ is that if you ignore the color curve information on a PQ stream, then everything is gonna look very dull, as PQ is a proper logarithmic curve, while HLG is indeed Hybrid and compromises details in the shadows to make it SDR compatible but still has room for the peak brightness to be way higher than 100 nits.

Out of curiosity, this is how you're gonna see the very same HDR HLG signal on two different monitors: one BT2020nc SDR which ignores the color curve and the other that reads and interprets the color curve:

https://i.imgur.com/S8pJItb.png
https://i.imgur.com/jQHYj7A.png

As you can clearly see, it's a compromise and it's never gonna look exactly as we intended it to look, which is something that kinda pisses us all off (broadcast encoders), but hey, we barely have the bandwidth to have 25 Mbit/s H.265 on all our 4K channels, so it would be madness to have two 4K channels for everything...

videoh

17th April 2020, 13:31

I quickly settled on HLG after reading about it, but couldn't find a conversion utility to do this. I cough like FranceBB too (no, not the virus!):

http://rationalqm.us/hdr/DGPQtoHLG_1.0.rar

It's an Avisynth filter that supports both SW mode and CUDA mode.

wswartzendruber

17th April 2020, 21:44

Yep, starting with the BBC Whitepaper is always a good starting point as it basically explains how the conversion should be done.
Am I crazy for wanting to add scene-specific tone mapping to compress anything over 1,000 nits via dynamic PQ metadata? This was a goal from the beginning, otherwise, I would have written this thing to just output a static LUT.

And what's this color curve you speak of? I don't remember reading anything about that. I know that I have to weigh R, G, and B by differing factors to determine the overall luminance of a PQ pixel, but that's about it. Everything else seems to treat R, G, and B independently of each other.

Oh, and here are some more screenshots.

wswartzendruber

30th July 2020, 07:04

I've made some modifications. Here's the new help screen:

PQ2HLG 0.1.0
Converts from HDR-PQ (HDR10) to HDR-HLG

USAGE:
pq2hlg [OPTIONS] <INPUT-FILE> <OUTPUT-FILE> --height <PIXELS> --width <PIXELS>

FLAGS:
--help Prints help information
-V, --version Prints version information

OPTIONS:
-h, --height <PIXELS> Height of the input video stream
-r, --ref-white <NITS> Brightness of the input video stream's reference white level [default: 203]
-w, --width <PIXELS> Width of the input video stream

ARGS:
<INPUT-FILE> Raw input video stream (PQ, Rec.2020, RGB48)
<OUTPUT-FILE> Raw output video stream (HLG, Rec.2020, RGB48)

This utility follows the BBC R&D method for converting PQ to HLG. Any pixel brighter than 1,000 nits will be clipped and
no tone mapping will be applied. If a reference white value is supplied, then the input signal will first be linearly
scaled so that the reference white level becomes 203 nits.

I would appreciate someone letting me know if linear scaling for reference white adjustment is the way to go. When it comes to Alita: Battle Angel, I've noticed that the darker areas of the picture are marginally darker than with the native Rec.709 grading from the 1080p Blu-ray. Again, this is when comparing in VLC on a SDR monitor without WCG.

As is the usual case, there are a few more screenshots. These were taken from an HLG encode that uses the new reference white scaling. I sampled the credits and determined that this movie's PQ grading uses 157 nits. So that was linearly scaled up to 203 across all three color channels.

In the future, I intend to add support for Kim-Kautz tone mapping.

EDIT: Imgur Album of Latest Screenshots (https://imgur.com/a/IN63ytr)

FranceBB

30th July 2020, 11:52

I'd suggest you to use an image host like imgur 'cause attached screenshot have to be approved first and that can take time (and it also uses space from the Doom9 server).

wswartzendruber

31st July 2020, 05:45

Done. :-)

VictorLS

31st July 2020, 12:58

wswartzendruber
Once you're strong in HLG may be you can help us (SAT-amateurs) in writing good working (giving near real colors) HLGtoSDR shader https://forum.doom9.org/showthread.php?t=176909 ?

Blue_MiSfit

31st July 2020, 20:40

wswartzendruber

31st July 2020, 23:06

Following up on some posts from above, it sounds like the plan is to have a single 4K BT. 2020 HLG stream that will serve two groups of customers:

1) Folks with modern 4K TVs with HLG and 2020 support who will see the extended dynamic range that HLG brings in addition to the wide 2020 color gamut

2) Folks with first generation 4K TVs that are SDR, but still support 2020/WCG. They get the wider gamut, but and will still be able to watch the content acceptably even being totally naive to HLG. I imagine folks with modern 4K TVs that have HDR10 and DoVi, but still lack HLG for some reason (like my older LG B6) would fall into this category as well.

This leaves everyone else, but you'll still uplink a standard HD SDR 709 feed for them?

So... the industry is going through the hassle and compromises of HLG just to help out that small group of 4K early adopters with SDR panels that support BT. 2020? Seems a little silly to me, honestly! How big is that user-base, and how much of even the P3-D65 gamut will they even see (forget about 2020)?

There must be some other reason for using HLG. Is it just the momentum around HLG in production workflows (cameras, etc)?
If you are speaking to me, then you misunderstand. This isn't about the industry or what they want, this is about me and what I want. With that said, I am also a very big believer in open source software and in sharing work. At least a few other people will likely find this useful as well. Eventually, that is.

I want a process to rip a 4K disc such that I have a single encode that can play back on any device with a competent player. Right now, I have Alita on my desktop (Rec.709), my phone (Rec.709), and it will also be on my HTPC when I get that, which will be Rec.709 first and then Rec.2020 later on. I want a single encode for all of these devices.

And this is because I find PQ to be an absolute joke for content distribution, and a bad one at that. I am utterly convinced that the reason Dolby persuaded everyone to go along with it was because it created an artificial need for Dolby Vision, which they would in turn profit from.

My goal is to create a single workflow for PQ content that allows me to adjust and transcode anything I have into a consistent HLG stream. This means, for example, that each transcode will have a reference white level of 75% signal strength (or as best as I can manage this). VLC and Kodi already handle 2020->709 amazingly well as they are, so I don't have a colorspace issue. And I don't have a luminosity problem, either, because I've determined that VLC clips HLG at precisely the 75% mark, which is exactly what I want it to do.

Something I'm really interested in at the moment is scene-specific tone mapping via dynamic metadata, since I have to stay within 1,000 nits.

But regardless of how I get to HLG, I will never need metadata during playback. What a farce. I seriously cannot believe everyone went along with this.

EDIT: Well, I guess the BBC and NHK didn't go along with it.

Blue_MiSfit

1st August 2020, 01:32

Oh no, I wasn't directing that at you :) I was directing it at FranceBB's description of their transmission process :)

To be honest though, as someone who's implemented PQ based workflows (HDR10 and Dolby Vision) for streaming services, it all works fine. For HDR10, Most of the displays just ignore the metadata and do whatever they want.

The Dolby Vision stuff is great, and works really well for preserving high luminance details and getting consistent color reproduction, especially on cheaper devices (TCL TVs etc). It also kicks in special rendering pipelines on the displays that do less bad things, and can result in sane output settings. People do wacky things to their TVs ;)

FranceBB

1st August 2020, 11:52

Derek was referring to this post (https://forum.doom9.org/showthread.php?p=1908011#post1908011)

As to the reason, believe it or not, that's one of the main reasons. What happened is that some companies reacted quicker than others in a period in which there was a huge technological switch and that made things complicated. The first thing is of course about manufacturers that released TV that were able to see BT2020 only without the HLG color curve as I mentioned above. In that period for instance (I believe 2015) we rolled out UHD in an H.265 stream with BT2020 SDR 100 nits. That worked like a charm. Many companies did the same but when we had to release it across the group as well like in Italy and other countries two years later, there were HLG and PQ already and we wanted to have a process as consistent as possible by launching HDR but also keeping old users with the old UHD BT2020 SDR option to be able to see contents. Being like "you can't watch UHD anymore unless you buy a new TV from today" after as little as two years wasn't really ideal for users... So we ended up with the HLG compromise as we didn't really want to air two separate 4K streams and waste twice the bandwidth, especially 'cause it's extremely costly on HotBird as it's really overcrowded. Sure, people may say that internet based decoders will be the future, but not everyone has a good connection and you're also moving the cost from a satellite to a CDN which isn't exactly cheap (although it's way cheaper than the satellite), which is why we're inviting customers to subscribe to our fiber internet offer.
Anyway, I'm digressing.
The thing is that this is our situation and many companies did the same. You also mentioned the cameras and that's true, for live sporting events they can shoot directly in BT2020nc HLG 10bit which is a big added bonus 'cause you can easily connect them all via SDI, route them to your beloved video server, adjust them and you're fine. You can also live downscale and tonemap the signal to FULL HD Linear BT709 SDR 100 nits and since it's a 50p stream it can be easily divided in fields to get a 25i all live with specialized hardware so that it can be aired in FULL HD as well (and in 720x576 25i Linear BT601 SD from the FULL HD stream for what matters). So it's a win win. If you wanna go to PQ instead, it becomes a bit trickier. Cameras don't generally shoot in PQ, they either use Linear BT709, HLG or Log (where Log can be SLog, CLog, LogC, FLog, VLog etc) depending on the manufacturers to get as many stops/nits as possible. Of course you can't air in Log (although I've had much fun laughing at other companies that did this wrong and ended up with things like highlights in Log on air by mistake xD) so you have to apply a matrix of Linear Transformation to go from Log to PQ and that has to be manually written every single time, for every event and possibly in the same even, adjusted for every camera. So you still have everything connected through SDI which goes to your hardware which performs the linear transformation and delivers it to the video server where you can adjust everything and go on air. Now, what about the ones with a TV that ignores the PQ colour curve? They're gonna have to watch the FULL HD Linear BT709 SDR 100 nits 25i stream... Right, speaking of which, how are you going to produce it? The same way as before? Oh no, that would be lame, you shouldn't start from an already linearly transformed signal (the PQ one) and tonemap it to Linear BT709 'cause your original signal is in Log, so you should start from the Log. Therefore, you're gonna need a second tech ops, encoding and ingest department that is gonna do exactly the same as the 1st one but instead of linearly transforming to PQ, they linearly transform to Linear BT709 and divide the 50p in fields to get the 25i. This is a mess, you have redundant people doing the same thing and you're gonna have to put extra stress on your departments and have two separate production teams... And for what, since many TV and cameras were not able to go past 780 nits leaving alone 1000 nits 'till not so long ago? If you take all these things into account, then HLG suddenly looks like a feasible option...

Side note: I think that BT Sport actually tried a similar workflow but I'm not sure 'cause I don't work there but apparently some former Sky employees are working there and these were the rumors, go figure...

Side note II: this is what I learned from my very personal experience, things might be different in other countries and other companies, but from what I saw I'm pretty confident that we're all in the same shoes more or less.

wswartzendruber

1st August 2020, 18:04

Wow, and here I am liking HLG simply because it's relative and not absolute.

videoh

1st August 2020, 18:10

Wow, and here I am liking HLG simply because it's relative and not absolute. I'm with you, bro. PQ requires a reference background brightness that diverse environments cannot all satisfy.

wswartzendruber

2nd August 2020, 04:05

Avengers is encoding now. I pegged reference white to the "MARVEL" text at the start. Hopefully it turns out good.

To be honest though, as someone who's implemented PQ based workflows (HDR10 and Dolby Vision) for streaming services, it all works fine.
I genuinely want to know why streaming these days is not done in HLG.

VictorLS

4th August 2020, 11:41

I genuinely want to know why streaming these days is not done in HLG.
I believe due to not all players (devices like TVsets etc) support HLG as well as SDR but in nearest future HLG must becomes standard de facto because declared backward compatibility with SDR.

wswartzendruber

4th August 2020, 19:08

I believe due to not all players (devices like TVsets etc) support HLG as well as SDR but in nearest future HLG must becomes standard de facto because declared backward compatibility with SDR.
I've concluded that HLG was never intended to be viewed on SDR devices unprocessed. The compatibility that has been mentioned refers to the production environment. Specifically, the BBC states that HLG can be viewed on existing devices for "non-critical monitoring."

From my experimentation, HLG can look really good on SDR (with 709 color) as long as:

1. The player is HLG-aware and tone maps accordingly.
2. The reference white level for the picture is at 75% signal strength.

The Avengers came out looking really good after using the "MARVEL" text at the beginning to set reference white. I'll post an Imgur link when I'm off work.

Screenshots: https://imgur.com/a/9nEZhlR

VictorLS

9th August 2020, 16:58

I've concluded that HLG was never intended to be viewed on SDR devices unprocessed.
Seems you're right.
Screenshots
Great colors.
Can you upload some PQ video and therefore HLG video after your PQ to HLG Converter?

Blue_MiSfit

9th August 2020, 22:21

I find it interesting that such a significant technical decision (PQ vs HLG) appears to have been made solely based on concern for such a small group of 4K early adopters with UHD SDR BT2020 displays.

If you are okay with giving that small cohort FHD SDR 709 along with all the other people with legacy HDTVs, then you can just have that legacy feed and a pure HDR feed.

You could make it HLG, sure, but there's a lot of very good HDR TVs that don't support HLG. That seems like a much more important cohort than the tiny number of UHD SDR TVs. Understood that production and acquisition must be done in LOG or HLG, fine, but what you COULD do is distribute in PQ that's compatible with Dolby Vision Profile 8.1, HDR10+, and HDR10.

This is compatible with every single HDR TV ever manufactured.

This is all doable today for VOD, and I think soon for live (tho not sure live HDR10+ is even a thing? Doesn't seem possible to me). Even if it's just HDR10, that works literally everywhere AFAIK. Show me a single HDR TV without support for HDR10.

For live via traditional satellite distribution where transponder capacity is so expensive, this seems like the ideal HDR WCG solution to me.

I guess, it comes down to -- who do you screw over? Someone with an old UHD SDR display, or someone with an earlier UHD HDR display that is excellent other than its lack of support for HLG? For example, I bought an LG B6 a few years ago. It's a great TV still to this day, but it only supports HDR10 and Dolby Vision. Do you screw me over, or the guy with a super expensive first generation UHD TV who, let's face it, has probably already upgraded because he wants HDR!

FranceBB

9th August 2020, 23:47

Understood that production and acquisition must be done in LOG or HLG, fine, but what you COULD do is distribute in PQ that's compatible with Dolby Vision Profile 8.1, HDR10+, and HDR10.

Technically yes, but that would require testing from scratch the whole playout ports behavior in all different scenarios and most importantly metadata... Yeah... metadata, which should change according to the content and would have to be included and aired as well and that could create issues as the whole thing would have to be tested again. Just think about ads in between, you would have a movie at a certain amount of nits with a reference display and then an ad with a totally different amount of nits and reference, with its own metadata... It's not exactly easy to do and test, while in HLG almost everybody actually don't air metadata other than the fact that it's HLG and BT2020nc and leave the TV to deal with the incoming feed, but that shouldn't really happen with PQ, which brings yet one added nag.

This is all doable today for VOD

Yep, 'cause VOD is a completely different thing, but who would ever encode the same movie or TV Series twice once it has been encoded in HLG already? No one (just like no one would do it in PQ only as playout channels are still in HLG). In other words, it can be done, but no TV Studio does it 'cause it would require two encodes, two mezzanine files and two QCs unfortunately... (but services that are VOD-only and not owned by a TV Company could potentially do it and are doing it already I think). Derek, I don't know if I can ask you directly out of curiosity, but you work for big D, is your + offer in PQ with HDR10? HDR10+? Dolby Vision? HLG? SDR only? Are you dragged behind by linear channels as well?

and I think soon for live (tho not sure live HDR10+ is even a thing? Doesn't seem possible to me). Even if it's just HDR10, that works literally everywhere AFAIK.

Well, HLG is not here to last in the very long run, but I think we'll have to live with it for quite some time...

I guess, it comes down to -- who do you screw over? Do you screw me over, or the guy with a super expensive first generation UHD TV who, let's face it, has probably already upgraded because he wants HDR!

lol your TV would still recognize the feed as BT2020nc SDR and you would see the same "just barely watchable" feed someone with an old BT2020 SDR UHD TV would see, so none is screwed (or both of you are, depending of the point of view hahahahahaha)

wswartzendruber

10th August 2020, 00:26

I find it interesting that such a significant technical decision (PQ vs HLG) appears to have been made solely based on concern for such a small group of 4K early adopters with UHD SDR BT2020 displays.
The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range

Blue_MiSfit

10th August 2020, 21:06

Technically yes, but that would require testing from scratch the whole playout ports behavior in all different scenarios and most importantly metadata... Yeah... metadata, which should change according to the content and would have to be included and aired as well and that could create issues as the whole thing would have to be tested again.

It's pretty interesting to see how many displays totally ignore the ST2086 metadata and just do their own thing :D

Yep, 'cause VOD is a completely different thing, but who would ever encode the same movie or TV Series twice once it has been encoded in HLG already? No one (just like no one would do it in PQ only as playout channels are still in HLG). In other words, it can be done, but no TV Studio does it 'cause it would require two encodes, two mezzanine files and two QCs unfortunately
This is always required. If you deliver both 2020 and 709 you've got two files to QC. If your 2020 file happens to use HLG or PQ that doesn't change this.

Derek, I don't know if I can ask you directly out of curiosity, but you work for big D, is your + offer in PQ with HDR10? HDR10+? Dolby Vision? HLG? SDR only? Are you dragged behind by linear channels as well?

I'll take a moment here to clearly state for the record that anything I have ever said or will say here on doom9 is NOT as a representative of Disney, but is from me as an individual :)

I don't work on D+, but I know the service offers SDR, HDR10, and Dolby Vision. There's not currently any HLG or HDR10+ support.

I DO work on Movies Anywhere, which supports the same formats.

Blue_MiSfit

10th August 2020, 21:09

The BBC and NHK wanted a scene-referred HDR system that did not depend on metadata.

https://www.bbc.co.uk/rd/projects/high-dynamic-range

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?

wswartzendruber

11th August 2020, 16:06

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?
FUD? You mean Fear, Unbelief, and Doubt?

My background is that I didn't get into HDR at all until HDR10+ was already a thing. So I'm very late to the game. Research and study (various docs from the ITU and BBC) revealed that for some reason, the industry had chosen to begin distributing content in a system that used absolute brightness coupled with metadata. Distribution to end users inherently entails differing viewing environments, which metadata doesn't address at all.

If you are going to point at HLG and call its rationale FUD, then I want you to defend distributing content in PQ instead. What advantages do you receive?

FranceBB

12th August 2020, 00:08

I'll take a moment here to clearly state for the record that anything I have ever said or will say here on doom9 is NOT as a representative of Disney, but is from me as an individual :)

Yeah, just like mine represent my own point of view and have nothing to do with the company I work for as they're my personal thoughts.

(Since you specified it, I specified it as well; better safe than sorry xD)

I don't work on D+, but I know the service offers SDR, HDR10, and Dolby Vision. There's not currently any HLG or HDR10+ support.

I DO work on Movies Anywhere, which supports the same formats.

Oh, ok, I see.

Interesting. These seem directly coupled. Is the desire to be scene referenced specifically because of the FUD surrounding the impact of metadata?

As long as nobody brings up the joke about broadcast guys screaming when they hear the word "metadata", I'm gonna keep posting in the thread xD
Jokes aside, non-static metadata on a linear channels with contents played by a playout port, encoded live and aired can be a pain in the... *forbidden word :P * so I understand why some decision were taken in the past (along with all the other reasons I stated before).

Blue_MiSfit

12th August 2020, 01:47

FUD? You mean Fear, Unbelief, and Doubt?

My background is that I didn't get into HDR at all until HDR10+ was already a thing. So I'm very late to the game. Research and study (various docs from the ITU and BBC) revealed that for some reason, the industry had chosen to begin distributing content in a system that used absolute brightness coupled with metadata. Distribution to end users inherently entails differing viewing environments, which metadata doesn't address at all.

If you are going to point at HLG and call its rationale FUD, then I want you to defend distributing content in PQ instead. What advantages do you receive?

FUD = fear, uncertainty, and doubt ;)

To be honest, I was just trolling a little :D I don't have any hard evidence whatsoever that HLG is inferior to PQ, or that using PQ offers some advantage over HLG --- aside from my unchallenged point that PQ offers wider device compatibility in the rec2020 case, especially on early HDR TVs.

Thankfully, almost everyone seems to support HLG on modern devices now from what I can tell. The notable exception is Roku set-top boxes (distinct from TVs with Roku's software like TCL). This is a ___HUGE___ market in the US, actually, and is the single biggest cohort for 10 foot viewing here, so that's a major factor.

Does anyone know of any other new devices that support a PQ format but not HLG?

The creatives I've spoken with with prefer PQ, and are in general fans of having an absolute signal that is display referenced. Their standpoint is that with relative luminance, for example, HLG can misrepresent contrast.

However, I absolutely do feel the pain of needing to worry about a 1000 nit grade or a 2000 nit grade, and worrying about how mapping these to a 4000 nit display is not defined, even if there are a lot of pretty good solutions to map either of those examples to a 600 nit display, for example. I can see the draw of sticking to a relative, scene-referenced signal, and having a reasonable degree of faith that it will display nicely on a broad swath of HDR WCG displays without needing to futz with metadata.

My point was that, to be totally pragmatic, many displays totally ignore metadata and just do their own thing. So like, I wouldn't be surprised if everything would mostly be fine if we just had no metadata at all (other than the transfer, primaries, and differencing flags to identify content as PQ or HLG, and 709 or 2020, of course!) hence the "metadata FUD" troll ;)

Personally I still prefer Dolby Vision Profile 5 for a number of reasons, but that's getting OT pretty fast!

wswartzendruber

12th August 2020, 07:22

The creatives I've spoken with with prefer PQ, and are in general fans of having an absolute signal that is display referenced. Their standpoint is that with relative luminance, for example, HLG can misrepresent contrast.
Compliant HLG displays are supposed to have an EOTF modifier that adjusts the gamma curve depending on ambient light, if that is what you are referring to. With that said, I've never actually gotten my grubby paws on a HLG set. My workflow has been:

1. Study spec sheets.
2. Write code.
3. Write tests for code (checking math values).
4. How does it look on SDR through a HLG-aware (tone mapping) player?

Nailing #4 is all about figuring out a good reference white value (in nits) for the PQ source material. That's everything. It'll look too dim if that's set too high and it'll look completely clipped if it's set too low. I think I've said this before, but I am going to say it again and again because of how important it is.

Nailing #4 is a massive step in having HLG content that can play back on 709 devices. Assuming of course, you have a competent color+tone mapping player, which even the open source community has access to.

My point was that, to be totally pragmatic, many displays totally ignore metadata and just do their own thing. So like, I wouldn't be surprised if everything would mostly be fine if we just had no metadata at all (other than the transfer, primaries, and differencing flags to identify content as PQ or HLG, and 709 or 2020, of course!) hence the "metadata FUD" troll ;)
BT.2408 makes an attempt at this, which puts reference white at 203 nits. Notice how we're taking something relative (reference white) and pinning it to something absolute (203 nits). This, of course, depends on movie studios following these conventions. And I can tell you with absolute certainty that this is not the case.

FOR FUN: Here's an attempt at Rogue One on VLC (SDR/709 tone mapping) with reference white set too low. https://imgur.com/gallery/AyQW7Pg

EDIT: I need to watch Rogue One again just to pick out a good reference white scene. That movie is really throwing me around.

Alita: easy
Avengers: easy
Age of Ultron: slightly difficult
Rogue One: "Ohhhhhhh no! You are NOT converting me to HLG!!!"

EDIT: I wonder if I could build a sort of predictive model that analyzes every pixel of every frame and then computes an approximation for where reference white should be...

Blue_MiSfit

13th August 2020, 19:53

Theoretically, if they were present and accurate wouldn't the MaxCLL / MaxFALL values in the ST2086 metadata be helpful here?

wswartzendruber

13th August 2020, 21:04

Theoretically, if they were present and accurate wouldn't the MaxCLL / MaxFALL values in the ST2086 metadata be helpful here?
Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.

In any event, I can compute this stuff myself. And that's what I'm already doing with pqstat (https://github.com/wswartzendruber/pqstat). I could have it compute MaxFALL as well, if I wanted.

But what I'm really after is reference white and I'm not seeing how MaxCLL and MaxFALL will help with that. By reference white, I mean the border between SDR and HDR brightness levels.

FranceBB

13th August 2020, 22:03

Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.

Really?! O_o Leaving aside the 1000 nits, even if they're lower, they should all have metadata and it's actually very important in HDR10 PQ. I would understand HLG .ts files, but not UHD-BD disk with PQ... Are you ripping the original m2ts file? What does ffprobe say? Perhaps it's just MediaInfo not showing them for some reason? I mean, it would be weird for an HDR10 official disk not to have metadata.

Blue_MiSfit

13th August 2020, 22:47

Half of my 4K discs don't even seem to have HDR10 metadata, and I'm not sure why not. Only one of them (so far), Rogue One, goes above 1,000 nits.

In any event, I can compute this stuff myself. And that's what I'm already doing with pqstat (https://github.com/wswartzendruber/pqstat). I could have it compute MaxFALL as well, if I wanted.

But what I'm really after is reference white and I'm not seeing how MaxCLL and MaxFALL will help with that. By reference white, I mean the border between SDR and HDR brightness levels.

I see, yeah that's the real trick. I wonder how companies like Colorfront do this in real-time with a pleasing result...

wswartzendruber

16th August 2020, 05:28

Can you upload some PQ video and therefore HLG video after your PQ to HLG Converter?
What you have asked for no doubt seems simple, but I do not know how to accurately do that. I suppose I could start by compressing the Rec.2020 colorspace into the Rec.709 triangle.

Really?! O_o Leaving aside the 1000 nits, even if they're lower, they should all have metadata and it's actually very important in HDR10 PQ. I would understand HLG .ts files, but not UHD-BD disk with PQ... Are you ripping the original m2ts file? What does ffprobe say? Perhaps it's just MediaInfo not showing them for some reason? I mean, it would be weird for an HDR10 official disk not to have metadata.
VLC has a metadata display. It's probably just not working. I haven't tried ffprobe. I've actually been using the pqstat program just to be sure.

I use MakeMKV to remux whatever streams I pick from the disc into Matroska.

I see, yeah that's the real trick. I wonder how companies like Colorfront do this in real-time with a pleasing result...
I have no idea, but the deeper I've gotten into this, the less it's become about math and the more it's become about eyeballing it. Which is what I did with Rogue One finally to put reference white at 180 nits. I'll link to screenshots of that when I'm off work. So maybe they have some fancy machine learning algorithm.

Speaking of which, my friend bought a HDR-capable phone and had no idea. So we sampled Rogue One (my latest attempt) on both our sets side-by-side. And while I didn't notice much difference in luminousity, the color on his was absolutely gorgeous compared to mine.

EDIT: Rogue One HLG Screenshots (https://imgur.com/gallery/T9PvLtT)

FranceBB

16th August 2020, 22:01

VLC has a metadata display. It's probably just not working. I haven't tried ffprobe. I've actually been using the pqstat program just to be sure.

I use MakeMKV to remux whatever streams I pick from the disc into Matroska.

Well, the remux shouldn't really get rid of the metadata as they're in the video essence itself, so if they put them there, they're still gonna be there after the remux. Anyway, VLC is probably one of the worst players for HDR contents, in fact my choice falls on MPV, especially if you have an SDR monitor. By the way, I actually found the MPV tone-mapping for PQ contents with metadata to be pretty much fine, while it's kinda sloppy for HLG materials without metadata...

videoh

16th August 2020, 22:38

VLC is probably one of the worst players for HDR contents You should provide evidence when making baseless claims that impugn others' work. And 'probably'? Is that only on random days or do you roll dice every day to decide?

...while it's kinda sloppy for HLG materials ... What does "kinda sloppy" mean here? Can you show it with screenshots?

When someone resorts to rhetoric we have to question their confidence in their technical arguments.

Don't mean to be harsh. ;)

FranceBB

17th August 2020, 00:27

You should provide evidence when making baseless claims that impugn others' work.

Well, when I tried it years ago it wasn't doing any kind of mapping, it was just wrongly displaying the data on SDR displays, outputting both BT2100 PQ and BT2020nc HLG without taking into account their color curve nor their color matrix (i.e grey-ish).

VLC: https://imgur.com/JktOMOt
MPV: https://imgur.com/ky2Kg6k

You can clearly see that VLC got it wrong and that it shouldn't really be displayed that way on SDR displays...
I gotta say that I don't have it installed anymore, so I don't know whether things improved or not, but I can say that MPV definitely applies the tone-mapping when it has to display HDR on an SDR monitor (and it did it years ago as well, while VLC didn't).

What does "kinda sloppy" mean here? Can you show it with screenshots?

Well, it's kinda hard to do it with screenshots 'cause they would look ok. What I mean by "sloppy" is that when I try it with 1000 nits HLG files without metadata (other than the fact that they're HLG and BT2020nc 10bit) that have frequent transitions between a scene with high lights and low lights, it struggles. When I was watching Riviera, for instance, there was a fairly dark room with sharp lights every few meters. When Cherry was walking and was not under the light, the picture was going to dim while, as she walked under the light, the picture was going all the way up to the maximum brightness allowed by SDR. This is fine, however imagine a person walking with the monitor going bright, dark, bright, dark, bright, dark all the time in a matter of seconds with the whole scene (not just the main character) changing (including the background). I never noticed such a thing with PQ with metadata in which a compromise between being "under the light" and "not under the light" was chosen and it was much "easier" to watch and way less stroboscopic. The absence of scene-referred metadata might play a very big role in some contests, especially with SDR monitors watching an HDR content which is supposed to be viewed on an HDR display. This of course doesn't really mean that the HLG tone-mapping of MPV is bad nor that the developers didn't do a good job; heck, the tone-mapping it's good and I don't even know whether it can be improved much further given that HLG doesn't have metadata, but I just pointed out that in my personal experience MPV generally behaves better with PQ files with metadata than it does with HLG ones without metadata and with very high spikes in terms of nits on some scenes.

When someone resorts to rhetoric we have to question their confidence in their technical arguments.

Don't mean to be harsh. ;)

lol, no problem at all, when I say things they're always based on my own personal usage and on what I found out in my examples, not as "absolute statement" or "dogma". They're definitely open to discussion. So... even if I don't specify anything, always take them as my personal point of view ;)

wswartzendruber

17th August 2020, 22:03

I've noticed that VLC tone maps HLG differently depending on OS, which is strange.

Anyway, I also remember the release notes for 3.0.0 saying that they corrected HLG tone mapping.

FranceBB

17th August 2020, 22:08

I've noticed that VLC tone maps HLG differently depending on OS, which is strange.

Anyway, I also remember the release notes for 3.0.0 saying that they corrected HLG tone mapping.

Since you have the latest version of VLC, can you try to compare its HLG tonemapping to the one done by MPV? It would be interesting to see how they look like and whether the VLC guys actually really improved it compared to years ago.

butterw2

17th August 2020, 23:59

VLC uses different video output APIs vs OS.
Default on Win10 is Dx11. If you check the changelog you will see there have been a number of HDR updates for this mode in the last year.

there is also an OpenGL for windows mode.

Dx9 doesn't do tonemapping without using a shader.

wswartzendruber

18th August 2020, 02:35

Since you have the latest version of VLC, can you try to compare its HLG tonemapping to the one done by MPV? It would be interesting to see how they look like and whether the VLC guys actually really improved it compared to years ago.

Here you go; VLC is on the top in each one while MPV is on the bottom: https://imgur.com/a/2JeNJvT

I was blown away by this at first. MPV seems to handle 2020->709 conversion with dramatic superiority along with having much more pleasing gamma. In fact with MPV, I can't even tell it wasn't graded natively in 709.

FranceBB

18th August 2020, 09:57

Here you go; VLC is on the top in each one while MPV is on the bottom: https://imgur.com/a/2JeNJvT

Thanks.

I was blown away by this at first. MPV seems to handle 2020->709 conversion with dramatic superiority along with having much more pleasing gamma. In fact with MPV, I can't even tell it wasn't graded natively in 709.

Yeah, that's exactly what I was talking about when I said that I switched to MPV years ago. Looks like VLC did improve, but it still lacks behind MPV when it comes to tone-mapping HDR contents.

wswartzendruber

28th August 2020, 05:59

Two things:

1. MPV on Linux is using luminance mapping. Bright specular areas will dim over a couple of seconds, and it's not part of the movie.

2. VLC on Windows doesn't exhibit this and looks much better than VLC on Linux.

wswartzendruber

11th September 2020, 07:18

I finally got to watch my encodes on a HLG display, a Samsung from either this year or last. It was absolutely abysmal. The TV correctly detected the gamma curve as HLG, and I disabled the TV's enhancements for everything, but it just looked bland and washed out.

Luminance

I could see the TV continually adjusting the brightness level depending on the average luminance of the frame being shown. I'm not understanding why it wouldn't just apply tone mapping similar to VLC and MPV, although much less aggressively. The tone mapping used would depend on what the TV's brightness is set to. The higher the brightness setting, the more quickly specular detail gets compressed. This...shouldn't be hard.

Color

I don't know how to describe this other than "underwhelming."

HEVC Decode

On my PC, the HEVC decoding is exceptional. For movies where I instructed x265 to preserve grain, that comes through nicely. Whereas with this television, each frame seemed more like a matte painting. I wasn't quite sure what I was even looking at or how messed up the decode filtering would have to be to produce this result.

This isn't the result I was looking for. My converter only does linear brightness adjustment followed by gamma conversion. That's it.