I Have Published a HDR10 to HLG Converter

[kolak]

UHD uses Rec.2020 when transmitted between devices. There is no such a thing like P3 in HDMI spec. This is why all BD UHD discs use 2020 primaries (and every TV station requires 2020 primaries for UHD) and any connection between 2 UHD devices over HDMI must be done with Rec.2020 (optionally Rec.709 is allowed if I'm correct).
Gamut is one thing and transmission standard another. Atomos is not an exception when it comes to HDMI in/out.

Resolve won't protect you against overshoots by default. FF has/will have some filtering built in (like many other pro tools). Its author meant o publish some technical doc about it.
There are tools which measure actual P3 gamut cover, but those are quite expensive.
Don't trust internet so much.

[Balling]

BTW, specifically for you red tulips from BT.2020 mdcv blu-ray of spears and munsil and if limited to P3 in color limiter of Davinci.

https://imgur.com/a/iHcmWdf

And of course AVIF tagged as Bt.2020/bt.2020nc/PQ with all those pixels. Open in chome, not in buggy mozilla. lossless avif with avif samples 420 lossless md5 with original and also not tagged as top left chroma siting, a bug in ffmpeg! Hope will be fixed!

So who is a genious! I am! Hah! (Rename to AVIF, can see it natively here: https://github.com/mpv-player/mpv/issues/10247) https://user-images.githubusercontent.com/104958042/174458649-ccff6eb4-0d6c-449c-a60a-7c0c8c817779.jpeg

>Don't trust internet so much.

Sigh. I do not need to. I generate most of this out of P3 comments on the internet under many different names.

[kolak]

Not sure what are you trying to prove?
That this disc has colors out of P3?
Very possible, you already mentioned it.

[Balling]

>Not sure what are you trying to prove?

Trying to? I proved it.

[wswartzendruber]

I'm not quite sure I'm following the scuffle you two have gotten yourselves into, but HEVC seems to allow a content author to record the "Mastering display color primaries."

Every 4K disc I have reads "Display P3."

In all cases, though, the color primaries signaled are BT.2020.

EDIT: Scratch that! The Matrix trilogy appears to have been graded natively in BT.2020.

[Balling]

> Every 4K disc I have reads "Display P3."

Spears and Munsil has BT.2020

Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color pri : BT.2020
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light : 400 cd/m2

What it was mastered upon has no impact what can be inside though.

[wswartzendruber]

Yeah, I just caught that. See my edit. All three Matrix movies are BT.2020-graded.

[Balling]

Spiderhead on Netflix has a lot of colors outside P3, so Netflix started using BT.2020 masters. Rescued by Ruby also has some scenes, but mostly inside 709. Planet Earth 2 has sometimes all the colors of Bt.2020 uniformely, which is nuts.

[FranceBB]

Planet Earth 2 has sometimes all the colors of Bt.2020 uniformly, which is nuts.

That's the BBC at work. ;)
I look forward to the day they'll upgrade the BBC iPlayer to have HDR on my Google Pixel 6 Pro too!

[kolak]

I'm not quite sure I'm following the scuffle you two have gotten yourselves into, but HEVC seems to allow a content author to record the "Mastering display color primaries."

Every 4K disc I have reads "Display P3."

In all cases, though, the color primaries signaled are BT.2020.

EDIT: Scratch that! The Matrix trilogy appears to have been graded natively in BT.2020.

Exactly what we said. Most discs are P3 limited or at least P3 graded ( so give correct preview on P3 screen, but have colors outside P3). All use 2020 as primaries as there is no such a thing like P3 transmission format. Single discs may be 2020 graded/tagged, so you are confirming yours and our claims.

Those HDR tags always been there and some TVs use them some not. Regardless, they should be accurate, not some random or max values as you will find on some discs ( like 10K MaxCCL). Early disc are not done 100% properly as industry was struggling with tools. It’s way better now, but exceptions still happen.
Btw. when you see 1000/400 for MaxCLL/MaxFALL then it's likely to be a false value as this is some default used by Resolve.

[kolak]

>Not sure what are you trying to prove?

Trying to? I proved it.

I already believed you that some discs may have colors outside P3 ( where some may be just falsely reported as such). I said that standard practice is to limit gamut to P3 (for broadcast it’s in many cases a hard rule, not just a recommendation).
It’s down to studio practice- if they turn on P3 limiter or not.
Content which is graded to full Rec.2020 on Rec.2020 capable device is very rare.
As I said- disc which is graded on P3 screen but kept as 2020 may actually be "problematic" when watched on 2020 capable display. You may find that some lights etc. are annoyingly saturated etc. as they were not seen by anyone to judge.
You can't just assume that that portion of the colors which you don't see during grading with P3 display will look perfectly (without seen it).
It's like if you were grading to P3 on Rec.709 display. I'm not a colorist, but I know no one does it.

[rwill]

I have seen my fair share of content and almost never have encountered content that was graded in P3D65 and delivered in BT.2020 that had problems. One just has to be somewhat careful.

I have seen problems with P3D65 being clipped to BT.2020 though.

[wswartzendruber]

I have seen problems with P3D65 being clipped to BT.2020 though.
What?

[rwill]

What?

P3D65 does not completely fit into the BT.2020 gamut. Its somewhat academical though.

If I remember correctly I only ever investigated things flagged by automated QC tools and nothing found by actual human guided QC.

[huhn]

well just compare them p3d65 just have a very tiny be of red that bt 2020 can't do.
compared to bt 2020 that red is more green.
it is so little it is kinda irrelevant.

[Balling]

well just compare them p3d65 just have a very tiny be of red that bt 2020 can't do.
compared to bt 2020 that red is more green.
it is so little it is kinda irrelevant.

I checked with 3D, it does not appear to be the case (in-gamut, liminance is not HDR) https://www.colour-science.org:8020/

(select as primary colorspace BT.2020 and as secondary P3-D65).

[kolak]

I found where the two SMPTE snippets above come from:

https://www.murideo.com/uploads/5/2/9/0/52903137/study_group_on_high-dynamic-range-hdr-ecosystem.pdf

This includes the algorithm for calculating MaxCLL. It demonstrates that MaxCLL is not necessarily the Y value of the brightest pixel in the stream as many sites suggest. Instead, MaxCLL tells the display what magnitude each color channel can reach, but all three don't have to reach this level, only one does. In practice, though, if the brightest pixel is truly white, then all color channels will reach this level.

EDIT: Working on hlg-tools-0.2.0 now... Version 0.1.0 is not even remotely correct.

Looks like industry is changing.
In one of the reference tools used in post MaxCLL is now calculated for compressed formats based on luminance (ITU-R BT.2020-2 Table 4). This is opposite to the max R, G or B component.

[huhn]

I checked with 3D, it does not appear to be the case (in-gamut, liminance is not HDR) https://www.colour-science.org:8020/

(select as primary colorspace BT.2020 and as secondary P3-D65).
please just move on it's very easy to prove.
https://abload.de/img/dcip3redisnotpartofbt6nku5.png

[wswartzendruber]

Looks like industry is changing.
In one of the reference tools used in post MaxCLL is now calculated for compressed formats based on luminance (ITU-R BT.2020-2 Table 4). This is opposite to the max R, G or B component.
Using Y for MaxCLL is outright irrational. Let's say the value is signaled at 593 nits. That can mean anything from 6% equal intensity across all three channels to 100% intensity on just blue.

[wswartzendruber]

Since I don't have a BT.2020 display at my house, I'll have to ask everyone (subjectively) how good Yavin IV looks here:

https://www.youtube.com/watch?v=oMHK7IbpI_I

[kolak]

Using Y for MaxCLL is outright irrational. Let's say the value is signaled at 593 nits. That can mean anything from 6% equal intensity across all three channels to 100% intensity on just blue.

No idea. I just read it in tool release notes.
6% on R,G,B gives bright white? I thought it's about black and Y will be low (at eg. 70 and 70 is not 593 nits for sure)?

[wswartzendruber]

Y(r = 0.06, g = 0.06, b = 0.06) = 593
y(r = 0.00, g = 0.00, b = 1.00) = 593

[kolak]

https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2020-2-201510-I!!PDF-E.pdf

table 4 has this:

Y'=0.2627R' + 0.6780G' + 0.0593B'

are they referring to it?

[wswartzendruber]

https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2020-2-201510-I!!PDF-E.pdf

table 4 has this:

Y'=0.2627R' + 0.6780G' + 0.0593B'

are they referring to it?
Those are the BT.2020 coefficients, so yes. But you can apply them to display light as well.

[kolak]

So how do they count light then if they refer to this table?

[wswartzendruber]

So how do they count light then if they refer to this table?
I have no idea. MaxCLL is specific to PQ, so how it's taken should reference BT.2100.

[kolak]

Have you implemented any filtering for compressed formats?
Like skip 1% of brightest pixels, low pass filter?
If you take properly 1K nit mastered master what MaxCLL is your tool reporting?

[wswartzendruber]

Have you implemented any filtering for compressed formats?
Like skip 1% of brightest pixels, low pass filter?
If you take properly 1K nit mastered master what MaxCLL is your tool reporting?
I haven't done any filtering. hlg-tools includes a utility called pqstat, but it doesn't filter. You'd need to do that via FFmpeg or somesuch.

I have heard rumor that there are defined specifics on how this is to be done, but I don't know what they are.

[kolak]

There are none.
Each company does own based on tests.
Paper math is not designed for compressed sources.

Quick test v210 vs ProRes gives on some random short file max Y as 922 vs 926, so it would translate to few nits, but not that bad actually.

[wswartzendruber]

There are none.
Each company does own based on tests.
Paper math is not designed for compressed sources.

Quick test v210 vs ProRes gives on some random short file max Y as 922 vs 926, so it would translate to few nits, but not that bad actually.
I guess I may as well start messing around.

[kolak]

Well this was ProResHQ. Now do the same with 30mbit h265 4:2:0 and things most likely will look way worse.
Maybe good solution is something which would average pixels so smallest one is made of 4 original. Who cares about place in the frame which is 1 pixel big? Maybe 4 is also too small?
Half size resize with area filter seems to give quite good (923) results on this sample :)
This is 30 seconds test, so don't rely on it :)

[wswartzendruber]

Well this was ProResHQ. Now do the same with 30mbit h265 4:2:0 and things most likely will look way worse.
Maybe good solution is something which would average pixels so smallest one is made of 4 original. Who cares about place in the frame which is 1 pixel big? Maybe 4 is also too small?
Half size resize with area filter seems to give quite good (923) results on this sample :)
This is 30 seconds test, so don't rely on it :)
I'm going to start by having FFmpeg blur the living crap out of it.

Eh, not that much, though.

[kolak]

Half size seems to work :)
Not sure if blur is a good way.

Interesting fact. h265 performs better than ProRes. Maybe it's not a surprise actually as it's way more complex codec and handles contrast much better. 924 max value, so not that big deal. Looks too good actually :)
Maybe 20Mbit is a lot for HD.

[wswartzendruber]

Half size seems to work :)
Not sure if blur is a good way.

Interesting fact. h265 performs better than ProRes. Maybe it's not a surprise actually as it's way more complex codec and handles contrast much better. 924 max value, so not that big deal. Looks too good actually :)
Maybe 20Mbit is a lot for HD.
Actually, yeah! Except I'm going to quarter-size 4K down to 540p.

EDIT: Quarter-size on each axis.

[kolak]

Yep, quarter size may be even better and faster to process (tried on HD as well and it was good). How much would it affect MaxFALL calculation?
Maybe I'm over-exaggerating it.
ProRes will do few levels overshoot, around 4-8 max (so far I've seen by simulating very high contrast edges), so it's not that much. How many nits is this in worse case?

5mbit h265 for HD is doing up to 20 Y levels, so this is bit high.
Wavelet based codecs (Cineform, Jpeg2000) behave bit better. DCT based always go up, where wavelet ones also bit down.

[wswartzendruber]

I just got from pqstat that Iron Man has a MaxCLL of 2,682 when metadata declares 826.

I'm going to try omitting the first and last several seconds in case there's some frame decoding error and artifacts are getting rendered.

[kolak]

Hmm...2,682 sounds bad. No one would master it to such a value. It doesn't sound 2K with overshoots, just random value, so something must be not right. Try some 20 min. portion- should be representative and give max as well (no guarantee of course).
Studios master mainly to 1K, 2K and 4K nits and typically grading software will clip max to this values (+- few nits). This is why value like 2682 is strange.
How old is this title? Try something from last 6 months.

Why can't we measure it this way.
All delivery codecs are YUV based, so just find max Y value and then count light by comparing it to PQ curve. Eg. I know that 769 (for 10bit) is 1000nits (this also perfectly agrees with Resolve when grading).
This sounds simple and should be correct (overshoots problems still apply though).
Why do we need some convoluted way to recreate RGB when original grading decision is bound to Y value inside your file?

[wswartzendruber]

Because MaxCLL is not properly calculated using Y values. It is properly calculated with max(R,G,B).

[FranceBB]

To be fair, I have a few ProRes HQ over here and I was checking the MaxCLL value by comparing it with the one sent via xml by the official provider.
For Harry Potter and the Chamber of Secrets, the official value provided via xml is 2856 Nits, however the one calculated by PQStats is 3204 which means that it's off by 348 nits due to compression overshooting (yes even if it's 900 GB worth of ProRes).

Hmm...2,682 sounds bad. No one would master it to such a value.

For that particular title, sure, but I've seen movies mastered at really arbitrary nits depending on which studio did it.
For instance, this is a series of titles we received over the years along with an xml depicting the MaxCLL:

Goodfellas 247 Nits
The Dark Knight Returns 1399 Nits
The Good Liar 244 Nits
Richard Jewell 1213 Nits
Harry Potter and the Half-Blood Prince 1915 Nits
The Dark Knight 1002 Nits
Harry Potter and the Order of the Phoenix 1135 Nits
Harry Potter and the Deathly Hallows: Part I 2632 Nits
Harry Potter and the Deathly Hallows: Part II 2811 Nits
Zack Snyder's Justice League 835 Nits
Batman Begins 992 Nits
Harry Potter and the Philosopher's Stone 552 Nits
Harry Potter and the Goblet of Fire 2848 Nits
Harry Potter and the Prisoner of Azkaban 1838 Nits
The Lego Movie 1687 Nits
Godzilla 561 Nits
The Town 1339 Nits
Just Mercy 1017 Nits
Birds Of Prey 4616 Nits
Harry Potter and the Chamber of Secrets 2856 Nits
Dune 945 Nits

so as you can see, it looks like the MaxCLL can be as arbitrary as people like, so beware of what the distributor puts on the BD 'cause it might differ from the actual value of the master.

[kolak]

Sorry, but MaxFALL is something totally different than I thought.
Not sure why it’s defined this way, but it has not much to do with fact that you graded to eg. 1K nits. Grading can be 1K where MaxFALL 1.5K ( I assume). Is it true MaxFALL will be always bigger ( or at least equal ) than max brightness coming from Y?
What I see in Resolve and on any other scopes etc. is a separate thing to MaxFALL. Titles are graded to rather round values and it’s 1,2,4K etc. but MaxFALL is totally separate thing. Why, no idea.

[kolak]

Regardless how "strange" MaxCLL and MaxFALL are there is this as an alternative measure (taking into account overshoots problem:

https://postimg.cc/YvJVJCfW

[kolak]

Top and bottom bars (as separate files) give similar MaxCLL (just bit above 1000nits).
In the same time max Y is 475 vs 705, so very different and not really appropriate for MaxCLL calculation.

https://i.postimg.cc/GmPcRHzN/cc.png

Grab is form QTX, so tone mapped to 500 nits Mac screen (irrelevant here).
Top row is just blue channel of the bottom bars.

This makes those metadata values defined in very confusing way, but I assume there is some idea behind them.
I've just learnt they are very different things than what I thought (even if I remembered them been defined in some specific way).

In the same time in Resolve scopes both peak at around 1000nits, so show up correctly. This means when I grade in Resolve (by scopes to 1000nits) MaxCLL should be 1000nits not more.
So I'm confused by numbers from those titles from the other thread. They should not be so "strange" and random. Very possible most of them have simply wrong values as metadata. Some do look fine though.

[wswartzendruber]

Regardless how "strange" MaxCLL and MaxFALL are there is this as an alternative measure (taking into account overshoots problem:

https://postimg.cc/YvJVJCfW
Well hello there...

[quietvoid]

The whole article (image based): https://0x0.st/oSpl.pdf

[kolak]

Hmm...looks like original recommendation wasn't really well thought out. I'm still not very convinced with current formula either.

[wswartzendruber]

Hmm...looks like original recommendation wasn't really well thought out. I'm still not very convinced with current formula either.
It's a bit late to be having this conversation, isn't it?

[kolak]

Why too late?
You can always change a formula :)
MaxCLL may stay as is, but I can see that companies started counting "titles' max brightness" in own ways- more intuitive/representative.
Also for your needs MaxCLL may be simply not the best measure.

[wswartzendruber]

If MaxCLL is retroactively reinterpreted to mean Y instead of maxRGB, television sets are going to suck at tome mapping.

[kolak]

They unlikely to change this, but you can ( if old way is not really representative).
Do TVs actually use it? Apparently many TVs don’t.

[wswartzendruber]

They unlikely to change this, but you can ( if old way is not really representative).
Do TVs actually use it? Apparently many TVs don’t.
MaxCLL tells you how aggressively you need to tone map.