How to analyze an HDR video for peak brightness level for the setting of metadata?

[TomArrow]

Originally Posted by suarsg

I was trying to keep it simple without much details convoluting the actual issue. This was an attempt to make him see how flawed his logic was without going into particulars. He was thinking way too much in sRGB-terms and without even considering how RGB relates to brightness or how to convert an RGB-tuple to luminance.

Originally Posted by suarsg

That's why I stopped responding to him, he made no attempt in trying to understand why that is but instead went on a rant about his life goals.

First of all, what are you talking about? You responded to me twice after that. Lol! Have you considered to stop being a <redacted> and just present your arguments? If you have something to say and are able to teach me what I'm wrong about, please by all means "go into the particulars", but so far you've been doing a shit job at it. I never claimed to be an expert about this stuff, I'm just trying to figure it out. No reason to be condescending. If you're not gonna say anything acually helpful - whether you are knowledgeable or not - you may as well shut up.

Originally Posted by suarsg

Hence for BT2020 (the color space of HDR10!) we have:

Y = 0.2627*R + 0.6780*G + 0.0593*B

Solving for R, G, B when Y=1.0 (=10,000nits) makes it clear there's only one possible tuple. And in SMPTE284, that is (940+,940+,940+).
Not sure why you're bringing 4000nits or display tech into this as it doesn't really matter for the theory. Obviously there's no consumer display out there that can do these values without tone mapping and each display's tech is completely different on a sub-pixel level and how they themselves decide to derive its pixel's brightness from that sub-pixel structure is their own individual job to figure out.
The important part is, if you have an RGB-tuple for a pixel, the pixel's brightness/luminance will not be based on the value of MAX(R,G,B) or AVG(R,G,B). My examples were there to demonstrate why it makes absolutely no sense.

Hypothetically assume that we had a subpixel structure whose primaries are 100% identical to the Rec2020 primaries. In that case, as I understand it, the nits value of each subpixel would be equal to its perceived brightness, as nits is candela/sqm and candela is already a perceptual value (aka measuring blue lower than green and so forth). The TV would, in this hypothetical case, and unless I'm missing something, only have to apply the reverse luminosity function to derive the required watt output of the subpixel and then probably compensate for energy conversion efficiency and voila, that should tell it how much to drive said subpixel.

See: https://en.wikipedia.org/wiki/Luminous_intensity
Also see: https://en.wikipedia.org/wiki/Luminosity_function
Also see: https://en.wikipedia.org/wiki/Radiant_intensity

Why would the brightness of an entire pixel (R, G and B combined) even matter to the TV? The TV, I think, has to be worried about not burning through any subpixel, and maybe energy saving regulations. As long as no individual subpixel gets too hot (or whatever else matters for its longevity), it hardly matters what the value of the other subpixels is. Of course you can argue that the heat from a subpixel affects subpixels around it, however then, depending on the subpixel layout, the TV might - e.g. in case of a red subpixel - be more worried about the blue subpixel of the neighboring pixel since its closer to that red subpixel than the blue subpixel of the same pixel. So knowing the brightness of a pixel as a whole is barely useful, unless I'm missing something.

Originally Posted by suarsg

I agree and this was my first post regarding this: In a YUV420 10bit encoded video (HDR10), doesn't the Y-component of the pixel already define the brightness of the pixel?

See my last post and the whole thing about constant vs. non-constant. In short, as I think I learned, no.

[TomArrow]

So I tested my plugin on some of the MaxCLL test patterns here:
https://www.avsforum.com/forum/139-d...terns-set.html

What I learned is that some of the patterns will read out as almost 10,000 nits despite supposedly being meant to have MaxCLL 1000 or 4000. Other patterns will read slightly above 1000 nits, which seems about right. Seemed like an error to me, so I took a screenshot and checked the RGB values in Photoshop. Turns out, the text superimposed over the colors creates some artifacts that lead to extreme overshoots in some cases, for example with the red testpattern. However I'm not sure if this is a problem with the chart itself or possibly with ffvideosource or ConvertToRGB64(matrix="Rec2020") or simply with the color subsampling, or with a combination of all of those.

Generally speaking, judging by the patterns that gave reasonable readings, MaxCLL indeed seems to be the maximum subpixel value, if those charts are correct. Different colors that were supposed to have identical MaxCLL all turned out to have just that with my method of reading.

I tried a few other videos but the results were very inconsistent and any resemblance between my readings and their data was spurious at best. For example I tested some JVC HDR test Blu Ray and it gave me 10,000 nits where the metadata said MaxCLL was 1020 cd/m2. I took a screenshot again and it was some very tiny highlight, a reflection in some shoe in a shoe store at night, that actually blew out the blue channel. Which leads me to believe that the metadata is either wrong or they used that kind of blurring someone mentioned, which might explain why the metadata is lower. However they also reused the same metadata for multiple clips so I'm not sure it can be trusted at all..

I also tested this video: https://www.youtube.com/watch?v=hsbM2c6-9Wg

My readings for FALL (I just looked at single frames) were a bit less than half of what the video said they were supposed to be in each place I measured. I wonder if this might have to do with that little point about only measuring the "active pixels" and whether I need to just discard all black pixels from the calculation, though that seems a little strange to me. I then cropped the video to only show the right half and measured that, but it didn't really change much.

Maybe someone here can run the same videos through another MaxFALL/MaxCLL application to compare the results.

[kolak]

If I remember well somewhere in the docs there was info about taking into account active area only (if aspect is different than frame aspect).

Those overshoots are present as you deal with compressed footage I assume (and present problem which I was talking about).
In high-end finish master are done as eg. 16bit TIFF so there should be no such a problem (but even encoding to ProRes etc. will introduce overshoots).

There is also some 'inconsistency' with range- some documents use 0-1023, others 4-1019 (due to timing reference been used for other values). As far as I understand eg. HDMI uses 4-1019 range.

[heisenberg]

Originally Posted by suarsg

In a YUV420 10bit encoded video (HDR10), doesn't the Y-component of the pixel already define the brightness of the pixel?

For example, there's a flashlight in a dark scene located at coordinates (x=1000,y=800) in a frame. Decoding this frame of that scene and getting the Y component of the pixel at those coordinates, gives you e.g. the value 643. Using 643 on the PQ EOTF equals 432.5cd/m2.

Am I missing something?

Seems right, for YUV420 10bit PQ transfer video, as you say.

[kedautinh12]

New ver:
https://github.com/TomArrow/MaxCLLFindAVS/releases

[quietvoid]

By the way, in the SMPTE August 2021 journal, there is a paper describing an alternative calculation for the metadata.
Pseudocode:

[kedautinh12]

0.36:
https://github.com/erazortt/MaxCLLFindAVS/releases

[kolak]

So does 0.36 include new formula with rejecting outliers ?

[quietvoid]

Doesn't seem to.
It should be pretty trivial to do using VapourSynth and numpy's percentile.

[kolak]

That's my impression as well. Could someone add it, please?
Other than this it seems to be accurate.

[kolak]

Originally Posted by benwaggoner

The metadata is derived from the uncompressed RGB source, not compressed output. So even if there were overshoots from compression, that shouldn't impact the metadata at all. Same deal when downscaling and chroma subsampling. Even though the scaling is a low-pass filter, the metadata is still supposed to be that of the full-rez uncompressed source.

You just forgotten to add: "in theory".
In practice it's not that often derived from correct stage of production. It's a fact and industry should react and change formula for MaxCLL (there is proposed version with outliers rejections). There are already tons of eg. BDs out there with wrong metadata.
My last investigation pointed that it's not really a compression (definitely not intermediate one), but chroma subsampling which is biggest enemy of MaxCLL calculation.
I also found out that quite many (what meant to be high quality A class tiles) eg. ProRes444 masters are actually gone through 422 at some stage and they are "fake 444". I'm rather disappointed with such a discovery.
These give you "random" MaxCLL values.

Make a simple test to 1K nits in Resolve, export to uncompressed, ProRes/DNxHR 444 and then to 422 equivalents and measure MaxCLL. As an extra test convert 422 to 444 and check then.
Intermediate compression should typically cause errors around 100nits (maybe 200 in extreme cases as PQ curve boosts it a lot compared to coded values). At least this is what I've seen on my tests, but it may not be full story.
Wavelet compression seems to be more 'HDR friendly' than DCT.

[quietvoid]

I made a first attempt at it in awsmfunc: https://github.com/OpusGang/awsmfunc

Example script:

from vapoursynth import core
import awsmfunc as awf
core.num_threads = 8

clip = core.ffms2.Source("video.mkv")
# Crop, or whatever

# Defaults to reject outliers, and not downscaling
awf.measure_hdr10_content_light_level(clip)

Run as: python script.vpy
It runs at ~20 fps for 2160p here, ~70 at 1080p.

Results:

Regular (max = 100% percentile):

2160p
MaxCLL: 1577.31 nits
MaxFALL: 91.02 nits

1080p (Spline36 to RGB)
MaxCLL: 1399.14 nits
MaxFALL: 91.02 nits

Outlier rejection (max = 99.99% percentile)
And then MaxCLL = 99.5% and MaxFALL = 99.75%

2160p
MaxCLL: 517.59 nits
MaxFALL: 90.67 nits

1080p (Spline36 to RGB)
MaxCLL: 517.37 nits
MaxFALL: 90.67 nits

At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.

[benwaggoner]

Originally Posted by quietvoid

At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.

There's been quite a lot of debate about how to properly measure, set, and apply MaxCLL. It wasn't very deeply considered before standardization. Some studios will use 99.99% percentile, which does violate the nominal spec, but yields better, brighter images on some TVs (particularly LG OLED, when MaxCLL would exceed the display's maximum brightness).

Static metadata is turning into a legacy thing, thank goodness, as dynamic metadata does everything it can and way way more.

[kolak]

Originally Posted by quietvoid

I made a first attempt at it in awsmfunc: https://github.com/OpusGang/awsmfunc

Example script:


Run as: python script.vpy
It runs at ~20 fps for 2160p here, ~70 at 1080p.

Results:

Regular (max = 100% percentile):


Outlier rejection (max = 99.99% percentile)
And then MaxCLL = 99.5% and MaxFALL = 99.75%



At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.

517 looks low then. It should be around 1K if this is a typical 1K grade.

[benwaggoner]

Originally Posted by kolak

517 looks low then. It should be around 1K if this is a typical 1K grade.

I see TONS of "1K" grades with MaxCLL below 500 even. HDR offers creatives a whole lot of range, and they vary a ton in how much of that they use.

[quietvoid]

It's not low in this case since these results I posted are from a 1000 frames clip trim.
And there are only a few frames with a single dot of specular highlight going over 1000 nits, so that's pretty small to be hardly be considered 99.99%.

The full analysis of the title was around 20 nits off the MaxCLL in the metadata.

This is the image from the posted analysis:
https://0x0.st/o1KO.png

The brightest pixel is probably somewhere in that "star" in the letter E.
At 100% percentile, it's over 1000 nits. At 99.99%, you get 510 nits.

[FranceBB]

Originally Posted by kolak

517 looks low then. It should be around 1K if this is a typical 1K grade.

Yep.

Originally Posted by quietvoid

This is the image from the posted analysis:
https://0x0.st/o1KO.png

The brightest pixel is probably somewhere in that "star" in the letter E.
At 100% percentile, it's over 1000 nits. At 99.99%, you get 510 nits.

That's interesting, but also differs from the officially provided value. The title you performed the analysis on is Dune which we've received a while ago as ProRes HQ 4:2:2 10bit planar and I have it right here (well, not "here" on my PC, that would be crazy, it's on an LTO tape, but you get the idea) and the metadata sent from the official provider via xml said MaxCLL 945 Nits, which is way closer to 1000 than the calculated 510. I noticed that when you calculated it, it skyrocketed to 1239 nits, which sounds about right given the compression artifacts and most importantly the chroma resampling, so if it has been mastered at 1000 nits, it effectively has 945 nits and in your calculations you've got 1239, I think that being 294 nits off on a distribution title encoded for consumers seems about right 'cause we're talking about 4:2:0 vs 4:2:2 and H.265 50 Mbit/s vs ProRes 700 Mbit/s.

On the other hand, the value you've got of 510 nits calculated by removing outliers looks way lower than it should be.

[kolak]

Originally Posted by benwaggoner

I see TONS of "1K" grades with MaxCLL below 500 even. HDR offers creatives a whole lot of range, and they vary a ton in how much of that they use.

Typical "Hollywood" dull and dark look.
Hard to imagine typical 90min movie not to have a scene which could not use of 1K nits (even 'dark' movie).

I would hardly call a title with 500nits max as HDR.
Almost non point to bother with this "so called" HDR grade then. Title like Dune has no scenes "deserving" 1K nits for highlights?

[quietvoid]

Originally Posted by FranceBB

On the other hand, the value you've got of 510 nits calculated by removing outliers looks way lower than it should be.

It's only 510 nits for this frame... because the highlight is very small and considered an outlier (using 99.99% percentile).

FWIW, the Dolby Vision metadata says that this shot is supposed to be 800 nits. This value is probably from analysis by the grading software.
So it probably doesn't do 99.99% either, which makes sense.

To be clear, I don't myself think 99.99% percentile is good as brightest pixel.
I've found better results with something like 99.9999%.

[kolak]

So is this 510 for whole movie or not?