How to analyze an HDR video for peak brightness level for the setting of metadata?

[kolak]

The metadata is derived from the uncompressed RGB source, not compressed output. So even if there were overshoots from compression, that shouldn't impact the metadata at all. Same deal when downscaling and chroma subsampling. Even though the scaling is a low-pass filter, the metadata is still supposed to be that of the full-rez uncompressed source.

You just forgotten to add: "in theory".
In practice it's not that often derived from correct stage of production. It's a fact and industry should react and change formula for MaxCLL (there is proposed version with outliers rejections). There are already tons of eg. BDs out there with wrong metadata.
My last investigation pointed that it's not really a compression (definitely not intermediate one), but chroma subsampling which is biggest enemy of MaxCLL calculation.
I also found out that quite many (what meant to be high quality A class tiles) eg. ProRes444 masters are actually gone through 422 at some stage and they are "fake 444". I'm rather disappointed with such a discovery.
These give you "random" MaxCLL values.

Make a simple test to 1K nits in Resolve, export to uncompressed, ProRes/DNxHR 444 and then to 422 equivalents and measure MaxCLL. As an extra test convert 422 to 444 and check then.
Intermediate compression should typically cause errors around 100nits (maybe 200 in extreme cases as PQ curve boosts it a lot compared to coded values). At least this is what I've seen on my tests, but it may not be full story.
Wavelet compression seems to be more 'HDR friendly' than DCT.

[quietvoid]

I made a first attempt at it in awsmfunc: https://github.com/OpusGang/awsmfunc

Example script:
from vapoursynth import core
import awsmfunc as awf
core.num_threads = 8

clip = core.ffms2.Source("video.mkv")
# Crop, or whatever

# Defaults to reject outliers, and not downscaling
awf.measure_hdr10_content_light_level(clip)


Run as: python script.vpy
It runs at ~20 fps for 2160p here, ~70 at 1080p.

Results:

Regular (max = 100% percentile):

2160p
MaxCLL: 1577.31 nits
MaxFALL: 91.02 nits

1080p (Spline36 to RGB)
MaxCLL: 1399.14 nits
MaxFALL: 91.02 nits


Outlier rejection (max = 99.99% percentile)
And then MaxCLL = 99.5% and MaxFALL = 99.75%

2160p
MaxCLL: 517.59 nits
MaxFALL: 90.67 nits

1080p (Spline36 to RGB)
MaxCLL: 517.37 nits
MaxFALL: 90.67 nits



At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.

[benwaggoner]

At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.
There's been quite a lot of debate about how to properly measure, set, and apply MaxCLL. It wasn't very deeply considered before standardization. Some studios will use 99.99% percentile, which does violate the nominal spec, but yields better, brighter images on some TVs (particularly LG OLED, when MaxCLL would exceed the display's maximum brightness).

Static metadata is turning into a legacy thing, thank goodness, as dynamic metadata does everything it can and way way more.

[kolak]

I made a first attempt at it in awsmfunc: https://github.com/OpusGang/awsmfunc

Example script:


Run as: python script.vpy
It runs at ~20 fps for 2160p here, ~70 at 1080p.

Results:

Regular (max = 100% percentile):


Outlier rejection (max = 99.99% percentile)
And then MaxCLL = 99.5% and MaxFALL = 99.75%



At the very least, using 99.99% percentile as "max pixel" in a frame makes a significant difference.
Especially when the highlights are very small.

It's also very likely that the values won't match pro tools, but they shouldn't be too far.

517 looks low then. It should be around 1K if this is a typical 1K grade.

[benwaggoner]

517 looks low then. It should be around 1K if this is a typical 1K grade.
I see TONS of "1K" grades with MaxCLL below 500 even. HDR offers creatives a whole lot of range, and they vary a ton in how much of that they use.

[quietvoid]

It's not low in this case since these results I posted are from a 1000 frames clip trim.
And there are only a few frames with a single dot of specular highlight going over 1000 nits, so that's pretty small to be hardly be considered 99.99%.

The full analysis of the title was around 20 nits off the MaxCLL in the metadata.

This is the image from the posted analysis:
https://0x0.st/o1KO.png

The brightest pixel is probably somewhere in that "star" in the letter E.
At 100% percentile, it's over 1000 nits. At 99.99%, you get 510 nits.

[FranceBB]

517 looks low then. It should be around 1K if this is a typical 1K grade.

Yep.


This is the image from the posted analysis:
https://0x0.st/o1KO.png

The brightest pixel is probably somewhere in that "star" in the letter E.
At 100% percentile, it's over 1000 nits. At 99.99%, you get 510 nits.

That's interesting, but also differs from the officially provided value. The title you performed the analysis on is Dune which we've received a while ago as ProRes HQ 4:2:2 10bit planar and I have it right here (well, not "here" on my PC, that would be crazy, it's on an LTO tape, but you get the idea) and the metadata sent from the official provider via xml said MaxCLL 945 Nits, which is way closer to 1000 than the calculated 510. I noticed that when you calculated it, it skyrocketed to 1239 nits, which sounds about right given the compression artifacts and most importantly the chroma resampling, so if it has been mastered at 1000 nits, it effectively has 945 nits and in your calculations you've got 1239, I think that being 294 nits off on a distribution title encoded for consumers seems about right 'cause we're talking about 4:2:0 vs 4:2:2 and H.265 50 Mbit/s vs ProRes 700 Mbit/s.

On the other hand, the value you've got of 510 nits calculated by removing outliers looks way lower than it should be.

[kolak]

I see TONS of "1K" grades with MaxCLL below 500 even. HDR offers creatives a whole lot of range, and they vary a ton in how much of that they use.

Typical "Hollywood" dull and dark look.
Hard to imagine typical 90min movie not to have a scene which could not use of 1K nits (even 'dark' movie).

I would hardly call a title with 500nits max as HDR.
Almost non point to bother with this "so called" HDR grade then. Title like Dune has no scenes "deserving" 1K nits for highlights?

[quietvoid]

On the other hand, the value you've got of 510 nits calculated by removing outliers looks way lower than it should be.

It's only 510 nits for this frame... because the highlight is very small and considered an outlier (using 99.99% percentile).

FWIW, the Dolby Vision metadata says that this shot is supposed to be 800 nits. This value is probably from analysis by the grading software.
So it probably doesn't do 99.99% either, which makes sense.

To be clear, I don't myself think 99.99% percentile is good as brightest pixel.
I've found better results with something like 99.9999%.

[kolak]

So is this 510 for whole movie or not?

[quietvoid]

So is this 510 for whole movie or not?

No, just this specific frame/shot.

[kolak]

Yep.



That's interesting, but also differs from the officially provided value. The title you performed the analysis on is Dune which we've received a while ago as ProRes HQ 4:2:2 10bit planar and I have it right here (well, not "here" on my PC, that would be crazy, it's on an LTO tape, but you get the idea) and the metadata sent from the official provider via xml said MaxCLL 945 Nits, which is way closer to 1000 than the calculated 510. I noticed that when you calculated it, it skyrocketed to 1239 nits, which sounds about right given the compression artifacts and most importantly the chroma resampling, so if it has been mastered at 1000 nits, it effectively has 945 nits and in your calculations you've got 1239, I think that being 294 nits off on a distribution title encoded for consumers seems about right 'cause we're talking about 4:2:0 vs 4:2:2 and H.265 50 Mbit/s vs ProRes 700 Mbit/s.

On the other hand, the value you've got of 510 nits calculated by removing outliers looks way lower than it should be.

This ProResHQ sounds still quite good. It's so easy for anything which touches 4:2:2 to go from 1K to even 2K nits.
Chroma subsampling is huge enemy of MaxCLL calculation (bigger than compression most likely).

I downloaded Netflix Sparks 1K TIFF master. And it was 1K when measured in Resolve. When exported as 12bit 444 lossless jpeg2000 it was still 1K. ProResXQ was 1085, where ProResHQ was 4208 (straight from TIFF export in Resolve and measured back in Resolve)!!! It also depends on content nature and also which tool you measure with. It should be all calculated in 32bit float as it quickly generates errors on rounding (PQ curve is strong so errors propagate heavily to final nits).

[kolak]

No, just this specific frame/shot.

Ok- makes more sense.
Best way to test all of it is to download eg. Netflix Sparks 1K nits TIFF sequence. This is 1K nits MaxCLL.
Then you can create different formats from it and analyse them and compare to reference value.
If you start with compressed title you are bit blind as you have no idea what it should be.

[benwaggoner]

Chroma subsampling is huge enemy of MaxCLL calculation (bigger than compression most likely).
Yep, it always requires a conversion to RGB AFAIK. Because Y~luma, not Y=luma. Darn it.

[FranceBB]

Hi Tom,
thanks for the function, I've been using it for several years and it works. Up until now I only really used it manually so I didn't mind to look at the various results and scroll down to the bottom to get the final one, however I'm now including it in a workflow to measure the values and write both MaxCLL and MaxFALL with a bitstream filter on already encoded files. This means that I'll be processing those in batch and given that I was only really interested in the last value, I took the chance to add a new parameter called LogIntermediateStats. It's a boolean and it can be specified by the user to signal whether all values need to be reported (i.e update the .txt with the values every 100 frame, which is the current behavior) or just the final calculated value.

In other words, this:


LWLibavVideoSource("Test_Source.mov")
ConvertToRGB64(matrix="Rec2020")
MaxCLLFind(maxFallAlgorithm=0, LogIntermediateStats=true)


is the same as this


LWLibavVideoSource("Test_Source.mov")
ConvertToRGB64(matrix="Rec2020")
MaxCLLFind(maxFallAlgorithm=0)


and maintains the current behavior of outputting the values every 100 frames, in fact the MaxCLLFind_Results1.txt shows the following values


Stats at frame 1:
MaxCLL: 0, raw value: 0 0 at X 0 Y 0 at frame 0
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 0 at frame 0
FALL Average: 0 across 1 frames.

Stats at frame 101:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 4.07527 across 101 frames.

Stats at frame 201:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 2.84126 across 201 frames.

Stats at frame 301:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.94568 across 301 frames.

Stats at frame 401:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.52747 across 401 frames.

Stats at frame 501:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.42005 across 501 frames.

Stats at frame 601:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.24852 across 601 frames.

Stats at frame 701:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.18646 across 701 frames.

Stats at frame 801:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.11332 across 801 frames.

Stats at frame 901:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 8.55332 at frame 41
FALL Average: 1.06956 across 901 frames.

Stats at frame 1001:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.569 at frame 991
FALL Average: 2.12734 across 1001 frames.

Stats at frame 1101:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 7.41441 across 1101 frames.

Stats at frame 1201:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 8.57353 across 1201 frames.

Stats at frame 1301:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 9.43403 across 1301 frames.

Stats at frame 1401:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 10.2929 across 1401 frames.

Stats at frame 1501:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 11.1018 across 1501 frames.

Stats at frame 1601:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 11.4985 across 1601 frames.

Stats at frame 1701:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 11.8852 across 1701 frames.

Stats at frame 1801:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 12.1784 across 1801 frames.

Stats at frame 1901:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 11.9798 across 1901 frames.

Stats at frame 2001:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 11.7196 across 2001 frames.

Stats at frame 2101:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 12.1415 across 2101 frames.

Stats at frame 2201:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 12.4846 across 2201 frames.

Stats at frame 2301:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 12.7015 across 2301 frames.

Stats at frame 2401:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 13.3252 across 2401 frames.

Stats at frame 2501:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 13.9658 across 2501 frames.

Stats at frame 2601:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 14.2071 across 2601 frames.

Stats at frame 2701:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 14.5209 across 2701 frames.

Stats at frame 2801:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 14.8495 across 2801 frames.

Stats at frame 2901:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 15.115 across 2901 frames.

Stats at frame 3001:
MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007
FALL Average: 15.0032 across 3001 frames.



while if we set the new boolean explicitly to false like this:


LWLibavVideoSource("Test_Source.mov")
ConvertToRGB64(matrix="Rec2020")
MaxCLLFind(maxFallAlgorithm=0, LogIntermediateStats=false)


then we only get the final output in the MaxCLLFind_Results0.txt, in fact:


MaxCLL: 400.011, raw value: 42767 0.652583 at X 0 Y 1101 at frame 7
MinCLL: 0, raw value: 0 0 at X 0 Y 2159 at frame 0
MaxFALL: 222.57 at frame 1007


FALL Average is disabled in this case.

Let me know if you're ok with this new parameter.
I've opened a pull request here: https://github.com/TomArrow/MaxCLLFindAVS/pull/15 just in case, but feel free to change/modify anything. :)

In the meantime, in case anyone wants to give it a go, I've made some temporary builds here: https://github.com/FranceBB/MaxCLLFindAVS/releases/tag/0.4

and, before anyone says anything, yes, it also works on Windows XP! :D