HDR10+ General Discussion

[Selur]

average CLL and MAX-CLL are calculated with the black bars, if I am right. so without it, it could cause playing somehow different, as with black bars... it's just my quess...
not according to:
MaxFALL/MaxCLL metadata is calculated on active picture only. Black mattes are not processed during the calculation.
source: https://www.linkedin.com/pulse/hdr-10-metadata-smpte-st2086-maxfall-maxcll-carlos-carmona

shoudn't be then all HDR video encoded without cropping?
How I see it:
1. Maximum Frame--Average Light Level (MaxFALL) and Maximum Content Light Level (MaxCLL) are maxima and thus should not change if you remove black borders.
2. The dynamic metadata of HDR10+ can change dynamically, based on the minimum, maximum and average luminance and gamut requirements for each scene. Thus resizing, cropping and other frame alternations would need to recalculate those metadata.



Cu Selur

[jlpsvk]

not according to:

source: https://www.linkedin.com/pulse/hdr-10-metadata-smpte-st2086-maxfall-maxcll-carlos-carmona


How I see it:
1. Maximum Frame--Average Light Level (MaxFALL) and Maximum Content Light Level (MaxCLL) are maxima and thus should not change if you remove black borders.
2. The dynamic metadata of HDR10+ can change dynamically, based on the minimum, maximum and average luminance and gamut requirements for each scene. Thus resizing, cropping and other frame alternations would need to recalculate those metadata.



Cu Selur

Thanks for that point of view. :)

[benwaggoner]

not according to:

source: https://www.linkedin.com/pulse/hdr-10-metadata-smpte-st2086-maxfall-maxcll-carlos-carmona


How I see it:
1. Maximum Frame--Average Light Level (MaxFALL) and Maximum Content Light Level (MaxCLL) are maxima and thus should not change if you remove black borders.
Correct that they do not change. This doesn't matter for MaxCLL. But for MaxFALL, everything outside of the active image area must be discarded or else the FALL will be really dragged down but big black rectangles.

2. The dynamic metadata of HDR10+ can change dynamically, based on the minimum, maximum and average luminance and gamut requirements for each scene. Thus resizing, cropping and other frame alternations would need to recalculate those metadata.
Incorrect. The dynamic metadata is calculated on the source, and needs to be identical for any adaptive bitrate streams irrespective of frame size or compression. Otherwise the tone mapper would do weird things when bitrate changes.

[Selur]

Incorrect. The dynamic metadata is calculated on the source, and needs to be identical for any adaptive bitrate streams irrespective of frame size or compression. Otherwise the tone mapper would do weird things when bitrate changes.
Okay, so resizing doesn't matter, but also cropping? (don't really know what how the dynamic data is calculated)

Correct that they do not change. This doesn't matter for MaxCLL. But for MaxFALL, everything outside of the active image area must be discarded or else the FALL will be really dragged down but big black rectangles.
As I understood it MaxCLL and MaxFALL both should only be calculated by taking the active picture into account so, cropping should not change anything for both of them,....

[benwaggoner]

Okay, so resizing doesn't matter, but also cropping? (don't really know what how the dynamic data is calculated)
Well, resizing matters. It just is ignored per spec. It yields some weird things, like calculating the metadata on an 8K master would yield higher values than on a 4K master for an HDR UHD Blu-ray. It doesn't seem right that the metadata should be determined by the master, but instead by the highest frame size that will be delivered.

As I understood it MaxCLL and MaxFALL both should only be calculated by taking the active picture into account so, cropping should not change anything for both of them,....
Well, black bars won't impact MaxCLL. But yes, MaxFALL should only be on active picture area. Which can yield some strange results; a movie than is 2.4:1 except for credits would theoretically have a much lower MaxFALL than one where everything is 2.4:1, since "active image" would be 16:9 if any is 16:9.

HDR10+ is way more useful than just having static MaxFALL and MaxCLL values for a whole title.

[blublub]

I have about 10 HDR movies and I cropped all of them. I haven't noticed anything weird and I look at pictures all day long in my job and usually spot imperfection before my friends do, so I guess cropping is fine

[jlpsvk]

I have about 10 HDR movies and I cropped all of them. I haven't noticed anything weird and I look at pictures all day long in my job and usually spot imperfection before my friends do, so I guess cropping is fine

cropping HDR10 is OK, cropping HDR10+ not.

[Variant]

I’ve read through this thread and have a question about discs that contain both HDR10+ and Dolby Vision. Is there some means of removing HDR10+ metadata entirely from a full disc backup while leaving Dolby Vision information intact for discs that offer both HDR formats?

An Oppo defaults to HDR10+ over Dolby Vision when the display supports both HDR10+ and DV. I am curious to see if it’s possible to simply remove the HDR10+ metadata/flags, so it would play DV.

[benwaggoner]

I’ve read through this thread and have a question about discs that contain both HDR10+ and Dolby Vision. Is there some means of removing HDR10+ metadata entirely from a full disc backup while leaving Dolby Vision information intact for discs that offer both HDR formats?

An Oppo defaults to HDR10+ over Dolby Vision when the display supports both HDR10+ and DV. I am curious to see if it’s possible to simply remove the HDR10+ metadata/flags, so it would play DV.
HDR10+ is a SMPTE 2094 SEI message, while DV is in NAL units. So if you have a tool that can strip specific types of SEI messages, that should do it.

I've not heard of anyone actually ever doing this, though.

[jlpsvk]

HDR10+ is a SMPTE 2094 SEI message, while DV is in NAL units. So if you have a tool that can strip specific types of SEI messages, that should do it.

I've not heard of anyone actually ever doing this, though.

yeah... for now the only way is to make lossless re-encode without dhdr10 metadata and mux to mp4 with DoVi layer.

[Selur]

So if you have a tool that can strip specific types of SEI messages, that should do it.
Never tried it, but it should be possible with FFmpegs Bitstream filter (https://ffmpeg.org/ffmpeg-bitstream-filters.html) and 'remove_types'.

Cu Selur

[SeeMoreDigital]

So if you have a tool that can strip specific types of SEI messages, that should do it.

I've not heard of anyone actually ever doing this, though.Never tried it, but it should be possible with FFmpegs Bitstream filter (https://ffmpeg.org/ffmpeg-bitstream-filters.html) and 'remove_types'.Back in the days when MPEG-2 video ruled the world, shh created a tool called ReStream (v0.9.0). It's a shame nobody got around to creating a similar type of tool for h264 and now h265 elementary video streams...

[imhh11]

should we use both --hdr10-opt and --dhdr10-opt for HDR10+ encoding or should we just use --dhdr10-opt ?

[hellgauss]

not according to:

source: https://www.linkedin.com/pulse/hdr-10-metadata-smpte-st2086-maxfall-maxcll-carlos-carmona


How I see it:
1. Maximum Frame--Average Light Level (MaxFALL) and Maximum Content Light Level (MaxCLL) are maxima and thus should not change if you remove black borders.
2. The dynamic metadata of HDR10+ can change dynamically, based on the minimum, maximum and average luminance and gamut requirements for each scene. Thus resizing, cropping and other frame alternations would need to recalculate those metadata.

Cu Selur

This seems strange. How the "active area" informations for maxfall is detected on the source and put into the output? There is some info in the source that should be handled? There is some information that should be put into the output stream?

How the player detects the active area?

Furthermore, a question on hdr10+ : in a rip, I found a parameter "AverageRGB" in the .json file. Does it check the "active area" informations as maxfall does?

As for the information I have, I think that hdr streams should not be re-encoded. Available references and guides are not clear.

[benwaggoner]

should we use both --hdr10-opt and --dhdr10-opt for HDR10+ encoding or should we just use --dhdr10-opt ?
They do different things; use both.

--hdr10-opt adjusts adaptive quantization to provide better compression efficiency using PQ in SMPTE 2100 content. Works equally well for classic HDR-10 and with HDR10+.

--dhdr10-opt reduces SEI overhead by only putting the HDR10+ dynamic metadata in only the IDR and frames where the values have changed. It saves a few bits and can help performance in the client's tonemapper.

[Blue_MiSfit]

HDR10+ is way more useful than just having static MaxFALL and MaxCLL values for a whole title.

Really?

I'm surprised by this. I haven't actually had any experience with it, but the dynamic metadata only seems useful (to me) when dealing with content mastered to very high light levels that exceed the capabilities of a given display. For example, content mastered to a 2000 nit peak being displayed on a ~700 nit display (like an LG OLED for example). In this case, only the brightest specular highlights would likely exceed 700 nits. Based on that, I could see having precise tone mapping metadata defined being helpful to improve highlight detail, but that's about it.

Is your experience different?

Everything I've read said its a very small improvement on HDR10, and that going to full fat Dolby Vision (e.g. Profile 5) is a much bigger improvement. I do have a lot of experience with that and am generally more sold on Dolby Vision as a solution for OTT in general, given the dynamic shaping into 10 bit IPT during encoding and 16 bit RGB reconstruction in the TV, plus Dolby having more control over the TV processing (versus often terrible processing rife with unnecessary format conversions that would otherwise engage).

[benwaggoner]

Really?

I'm surprised by this. I haven't actually had any experience with it, but the dynamic metadata only seems useful (to me) when dealing with content mastered to very high light levels that exceed the capabilities of a given display. For example, content mastered to a 2000 nit peak being displayed on a ~700 nit display (like an LG OLED for example). In this case, only the brightest specular highlights would likely exceed 700 nits. Based on that, I could see having precise tone mapping metadata defined being helpful to improve highlight detail, but that's about it.
HDR10+ specifies the metadata, not how the tone mapper will use that information, so its behavior will be less predictible than Dolby Vision, where Dolby provides the tone mapper and certification as well.

There are two ways static metadata can be used - on-device player and over HDMI. Over HDMI is more limited as you don't get much lookahead - maybe you get the metadata one frame ahead. When it's in-player the bitstream itself stores the metdata in the clear, and it can be extracted from the whole GOP in advance, or even from future GOPs. That allows the tone mapper to know how much headroom it needs to leave for future variability. Take, for example, your 700 nit display playing content with a MaxCLL of 2000. Using just static metadata, it might have 700 nits map to 600 nits, and then save 600-700 as a rolloff region so you don't get a big flat white blob on highlights. But if the tonemapper knows that the whole shot doesn't have anything above 600, it can go all the way up to 700 for that shot.

Also, it's not just about luminance but also chrominance. Because of how RGB color volumes work, the higher the peak brightness that can be played, the more saturated colors can be farther from the luma midpoint. And plenty of cheaper HDR displays can't do P3 all the way up past 1000 nits. Those displays also need headroom for chroma, so maybe it'll only go up to 90% of saturation most of the time in case there's some stuff up against the master-display color volume. If the tonemapper knows what the most saturated colors in a given shot are, it can go right up to the maximum the display can show without leaving any headroom. This can be a big deal, since limited chroma gets into tradeoffs of reducing saturation (better) or shifting hue (worse).

Everything I've read said its a very small improvement on HDR10, and that going to full fat Dolby Vision (e.g. Profile 5) is a much bigger improvement. I do have a lot of experience with that and am generally more sold on Dolby Vision as a solution for OTT in general, given the dynamic shaping into 10 bit IPT during encoding and 16 bit RGB reconstruction in the TV, plus Dolby having more control over the TV processing (versus often terrible processing rife with unnecessary format conversions that would otherwise engage).
This is really going to depend on content, display, and tonemapper. Dolby Vision is going to be more reliably decent. But equally good HDR10+ and Dolby Vision tonemappers should produce pretty similar quality results.

One other advantage of HDR10+ is it has lower complexity than DoVi. DoVi Profile 5 requires constructing a new intermediate frame based on the non-backwards compatible base layer, the dynamic metadata, and display characteristics. While that can be done with HDR10+, it's also possible to just decode the HDR10 base layer and then use the tonemapper for everything else. Which is why some devices might only be able to do DoVi Profile 3 to 24p, Profile 5 to 30p, and HDR10+ to 60p.

This can also save power on mobile devices.

HDR10+ can save power/compute relative to HDR10 original flavor, because the decoded frame doesn't need to be analyzed to determine its luma and chroma ranges; that can be just read from the metadata.

[Blue_MiSfit]

Great info, thanks for sharing. I do see how HDR10+ vs HDR10 can be significantly better when dealing with tone mapping. You brought up some interesting points regarding HDR10+ vs Dolby Vision.

I do have one question though:

some devices might only be able to do
DoVi Profile 3 to 24p, Profile 5 to 30p, and HDR10+ to 60p.


Just a sanity check, on the above you meant level 3 / 5 etc, not profile, right? Profile 3 isn't a thing, but levels 3 and 5 both make sense in the context of DoVi Profile 5.

https://www.dolby.com/us/en/technologies/dolby-vision/dolby-vision-profiles-levels.pdf

Are you aware of any DoVi supporting devices that are Profile 5 compatible that are not at least Level 5 compliant (4k @ 30 fps)

Your note about power consumption is quite interesting. I wonder how a mobile device with good DoVi support (like a current iOS device) would fare with DoVi vs HDR10. If only Apple supported HDR10+ :)

I also got the sense that DoVi would do a better job than HDR10 / HDR10+ when below the tone mapping curve, due to the higher quality offered by their reconstructed frame, at least theoretically. In practice the wildly different shaped IPT frame required some special tuning on our side to avoid additional banding relative to an HDR10 reference!

[benwaggoner]

Just a sanity check, on the above you meant level 3 / 5 etc, not profile, right? Profile 3 isn't a thing, but levels 3 and 5 both make sense in the context of DoVi Profile 5.

https://www.dolby.com/us/en/technologies/dolby-vision/dolby-vision-profiles-levels.pdf
Profile 3 was the original HEVC 8-bit base layer + 8-bit enhancement layer profile that no one uses anymore.

Are you aware of any DoVi supporting devices that are Profile 5 compatible that are not at least Level 5 compliant (4k @ 30 fps)
No, they can all do 30p at least. There are some devices that can only do Profile 3 up to 24p, Profile 5 to 30p, and HDR10 to 60p.

Your note about power consumption is quite interesting. I wonder how a mobile device with good DoVi support (like a current iOS device) would fare with DoVi vs HDR10. If only Apple supported HDR10+ :)
Apple writes their own DoVi stack; the only vendor I know of who doesn't use Dolby's. So I'm not sure how their model works. Apple can do some deeper hardware integration that is generally feasible on Android without a lot of work from the OEM.

I also got the sense that DoVi would do a better job than HDR10 / HDR10+ when below the tone mapping curve, due to the higher quality offered by their reconstructed frame, at least theoretically. In practice the wildly different shaped IPT frame required some special tuning on our side to avoid additional banding relative to an HDR10 reference!
In theory, yes. Although I don't know how many systems really follow the best practices to render into a 12-bit intermediate space prior to tone mapping. Given how few devices have a truly >8-bit display controller and panel, some dithering is pretty much always in play that would keep the 12-bit from being really that much better. But there is so much weird handwaving in the bitstream-to-glass process in real devices.

The future really would be a 12-bit base layer with dynamic metadata going into a half-float integrated processing unit that directly renders to panel color volume.

[Blue_MiSfit]

Profile 3 was the original HEVC 8-bit base layer + 8-bit enhancement layer profile that no one uses anymore.


I see. Yikes. I love how it's not even in the Dolby Vision documentation anymore!


The future really would be a 12-bit base layer with dynamic metadata going into a half-float integrated processing unit that directly renders to panel color volume.


That would indeed be the real deal. As long as all the optional display processing features stay in that half-float space and don't do any conversions!

I wonder what it would take to have a truly high end rendering + processing pipeline on a TV a-la MadVR. I've heard that MadVR is going commercial and making hardware, which is kind of fascinating....

[SeeMoreDigital]

The future really would be a 12-bit base layer with dynamic metadata going into a half-float integrated processing unit that directly renders to panel color volume.....And 12-bit OLED panels ;)

[benwaggoner]

I see. Yikes. I love how it's not even in the Dolby Vision documentation anymore!
It was a clever solution back in 2014, before >8-bit decoders became common. Plus it was a relatively little overhead on top of a backwards-compatible SDR stream, so well suited to channel/capacity limited delivery mechanisms like cable/sat/broadcast and optical disc.

I wonder what it would take to have a truly high end rendering + processing pipeline on a TV a-la MadVR. I've heard that MadVR is going commercial and making hardware, which is kind of fascinating....
It would take a whole lot of unprecedented collaboration between the display panel vendor and the SoC/OS signal processing chain. Which is surprisingly hard, even when both are in the same company.

[Fitik]

Hello,
have a question.
If source is HDR10+ and i dont specify --dhdr10-info with x265, do I get correct HDR10 video?
In other words, difference between HDR10+ and HDR10 is only SEI metadata and result playback is the same whether they are not present or player can not use them?
Thanks a lot.

[benwaggoner]

Hello,
have a question.
If source is HDR10+ and i dont specify --dhdr10-info with x265, do I get correct HDR10 video?
In other words, difference between HDR10+ and HDR10 is only SEI metadata and result playback is the same whether they are not present or player can not use them?
Thanks a lot.
For a non-HDR10+ capable player, you'll see no difference with or without the metadata. With a capable player, you'll get an improvement with the metadata.

[Ts9001]

I hope this is correct thread for my question. I have issues with parsing hdr10+ metadata from some mkv files using ffmpeg+hdr10plus_parser.

First, when I run the verification I get this which looks promising:
←[34mParsing HEVC file for dynamic metadata... ←[0m
←[34mDynamic HDR10+ metadata detected.←[0m

But then when I start the parsing it stops at some point and I get this:
←[34mParsing HEVC file for dynamic metadata... ←[0m
frame=193656 fps=864 q=-1.0 Lsize=36697460kB time=02:14:37.02 bitrate=37219.8kbits/s speed= 36x
video:36697460kB audio:0kB subtitle:0kB other streams:0kB global headers:1kB muxing overhead: 0.000000%
←[34mReading parsed dynamic metadata... ←[0mthread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Parse("not enough data: expected 12 bits got 2 bits")', src/hdr10plus/parser.rs:233:71
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I have to say that with my limited understanding on these things I have no idea what is the issue here. Well, it says something about "not enough data" but what does that mean? Is there anything I can do to get the HDR10+ metadata out of these file?

This is the command line I'm using:

ffmpeg -i input.mkv -c:v copy -vbsf hevc_mp4toannexb -f hevc - | hdr10plus_parser -o metadata.json -

[foxyshadis]

I have to say that with my limited understanding on these things I have no idea what is the issue here. Well, it says something about "not enough data" but what does that mean? Is there anything I can do to get the HDR10+ metadata out of these file?

This is the command line I'm using:

ffmpeg -i input.mkv -c:v copy -vbsf hevc_mp4toannexb -f hevc - | hdr10plus_parser -o metadata.json -

Yep, that was a major bug, it was fixed a couple of weeks ago so you just need to get the new version: https://github.com/quietvoid/hdr10plus_parser/issues/26

[Ts9001]

Yep, that was a major bug, it was fixed a couple of weeks ago so you just need to get the new version: https://github.com/quietvoid/hdr10plus_parser/issues/26

Thanks, that fixed the issue.

Then I would have another issue in regards to HDR10+ metadata. When I try to encode one file using FastFlix it immediately stops and the following lines can be seen in the log. If I encode the same file without the HDR10+ metadata then it works. What could be wrong with the metadata?

INFO x265 [error]: Unable to open tone-map file.
INFO [libx265 @ 0000020444ccdec0] Cannot open libx265 encoder.
INFO Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
INFO Conversion failed!
INFO
INFO
WARNING Error during conversion

[quietvoid]

Maybe the libx265 version you're using wasn't compiled with the HDR10+ option?

[ghostshadow]

FastFlix supports using generated or extracted JSON HDR10+ Metadata with HEVC encodes via x265. However that is highly dependant on a FFmpeg version that has been compiled with x265 that has HDR10+ support. BtbN's Windows FFmpeg builds have this support as of 10/23/2020 and may require a manual upgrade.

[zweifingerjoe]

Is there any document or specification publicly available that describes how hdr10+ metadata is mapped to the json file? There is a link in x265's manual but it doesn't work anymore.

[quietvoid]

There are example files in the x265 repo: https://bitbucket.org/multicoreware/x265_git/downloads

Not all the fields are used in the x265 code, however.

[zweifingerjoe]

thanks

[telemO]

Hello quietvoid,
and thanks again for your amazing work in developping this metadata parsing tools.

I've seen that you've recently updated the HDR10+ parsing tool with the similar principle than for the dovi one : does it have a "crop" option as well to be activated? Or as it should only concern the "active area", according to the official documentation, it is no problem to transcode with cropped bars?

[asarian]

Anything that doesn't change the video elementary stream should retain HDR10+ metadata.

I understand I'm replying to an older post, but the issue is still relevant to me, as I plan to denoise some UHD material, and don't want to lose metadata while running x265. Are there specific options I need to use to retain HDR data? Or will x265 default to keeping the metadata itself?

Thanks.

[benwaggoner]

I understand I'm replying to an older post, but the issue is still relevant to me, as I plan to denoise some UHD material, and don't want to lose metadata while running x265. Are there specific options I need to use to retain HDR data? Or will x265 default to keeping the metadata itself?
x265 doesn't support passthrough itself, although some tools that incorporate it presumably do so.

You'll need manually set the metadata to the same as the source if using standard x265.exe.

[asarian]

x265 doesn't support passthrough itself, although some tools that incorporate it presumably do so.

You'll need manually set the metadata to the same as the source if using standard x265.exe.


Thx.

Speaking of which, seems DGindexNV already tags a line to its .dgi files, like

X265_CL --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "10000,724" --frames 181104 --chromaloc 2


Is that what I think it is, a courtesy to show ppl how to preserve the correct color info?! Not sure what the X265_CL stands for (Command Line?), but it really looks interesting.

[Boulder]

Thx.

Speaking of which, seems DGindexNV already tags a line to its .dgi files, like

X265_CL --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "10000,724" --frames 181104 --chromaloc 2


Is that what I think it is, a courtesy to show ppl how to preserve the correct color info?! Not sure what the X265_CL stands for (Command Line?), but it really looks interesting.

Yes, that's something you put directly in your x265 command line to preserve those properties from the source. Don kindly added this feature upon my request, it saves a nice amount of time when you don't have to find out and type the info manually.

[asarian]

Yes, that's something you put directly in your x265 command line to preserve those properties from the source. Don kindly added this feature upon my request, it saves a nice amount of time when you don't have to find out and type the info manually.


Well, thank you kindly for that request. :thanks: Saves me a ton of work trying to figure it out each time. And thanks be to Don, of course, for adding it.

[Ripmann]

cropping HDR10 is OK, cropping HDR10+ not.

Is this still valid in today? No new tools since early 2020 that can work around the HDR10+ crop problem?

[benwaggoner]

Is this still valid in today? No new tools since early 2020 that can work around the HDR10+ crop problem?
Not that I know of. HDR10+ is largely used in professional encoding from a mezzanine, so reencode-with-crop isn't a scenario that the professional tools consider.