Add HDR metadata without re-encoding?

[albt]

Is there any way to add the "Master display" info to a mkv or hevc file without the need to re-encode it?

I know there is a software "SEIEdit" but it costs money.

[sneaker_ger]

It's possible to add HDR metadata as MKV container elements using mkvtoolnix (https://github.com/youtubehdr/hdr_metadata#ingest-metadata-with-mkvmerge). Don't know about HEVC bitstream.

[albt]

It's possible to add HDR metadata as MKV container elements using mkvtoolnix (https://github.com/youtubehdr/hdr_metadata#ingest-metadata-with-mkvmerge). Don't know about HEVC bitstream.

will it play ok in tv if i add in mkv?

i don't have any problem in the pc, it plays very well with madvr without the metadata, as it's encoded in bt2020 hdr.

i tried hex editor for hevc, but i don't know how to use it well.

[sneaker_ger]

will it play ok in tv if i add in mkv?
Most likely not.

[TEB]

AFAIK, the HDR metadata needs to recide in the Elementary stream itself.. since its the decoder that passes this on to the rest of the display chain.

[Selur]

I agree for hevc hdr meta data needs to be inside the elementary stream.
-> one would need something similar to NVENC HEVC Stream Patcher (https://github.com/SK-Hardwired/nv_hevc_hdr_patcher) for general H.265 streams. For the nvenc created streams the HDR info is only added to the first headers, so for x265 and other H.265 encoders which have the hdr data repeatedly throughout the stream one would need another tool.

[sneaker_ger]

so for x265 and other H.265 encoders which have the hdr data repeatedly throughout the stream one would need another tool.
I thought the problem here was the absence of such metadata? By default x265 doesn't repeater headers.

[Selur]

Yes, the problem is that the meta data is missing, but iirc nevenc only adds those at the beginning and x265&co add them multiple times. (<- not totally sure, since I normally enable '--stitchable', this his might be just because of that,.. to long since I looked into that)
Afaik vshsp only patched the first header which doesn't work for x265, but it might be that nevenc does add empty headers which nvhsp 'simply' fills. :)

[nevcairiel]

The HDR SEI should always be repeated for practically every GOP, otherwise the stream is easily broken. Seek somewhere and never see it.
Its really the same concept as SPS/PPS/VPS in AnnexB streams, its also repeated every GOP so a decoder can find it even if starts with random access.

So if nvenc only writes it once, its practically broken.

[sneaker_ger]

Yes, seems you are correct. Or the muxer would need to put the SEI into the header of e.g. mkv. But mkvmerge doesn't do this (maybe it should?). So for x265 users planning on using HDR the --repeat-headers parameter is recommended.

[Selur]

small side note: if you use '--repeat-headers' and '--no-opt-qp-pps --no-opt-ref-list-length-pps' don't use MP4Box (see: https://bitbucket.org/multicoreware/x265/issues/309/mp4box-incompatibility and https://github.com/gpac/gpac/issues/705)

[nevcairiel]

Yes, seems you are correct. Or the muxer would need to put the SEI into the header of e.g. mkv. But mkvmerge doesn't do this (maybe it should?). So for x265 users planning on using HDR the --repeat-headers parameter is recommended.

SEI is not technically meant to go into the headers, only SPS/PPS/VPS are (ie. check the MP4 H264 config box, it doesn't hold SEI, and Matroska models after the MP4 format, although it appears HEVC does, but limits the types of SEI that are allowed to be in there). I do not know if --repeat-headers is required to repeat SEI, but from this logic it shouldn't be.

The reason SEI is in-band with the bitstream is that it can change on a per-GOP basis, but unlike SPS/PPS etc its not directly referenced by the frames with a numeric ID, so the active SEI is infered by the position in the bitstream (ie. the most recent one seen).
Going back to the MP4 HEVC spec, it specifically states only SEIs are allowed to go into the header which apply to the entire stream without exception - and HDR metadata for example can technically change (even if current HDR10 standards don't allow that, the bitstream does).

[sneaker_ger]

SEI is not technically meant to go into the headers, only SPS/PPS/VPS are (ie. check the MP4 H264 config box, it doesn't hold SEI, and Matroska models after the MP4 format, although it appears HEVC does, but limits the types of SEI that are allowed to be in there).
Maybe. But for the longest time "Matroska spec" was mainly whatever mkvmerge did. I think it even puts x265 custom SEI (with encoding settings) into CodecPrivate.


I do not know if --repeat-headers is required to repeat SEI, but from this logic it shouldn't be.
It is, I checked.

Going back to the MP4 HEVC spec, it specifically states only SEIs are allowed to go into the header which apply to the entire stream without exception - and HDR metadata for example can technically change (even if current HDR10 standards don't allow that, the bitstream does).
Doesn't MP4 use different codec IDs or something when headers of a stream can change? I vaguely remember such a thing. Matroska doesn't have that.

[nevcairiel]

I think it even puts x265 custom SEI (with encoding settings) into CodecPrivate.
That is one of those global user-data SEIs the MP4 spec specifically mentions can go into there, though.


It is, I checked.

Then that should be fixed instead. --repeat-headers is even only documented to repeat SPS/PPS/VPS, and HDR SEIs (and various other SEIs) should still be repeated at all times.

Even if it wouldn't apply to MKV because it does whatever it wants, it still applies to MP4.

Decoders might also not expect a HDR SEI in private data, so it seems saner to fix one thing instead of changing all the things everywhere. (fwiw FFmpeg avcodec would not handle this well right now)

[TEB]

Regarding HDR signalling in HEVC ES, heres a good article on this :

https://ngcodec.com/news/2015/11/16/fpgas-are-best-for-encoding-high-dynamic-range-hdr-content-with-hevch265

[sneaker_ger]

@nevcairiel
Yes, you seem to be correct. The spec says:
The mastering display colour volume SEI message persists in decoding order from the current access unit until the end of
the coded video sequence.

Reported to x265 team.
https://bitbucket.org/multicoreware/x265/issues/333/hdr-metadata-sei-not-repeated-for-every

[pradeeprama]

@nevcairiel
Yes, you seem to be correct. The spec says:


Reported to x265 team.
https://bitbucket.org/multicoreware/x265/issues/333/hdr-metadata-sei-not-repeated-for-every

x265 repeats VPS, SPS, PPS, and SEI messages (which include HDR metadata) at every IDR frame if --repeat-headers option is included in the command-line. Since the default encoder settings is for open-gop, this option is disabled by default.

[nevcairiel]

x265 repeats VPS, SPS, PPS, and SEI messages (which include HDR metadata) at every IDR frame if --repeat-headers option is included in the command-line. Since the default encoder settings is for open-gop, this option is disabled by default.

The point is that HDR metadata SEI should be repeated even if --repeat-headers is not set, unlike SPS/PPS/VPS, which are fine to have globally.
This probably applies to various other SEI messages as well.

As sneaker_ger has quoted from the spec, various SEI messages have varying lifetime, and the HDR metadata applies to the current CVS (HEVCs fancy name for GOPs), and not any further, hence it needs to be repeated at the start of every CVS.
Refer to D.3.27 in the HEVC specification for the exact lifetime description of the HDR SEI, as well as Table D.1 in section D.3.1 for an overview of SEI persistence scope.

[nevcairiel]

This would be even more important to get right with HDR10+ coming up, should take the opportunity to also get the baseline HDR10 implementation quite right.

[kolak]

Looks decent and also expensive:

https://hdr.avtop.com/hdr_solutions_avtop

[_kermit]

I thought I add something to this old thread, maybe a search engine picks it up...

I had the same issue, wrong metadata info (done by me) while encoding.
As far as I can tell, while it's there, it's not important for encoding itself, it just has to be properly added.
Hence this might help:

https://github.com/SK-Hardwired/nv_hevc_hdr_patcher
https://github.com/YouTubeHDR/hdr_metadata

I've used the latter to update it. It goes through the whole file, not just at the beginning

[Selur]

@_kermit: https://github.com/YouTubeHDR/hdr_metadata just adds HDR MetaData to the mkv container, it doesn't adjust the video stream at all.