how to use ffmpeg to encode to mp4-avc-HDR-4k

[Grojm]

The ffmpeg team admitted that not transferring the metadata is bug, they just refuse to work on this. A bug report is open for a year without any attempts to solve this:
https://trac.ffmpeg.org/ticket/7037
Sad.

[nevcairiel]

No one "refuses" to work on it, but most in the project donates their time for free, which means everyone works on things they are personally interested in, or have a need for in their use of ffmpeg.
Insulting the developers and demanding they work on your pet project is not going to make you any friends, or get your issue adressed any sooner.

If you want it fixed so badly, I suggest you put a bounty on it, ie. pay for someones time, instead of demanding how someone else spends their free time - after insulting them, apparently.

Or, you know, just fix it and send a patch.

[Grojm]

This is not a pet project - this is absolutely necessary!

"ffmpeg HDR metadata" is the among the top 4 (!) suggestions in Google if you type "ffmpeg HDR". Because everyone who wants to encode HDR will stumble across this. HDR is already in the mass market. The demand for such a bugfix is huge.

I didn't insult the developers, I just told the truth. If you think the truth is insulting, this is your problem. I only suggested that we should stop glorifying developers for just having fun. I mean you wouldn't give a skydiver credit because he had fun, and you wouldn't say he "donated" his time to skydiving. But it's still ok that people have fun, don't get me wrong. I also like to have fun. You just can't develop software with this attitude. What's necessary is that we empathize the importance of bugfixing and start to develop a culture in which fixing bugs gets you lot's of honor and fame, and people who are just dropping their cool innovative but buggy piece of code should not be given the same amount of honor.
There is even something the leadership can do about this. Look at Linus Torvalds and how he is telling people if they produce buggy code quite clearly. Every project needs someone like Linus Torvalds.
One could even build some gamification elements in this. Like every new feature submission has to be followed by one bugfix submission, otherwise you can not submit new features any more. Bugs like the present one would be fixed within weeks. The ffmpeg devs are really fast. Just look how fast cehoyos was to downplay the importance of this very serious issue. It was like one minute reaction time - on a Sunday! They have a lot of spare time but the incentives must be misaligned if this time and energy is not converted into good code. This is what I wanted to tell right from the beginning.

[benwaggoner]

The broader problem is that no one is using H.264 for HDR 4K. HEVC is much more efficient for that kind of encoding, and there are plenty of HDR capable devices out there that can decode 10-bit HEVC but only 8-bit H.264. The only reason I could see to encode H.264 for HDR is if targeting a specific bit of oddball hardware that requires it. But a HDR H.264 stream will not work in many places where HDR HEVC would. And even when it worked well, it would still requires >2x the bitrate for 2160p 10-bit HDR. There is no --hdr-opt for x264 like there is in x265, for example.

[Grojm]

H.265 ist also affected. It's independent of the codec. ffmpeg destroys any HDR related metadata. I'm interested in H.265 of course, but I just saw that H.264 users had the exact same problem.
HDR always comes with metadata telling the color profile etc. without it you can't use it effectively. To put it in other words: ffmpeg is broken for HDR. In general. That's the issue.

[benwaggoner]

Originally Posted by Grojm

H.265 ist also affected. It's independent of the codec. ffmpeg destroys any HDR related metadata. I'm interested in H.265 of course, but I just saw that H.264 users had the exact same problem.
HDR always comes with metadata telling the color profile etc. without it you can't use it effectively. To put it in other words: ffmpeg is broken for HDR. In general. That's the issue.

Ah. Well, THAT certainly is a bug! I presume one could re-add the metadata using x265 parameters, but a pass through would be nice.

[Blue_MiSfit]

also worth noting, maxcll and maxfall describe the light peak and average light levels of the mastered content.

The ST2084 metadata (i.e. master-display-parameters in x265 speak) describe the characteristics of the mastering display. For example, a 4000 nit Dolby Pulsar is sometimes used for grading content. So you would set the mastering display metadata to describe this. However, the content is most likely not graded up to 4000 nits. Content is quite frequently graded down to 1000 nits or less. THAT is what MaxCLL and MaxFALL describe.

[hydra3333]

Just tested and ffmpeg situation seems unchanged.
It seems even with latest nvidia 2060 Super gpu and ffmpeg build with the latest nvenc gpu encoder, nvenc based output omits the HDR metadata ... with apparently no possibility of fixing it.
Oh well.

[benwaggoner]

Originally Posted by hydra3333

Just tested and ffmpeg situation seems unchanged.
It seems even with latest nvidia 2060 Super gpu and ffmpeg build with the latest nvenc gpu encoder, nvenc based output omits the HDR metadata ... with apparently no possibility of fixing it.
Oh well.

Piping from ffmpeg into x265 works.

[hydra3333]

Thank you !
However, one then loses the "massive?" benefit of gpu-speed encoding
I did a terrible, loose, undocumented, once-off 4k 10 bit test and thought it was near realtime in encoding fps, depending. Haven't tried x265, however suspect 4k won't be anywhere near gpu speed.

[benwaggoner]

Originally Posted by hydra3333

Thank you !
However, one then loses the "massive?" benefit of gpu-speed encoding
I did a terrible, loose, undocumented, once-off 4k 10 bit test and thought it was near realtime in encoding fps, depending. Haven't tried x265, however suspect 4k won't be anywhere near gpu speed.

x265 can itself use the intel svt libraries for rapid acceleration. But I doubt you'll get realtime 2160p in software on anything less than a monster workstation.

GPU is fast, but it has substantial quality/efficiency limitations compared to even a performance tuned software encode.

[Loomes]

Originally Posted by benwaggoner

Piping from ffmpeg into x265 works.

That's what I just tried using

ffmpeg -i input.mkv -c:v libx265 -x265-params crf=20:no-sao=yes:hdr=yes -pix_fmt yuv420p10le -bsf hevc_mp4toannexb -f rawvideo output.h265

but none of the HDR metadata got taken to the transcoded videofile. Maybe I am missing something? Is there a way to extract metadata from an HDR10 file (non plus) to a json file (like hdr10plus_parser does) and import it back in while piping with ffmpeg? The HDR Info in the source file are:

Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000 cd/m2
Maximum Frame-Average Light Level        : 288 cd/m2

[benwaggoner]

Originally Posted by Loomes

That's what I just tried using

ffmpeg -i input.mkv -c:v libx265 -x265-params crf=20:no-sao=yes:hdr=yes -pix_fmt yuv420p10le -bsf hevc_mp4toannexb -f rawvideo output.h265

but none of the HDR metadata got taken to the transcoded videofile. Maybe I am missing something? Is there a way to extract metadata from an HDR10 file (non plus) to a json file (like hdr10plus_parser does) and import it back in while piping with ffmpeg? The HDR Info in the source file are:

Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000 cd/m2
Maximum Frame-Average Light Level        : 288 cd/m2

I mean like:

ffmpeg.exe -i foo.mov -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m --profile main10 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --hdr --hdr-opt --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --max-cll "1005,894" -o foo.hevc

[Loomes]

I see, yes, I could also pass the HDR metadata like -x265-params colorprim=9:transfer=16:colormatrix=9 but thats not exactly what I meant. I am looking for a way to extract and then pass them to the transcoded file automatically, like "hdr10plus_parser.exe" does.

[benwaggoner]

Originally Posted by Loomes

I see, yes, I could also pass the HDR metadata like -x265-params colorprim=9:transfer=16:colormatrix=9 but thats not exactly what I meant. I am looking for a way to extract and then pass them to the transcoded file automatically, like "hdr10plus_parser.exe" does.

MediaInfo can tell you the values at least, although you might have to reformat them syntactically.

[Blue_MiSfit]

Just for fun it's worth mentioning that Dolby Vision has an 8 bit AVC profile with an enhancement layer

It's designed to be implemented on devices without a working HEVC decoder (or one that's present but just not licensed / activated).

[benwaggoner]

Originally Posted by Blue_MiSfit

Just for fun it's worth mentioning that Dolby Vision has an 8 bit AVC profile with an enhancement layer

It's designed to be implemented on devices without a working HEVC decoder (or one that's present but just not licensed / activated).

Yeah, there are a bunch of legacy profiles I've not seen in real-world deployments. I played around with that one myself ~5 years ago.

Everything for current content seems to have converged on Profile 5 (non-backwards compatible 10-bit HEVC) for VOD and Profile 8.1 (HDR-10 base layer) for Live that I know of.

[Krautmaster]

having the same issue for 2 years now, i can't reencode with ffmpeg as its not able to keep the HDR metadata in place.

Any how, my ffmpeg probe does not print the meta data at all, so I cant even use that massive batch posted before to do a mapping within the batch process itself.

in ffprobe i get

[STREAM]
index=0
codec_name=hevc
codec_long_name=H.265 / HEVC (High Efficiency Video Coding)
profile=Main 10
codec_type=video
codec_time_base=1001/24000
codec_tag_string=[0][0][0][0]
codec_tag=0x0000
width=3840
height=2160
coded_width=3840
coded_height=2160
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p10le
level=153
color_range=tv
color_space=bt2020nc
color_transfer=smpte2084
color_primaries=bt2020
chroma_location=unspecified
field_order=unknown
timecode=N/A
refs=1
id=N/A
r_frame_rate=24000/1001
avg_frame_rate=24000/1001
time_base=1/1000
start_pts=17
start_time=0.017000 s
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=N/A
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A

while mediainfo tells me

Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 944 cd/m2
Maximum Frame-Average Light Level : 143 cd/m2

Edit:

Okay the -show_entries side_data switch in ffprobe does extract the data itself but it takes ages. And anyway, then a custom massive manual mapping is required to pass that data to x265. Sucks. A fix should be implemented

[sneaker_ger]

You can limit the amount of frames ffprobe reads via -read_intervals.

ffprobe -read_intervals "%+#2" -show_frames -show_entries side_data "video.mkv"

[Krautmaster]

thanks !

Edit: Working fine now for me using ffprobe and that awesome mapping of a page before. That preserves my hdr metadata.