Is there a simple way to conserve HDR10 through adobe/handbrake?

[BobbyBoberton]

Hey there,

I have been trying to learn how to conserve HDR10 metadata that a film/file already has on it but I haven't found a good answer.

The most certain answer I have found is to take the original UHD and transcode to DNxHR then use 3DLut plugin through davinci resolve 15/16 to create a tone map and then use davinci resolve 15/16 to combine the tone map with the original video stream and export again as DNxHR (which is HUGE, like 700GB huge, so we are talking 1.4 TB just for the middle workflow) and then finally encoding that DNxHR file into HEVC 10-bit HDR10.

There is also a person that claims that they get HDR10 conserved by using this FFmpeg script/code:

ffmpeg -hide_banner \
-i "../storage/My.Source.Movie.2019.2160p.BluRay.REMUX.HEVC.DTS-HD.MA.TrueHD.7.1.Atmos.mkv" \
-pix_fmt yuv420p10le \
-max_muxing_queue_size 9999 \
-map_chapters 0 \
-metadata title="My Awesome Movie (2019)" \
-map 0:0 -metadata:s:v:0 language=eng -metadata:s:v:0 title="My Awesome Movie (2019)" \
-metadata:s:t:0 filename="" -metadata:s:t:0 mimetype="image/jpeg" \
-c:v libx265 -preset fast \
-x265-params crf=18:no-sao=1:aq-mode=3:max-merge=4:keyint=60:bframes=3:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display="G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)" \
My.Movie.2019.2160p.BluRay.REMUX.HEVC.DTS-HD.MA.TrueHD.7.1.Atmos.mkv

but I have to compile a FFmpeg myself for windows so that it includes libx265 (I can definitely do this, I just don't want to stop the encoding jobs I have going to try this out right this moment) and all the extras. I am aware of the autobuild that was put together for this purpose so I am looking into that but supposedly handbrake is not able to conserve HDR10 data even with the extra settings of:

hdr:max-cll=4000,1000:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):deblock=-2,-3:strong-intra-smoothing=0:sao=0:hdr-opt

using HEVC 10-bit main 10 encoder, I am told this has something to do with the handbrake workflow being only 8-bit at some point therefore it is not able to properly conserve HDR10 metadata.

Can anyone give me any advice on whether the FFmpeg script above will conserve HDR10 metadata (I know the script is just video, I can easily mux together the audio after the video encodes) before I try it for myself?

If not can someone either point me to a thread that goes through how to extract the HDR10 metadata from a UHD file and then re-encode the base video and tone map? The person that posted the info I found on redfox.bz was evidently made into a video that was over 2 hours long to explain the workflow but the video is no longer available and the author stopped replying over 2 years ago now.

I can't even get the HDR10 metadata conserved on personal video shot with my Samsung Note10+, if I try to re-encode in anyway the HDR10 info is gone. It's still 10-bit BT.2020 SMPTE ST 2086 HDR10 'compatible' but the HDR10 metadata is gone.

I have been looking for an answer to this for a long time, please help point in the right direction if its not too much trouble.

Thanks for your time.

EDIT: I went ahead and compiled my own FFmpeg for windows with all the extra's and it 'seems' to conserve the HDR10 tags on UHD video but I would still like to hear from community members as to what you do personally to conserve HDR10 metadata in a UHD HDR10 video taken, like I said, for instance, from my Samsung Note10+.

Thanks

[benwaggoner]

I believe Zeranoe's GPL builds include x265. That might be simpler for you.

https://ffmpeg.zeranoe.com/builds/

I'm not quite parsing that script mentally. The metadata is in SEI messages in the elementary stream, so those could be copied forward. Alternatively it could be read from the file with MediaInfo or whatever and then those values specified in the x265 parameters. The static 2086 metadata (NOT HDR10+) shouldn't vary across a title.

If you are preserving HDR, a tone mapping step shouldn't be needed. If you're changing the color volume or adjusting video levels, you're changing the metadata and will need to calculate new values. What is your actual source?

[excellentswordfight]

I'm not sure if I understand the issue, just set the flags that matches the source in your re-encode? I dont understand why you wanna add tone map in the mix? I dont use handbreak though, but I have a hard time believing that newer versions cant handle 10bit input properly.

These are the relevant HDR10 metadata flags in x265 (max-cll and master-display values may differ from source to source), and you can just "extract" them by using mediainfo.

--colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --max-cll "1000,400" --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"

If you dont want to compile ffmpeg, just use an zeranoe build, they include x265 and are publicly available.

[benwaggoner]

I'm not sure if I understand the issue, just set the flags that matches the source in your re-encode? I dont understand why you wanna add tone map in the mix? I dont use handbreak though, but I have a hard time believing that newer versions cant handle 10bit input properly.

These are the relevant HDR10 metadata flags in x265 (max-cll and master-display values may differ from source to source), and you can just "extract" them by using mediainfo.

If you dont want to compile ffmpeg, just use an zeranoe build, they include x265 and are publicly available.
Master Display paramaters themselves can change when a different mastering monitor is used. There was pretty much just the one Sony model for the early years of HDR, but we're seeing some more options now.

That said, I'm not aware of any displays doing anything with the mastering display metadata. MaxCLL is the most used.

[BobbyBoberton]

I'm not sure if I understand the issue, just set the flags that matches the source in your re-encode? I dont understand why you wanna add tone map in the mix? I dont use handbreak though, but I have a hard time believing that newer versions cant handle 10bit input properly.

These are the relevant HDR10 metadata flags in x265 (max-cll and master-display values may differ from source to source), and you can just "extract" them by using mediainfo.



If you dont want to compile ffmpeg, just use an zeranoe build, they include x265 and are publicly available.

I did compile FFMpeg myself, I just didn't want to interrupt the 50+ files I had in queue in handbrake, and no, if I sent colorprim, transfer, colormatrix, master-display, max-cll in handbrake (using x265 main 10) it will spit out a file with those 'HDR tags' but the colorprim/colormatrix is set to bt.709 even if I specify smpte2084 in the extra opts.

I don't want to bring in the tone mapping stuff if it is not necessary, it is just the best info I have bee able to find or been told on other forums for preserving HDR10 metadata.

If I run a video that I shot from my phone through my nVidia shield on my TV it will go into 'HDR' mode but if I run it through handbrake with all the correct options set,
(options being: crf=23:no-sao=1:aq-mode=3:max-merge=4:keyint=60:bframes=3:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display="G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)":strong-intra-smoothing=0:hdr10=1:deblock=-2,-3)

and then play that reencoded file on my TV through the same means it does not pick up any 'hdr' info and just plays and an SDR video.

I have gotten CLI FFmpeg working, I will see if using that properly conserves HDR metadata, but, what I am hearing from you is that I should not go down the road of using UHD.mkv -> DNxHR -> 3dLut with DaVinci Resolve -> export DNxHR with lut applied -> reencode to HEVC. This seemed like a ton of work for something that is relatively simple, its not like its HDR10+ dynamic metadata, so I thought there should be a simple way to do it.

:thanks:

[BobbyBoberton]

Also can someone advise me why I cannot set the profile to Main10 for the CLI version of FFmpeg when I just used it to encode some 4k video without issue. I know that that the hdr10=1 and other tags would auto trigger this profiles use but when I try -profile:v main10 before -x265 params but after -c:v libx265 it doesn't give any errors but the encode still says using main profile, not main 10. Also setting -x265-params profile=main10 or profile="main10" I get a profile option not found error and it defaults back to main.

this is the command I used:

ffmpeg.exe -hide_banner -i "D:\Convert\Home.Movie.1080p.HDR.mkv" -max_muxing_queue_size 9999 -map_chapters 0 -metadata title="Awesome Show E01" -map 0:0 -metadata:s:v:0 language=eng -metadata:s:v:0 title="Awesome Show E01" -metadata:s:t:0 filename="" -metadata:s:t:0 mimetype="image/jpeg" -c:v libx265 -preset medium -profile:v main10 -x265-params max-cll="1000,400":crf=23:no-sao=1:aq-mode=3:max-merge=4:keyint=60:bframes=3:repeat-headers=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:strong-intra-smoothing=0:deblock=-2,-3:range="limited":me="star" "D:\Converted\Home.Movie.1080p.10-bit.no_audio.mkv"

not sure what I am doing wrong, I could convert it with handbrake to 10-bit for sure but I would rather get use to the CLI version. Does it have anything to do with the colorprim,transfer,colormatrix stuff I have?

Thanks


EDIt: didn't expect this to post so if mod sees it can you release the message i replied with before this one? Thanks, its in reply the first and second reply.

EDITEDIT: I should say the settings that auto triggered the main10 profile was:

ffmpeg.exe -hide_banner -i "D:\Convert\Home.Move.HDR.Samsung.Note10+.mkv" -pix_fmt yuv420p10le -max_muxing_queue_size 9999 -map_chapters 0 -metadata title="My Awesome Movie (2019)" -map 0:0 -metadata:s:v:0 language=eng -metadata:s:v:0 title="My Awesome Movie (2019)" -metadata:s:t:0 filename="" -metadata:s:t:0 mimetype="image/jpeg" -c:v libx265 -preset fast -x265-params max-cll="1000,400":crf=23:no-sao=1:aq-mode=3:max-merge=4:keyint=60:bframes=3:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display="G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)":strong-intra-smoothing=0:hdr10=1:deblock=-2,-3 Home.Movie.HDR.Samsung.Note10+.mkv

[Blue_MiSfit]

-pix_fmt yuv420p10le is the key ;)

[BobbyBoberton]

-pix_fmt yuv420p10le is the key ;)

Um ok, maybe I misunderstood the documentation but I thought -pix_fmt was to explicitly tell FFmpeg what the format of the incoming file, not the output format, but I guess that makes sense because if I drop the -pix_fmt yuv420p10le then it defaults to yuv420p so ok, thank you.

[benwaggoner]

Um ok, maybe I misunderstood the documentation but I thought -pix_fmt was to explicitly tell FFmpeg what the format of the incoming file, not the output format, but I guess that makes sense because if I drop the -pix_fmt yuv420p10le then it defaults to yuv420p so ok, thank you.
I think it specifies the internal color space. yuv420p10le needs to get set for any HDR work, for example.

[nevcairiel]

Um ok, maybe I misunderstood the documentation but I thought -pix_fmt was to explicitly tell FFmpeg what the format of the incoming file, not the output format, but I guess that makes sense because if I drop the -pix_fmt yuv420p10le then it defaults to yuv420p so ok, thank you.

It depends where you put the option. And what kind of file it is.

If you want to specify any properties for the input file, you need to put them before the -i option. Anything after the -i option specifies the output.

An MKV file with presumably h264 or h265 inside, you cannot override the "incoming" format anyway, the encoded bitstream clearly defines that, so that isn't often useful.

So yeah, you are specifying the output format, which is going to be passed to x265. Although if your input was already 10-bit, it should've just stuck with that. So your input must've been something else then?

[BobbyBoberton]

It depends where you put the option. And what kind of file it is.

If you want to specify any properties for the input file, you need to put them before the -i option. Anything after the -i option specifies the output.

An MKV file with presumably h264 or h265 inside, you cannot override the "incoming" format anyway, the encoded bitstream clearly defines that, so that isn't often useful.

So yeah, you are specifying the output format, which is going to be passed to x265. Although if your input was already 10-bit, it should've just stuck with that. So your input must've been something else then?

Yes that issue was going from 1080p 8-bit to 1080p 10-bit, after using -pix_fmt yuv420p10le it worked out fine so you're all right. Also, the bitstream defining the incoming format makes sense, I guess I wasn't thinking too critically about it. I was only messing around with the encoder for this portion because I found a post from someone saying that encoding in 10-bit vs 8-bit even if the source is 8-bit will compress better (assuming smaller file size is meant by this) without loss of quality with the exact same encoder settings, I haven't found this to be generally true, for the files I tried it on the 10-bit version was usually about 10-15MB larger but then I found if I turn on hevc-aq and use 10-bit pixel depth I get at least 25% smaller file sizes that do not look discernably better or worse than the larger file. Not sure why that would be but I know hevc-aq disables all other aq-modes so perhaps hevc-aq should be enabled by default if running libx265 encode? Just a thought.

EDIT: So the original was yuv420p and without explicitly setting pix_fmt to yuv420p10le it defaulted to the same colorspace as the input stream

[Asmodian]

I was only messing around with the encoder for this portion because I found a post from someone saying that encoding in 10-bit vs 8-bit even if the source is 8-bit will compress better (assuming smaller file size is meant by this) without loss of quality with the exact same encoder settings, I haven't found this to be generally true, for the files I tried it on the 10-bit version was usually about 10-15MB larger but then I found if I turn on hevc-aq and use 10-bit pixel depth I get at least 25% smaller file sizes that do not look discernably better or worse than the larger file.

This is not a good way to figure out this relation. You do not actually get the same quality with the same settings when you change the bitdepth. You are comparing files with different quality and different sizes, which makes it very hard to learn anything.

Ideally you would use a two pass encode while testing so the files were the same size and you can judge them on quality. To do this you also need to compress enough that you can judge differences in quality.

From forum user reports and my limited testing I believe that with x265 the difference in efficiency between 8 and 10 bit is not very significant, it is with x264 where you get a big improvement. However, 10 bit is still better and hardware decoding is much more common for 10 bit HEVC so it is also reasonably practical, unlike 10 bit AVC or 12 bit HEVC. I always encode to 10 bit HEVC, when I encode to HEVC, with 8 or 10 bit sources.

[BobbyBoberton]

This is not a good way to figure out this relation. You do not actually get the same quality with the same settings when you change the bitdepth. You are comparing files with different quality and different sizes, which makes it very hard to learn anything.

Ideally you would use a two pass encode while testing so the files were the same size and you can judge them on quality. To do this you also need to compress enough that you can judge differences in quality.

From forum user reports and my limited testing I believe that with x265 the difference in efficiency between 8 and 10 bit is not very significant, it is with x264 where you get a big improvement. However, 10 bit is still better and hardware 10 bit decoding is much more common for 10 bit HEVC so it is also reasonably piratical, unlike 10 bit AVC or 12 bit HEVC. I always encode to 10 bit HEVC, when I encode to HEVC, with 8 or 10 bit sources.

Ok, I can do that but I don't really understand why. I do not do video processing professionally so my knowledge is in the early-mid home enthusiast-level (I am starting to understand colorspaces and HDR metadata and stuff like that but the best way to test a video's quality to another isn't something I can easily answer besides if it looks good to you that's all that matters if it's for your personal use only obviously) so it is not obvious to me why encoding a file via CRF with only changing aq-mode=3 to hevc-aq=1 is not a valid method of comparison if you are using the same source file, shouldn't the encoder pull out as much quality (information) from the input either way? I know that making a CBR encode usually can help show you which parts of a video require the most bitrate to show the proper detail but if you are interested in getting the 'most for your money', money being time and actual electricity and you are not planning on showing this on a network with limited bandwidth (like broadcast TV) then I was told that CRF is the way to go.

Can you explain more? or point me to a resource that would help explain why?

:thanks:

[Asmodian]

Because as soon as you change any settings you end up with different average bitrates or file sizes.

With different size files how do you judge the results? If one file is larger it might look only a little bit better, so the setting is equivalent to lowering CRF a tiny bit (performance becomes the deciding factor). It might instead look a lot better and the setting improves quality even if a slightly higher CRF was used (a good setting, it could be worth lower performance). It could be worse too, increasing both the size and quality but making the encode less efficient, decreasing the CRF value slightly would have produced better quality at the same size. It is really hard to judge if the quality improved enough compared to the bitrate increase to be able to separate these. The same is true with a smaller file, is it simply slightly worse quality? The option might make the encode less efficient while also lowering the size, so is worse then simply raising CRF slightly.

The only situation you can judge with any accuracy would be if the larger file was worse quality. The only accurate way is to compare quality at the same average bitrate.

As an aside, be careful of judging still frames too. Good quality comparison is actually really hard. I can end up focusing on one aspect and tuning to improve that while missing some other aspect getting worse at the same time. I then come back in a few years and feel like an idiot, all that time figuring out the perfect settings and in total they were probably slightly worse than the preset would have been. Especially when I had less experiance but it is still really easy to do. I use more aggressive compression and two pass encoding when testing settings and then run high bitrate CRF encodes after I have them figured out.

[-QfG-]

I Only known 2 different HDR10 metadatas.

BT2020 and P3D65

If you will reencode a HEVC 10-Bit HDR Stream, use following custom strings for x265:

Check in MediaInfo this entry:

Mastering display color primaries : BT.2020

BT2020:
--hdr --output-depth 10 --hdr-opt --max-cll "1000,400" --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)"

P3D65:
--hdr --output-depth 10 --hdr-opt --max-cll "1000,400" --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"

You must change the blue and green entries, use the entries you see in MediaInfo of the Stream/file.

The above examples looks so in MediaInfo:

Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light Level : 400 cd/m2

Another thing is, you must use a 10-Bit pipeline! You see it in the encoder Window on the top, there must stand: i420p10!.
I use LSMash for my DNxHR Intermediates, for HEVC Input in Mkv Container i use DGIndesNV or LWLibAv.

I'm using for Encoding StaxRip as Gui and x265 as Encoder. It's faster as FFMPEG.

[LeXXuz]

Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light Level : 400 cd/m2


What do I do when these values are missing in the source video metadata?

F.e. in the source video I have here for testing purposes, I only have these values:

Video
Format : HEVC
Format/Info : High Efficiency Video Coding
Format-Profil : Main 10@L5.1@High
HDR_Format/String : SMPTE ST 2086, HDR10 compatible
Breite : 3 840 Pixel
Höhe : 2 160 Pixel
Bildseitenverhältnis : 16:9
Bildwiederholungsrate : 23,976 (24000/1001) FPS
ColorSpace : YUV
ChromaSubsampling/String : 4:2:0 (Type 2)
BitDepth/String : 10 bits
verwendete Encoder-Bibliothek : ATEME Titan File 3.7.8 (4.7.8.0)
colour_range : Limited
colour_primaries : BT.2020
transfer_characteristics : PQ
matrix_coefficients : BT.2020 non-constant
MasteringDisplay_ColorPrimaries : Display P3
MasteringDisplay_Luminance : min: 0.0000 cd/m2, max: 1000 cd/m2

I assume "Display P3" is the P3D65 you mentioned?

[Boulder]

I've always set --max-cll "0,0" if that one is missing. It seems to be quite common that it is.

[benwaggoner]

I've always set --max-cll "0,0" if that one is missing. It seems to be quite common that it is.
It is definitely better to use undefined values than incorrect values!

[LeXXuz]

It is definitely better to use undefined values than incorrect values!

So best would be to not set max-cll at all?

Also I noticed this line:
ChromaSubsampling/String : 4:2:0 (Type 2)

What does "type 2" mean? My transcoded sample just has the usual 4:2:0 string. :confused:

EDIT: Never mind, I found the explanation to that type 2 subsampling.

[Boulder]

It is definitely better to use undefined values than incorrect values!

I believe "0,0" is safer than NULL.

"The first value is the max content light level (or 0 if no maximum is indicated), the second value is the maximum picture average light level (or 0)."

[LeXXuz]

I believe "0,0" is safer than NULL.

"The first value is the max content light level (or 0 if no maximum is indicated), the second value is the maximum picture average light level (or 0)."

I found that in the docs, too. I would say you are right.

Btw how do you guys handle a video source with dynamic tone mapping?

HDR_Format/String : SMPTE ST 2094 App 4, Version 1, HDR10+ Profile A compatible


:confused:

[nevcairiel]

I believe "0,0" is safer than NULL.

"The first value is the max content light level (or 0 if no maximum is indicated), the second value is the maximum picture average light level (or 0)."

Not indicating values is generally better then indicating "wrong" values.

[Boulder]

Not indicating values is generally better then indicating "wrong" values.

I agree with that. Unless the docs or specifications state otherwise, it's safer.

I think that x265 treats "0,0" the same way as leaving the value unspecified. At least it won't write the information in the metadata if you use those zeroes.

[benwaggoner]

I found that in the docs, too. I would say you are right.

Btw how do you guys handle a video source with dynamic tone mapping?

HDR_Format/String : SMPTE ST 2094 App 4, Version 1, HDR10+ Profile A compatible


:confused:
You would copy each frame's HDR10+ metadata into the reencoded version of that frame. It's just an SEI message. I'm not sure if there are any tools that can do that simply. Probably mp4box, as it can do pretty much anything with long enough a command line and patient enough trial and error ;).

But it needs to be done correctly. A four frame offset could lead to some quite strange behavior around edits. Also, it only matters if you have or are planning to have a display device that supports HDR10+. The nubmer of which are growing. But if you skip it, you'll still have a normal HDR-10 file with static metadata.

[LeXXuz]

Damn, I was afraid this might be a lot of work. :D

So basically we need a tool that reads all metadatas from the source file and copies them over to the encoded file. Therefor the encoded file has to have the very same frame structure as the source, I guess?

I'll pass on these files for now unless there is some kind of automated tool/solution one day. :rolleyes:

[Boulder]

Isn't quietvoid's tool suited for HDR10+? Or do you mean adding the metadata without re-encoding?

https://github.com/quietvoid/hdr10plus_parser/releases

[LeXXuz]

Isn't quietvoid's tool suited for HDR10+?
That sounds awesome. :eek: I have to check that out. :D

Or do you mean adding the metadata without re-encoding?

No I meant re-encoding to a smaller filesize and adding back that dynamic metadata of the source video.

[Boulder]

Then that's the thing for you. Get the parsed metadata and feed it to x265, there's a separate parameter for it.

[-QfG-]

What do I do when these values are missing in the source video metadata?

Then delete this entries, they will be not written, if they not exist.
Do not set Fake entries how 0,0 or something.



I assume "Display P3" is the P3D65 you mentioned?

Display P3 = P3D65 correct!

Isn't quietvoid's tool suited for HDR10+? Or do you mean adding the metadata without re-encoding?

This is a HDR10+ Metadata parser, who reads the bitstream and save all HDR10+ metadata in a JSON textfile. While encoding you must write this Data into the bitstream.
No chance to implent HDR10+ metadata after Encoding in a HDR10 bitstream.

So basically we need a tool that reads all metadatas from the source file and copies them over to the encoded file. Therefor the encoded file has to have the very same frame structure as the source, I guess?

I have build a tool for me, that reads HDR10 and HDR10+ metadata and write the commands for x265 in a txt file. The Problem is, i must solve the P3 / BT2020 Alghorythm, i use DGindexNV for this,
so this tool works only with a NVideo Geforce GTX 1050 and above.

[BobbyBoberton]

I Only known 2 different HDR10 metadatas.

BT2020 and P3D65

If you will reencode a HEVC 10-Bit HDR Stream, use following custom strings for x265:

Check in MediaInfo this entry:



BT2020:


P3D65:


You must change the blue and green entries, use the entries you see in MediaInfo of the Stream/file.

The above examples looks so in MediaInfo:



Another thing is, you must use a 10-Bit pipeline! You see it in the encoder Window on the top, there must stand: i420p10!.
I use LSMash for my DNxHR Intermediates, for HEVC Input in Mkv Container i use DGIndesNV or LWLibAv.

I'm using for Encoding StaxRip as Gui and x265 as Encoder. It's faster as FFMPEG.

ok awesome, thank you for that information. I don't see any difference in the "green" text for luminance values between BT2020 and P3 but the G(x,y) parameters are different, so is that what you meant to pay attention to?

All my HDR so far has been in P3 (as far as I am aware) so I have been using the proper G(x,y) values but what would happen if I had a video with the BT2020 master display primaries and used G(13250,34500) instead of G(8500,39850), the max-cll is the same in both of these so I am not sure why you point them out other than to say 'makes sure that this is set'?

EDIT: err i guess the whole G(x,y)B(x,y)R(x,y)WP(x,y) numbers change, but the question at the end still stands, what if I use the GBRWP values for P3 on a BT2020 source? I figure nothing in a non-hdr monitor but what effect to HDR would this have? Would it be terribly off, or just annoying or completely unwatchable? Thanks for any answers.

Also I have been using max-cll=1000,400 but the source files can sometimes be above this like one I current have through mediainfo is

Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 5847 cd/m2
Maximum Frame-Average Light Level : 1355 cd/m2


So max-cll would be 5847,1355 if I went by the source material, and luminance would be different as well but I know that there are not many displays that go above 1000 nits so I don't really get these settings, wouldn't this blow out the image in an HDR display to be super bright all the time?

[-QfG-]

Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 5847 cd/m2
Maximum Frame-Average Light Level : 1355 cd/m2

The x265 commandline for this:

--hdr --output-depth 10 --hdr-opt --max-cll "5847,1355" --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)"

[BobbyBoberton]

Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 5847 cd/m2
Maximum Frame-Average Light Level : 1355 cd/m2

The x265 commandline for this:

--hdr --output-depth 10 --hdr-opt --max-cll "5847,1355" --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)"

Thanks but I did say I know that that would be the command, the 5847 cd/m^(2) = 5847 nits and the 'best' consumer HDR displays are 1000 cd/m^(2) or 1000 nits peak brightness so wouldn't having the Frame-Average Light Level = 1355 cd/m^(2) / nits mean that the image is super bright and washed out all the time? I mean that's 100% brightness on average from what I understand about HDR and nits output. I know how to change the command line but i am asking if that make sense to use those settings or does the 1000,400 level make more sense for not professional grade video equipment. (Its "pro-sumer" stuff I have but its not professional grade AV gear).

EDIT: Also, I forgot to ask about why you didn't address the what if I encode a source with P3 master display primaries when it was originally encoded with BT2020 primaries or vice versa? Would this be barely noticeable, very noticeable, dramatically noticeable or produce an indecipherable picture in HDR mode?

Thanks

[Asmodian]

No. You are telling the device what the mastering was, not what it should do when displaying it. You do not change the metadata for your display. The entire reason for metadata is because the file represents colors too bright for your display, you are telling it exactly how bright it represents so your device can figure out the best way to compress it into the range it can actually display.

P3 is very not the same as BT.2020, encoding a BT.2020 source and lying to the display that it is P3 will make it very washed out. The same is true in reverse, P3 content flagged as BT.2020 will be very over saturated. Very noticeable but a decipherable picture.

[BobbyBoberton]

No. You are telling the device what the mastering was, not what it should do when displaying it. You do not change the metadata for your display. The entire reason for metadata is because the file represents colors too bright for your display, you are telling it exactly how bright it represents so your device can figure out the best way to compress it into the range it can actually display.

P3 is very not the same as BT.2020, encoding a BT.2020 source and lying to the display that it is P3 will make it very washed out. The same is true in reverse, P3 content flagged as BT.2020 will be very over saturated. Very noticeable but a decipherable picture.

Oh wow yeah that does make perfect sense, just means I will have to do some re-encoding of material as I was keeping a constant max-cll=1000,400 on my HDR encodes even when they were not the source values. I don't current have an HDR monitor to try this stuff out on besides a Samsung Note10+ phone so when I play the video files I have 'wrongly "HDRd" on it they don't seem weird but I am not 100% that it is picking up the HDR data to begin with. Thats why I can't wait to get a HDR10/HDR10+/Dolby Vision compatible TV/Monitor for downstairs and I don't want all my hard work encoding to look terrible, it'll take a bit of time but I can redo the damage pretty quick, maybe a week or two and I should be ok.

EDIT: Also, due to some sloppy batch controlling I accidently encoded a set of 1080p SDR files as HDR ones. I know that this wont effect playback on SDR device but what effect will this have on an HDR monitor? Will the video look 'wrong' due to that extra HDR data? Also is that "HDR metadata" the "global tags" that are found in MKVtoolNiX? If so can I just strip the global tags from the files without having to re-encode the entire file? This would save me a ton of time if possible.

EDIT2: Also this only works for HDR10 static metadata right? As others were discussing above you would need a HDR10+ parser to 'map the dynamic HDR10+ metadata' and then use that file with the appropriate options to intake the HDR10+ metadata (I have seen these options on the x265 options page I just don't remember the specific command off the top of my head). Is that all correct?

Thank you!

[benwaggoner]

Thanks but I did say I know that that would be the command, the 5847 cd/m^(2) = 5847 nits and the 'best' consumer HDR displays are 1000 cd/m^(2) or 1000 nits peak brightness so wouldn't having the Frame-Average Light Level = 1355 cd/m^(2) / nits mean that the image is super bright and washed out all the time?
Higher-end LCD HDR TVs have done well above 1000 nits for a few years now, and some can hit >2000 today. Although that's for highlights. Not many can do >1000 nits as an average light level due to power/heat limitations. Also, that's verging on being a tanning bed more than a TV. Even 500 nits FALL can be BLINDING.

I mean that's 100% brightness on average from what I understand about HDR and nits output. I know how to change the command line but i am asking if that make sense to use those settings or does the 1000,400 level make more sense for not professional grade video equipment. (Its "pro-sumer" stuff I have but its not professional grade AV gear).
If the content really gets that bright, it's probably good to stick to the real values. Although I wouldn't expect FALL >1000 having much practical impact.

EDIT: Also, I forgot to ask about why you didn't address the what if I encode a source with P3 master display primaries when it was originally encoded with BT2020 primaries or vice versa? Would this be barely noticeable, very noticeable, dramatically noticeable or produce an indecipherable picture in HDR mode?

Most HDR content uses the P3 subset of 2020 primaries. The code values will produce very different results depending on what the primaries are set to. If the mastering display was P3, that doesn't impact anything. What matters is what primaries were used, which almost certainly were 2020.

[BobbyBoberton]

Higher-end LCD HDR TVs have done well above 1000 nits for a few years now, and some can hit >2000 today. Although that's for highlights. Not many can do >1000 nits as an average light level due to power/heat limitations. Also, that's verging on being a tanning bed more than a TV. Even 500 nits FALL can be BLINDING.


If the content really gets that bright, it's probably good to stick to the real values. Although I wouldn't expect FALL >1000 having much practical impact.


Most HDR content uses the P3 subset of 2020 primaries. The code values will produce very different results depending on what the primaries are set to. If the mastering display was P3, that doesn't impact anything. What matters is what primaries were used, which almost certainly were 2020.

Yeah some of the source material I am working with has a CLL=10,000 and FALL=2427 which sounds crazy to me especially considering the actual video is quite dark but maybe that's why? IDK.

Thanks for the answers.

[BobbyBoberton]

I asked above but it was kind of hidden,

I wasn't paying attention to a batch encode of videos that were 10-bit SDR (BT.709) source and I mistakenly left the HDR settings by mistake so now these "SDR" videos have HDR metadata (HDR10, not HDR10+), is there a way to remove this data without re-encoding the files completely like in MKVToolNiX after I encode a video with HDR it will have "global tags", is this the HDR metadata? Or is there some other way or do I just have to re-encode so that it displays properly on an HDR monitor?

:thanks:

Also, to confirm I asked the people over at the handbrake official forums and handbrake IS NOT a 100% 10-bit pipeline/workflow. ALL input is seen as 8-bit SDR even if its 10-bit HDR, it can encode that 8-bit YUV420p stream into a 10-bit YUV420p10le stream but there is no chance of the extra info from the original source making it through, so as others have said, if you are working with 10-bit source material then make sure your entire pipeline/workflow is 10-bit!!

[BobbyBoberton]

Try Cine Encoder (https://www.pling.com/p/1406740/) (there is a Linux-only version.)

I can deal with Linux only, that isn't that huge of a deal and I have been planning on doing a Win 10/Gentoo Linux dual boot on my new system but my real question is how would this help exactly?

According to the site it says that it was built to preserve HDR signals, the last question I asked was how to remove HDR from a source that was SDR and had a HDR10 static metadata stuck to it by mistake (not paying attention to the script).

If this was in response to the OP then yeah, it may be worth a look for sure.

Thanks!

[ReciprocalUniverse]

Master Display paramaters themselves can change when a different mastering monitor is used. There was pretty much just the one Sony model for the early years of HDR, but we're seeing some more options now.

That said, I'm not aware of any displays doing anything with the mastering display metadata. MaxCLL is the most used.

From the command line run dxdiag and you can poll the actual mastering monitor coefficients in use from the display card.

[BobbyBoberton]

I know this is kind of out of the blue but I am trying to get staxrip set up and I don't see where I am going wrong, I set up a template, I import a source, set a destination, set all the variables for x265 and mkv container, the only AVS filter checked is 'source' and that is set to FFVideoSource (DXVA2 accel, but tried without it as well), gets the same error:

x265 M-3.4+6-g73f96ff39-gcc11.0.0 Patman

"C:\Program Files\StaxRip-x64-2.1.3.0-stable\Apps\Encoders\ffmpeg\ffmpeg.exe" -threads 1 -hwaccel dxva2 -i "D:\Videos\TOS 2160p HDR.mkv" -f yuv4mpegpipe -strict -1 -loglevel fatal -hide_banner - | "C:\Program Files\StaxRip-x64-2.1.3.0-stable\Apps\Encoders\x265\x265.exe" --crf 19 --output-depth 10 --hevc-aq --aq-mode 3 --me star --max-merge 4 --fades --frame-threads 3 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --hdr10 --colorprim bt2020 --colormatrix bt2020nc --transfer smpte2084 --range limited --max-cll "10000,1992" --hdr10-opt --repeat-headers --hrd --aud --input-depth 10 --deblock -2:-3 --frames 64297 --y4m --output "D:\Videos\TOS 2160p HDR_temp\TOS.Atmos.5.1.mkv_out.hevc" -

x265 [error]: unable to open input file <->

technically gives me error code 0x1 but says that its meaning is pretty vague.

Any help? Its like its not putting output to SDOUT

If I just take that code and run it in CLI I get a similar error saying x265.exe cannot open the input stream so whats going on? it should grab the y4m stream from FFmpeg and then output a .hevc file that is then muxed together by the other utilities in staxrip (probably mkvmerge).

[Asmodian]

Try running only the ffmpeg part and see what it says. It sounds like ffmpeg cannot open the file, so yes, not outputting to SDOUT. Changing to -loglevel info instead of -loglevel fatal would also help gain insight into what is going wrong.

[BobbyBoberton]

Try running only the ffmpeg part and see what it says. It sounds like ffmpeg cannot open the file, so yes, not outputting to SDOUT. Changing to -loglevel info instead of -loglevel fatal would also help gain insight into what is going wrong.

it works if I specify the -pix_fmt as yuv420p10 on the FFmpeg command but I cant figure out how to generate that from staxrip so its still not really able to automate it...

actually tried putting -pix_fmt yuv420p10le (as that is what the source is but I still get the error:)

Guessed Channel Layout for Input Stream #0.1 : 5.1
[yuv4mpegpipe @ 0000020f7ba0a740] Warning: generating non standard YUV stream. Mjpegtools will not work.
y4m [info]: 3840x1920 fps 24000/1001 i420p10 sar 1:1 unknown frame count