Video Edit: AviSynth+ - HEVC HDR

[tolias63]

Hello!

This is my first time dealing with 4K HEVC HDR and I would like to ask you a few things.

To begin with, I have been involved in video editing for years (filtering, encoding, etc.), but with 1080p x264 SDR videos.
Usually, I edit the video in VirtualDub2, creating a lossless with FFVH codec and then encoding the lossless with MeGUI.

Can I edit an HDR video the same way? For example, apply filtering, create lossless, etc...
Do I need to convert it to SDR, edit it as I want and convert it back to HDR?

I think it's wrong to edit it directly on HDR, right?

The tools I use are Avisynth+, VirtualDub2 and MeGUI.

Thanks for your help!

[videoh]

Can I edit an HDR video the same way? For example, apply filtering, create lossless, etc...
Yes.

Do I need to convert it to SDR, edit it as I want and convert it back to HDR?
No.

I think it's wrong to edit it directly on HDR, right?
No. Why would it be wrong?

[tolias63]

Thank you for your answer!

Would it be easy for you to give me some information about the process? If I open the .m2ts file with VirtualDub2, it has a gray layer on top because of HDR. For example, if I want to apply the Levels filter, I don't see the true colors. So, how do I edit it correctly?

[MeteorRain]

If you don't have a HDR monitor / TV, how can you see the true colors? Your monitor can't even display them.

[tolias63]

I have a HDR TV. If I plug it into my pc, is the process the same? Will I open VirtualDub2 and apply whatever filter I want?

[MeteorRain]

If you can somehow trick the TV to display HDR then you might be able to see the right color?

Also I'm not sure what kind of process you are doing here. The source is carefully mastered by the publisher, so usually you shouldn't be changing the colors. If you just want to denoise or do some video effects, colors shouldn't be a big deal.

Anyway, you can mostly process them in the same way, but not likely watching them correctly until you find a way to display them.

[tolias63]

Thank you for your time and your help!

I tried to connect the TV to the computer, but it does not recognize it as HDR.

Anyway, for now I will be using videoh's DGHDRtoSDR to experiment a little with 10bit 4K videos.

Let me ask you something else, what source plugin can I use in AviSynth+ to open a 10bit video? If I use FFMS2, will it open it as 10bit or wiill it convert it to 8bit?
I'm reading that I need a "hack" version of FFMS2 to load 10bit sources. Is this true or not?

[videoh]

Let me ask you something else, what source plugin can I use in AviSynth+ to open a 10bit video? DGSource() [part of DGDecNV] is one option.

[MeteorRain]

LSMASH family returns high bit depth video. DGdecodeNV also by default returns full depth if source is high bit depth.

I haven't used FFMS2 for a few years so can't speak for it.

[tolias63]

I made a lossles 4K video (~280 GB), but neither DGDecNV or LSMASH was able to open it. Only FFMS2 open it.

So I'm looking to see if FFMS2 is processing it properly, or if I need to use another one...

Edit

I created the lossless with HuffYUV codec. It's okay or I did wrong?

[videoh]

I created the lossless with HuffYUV codec. It's okay or I did wrong? DGSource supports only AVC, HEVC, MPG2, and VC1 video.

[FranceBB]

Let me ask you something else, what source plugin can I use in AviSynth+ to open a 10bit video? If I use FFMS2, will it open it as 10bit or wiill it convert it to 8bit?

Yes, ffms2 is also gonna be fine with high bit depth stuff.

LSMASH family returns high bit depth video. DGdecodeNV also by default returns full depth if source is high bit depth.

I haven't used FFMS2 for a few years so can't speak for it.

modern/recent version of ffms2 output the real bit depth (i.e 10bit planar if it's 10bit planar, or 12bit planar if it's 12bit planar and so on).


I'm reading that I need a "hack" version of FFMS2 to load 10bit sources. Is this true or not?

No. The old "hack" was used to display high bit depth videos in the old Avisynth, back when it didn't have native planar high bit depth support. Back then there were two version: 16bit stacked with MSB and LSB stacked one on top of the other OR 16bit interleaved with MSB and LSB interleaved one next to the other (double width). ffms2 used the first (16bit stacked) for 10bit or higher, while LSMASH used the latter (16bit interleaved) for 10bit or higher. Nowadays, both ffms2 and lsmash are able to output 10/12/14/16bit planar if required.

A few side notes:

Side note 1: ffms2 and LSMASH will output an 8bit dithered output if they detect Avisynth 2.6 which doesn't support planar high bit depth.

Side note 2: if you're using Windows XP and you want a 10bit planar output from your indexer, you must use ffms2 'cause it's been kept updated for a bit more time, while lsmash support was dropped almost immediately and the last xp compatible version of lsmash will always return 16bit interleaved.


I created the lossless with HuffYUV codec. It's okay or I did wrong?

It's fine. Did you make it with the huffyuv implementation of ffmpeg? The reason why I'm asking this is that it's important that you preserve 10bit bit depth while encoding in lossless huffyuv. As a side note, though, HuffYuv will NOT keep track of any metadata whatsoever, therefore you're gonna lose all the HDR Metadata afaik. Just keep track of them and store them somewhere 'cause they're gonna be useful as you will have to put them back when you encode in either H.264 HDR or H.265 HDR.

[MeteorRain]

Thanks TIL.

[tolias63]

Thank you, FranceBB, for your information!

It's fine. Did you make it with the huffyuv implementation of ffmpeg? The reason why I'm asking this is that it's important that you preserve 10bit bit depth while encoding in lossless huffyuv. As a side note, though, HuffYuv will NOT keep track of any metadata whatsoever, therefore you're gonna lose all the HDR Metadata afaik. Just keep track of them and store them somewhere 'cause they're gonna be useful as you will have to put them back when you encode in either H.264 HDR or H.265 HDR.

I tried to connect my TV to the computer, but it does not recognize it as HDR. So, right now, I am experiment with SDR.

I used MeGUI's pre-render to create the lossless. If I'm not mistaken, it's implementation of ffmpeg, right?

I want to work with HDR as well, so I will buy a HDR monitor. Can you tell me how I'm going to extract the HDR metadata and store them? Do I need a special software?

Edit

I think I found it.

DGDecNV's dgi file contains the following lines:
MASTERING 8500 39850 6550 2300 35400 14600 15635 16450 10000000 0
LIGHT 711 617
That's the HDR metadata, right?

So, if I want to put them back when encode to x265, I will add the following commands, correct?
--master-display G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,0) --max-cll 711,617

[FranceBB]

I don't really use MeGUI so I have no idea about what it does under the hood. Ideally, you should do everything from the command line and use bat files and thus avoid MeGUI. The idea is that you don't really want someone else to make a decision for you without knowing what it does. I remember years ago when I first encountered some x264 encoded files spreading across the internet with "weird" and rather unusual parameters only to find out later on that they were made by MeGUI presets. A few things I'm particularly concerned about MeGUI handling HDR HEVC sources is:

1) Chroma placing: is it correctly interpreting 4:2:0 type2 and converts it properly or does it just think it's the old fashioned 4:2:0 and screws everything up?

2) HDR Metadata. Does it even take them into account?

As to the Metadata, if your source it's an HLG from a broadcaster or an HDR10 PQ from a Blu-ray, you'll find them in mediainfo. Generally, with HLG stream you won't find anything other than the color curve, the color matrix and the transfer characteristics, so you're gonna be fine by just re-encode with the same three flags in x265 or x264, you'll find them in the documentation or in several other topics here on Doom9. If it's an HDR10 PQ, in addition to the former three, you'll also find master display values and Max,min CLL. Those are very important and have to match exactly the values reported because not only it tells your monitor how many nits (cd/m²) it is, but it also tells it how to correctly display the file. If your file is in HDR10+ or in DolbyVision, then things get a bit more complicated because those Metadata "change" over time from scene to scene, so you have to extract the Metadata file and then use it to re-encode, you can't just put a flag and some values like before. It can be done, but it's not straightforward and again the Doom9 community made a huge effort and an outstanding job in covering the new specs and make it possible to handle these standard with open source softwares (you'll find many things about HDR10+ in the x265 section).

As to your TV, is it HDR compatible? I gotta say that I generally work with a not so good HDR monitor for windows stuff and I have a dedicated 1000 nits monitor which displays only the output from the dedicated Black magic design SDI chip output. (I wish I had the money to buy the Sony $50k monitor I have at work... T_T). Anyway, if it is HDR Compatible and you're using Windows 10, then make sure that you turn on the HDR mode on Windows. If it isn't, then you're in no luck.

HDR:
If it is HDR and you wanna mess with levels and colours, remember that you can only edit contents with as many nits as your monitor, otherwise you won't get a real representation of what you're watching. Besides, if you do mess up with the grading for artistic purposes, you're gonna have to make new Metadata that match your modified source. For instance, in the studio I work for, we always edit the masterfile in Log (mainly either Slog3 or Clog3) and then we grade it to PQ and generate Metadata. It's almost always the case that I DO NOT grade a PQ file that has already been graded. Anyway you can do it as long as you then generate the right Metadata for it, just don't expect things like Levels, Tweak etc to work in Avisynth as they only work with linear BT601, BT709 and don't support logarithmic curves or hybrid curves. You're better off with Davinci Resolve for grading; you could even make a LUT on Davinci and then use it to grade your HDR file in Avisynth.

NOT HDR:
You can either leave it as it is for the future and leave the player handle the SDR conversion for now on your current monitor (MPV does a good job most of the time) or you could convert it to linear BT709 yourself by either using a tonemapping algorithm (HDR Tools by Jean Philippe or DGHDRToSDR by Donald Graft) or using a matrix of linear transformation (a LUT).

[tolias63]

Chroma placing: is it correctly interpreting 4:2:0 type2 and converts it properly or does it just think it's the old fashioned 4:2:0 and screws everything up?
I think you are right. On original source:
Chroma subsampling: 4:2:0 (Type 2)
On lossless:
Chroma subsampling: 4:2:0
So, it doesn't converts it properly. Because I've never used a bat file to create a lossless, can you give me an example of a bat file that it will properly create a lossless?

As to the Metadata, if your source it's an HLG from a broadcaster or an HDR10 PQ from a Blu-ray, you'll find them in mediainfo. Generally, with HLG stream you won't find anything other than the color curve, the color matrix and the transfer characteristics, so you're gonna be fine by just re-encode with the same three flags in x265 or x264, you'll find them in the documentation or in several other topics here on Doom9. If it's an HDR10 PQ, in addition to the former three, you'll also find master display values and Max,min CLL. Those are very important and have to match exactly the values reported because not only it tells your monitor how many nits (cd/m²) it is, but it also tells it how to correctly display the file. If your file is in HDR10+ or in DolbyVision, then things get a bit more complicated because those Metadata "change" over time from scene to scene, so you have to extract the Metadata file and then use it to re-encode, you can't just put a flag and some values like before. It can be done, but it's not straightforward and again the Doom9 community made a huge effort and an outstanding job in covering the new specs and make it possible to handle these standard with open source softwares (you'll find many things about HDR10+ in the x265 section).
My source is HDR10+ PQ. I found in the x265 section about hdr10plus_parser, which provides a JSON file with the metadata. If I use it while encode, will it be okay or do I need to use master display and CLL values as well?

As to your TV, is it HDR compatible? I gotta say that I generally work with a not so good HDR monitor for windows stuff and I have a dedicated 1000 nits monitor which displays only the output from the dedicated Black magic design SDI chip output. (I wish I had the money to buy the Sony $50k monitor I have at work... T_T). Anyway, if it is HDR Compatible and you're using Windows 10, then make sure that you turn on the HDR mode on Windows. If it isn't, then you're in no luck.
My TV is HDR compatible. This is it (https://www.samsung.com/uk/tvs/uhd-ku6000/UE43KU6000KXXU/). But when I plug it into the computer, it doesn't recognize it as HDR. Maybe it's because it's an old model or something, I don't know...

If it is HDR and you wanna mess with levels and colours, remember that you can only edit contents with as many nits as your monitor, otherwise you won't get a real representation of what you're watching. Besides, if you do mess up with the grading for artistic purposes, you're gonna have to make new Metadata that match your modified source. For instance, in the studio I work for, we always edit the masterfile in Log (mainly either Slog3 or Clog3) and then we grade it to PQ and generate Metadata. It's almost always the case that I DO NOT grade a PQ file that has already been graded. Anyway you can do it as long as you then generate the right Metadata for it, just don't expect things like Levels, Tweak etc to work in Avisynth as they only work with linear BT601, BT709 and don't support logarithmic curves or hybrid curves. You're better off with Davinci Resolve for grading; you could even make a LUT on Davinci and then use it to grade your HDR file in Avisynth.
If I understood well, you mean to edit the video in DaVinci, export the 3D LUT file and use a plugin like avscube to load the file in AviSynth+, right? But what about the new metadata? Do I also export them from DaVinci?

Sorry for the many questions, but this is all new to me. Like discovering a new world!

[FranceBB]

My source is HDR10+ PQ. I found in the x265 section about hdr10plus_parser, which provides a JSON file with the metadata. If I use it while encode, will it be okay or do I need to use master display and CLL values as well?


Yes, you have to extract the JSON file and use the parser so that you can use it for the new encode, but you also have to specify all the settings you find in mediainfo as well.


My TV is HDR compatible. But when I plug it into the computer, it doesn't recognize it as HDR.


It doesn't detect it as HDR 'cause Windows is outputting SDR by default. Can you try to turn the HDR Mode on on Windows?


you mean to edit the video in DaVinci, export the 3D LUT file and use a plugin like avscube to load the file in AviSynth+, right?


Precisely. Which reminds me that I still have to make a LUT for the poor guys who asked me for one several days ago on my topic...


But what about the new metadata? Do I also export them from DaVinci?


This is a problem as Davinci doesn't generate them. Perhaps someone else can comment on this part with some open source solution to generate a JSON file.

[tolias63]

It doesn't detect it as HDR 'cause Windows is outputting SDR by default. Can you try to turn the HDR Mode on on Windows?
The only option I found is "Windows HD Color" and on "Display capabilities" it says "Stream HDR video: No". Is there any other option I haven't found?

Precisely. Which reminds me that I still have to make a LUT for the poor guys who asked me for one several days ago on my topic...
Great! But how do I open the original video in DaVinci? I extracted the HEVC video stream, but doesn't open it. I remuxed it into a mp4 file, but it says "Media Offline"...

Edit

I downloaded "HEVC Video Extensions from Device Manufacturer" from Microsoft and fixed the "Media Offline" problem.

Also, I think I need to buy DaVinci Resolve Studio to properly edit a HDR video.

One last thing. Can you tell me about the properly creation of a lossless so that doesn't mess with Chroma?

[FranceBB]

Can you tell me about the properly creation of a lossless so that doesn't mess with Chroma?


The Chroma location for 4:2:0 type2 is 0:0 while in classic 4:2:0 is 0:128.

You can try this to go from 4:2:0 type2 to 4:2:0 in ffmpeg before you encode to HuffYUV or UTVideo or FFV1 or Lagarith or MagicYUV (they are all lossless):


-vf 'scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=128'


And then this after you have done what you have done in your Avisynth script and you wanna go back to 4:2:0 type2 before encoding with x265:


ffmpeg.exe -i "avs_script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe


This should allow you to work with a lossless file and use Avisynth in 4:2:0 but then output a 4:2:0 type2 stream. In all this I wonder when Ferenc will introduce support to 4:2:0 type2 in Avisynth. :rolleyes:

[tolias63]

Thank you so much for your help!!

One last question. Because, for now, I can't work with HDR, I will use DGHDRToSDR by Donald Graft to convert it to SDR.

In order to create properly the lossless, I will use the following command?
ffmpeg.exe -i "hdr_to_sdr.avs" -vf 'scale=out_color_matrix=bt709nc:out_h_chr_pos=0:out_v_chr_pos=128'

And then, the following command to change it back to type 2?
ffmpeg.exe -i "avs_script.avs" -vf scale=out_color_matrix=bt709nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe

[FranceBB]

No. If you wanna go to SDR with DGHDRToSDR and Avisynth you don't need to do that conversion, just use FFMpegSource2("something.m2ts", atrack=-1") to open your source and then DGHDRToSDR. The output is already going to be in 4:2:0 'cause FFMpegSource2 took care of the conversion while indexing. There's no need to make a lossless with ffmpeg before indexing it with Avisynth. That's pretty much it.

[tolias63]

I can't use FFMpegSource2 because the DGHDRToSDR requires a dgi file and instead I am using DGSource. Neither in this case Ι need some conversion?

And something last, I swear. SDR support Type2 or not? I can't find something. Before the last encode with HEVC, can I do the conversion "out_h_chr_pos=0:out_v_chr_pos=0" or it doesn't matter?

[videoh]

I can't use FFMpegSource2 because the DGHDRToSDR requires a dgi file and instead I am using DGSource. Neither in this case Ι need some conversion? It's false that DGHDRtoSDR requires a DGI file! Whatever gave you that idea? DGHDRtoSDR requires only YUV420P16 delivered from Avisynth+. Of course, you can use DGSource() but as I mentioned earlier it does not support lossless codecs, such as HUFYUV.

[tolias63]

Yes, you are right, I am sorry! I was wrong.

Τhe plugin will not work with the following script:
FFMpegSource2("hdr_video.m2ts")
DGHDRtoSDR(...)

But it will work with the following script:
FFMpegSource2("hdr_video.m2ts")
ConvertTo16bit()
DGHDRtoSDR(...)

[videoh]

That's better. Are you sure FFMpegSource2() is delivering high-bit depth and not 8-bit? You should be able to configure it to deliver 16-bit (padded with zeros in the lower bits).

[tolias63]

modern/recent version of ffms2 output the real bit depth (i.e 10bit planar if it's 10bit planar, or 12bit planar if it's 12bit planar and so on).

So, I think it's okay.

[videoh]

Good to go!

[FranceBB]

Yes, as I said, new ffms2 version are fine, but just to be 100% sure...
Do you have AVSPmod? If not, please download it and "compile" your script with it.
When the preview pops up, check that it says "YUV420P10" (bottom right) and NOT "yv12".
If it is, then it's outputting the real bit depth and you can safely convert it to 16bit with the Avisynth built in converter.
Alternatively, as Donald said, you can configure ffms2 itself to output 16bit planar like so:


video=FFVideoSource("I:\Production\aaa\test.mxf", colorspace="YUV420P16")
audio=FFAudioSource("I:\Production\aaa\test.mxf")
AudioDub(video, audio)


As a side note here, since you have not so many messages in this forum and you may not know, I referred to videoh as Donald 'cause he's... Donald Graft, the mind behind DGIndex, DGIndexNV, DGHDRToSDR among other remarkable things, so... you're lucky, you're kinda talking to the developer xD

[videoh]

Thank you, FranceBB, for the kind words.

[tolias63]

Do you have AVSPmod? If not, please download it and "compile" your script with it.
When the preview pops up, check that it says "YUV420P10" (bottom right) and NOT "yv12".
I downloaded AVSPmod and I open the script, but at the bottom right it says only the resolution and the fps. Also, the Colorspace in Video Information is blank. I will go with DGSource to be sure.

As a side note here, since you have not so many messages in this forum and you may not know, I referred to videoh as Donald 'cause he's... Donald Graft, the mind behind DGIndex, DGIndexNV, DGHDRToSDR among other remarkable things, so... you're lucky, you're kinda talking to the developer xD
I was thinking he might be the developer, but I wasn't sure.
And of course you are right, without all his awesome work, we wouldn't be able to do what we do! Thank you, Donald!!!

[MeteorRain]

A side note, VirtualDub2 also supports high bit depth and will display the colorspace inside file information.

[videoh]

You're welcome, tolias63! The forum needs a 'blushing' smilie. :)

[FranceBB]

Great thread, guys! Thanks a lot for discussing this question because I have to edit 4K HEVC HDR video as well, and had the same problem as op had.))

It's remarkable how many times in the past I was wondering about something and a Google Search led me to Doom9 where my question already had an answer.
Doom9 didn't become what it is today by chance, it was thanks to the dedication of all its users and I'm glad that this time I've have been part of it. :)