The HDR-to-SDR process

[wonkey_monkey]

I'm finally learning about HDR, and it seemed like a good idea to implement my own HDR-to-SDR converter to check that my understanding was correct.

Ignoring tonemapping for now, I've been able to get a result that's close to, but not identical to, DGHDRtoSDR(tm = 0):

https://imgsli.com/NDQ1MDc2

Mine has higher contrast and DGHDRtoSDR's has higher saturation.

I just wanted to run through the process I'm using to see if anyone can spot any misunderstandings:

----

1. Scale the limited range HDR luma from [64, 940] to full range [0,1], and chroma from [64, 960] to [-0.5, 0.5]

2. Convert to RGB using BT2020 coefficients:

float Kr = 0.2627f, Kb = 0.0593f;
float Kg = 1 - Kr - Kb;

float r = y_channel + v_channel * (2 - 2 * Kr);
float g = y_channel - u_channel * (Kb / Kg) * (2 - 2 * Kb) - v_channel * (Kr / Kg) * (2 - 2 * Kr);
float b = y_channel + u_channel * (2 - 2 * Kb);

These values, as I understand it, are non-linear PQ, ranging from [0, 1], and representing a brightness of 0-10,000 nits.

3. Apply the PQ EOTF to make the RGB values linear:

float m1 = 2610.0f / 16384;
float m2 = 2523.0f / 4096 * 128;
float c1 = 3424.0f / 4096;
float c2 = 2413.0f / 4096 * 32;
float c3 = 2392.0f / 4096 * 32;

float n = std::pow(pq, 1.0f / m2);
return std::pow((std::max)(n - c1, 0.0f) / (c2 - c3 * n), 1.0f / m1);

[ Edit: missed step: convert the linear RGB values between different chromaticities? Not yet understood ]

4. Taking 100 nits as SDR white, I multiply these values by 100 (HDR's 10,000 nit peak divided by SDR's 100 nit peak) and clamp to [0,1].

5. Apply Rec709's OETF to make the values non-linear (done on each channel):

float HDRtoSDR::Rec709_OETF(float x) {
return (x < 0.018f) ? 4.5f * x : 1.099f * std::pow(x, 0.45f) - 0.099f;
}

6. Apply the Rec709 matrix to convert RGB to YUV.

7. Scale to 8-bit limited range (luma: [0,1] -> [16,235], chroma: [-0.5,0.5] -> [16,240]

----

I get that (annoyingly, for my precision-wanting brain) there's no single "correct" way to do this, but does that look right as a baseline HDR-to-SDR process?

[FranceBB]

Yep, that's essentially correct. By the way, Jean Philippe Scotto di Rinaldi wrote a nice little paper about this when he started working on HDR Tools. If you download the latest release from here https://github.com/jpsdr/HDRTools/releases you're gonna find a folder called "Document" in which you can find the PDF from 2018 in which he describes this exact process and the formulas used including the conversion to RGB (and the alternative XYZ conversion roundtrip).

[wonkey_monkey]

Thanks - I had found that document but got a bit lost in it. I looked at the flowchart at the end just now though, and it seems to suggest I'm missing a PQ OOTF^-1 step which I haven't got my head round yet... edit: or is that just tone mapping?

I also tried your PQ_to_BT709_v2.cube, DGHDRtoSDR, and mine on an HDR video which I also have an SDR version of (produced at the same time) - shadow levels differ between all of them, but mine are the darkest:

https://i.ibb.co/RTBGb5KB/image.png

But would you say they're still all "correct" as far as conversions go?

[FranceBB]

or is that just tone mapping?


That's the tonemapping which you haven't implemented yet, which is also why you're getting the results you're getting. I mean, ideally after step 3 you would have tonemapping using a Reinhard operator. Generally, Reinhard offers a soft appearance, similar to what you would see in a native SDR grading, Hable makes everything appear "HDR" in an SDR output, namely strong contrast and saturated colors, Mobius is something in between.

Personally, if I have to tonemap, I use Reinhard.


I also tried your PQ_to_BT709_v2.cube, DGHDRtoSDR, and mine on an HDR video which I also have an SDR version of (produced at the same time) - shadow levels differ between all of them, but mine are the darkest:

https://i.ibb.co/RTBGb5KB/image.png

But would you say they're still all "correct" as far as conversions go?

Yep, they look right, but it's always hard to compare real footage. Can you try to apply your conversion to the following image? It represents colorbars at 400 nits in BT2020 HDR PQ.

https://i.postimg.cc/HWz1M3hr/New-File-(25)000000.png

in fact:
https://i.postimg.cc/4xTb5pd2/image.png

As you can see, the conversion changes between the PQ_to_BT709_v1.cube and PQ_to_BT709_v2.cube as the v1 assumes a lower MaxCLL than v2.

PQ_to_BT709_v1.cube (top)
PQ_to_BT709_v2.cube (bottom)

https://i.postimg.cc/rwB6gnW8/image.png

Once again, there's no perfect way to perform the conversion, but both of them would be technically correct, it's just that the MaxCLL of 400 nits wasn't brought down to 100 nits as they both assumed it was higher.

I'm curious to see what your code outputs and it's assuming 'cause if it's assuming 10,000 nits as MaxCLL, then it will always be darker than any other conversions as most titles are within 1200 nits and in some occasions 2000 nits only on things like special effects (like for the Remaster of Harry Potter in which pretty much everything is low as it was shot with SDR in mind and then the spells skyrocket to 2000 nits).

[wonkey_monkey]

That's the tonemapping which you haven't implemented yet, which is also why you're getting the results you're getting.

I was hoping DGHDRtoSDR with tm = 0 could serve as reference, but it's a little different and there's probably no way to know exactly why. It's pretty close though.

It represents colorbars at 400 nits in BT2020 HDR PQ.

Ah... represents how? My lazy code expects YUV420P10.

What do you use for your histograms?

As you can see,

I will try to see, but that's all I can promise at my current level of understanding :D

[Columbo]

I was hoping DGHDRtoSDR with tm = 0 could serve as reference, but it's a little different and there's probably no way to know exactly why. It's pretty close though. It makes some heuristic tweaks in favor of maximum performance on CUDA. I thought DG released the code. I'll check on that.

tm=0 doesn't disable tone mapping completely IIRC.

[Z2697]

Is zimg's transfer function conversion a good reference?

[GeoffreyA]

zimg or fmtconv: both aim to operate "by the book." libplacebo, too, is an exceptional tool to experiment with.

[FranceBB]

Ah... represents how? My lazy code expects YUV420P10.


That's not actually a problem, using one of my LUT you can get roughly the same result which is gonna be ok for testing purposes in this case as well.


ColorBars(848, 480, pixel_type="YV24")

ConvertBits(16)
ConvertToPlanarRGB(matrix="Rec709")
Cube("C:\Program Files (x86)\AviSynth+\LUTs\BT709_to_PQ.cube", interp=1, fullrange=1)
ConverttoYUV420(matrix="Rec2020")
ConvertBits(bits=10, dither=1)


This way you can generate a BT2020 HDR PQ output at 400 nits directly on your own and use that to perform the tests. :)


What do you use for your histograms?


VideoTek(Mode="PQ", Type="nits", Detailed=true)

which you can find here https://github.com/FranceBB/VideoTek/tree/master

I started working on it in 2018 because when I wasn't in the office and I didn't have access to an actual hardware waveform monitor from Tektronix I was essentially blind. Then, when the pandemic arrived, it was a blessing 'cause it allowed me to see exactly what I was working on during the first month, before we were all declared "essential workers" and went back to the office wearing an FFP3 mask all day. Odd times.

StainlessS, magiblot and erazortt helped me write it too.



I will try to see, but that's all I can promise at my current level of understanding :D

It's actually pretty easy to watch the waveform once you get use to it. By default VideoTek() shows everything in volts as if you were dealing with a signal carried via an SDI cable. In the case of PQ,

https://images2.imgbox.com/0c/a6/N0DMYwoT_o.png

where 0.40V (or, to be precise, 406 millivolts) is the 58% reference white which would correspond to 203 nits with:

VideoTek(Mode="PQ", Type="nits", Detailed=true)

https://images2.imgbox.com/c3/9a/uYaAN467_o.png


If you look on the left hand side, you can see a flat line which is higher than all the other. That corresponds to the white which is 400 nits in the PQ example. If you move back and forth on a real file with the waveform, you're gonna be able to find out what the actual peak is and how it behaves through the length of the footage. Ideally, when you perform the conversion, you want that peak to become 0.7V (or 100 nits) in SDR. In this case, given that the peak is 400 nits, it would become like this:

VideoTek()

https://images2.imgbox.com/06/fe/au8gkIOM_o.png

which, expressed as nits, would be

VideoTek(Mode="SDR", Type="nits", Detailed=true)

https://images2.imgbox.com/6a/6a/fDm1MxWK_o.png

To find out the MaxCLL when it's not reported in the stream or in the container, you can use MaxCLLFind() https://github.com/FranceBB/MaxCLLFindAVS/releases


ConvertToRGB64(matrix="Rec2020")
MaxCLLFind(maxFallAlgorithm=0, LogIntermediateStats=false)

and then use that as your reference to bring the peak to 100 nits, but of course this creates problems on its own when you have a lot of dynamic range.

I made the Harry Potter examples before and I've actually checked now:

Harry Potter and the Philosopher's stone is 552 Nits.
Harry Potter and the goblet of fire is 2848 Nits.
Harry Potter and the azkaban prisoner is 1838 Nits.

Obviously all of them where shot with SDR in mind but have the spells going high, so if you were to take 2848 nits from the Goblet of Fire and bring it down to 100 nits, then everyone's faces would be far too dark, while if you preserve the faces then the spells would clip out and be white in some scenes (which is preferrable but still not ideal). This is where dynamic tonemapping comes in hand as you can define a threshold based on which a scene change is defined and what it will do is basically computing the MaxCLL of the scene and use that as the reference to bring it to 100 nits. If you're curious, this is exactly what MPV is doing when it tries to decode a BT2020 HDR PQ file and the monitor only supports BT709 SDR 100 nits. You can even press Shift + I and see the MaxCLL computed for that scene and see it updated as the movie progresses. Obviously MPV uses libplacebo to do this which is available in Avisynth too as asd-g actually ported it a while ago (even though I'm not personally using it, but I know Tormento is).

[wonkey_monkey]

Never mind, I had forgotten to change some test code back. I'll fix it.

[wonkey_monkey]

Okay, so, my colour bars don't seem to be coming out as they should (this is from calling my plugin with white = 400 nits):

https://i.ibb.co/svJKb16h/image.png

The input pixels in the white square have a value of 636 (I turned off dithering). That normalises to (636 - 64) / (940 - 64) = 0.641553, and after running through my EOTF function, this comes out to 0.0401452 - and since that's a range of [0,1] equating to 0 nits - 10000 nits, that correctly comes out to 400 nits (if I run it with white = 410, the SDR value of that white box comes down just a bit, as you'd expect).

The input pixels in the 75% grey bar have a value of 513. After normalising and EOTFing, this comes out to only 0.0104555, or 100 nits. It should be 300 nits, right?

Here's my EOTF function in graph form: https://www.desmos.com/calculator/czvbe1vi2n

with the two vertical lines representing the normalised values of 75% and 100% pixels.

Have I misunderstood your PQ-colorbars creating code? DGHDRtoSDR produces something with similarly off colours (but higher brightness).

[Z2697]

512 is roughly 100 nits in PQ.
And it's fixed, your target peak is only for (re-)interpretation of that value.

[FranceBB]

uhmmm interesting...
From my side, the results look as follows:


ColorBars(848, 480, pixel_type="YV24")

ConvertBits(16)
ConvertToPlanarRGB(matrix="Rec709")
Cube("C:\Program Files (x86)\AviSynth+\LUTs\BT709_to_PQ.cube", interp=1, fullrange=1)
my_pq=last

ConverttoYUV444(my_pq, matrix="Rec2020")
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:709:709:full", dither_type="error_diffusion")
ConverttoYUV444(matrix="Rec709")
VideoTek(Mode="SDR", Type="Nits", Detailed=true)
Crop(0, 0, -0, -520)
Subtitle("zlib")
zlib=last

Cube(my_pq, "C:\Program Files (x86)\AviSynth+\LUTs\PQ_to_BT709_v1.cube", interp=1, fullrange=1)
ConverttoYUV444(matrix="Rec709")
VideoTek(Mode="SDR", Type="Nits", Detailed=true)
Crop(0, 0, -0, -520)
Subtitle("LUT")
LUT=last

ConverttoYUV444(my_pq, matrix="Rec2020")
ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Hable_HDRtoSDR()
ConvertXYZtoYUV(pColor=0)
VideoTek(Mode="SDR", Type="Nits", Detailed=true)
Crop(0, 0, -0, -520)
Subtitle("Hable")
hable=last

ConverttoYUV444(my_pq, matrix="Rec2020")
ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Mobius_HDRtoSDR(exposure_X=5.1, exposure_Y=5.1, exposure_Z=5.1)
ConvertXYZtoYUV(pColor=0)
VideoTek(Mode="SDR", Type="Nits", Detailed=true)
Crop(0, 0, -0, -520)
Subtitle("Mobius")
mobius=last

ConverttoYUV444(my_pq, matrix="Rec2020")
ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=4.1, exposure_Y=4.1, exposure_Z=4.1)
ConvertXYZtoYUV(pColor=0)
VideoTek(Mode="SDR", Type="Nits", Detailed=true)
Crop(0, 0, -0, -520)
Subtitle("Reinhard")
reinhard=last


StackVertical(zlib, LUT, Hable, Mobius, Reinhard)


https://images2.imgbox.com/df/8f/YYaXF3TT_o.png

which are very different...

[wonkey_monkey]

ColorBars(848, 480, pixel_type="YV24")

ConvertBits(16)
ConvertToPlanarRGB(matrix="Rec709")

A quick aside - shouldn't that be Rec601 for regular ColorBars, and only Rec709 for ColorBarsHD?

[FranceBB]

A quick aside - shouldn't that be Rec601 for regular ColorBars, and only Rec709 for ColorBarsHD?

right, yes, well spotted. I mean, it's not the cause of the issue nor the reason why we have the huge difference in the output in this case, but still, yes.

[wonkey_monkey]

Did I miss a step where I need to change the linear RGB values before applying gamma followed by the Rec709 matrix? Because Rec2020 and Rec709 have different primaries and cover different real colours?

And is this why the channels in the PQ_to_BT709_v1.cube aren't independent of each other?

So much to learn...

[GeoffreyA]

Did I miss a step where I need to change the linear RGB values before applying gamma followed by the Rec709 matrix? Because Rec2020 and Rec709 have different primaries and cover different real colours?

I think that's it.

Here's an old VS script that used to give correct output:

clip = clip.fmtc.bitdepth(bits=32)
clip = clip.fmtc.resample(css="444")
clip = clip.fmtc.matrix(mats="2020", matd="rgb")
clip = clip.fmtc.transfer(transs="2084", transd="linear", match=1)

clip = clip.tonemap.Reinhard(exposure=1, peak=100)
#clip = clip.tonemap.Mobius(exposure=0.7, transition=0.3, peak=100)
#clip = clip.tonemap.Hable(exposure=1.9)

clip = clip.resize.Spline36(1920, 1080)

clip = clip.fmtc.primaries(prims="2020", primd="709")
clip = clip.fmtc.transfer(transs="linear", transd="709")
clip = clip.fmtc.matrix(mats="rgb", matd="709")
clip = clip.fmtc.resample(css="420", cplaced="left")
clip = clip.fmtc.bitdepth(bits=8)


So the primaries are converted while still in linear RGB.

[wonkey_monkey]

Thanks, I implemented the conversion and now my saturation looks more like DGHDRtoSDR, and my colorbars colours look better (though the brightness is still way off on that test, unlike the movie tests). I've still got a bit more contrast compared to DGHDRtoSDR though.

Also implemented Reinhard. Surprised how well it works!

[Ingram112]

The step of converting linear HDR RGB values to Rec.709 can sometimes introduce subtle color shifts. Applying a chromatic adaptation transform before scaling to SDR nits often helps preserve the original hue and saturation. It's interesting to see a hands-on approach to HDR-to-SDR without relying on presets.