HDR10 to SDR with Hable tone-mapping

[poisondeathray]

@lansing - does the original stream have metadata and parameters listed ? That can help guide what settings to use (or at least as a starting point)

[lansing]

Here's the sample clip (http://www.mediafire.com/file/p2pzv2k722l8o33/4kbs_trim.mp4/file), there's no metadata with the stream.

script I've tested:


rgb_clip = core.resize.Bicubic(clip, height=1080, width=1920, matrix_in_s="709", format=vs.RGBS)

# hable
tonemap_clip = core.tonemap.Hable(rgb_clip, exposure=7.0, a=0.10, b=0.50, c=0.10, d=0.20, e=0.01, f=0.08, w=5.2)

# mobius
tonemap_clip = core.tonemap.Mobius(rgb_clip, exposure=1.5, transition=0.9, peak=1.0)

#reinhard
tonemap_clip = core.tonemap.Reinhard(rgb_clip, exposure=1.5, contrast=0.5, peak=1.0)



The hable method is the only one that came close, but color are still off. The other two failed completely, as they have nothing to tweak.

[ifb]

The hable method is the only one that came close, but color are still off. The other two failed completely, as they have nothing to tweak.
tonemap requires linear input and you still need to convert from 2020 to 709.

<edit>
I'm pretty sure it's HLG but with Rec.709 color, which is weird but I suppose they are worried about backward compatibility.

This is with zero tweaking, just copied from the tonemap examples:
c = core.ffms2.Source(source = "4kbs_trim.mp4")
c = core.resize.Lanczos(clip=c, format=vs.RGBS,
width=1920, height=1080,
matrix_in_s="709",
transfer_in_s="std-b67", transfer_s="linear",
nominal_luminance=1000)
c = core.tonemap.Mobius(clip=c, exposure=4)
c = core.resize.Lanczos(clip=c, format=vs.YUV420P10, matrix_s="709",
primaries_in_s="709", primaries_s="709",
transfer_in_s="linear", transfer_s="709")
https://i.imgur.com/PmoiNZG.jpg

[poisondeathray]

It seems that the stream is actually SDR rather than HDR even though the resolution is 4K. By definition, HDR video should have at least 10-bit color depth. And the stream is encoded in AVC, not HEVC. AVC-encoded HDR video is very uncommon.


^ this

If the goal was to "match" the other version, I'd probably just use normal color manipulation filters .

[age]

Hi :-)
I would like to use normal color manipulation filters too for this video.
I've made a.cube file for the sdr match.
https://filebin.net/uu8l2zh4xso2gqd2

https://i.imgur.com/ohvW0R3h.jpg


import vapoursynth as vs


core = vs.get_core()

c = core.lsmas.LibavSMASHSource(source= "C:/Users/..../4kbs_trim.mp4")




c=core.resize.Bicubic(clip=c, format=vs.RGB48, chromaloc_in_s="left", chromaloc_s="center", filter_param_a=0.0, filter_param_b=0.75, matrix_in_s="2020ncl", range_in_s="limited",dither_type="none")
c=core.timecube.Cube(c, cube="C:/..../Videos/4kbshdr.cube")
c=core.resize.Bicubic(clip=c, format=vs.YUV420P8, chromaloc_in_s="center", chromaloc_s="left",matrix_s="709", filter_param_a=0.0, filter_param_b=0.75, range_in_s="full", range_s="limited",dither_type="none")


c.set_output()

[ifb]

HLG has a different look ("natural" to use BBC/BT.2408 terminology) versus the "traditional" look of SDR. To convert, you need to do a scene-referred conversion, which is not exposed in z.lib/vapoursynth. You can patch z.lib and build your own custom vapoursynth.dll, but really you're just better off using a LUT like shown.

The BBC has some official .cube files they provide for doing this sort of thing.

[age]

That video doesn't look hdr, but more like an hdr video converted to rec709 without tonemapping
for example with hable tonemapping for 1000 nits

import vapoursynth as vs


core = vs.get_core()
c = core.lsmas.LibavSMASHSource(source= "C:/Users//Desktop/4kbs_trim.mp4")




c=core.resize.Bicubic(clip=c, format=vs.RGBS,filter_param_a=0.0,filter_param_b=0.75, range_in_s="limited", matrix_in_s="2020ncl", chromaloc_in_s="left", chromaloc_s="center", transfer_in_s="709", transfer_s="linear",dither_type="none")

#insert here tonemapping 1000nits
#exposure_bias=10
#tm = core.std.Expr(c, expr="x {exposure_bias} * 0.15 x {exposure_bias} * * 0.05 + * 0.004 + x {exposure_bias} * 0.15 x {exposure_bias} * * 0.50 + * 0.06 + / 0.02 0.30 / - ".format(exposure_bias=exposure_bias))
#w=((exposure_bias*(0.15*exposure_bias+0.10*0.50)+0.20*0.02)/(exposure_bias*(0.15*exposure_bias+0.50)+0.20*0.30))-0.02/0.30
#tm = core.std.Expr(clips=[tm,c], expr="x 1 {w} / * ".format(exposure_bias=exposure_bias,w=w))



c=core.resize.Bicubic(clip=c, format=vs.RGBS, transfer_in_s="linear", transfer_s="709",dither_type="none")
c=core.resize.Bicubic(clip=c, format=vs.YUV420P8, chromaloc_in_s="center", chromaloc_s="left", filter_param_a=0.0, filter_param_b=0.75, range_in_s="full", range_s="limited", matrix_s="709",dither_type="none")





c.set_output()


it gives me this
https://i.imgur.com/FRpontwh.jpg

[lansing]

That video doesn't look hdr, but more like an hdr video converted to rec709 without tonemapping

I think you're right, it turns out that the recorder has an older model 4K TV that doesn't support HDR, and the tv just converted it into something else.

[lansing]

Ok after reading the hable reference article from ifb's tonemap page (https://github.com/ifb/vapoursynth-tonemap), I just realized that hable is not for 4k hdr tv to sdr conversion. It was created for images and SDR games to make them looks HDR. So this whole approach to use it for "4K HDR TV content -> SDR" is just wrong. The right tool for this in vapoursynth is DGHDRtoSDR.

[asarian]

So this whole approach to use it for "4K HDR TV content -> SDR" is just wrong. The right tool for this in vapoursynth is DGHDRtoSDR.

So, maybe you know a solution to my problem too. :devil: See: DGHDRtoSDR lacking in contrast (https://forum.doom9.org/showthread.php?p=1862404#post1862404)

Which also makes me wonder, would there be a way for DGHDRtoSDR to extract values like 'Max luminance: 1000' from the metadata itself?

[lansing]

So, maybe you know a solution to my problem too. :devil: See: DGHDRtoSDR lacking in contrast (https://forum.doom9.org/showthread.php?p=1862404#post1862404)

Which also makes me wonder, would there be a way for DGHDRtoSDR to extract values like 'Max luminance: 1000' from the metadata itself?

I don't have UBD content, so I don't know what they look like.

For HDR TV, HDR was suppose to look different from SDR, as it preserves detail on both highlight and shadow instead of "white" being blown out in SDR in comparison. But as a result, it will look washier and less contrast than SDR. You HDR result looks ok to me except for the saturation.

[Alexkral]

Ok after reading the hable reference article from ifb's tonemap page (https://github.com/ifb/vapoursynth-tonemap), I just realized that hable is not for 4k hdr tv to sdr conversion. It was created for images and SDR games to make them looks HDR. So this whole approach to use it for "4K HDR TV content -> SDR" is just wrong. The right tool for this in vapoursynth is DGHDRtoSDR.

I don't think that matters at all, it's a curve as valid as any other and as you can see mpv is using it for this purpose.

[lansing]

I don't think that matters at all, it's a curve as valid as any other and as you can see mpv is using it for this purpose.

And maybe that's why it gives the obvious wrong color on HDR content?

[asarian]

I don't have UBD content, so I don't know what they look like.

For HDR TV, HDR was suppose to look different from SDR, as it preserves detail on both highlight and shadow instead of "white" being blown out in SDR in comparison. But as a result, it will look washier and less contrast than SDR. You HDR result looks ok to me except for the saturation.

I do not have (1080p) Blu-Rays for all UHD discs I own, so I will only seldom be able to do a comparison. Hence, my initial question, "would there be a way for DGHDRtoSDR to extract values like 'Max luminance: 1000' from the metadata itself?" And then for contrast too, of course.

[Alexkral]

And maybe that's why it gives the obvious wrong color on HDR content?
I don't see those wrong colors in mpv, so the problem is in the script, not in the curve. It may be because the script is tone mapping the RGB channels instead of the luminance. I don't know what DGHDRtoSDR does in that sense, but it seems to be using the Reinhard TMO which is much simpler.

[Alexkral]

@asarian

DGHDRtoSDR doesn't even have a parameter for that.

[asarian]

@asarian

DGHDRtoSDR doesn't even have a parameter for that.

Why would it need a parameter for that?! Preferably, it would find and extract the info on its own from the UHD Blu-Ray structure (DV layer and what not; or, if it needs a parameter per se, a location of the meta-data file).

[Alexkral]

It would still need a parameter internally for the math.

[age]

I was testing the tonemapping using the hue-constant curves from adobe
https://github.com/Beep6581/RawTherapee/blob/3ff2519302e3bc529b111462a4697ac4dfba30c4/rtengine/curves.h#L919
https://github.com/Beep6581/RawTherapee/blob/3ff2519302e3bc529b111462a4697ac4dfba30c4/rtengine/curves.h#L974

Here's a new script
tonemapping.py

import vapoursynth as vs



def tm_hable(clip="",source_peak=1000 ) :
core = vs.get_core()
c=clip

source_peak=source_peak
LDR_nits=100
exposure_bias=source_peak/LDR_nits
o=c
c=core.std.Limiter(c, 0)
r=core.std.ShufflePlanes(c, planes=[0], colorfamily=vs.GRAY)
g=core.std.ShufflePlanes(c, planes=[1], colorfamily=vs.GRAY)
b=core.std.ShufflePlanes(c, planes=[2], colorfamily=vs.GRAY)
lum = core.std.Expr(clips=[r,g,b], expr="0.216 x * 0.715 y * + 0.0722 z * +")
lum=core.std.Limiter(lum, 0)
lum=core.std.ShufflePlanes(lum, planes=[0], colorfamily=vs.RGB)




rr="x {exposure_bias} * 0.15 x {exposure_bias} * * 0.05 + * 0.004 + x {exposure_bias} * 0.15 x {exposure_bias} * * 0.50 + * 0.06 + / 0.02 0.30 / - 1 {exposure_bias} 0.15 {exposure_bias} * 0.05 + * 0.004 + {exposure_bias} 0.15 {exposure_bias} * 0.50 + * 0.06 + / 0.02 0.30 / - / * ".format(exposure_bias=exposure_bias)
gg="y {exposure_bias} * 0.15 y {exposure_bias} * * 0.05 + * 0.004 + y {exposure_bias} * 0.15 y {exposure_bias} * * 0.50 + * 0.06 + / 0.02 0.30 / - 1 {exposure_bias} 0.15 {exposure_bias} * 0.05 + * 0.004 + {exposure_bias} 0.15 {exposure_bias} * 0.50 + * 0.06 + / 0.02 0.30 / - / * ".format(exposure_bias=exposure_bias)
bb="z {exposure_bias} * 0.15 z {exposure_bias} * * 0.05 + * 0.004 + z {exposure_bias} * 0.15 z {exposure_bias} * * 0.50 + * 0.06 + / 0.02 0.30 / - 1 {exposure_bias} 0.15 {exposure_bias} * 0.05 + * 0.004 + {exposure_bias} 0.15 {exposure_bias} * 0.50 + * 0.06 + / 0.02 0.30 / - / * ".format(exposure_bias=exposure_bias)


r1 = core.std.Expr(clips=[r,g,b], expr="x y >= y z > {rr} z x > x y - z y - / {bb} {gg} - * {gg} + z y > {rr} {rr} ? ? ? x z >= x z - y z - / {gg} {bb} - * {bb} + z y > {rr} {rr} ? ? ? ".format(exposure_bias=exposure_bias,rr=rr,gg=gg,bb=bb))
g1 = core.std.Expr(clips=[r,g,b], expr="x y >= y z > y z - x z - / {rr} {bb} - * {bb} + z x > {gg} z y > {gg} {gg} ? ? ? x z >= {gg} z y > y x - z x - / {bb} {rr} - * {rr} + {gg} ? ? ? ".format(exposure_bias=exposure_bias,rr=rr,gg=gg,bb=bb))
b1 = core.std.Expr(clips=[r,g,b], expr="x y >= y z > {bb} z x > {bb} z y > z y - x y - / {rr} {gg} - * {gg} + {bb} ? ? ? x z >= {bb} z y > {bb} z x - y x - / {gg} {rr} - * {rr} + ? ? ? ".format(exposure_bias=exposure_bias,rr=rr,gg=gg,bb=bb))

crgb=core.std.ShufflePlanes(clips=[r1,g1,b1], planes=[0,0,0], colorfamily=vs.RGB)


lumtm = core.std.Expr(clips=[r1,g1,b1], expr="0.216 x * 0.715 y * + 0.0722 z * +")
lumtm=core.std.Limiter(lumtm, 0)
lumtm=core.std.ShufflePlanes(lumtm, planes=[0], colorfamily=vs.RGB)
clum = core.std.Expr(clips=[o,lum,lumtm], expr=" x {exposure_bias} * y {exposure_bias} * / z * ".format(exposure_bias=exposure_bias))
clum=core.std.Limiter(clum, 0)

mask=core.std.Expr(clips=[lum,lumtm], expr=" x {exposure_bias} * y - abs {exposure_bias} 1 - / ".format(exposure_bias=exposure_bias))
mask=core.std.Limiter(mask, 0,1)


ctemp=core.std.MaskedMerge(crgb, lumtm, mask)
c=core.std.Merge(clum, ctemp, 0.5)

return c



It looks similar to the new algo used by madvr at 200 nits target, not as good as madvr because the script needs a gamut compression when converting to rec.709 color primaries

http://screenshotcomparison.com/comparison/133859
http://screenshotcomparison.com/comparison/133858
http://screenshotcomparison.com/comparison/133861

It has only a parameter source_peak

this is the usage of the script

import tonemapping
import vapoursynth as vs

core = vs.get_core()
c = core.ffms2.Source(source = 'C:/Users/.../videos/HDR.mp4')

source_peak=1200
matrix_in_s="2020ncl"
transfer_in_s="st2084"


c=core.resize.Bicubic(clip=c, format=vs.RGBS, filter_param_a=0, filter_param_b=0.75, matrix_in_s=matrix_in_s,chromaloc_in_s="center",chromaloc_s="center", range_in_s="limited",dither_type="none")

c=core.resize.Bilinear(clip=c, format=vs.RGBS, transfer_in_s=transfer_in_s, transfer_s="linear",dither_type="none", nominal_luminance=source_peak)



c=tonemapping.tm_hable(c,source_peak=source_peak )

c=core.resize.Bilinear(clip=c, format=vs.RGBS, primaries_in_s="2020", primaries_s="709",dither_type="none")
c=core.resize.Bilinear(clip=c, format=vs.RGBS, transfer_in_s="linear", transfer_s="709",dither_type="none")

c=core.resize.Bicubic(clip=c, format=vs.YUV420P8,matrix_s="709", filter_param_a=0, filter_param_b=0.75, range_in_s="full",range_s="limited", chromaloc_in_s="center", chromaloc_s="center",dither_type="none")

c.set_output()

[chipseps]

Thank you all for your suggestion and the hdr samples.
Well I've discovered some interesting things....
First, a better version of the hable tonemapping that gives better contrast,brightness and saturation.
http://screenshotcomparison.com/comparison/203550


I've tried different algorithms, the aces tonemapping too but I don't like it very much.
Here a good link for a lot of open source tm operators :

https://github.com/tizian/tonemapper/tree/master/src/operators

And a different version of aces:

https://github.com/keijiro/PostProcessingUtilities/blob/master/Assets/PostProcessing%20Imported/Resources/Shaders/Tonemapping.cginc


new script with correct parameters

import vapoursynth as vs


core = vs.get_core()
c = core.ffms2.Source(source = 'C:/bla')

source_peak=1000 #set manually
LDR_nits=100 #set 150 or 200 for lowering the brightness

c=core.resize.Bilinear(clip=c, format=vs.YUV444PS, range_in_s="limited", range_s="full",chromaloc_in_s="center",dither_type="none")

c=core.resize.Bicubic(clip=c, format=vs.RGBS, matrix_in_s="2020ncl", range_in_s="full",dither_type="none")

c=core.resize.Bilinear(clip=c, format=vs.RGBS, transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=source_peak)

exposure_bias=(source_peak/LDR_nits)

#hable tone mapping correct parameters
#["A"] = 0.22
#["B"] = 0.3
#["C"] = 0.1
#["D"] = 0.2
#["E"] = 0.01
#["F"] = 0.3
#["W"] = 11.2
#((x * (A*x + C*B) + D*E) / (x * (A*x+B) + D*F)) - E/F"
tm = core.std.Expr(c, expr="x {exposure_bias} * 0.22 x {exposure_bias} * * 0.03 + * 0.002 + x {exposure_bias} * 0.22 x {exposure_bias} * * 0.3 + * 0.06 + / 0.01 0.30 / - ".format(exposure_bias=exposure_bias),format=vs.RGBS)#12=1200 nits / 100 nits



w = core.std.Expr(c, expr="{exposure_bias} 0.22 {exposure_bias} * 0.03 + * 0.002 + {exposure_bias} 0.22 {exposure_bias} * 0.3 + * 0.06 + / 0.01 0.30 / - ".format(exposure_bias=exposure_bias),format=vs.RGBS)

c = core.std.Expr(clips=[tm,w], expr=" x 1 y / * ",format=vs.RGBS)

#c=core.fmtc.primaries(clip=c, prims="2020", primd="709")
c=core.resize.Bilinear(clip=c, format=vs.RGBS, primaries_in_s="2020", primaries_s="709",dither_type="none")
c=core.resize.Bilinear(clip=c, format=vs.RGBS, transfer_in_s="linear", transfer_s="709",dither_type="none")

c=core.resize.Bilinear(clip=c, format=vs.YUV420P8, matrix_s="709", range_in_s="full",range_s="limited",chromaloc_in_s="center")


c.set_output()



Hi you! With color filtering like this, what command must we use to crop and resize to 1080p? Because I'm newbie encodec so I don't know! Hope to help you! thank you very much!

[gonca]

I do not have (1080p) Blu-Rays for all UHD discs I own, so I will only seldom be able to do a comparison. Hence, my initial question, "would there be a way for DGHDRtoSDR to extract values like 'Max luminance: 1000' from the metadata itself?" And then for contrast too, of course.

The master display data, CLL, and FALL is "extracted by DGIndexNV

[videoh]

The master display data, CLL, and FALL is "extracted by DGIndexNV ... and recorded in the DGI file.