HDR ColorBars - bug at transitions ?

[StvG]

rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=false, HLGLw=1000) - use OutputMode=2 for float output like avsresize. Also OOTF should be true to be scene-referred according to the docs? Also EOTF doesn't have effect for HDRMode=1/2 and Color=0 according to docs but changing between true/false there is noticeable diff. EOTF should be set to false? Also HLGLw should be set to 12 (to match avsresize).

Here what I have without any rgbadjust - avsresize and fmtc are close but hdrtools differs noticeably:

# jpsdr_plugin
rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=true, EOTF=false, outputmode=2, HLGlw=12, hlglb=0)

# avsresize
#rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020=>rgb:linear:2020", scene_referred=true) # nominal_luminance does not have effect when scene_referred=true

#fmtconv
#rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
#rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)

rgb_lin= SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
#return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709=>rgb:709:709").ConvertToRGB()
#return fmtc_transfer(rgb_lin,transs="linear",transd="709").fmtc_bitdepth(8, dmode=1).ConvertToRGB()
return ConvertLinearRGBToYUV(rgb_lin,Color=2, OOTF=false, EOTF=true).ConvertToRGB24(matrix="PC.709")

Edit: Actually it's expected the different result from HDRTools because ConvertLinearRGBToYUV(..., Color=2) converts matrix, transfer and primaries to 709 while with avsresize and fmtc the matrix and primaries aren't converted.

[DTL]

rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=true, EOTF=false, outputmode=2, HLGlw=12, hlglb=0)
return ConvertLinearRGBToYUV(rgb_lin,Color=2, OOTF=false, EOTF=true).ConvertToRGB24(matrix="PC.709")

It outputs both distorted transient and R'G'B' sustained big about 230+ and unaligned.

rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020=>rgb:linear:2020", scene_referred=true) # nominal_luminance does not have effect when scene_referred=true
return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709=>rgb:709:709").ConvertToRGB()

It makes very distorted transient and R'G'B' clipped to 0 and 255.

rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)

return fmtc_transfer(rgb_lin,transs="linear",transd="709").fmtc_bitdepth(8, dmode=1).ConvertToRGB()

Transient is a bit dark, R'G'B' clipped to 0..255 (full range, not narrow ?)

This assumed current 'reference' conversion:

rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", scene_referred=true)
rgb_lin=RGBAdjust(rgb_lin,r=0.157,g=0.157,b=0.157)
..scale..
return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709:l=>rgb:709:709:l").ConvertToRGB()

With R'G'B' result of

[StvG]

About HDRTools - after HLG->linear conversion the green is shifted. Compare:

# jpsdr_plugin
return ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=true, EOTF=false, outputmode=2, HLGlw=12, hlglb=0)

# avsresize
#return z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", scene_referred=true)

# fmtconv
#rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
#return fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)

Edit: Here fmtc equivalent of the avsresize 'reference' conversion:

#fmtconv
#rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
#rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)
#rgb_lin=RGBAdjust(rgb_lin, r=0.5, g=0.5, b=0.5)

rgb_lin= SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
#return fmtc_transfer(rgb_lin, transs="linear",transd="1886").fmtc_bitdepth(8, fulls=true, fulld=false).ConvertToRGB()

fmtc transfer "709" is scene-referred - same as z_convertformat(colorspace_op="x:x:x=>x:709:x", scene_referred=true).
fmtc transfer "1886" is display-referred - same as z_convertformat(colorspace_op="x:x:x=>x:709:x").

Also for avsresize add approximate_gamma=false in the latest step - return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709:l=>rgb:709:709:l", approximate_gamma=false).ConvertToRGB()

[DTL]

" fmtc equivalent of the avsresize 'reference' conversion:

#fmtconv
rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)
rgb_lin=RGBAdjust(rgb_lin, r=0.5, g=0.5, b=0.5)

rgb_lin= SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
return fmtc_transfer(rgb_lin, transs="linear",transd="1886").fmtc_bitdepth(8, fulls=true, fulld=false).ConvertToRGB()

"

It is good too. But only not understand why "fmtc_bitdepth(8, fulls=true, fulld=false)" - it mean RGB linear float always full range (in fmtc) ? Or it is a internal feature of fmtc to convert narrow input range to full at some point ? Also the 'magic' amplitude correction of 0.5 looks less magic.

I test fmtc_bitdepth(8, fulls=false, fulld=false) - it returns the same result. Looks like 'fulls' with float input is ignored in fmtc ?
Same result with simple fmtc_bitdepth(8, fulld=false).

Addition: it looks difference between gamma 2.2 and 2.4 is good visible at the 'look' of transient. Also the real physical output of display depends on its physical transfers in RGB channels, so at LCD the view of transient also depends on viewing angle.

[StvG]

Originally Posted by DTL

" fmtc equivalent of the avsresize 'reference' conversion:

#fmtconv
rgb_lin=fmtc_bitdepth(yuv444, 32).fmtc_matrix(mat="RGB", mats="2020")
rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true)
rgb_lin=RGBAdjust(rgb_lin, r=0.5, g=0.5, b=0.5)

rgb_lin= SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
return fmtc_transfer(rgb_lin, transs="linear",transd="1886").fmtc_bitdepth(8, fulls=true, fulld=false).ConvertToRGB()

"

It is good too. But only not understand why "fmtc_bitdepth(8, fulls=true, fulld=false)" - it mean RGB linear float always full range (in fmtc) ? Or it is a internal feature of fmtc to convert narrow input range to full at some point ? Also the 'magic' amplitude correction of 0.5 looks less magic.

rgb_lin=fmtc_bitdepth(yuv444, 32) - here the range is converted from tv range to full because input is int (assumed tv range) and output is float (always [0,1]). Then everything is done in full range. fmtc_bitdepth(8, fulls=true, fulld=false) - float->8 but rgb, this makes the output [0,255], so, yes, fulls is not needed, fulld is enough for [0,255]->[16,235].

About the 'magic' amplitude correction:
- fmtconv - after rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true) the ouput is within range [0,~1]. After applying RGBAdjust values of 0.5, the output becomes within range [0, ~0.5].
- avsresize after rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", scene_referred=true) the output is within range [0, ~3.188]. After applying RGBAdjust values of 0.157, the output becomes within range [0, ~0.5].