Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th September 2021, 23:52   #1  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
HDR ColorBars - bug at transitions ?

It looks I find fun bug with HDR but still can not understand how to fix it to get ColorBars test pattern in HDR transfer encoded.

Code:
LoadPlugin("plugins_JPSDR.dll")
LoadPlugin("fmtcavs.dll")

ColorBarsHD(9600,1000, pixel_type="YUV444P16")
Crop(0,0,9600,500)
AddBorders(100,100,100,100)
#fmtc_matrix(mats="709", matd="rgb", fulls=false, fulld=false, csp="RGBP16")
#ConvertToRGB48(matrix="PC.709")
#ConvertToPlanarRGB()
#fmtc_transfer(transs="709", transd="linear", bits=16, flt=false, fulls=false, fulld=false)

#ConvertYUVtoLinearRGB(Color=2, OutputMode=2)
ConvertYUVtoLinearRGB(Color=2)

GaussResize(width/10,height/3, p=2)
# Linear RGBfloat planar in out size 


#ConvertToPlanarRGB()
#fmtc_transfer(transs="linear", transd="2084",bits=16, flt=false, fulls=false, fulld=false)

#ConvertToYUV444().Tweak(sat=0)
#return ConvertBits(8).ConvertToRGB24()



#ConvertToYUV444()
#ConvertLinearRGBtoYUV(Color=2)
ConvertLinearRGBtoYUV(Color=0, HDRMode=0) # PQ
#ConvertLinearRGBtoYUV(Color=0, HDRMode=2) # HLG
# YUV 444 
#Tweak(sat=0)
return ConvertBits(8).ConvertToRGB24()
It looks like appear in both jpsdr's linear to HDR convertor and in fmtconv plugin (with fmtc_transfer()). But I still can not understand how in appear in simple transfer operation in 4:4:4.

The bug looking - white lines at colour transitions of ColorBars pattern. And the width of white line depends on the data transition between colors (if lower GaussResize p-param the transient become wider and bug line too).
And it happens only with HDR transfers and not old bt.709 (or may be simple as low as invisible). In HDR transfers mostly visible at green-magenta transition.

Results:


If process only Y black and white - the bug not happens. So I assume it depends on RGB<->YUV calculations and for some reason occur even with 4:4:4 without even going to sub-sampled colour and back.
Maybe something wrong with luma/chroma placement from ColorBarsHD() source and it become amplified when transformed to much more non-linear transfer like PQ (in compare with gamma about 2 at bt.709).
DTL is offline   Reply With Quote
Old 16th September 2021, 04:33   #2  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
I think it's because you didn't adjust primaries;

zimg (avsresize) works ok, and you get similar artifact if you don't adjust primaries

Code:
ColorBarsHD(9600,1000, pixel_type="YUV444P16")
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:709:f")

GaussResize(width/10,height/3, p=2)

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:709:f=>2020ncl:st2084:2020:l")

#don't adjust primaries for 2020
#z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:709:f=>2020ncl:st2084:709:l")

#tweak(sat=0)

#convert to 8bit srgb preview
z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:2020:l=>rgb:srgb:709:f")

#convert to 8bit srgb preview 709 primaries
#z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:709:l=>rgb:srgb:709:f")
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 11:51   #3  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Oh - you almost save the world. But it looks either my misunderstanding of avsresize conversions or something strange with z_convert format:

Code:
LoadPlugin("avsresize.dll")

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

#z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:709:f")
z_convertformat(pixel_type="rgbp", colorspace_op="709:709:709:l=>rgb:linear:2020:l") # attempt to have primaries in 2020 before conditioning - no help

GaussResize(width/10,height/10, p=10)

#testing source
#return Crop(300,0, width-300, height).SincResize(width*4, height*4, taps=16).ConvertToRGB24(matrix="PC.709")

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")
#PQ HDR here

# attempt to resize in PQ HDR transfer
Crop(300,0, width-300, height).SincResize(width*4, height*4, taps=16)
return z_convertformat(pixel_type="yuv444p16", colorspace_op="2020ncl:st2084:2020:l=>709:709:709:l").ConvertToRGB24(matrix="PC.709")

# attempt to resize in linear
  rgb_lin=z_convertformat(pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincResize(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  z_convertformat(rgb_lin, pixel_type="YUV444P16", colorspace_op="rgb:linear:2020:f=>709:709:709:l")

return Crop(300,0, width-300, height).ConvertToRGB24(matrix="PC.709")
When I try to test linear scaling of the produced by z_convertformat PQ HDR output it cause severe vertical ringing (on black border, though colour patches almost clear - that is strange), but the source linear RGB before convert to PQ HDR and back to linear was good enough conditioned:


Also the ringing is colored so I assume in something inside z_convertformat at primaries/matrix/transfer conversion happens. May be need more arguments in conversions to switch z_convertformat logic to convert in 'linear' ? May be avsresize can not completely revert HDR transfer and still leave OOTF part of transform and it distorts data before sinc scaling ? Though OOTF gamma is not very great so I do not think it can cause so significant distortions.

When trying to resize in HDR PQ domain the ringing swtches to colour patches from black border.
DTL is offline   Reply With Quote
Old 16th September 2021, 13:31   #4  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Well - the only that works as it expected is combination of avsresize and HDRtools:

For PQ to 709:
Code:
  rgb_lin=z_convertformat(pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincResize(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  yuv_out=ConvertLinearRGBToYUV(rgb_lin,Color=2)
Same is for HLG input with changed colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:f" at z_convertformat. So it looks linear output from avsresize is enough linear for sincresize. But may I miss some options for back convert from linear to bt.709/srgb for monitoring ?
DTL is offline   Reply With Quote
Old 16th September 2021, 13:38   #5  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
Quote:
Originally Posted by DTL View Post
Code:
LoadPlugin("avsresize.dll")

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

#z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:709:f")
z_convertformat(pixel_type="rgbp", colorspace_op="709:709:709:l=>rgb:linear:2020:l") # attempt to have primaries in 2020 before conditioning - no help

GaussResize(width/10,height/10, p=10)

#testing source
#return Crop(300,0, width-300, height).SincResize(width*4, height*4, taps=16).ConvertToRGB24(matrix="PC.709")

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")
#PQ HDR here

# attempt to resize in PQ HDR transfer
Crop(300,0, width-300, height).SincResize(width*4, height*4, taps=16)
return z_convertformat(pixel_type="yuv444p16", colorspace_op="2020ncl:st2084:2020:l=>709:709:709:l").ConvertToRGB24(matrix="PC.709")

# attempt to resize in linear
  rgb_lin=z_convertformat(pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincResize(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  z_convertformat(rgb_lin, pixel_type="YUV444P16", colorspace_op="rgb:linear:2020:f=>709:709:709:l")

return Crop(300,0, width-300, height).ConvertToRGB24(matrix="PC.709")
Replacing sinc kernel with another or using lower taps gives good result.
StvG is offline   Reply With Quote
Old 16th September 2021, 15:57   #6  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
I assume good quality moving pictures display must be based on simple sinc kernel and enough number of taps to restore most of high valid frequencies. Unfortunately using simple sinc without any adjustments works only for luma but not for chroma (it need to be de-ringed at final end / display point with current design of common-use digital video systems).

And 'production' workflow must provide data, which do not cause (significant visible) ringing with (infinite taps) sinc interpolators.
DTL is offline   Reply With Quote
Old 16th September 2021, 16:06   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by DTL View Post
Well - the only that works as it expected is combination of avsresize and HDRtools:

For PQ to 709:
Code:
  rgb_lin=z_convertformat(pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincResize(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  yuv_out=ConvertLinearRGBToYUV(rgb_lin,Color=2)



Not really

1) you're not "test linear scaling of the produced by z_convertformat PQ HDR output it cause severe vertical ringing" - because now you're now performing the sinc scaling step in RGB linear, whereas before you were testing in PQ HDR YUV444P16.

2) The bars are way off in color now ; pay attention to what color they are supposed to be, not just the artifacts

Proper way to do this is perform the scaling steps in RGB linear
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 16:49   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
1) Is the vapoursynth placebo shader sinc kernel the same as avs internal sinc? or fmtc sinc? There seem to be fewer errors, more accurate colors , even when feeding the YUV444P16 avs input to perform the sinc scaling step in vapoursynth

2) Same with fmtc r22 (r24 has a srgb transfer issue, there was a fix commit, but no new binary). sinc kernel seems more accurate

3) Same with avsresize in YUV/RGB conversion - seems to be less accurate in avs version. But both use zimg library, you'd expect same results

e.g. 8bit sRGB converted "red" should 191,0,0 . AVS version is 188,0,0. VPY version is 191,0,0

Maybe StvG can look into it because he can compile both



AVS, Sinc scaling step in RGB linear float
Code:
ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

SincResize(last.width*4, last.height*4, taps=16)

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")

#8bit sRGB Preview
z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:2020:l=>rgb:srgb:709:f")
note "red" is RGB is 188,0,0, "mg" is 187,0,189 - ie. everything a bit off - but absense of large ringing artifacts


AVS, Sinc scaling step in YUV444P16
Code:
ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")

SincResize(last.width*4, last.height*4, taps=16)

#8bit sRGB Preview
z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:2020:l=>rgb:srgb:709:f")
Same, but ringing artifacts


AVS YUV444P16 input script into VPY, to check VPY sinc and RGB conversions
Code:
ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")

VPY, avs input, sinc scaling step in YUV444P16
Code:
clip = core.avisource.AVISource(r'avs_input.avs')

#clip = core.placebo.Resample(clip, width=clip.width*4, height=clip.height*4, filter ="sinc", param1=16)
clip = core.fmtc.resample(clip, w=clip.width*4, h=clip.height*4, kernel="sinc", taps=16)

#8bit RGB preview
clip = core.resize.Point(clip, format=vs.RGB24, matrix_in_s="2020ncl", matrix_s="rgb", transfer_in_s="st2084", transfer_s="709", primaries_in_s="2020", primaries_s="709")

clip.set_output()
No large ringing with either placebo.Resample sinc (not sure if taps are correct with param1), or fmtc

Colors are slightly more accurate, e.g. "red" is 191,0,0, "mg" is 191,0,192
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 18:51   #9  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
"AVS, Sinc scaling step in RGB linear float
z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l")
GaussResize(width/10,height/10, p=10)
SincResize(last.width*4, last.height*4, taps=16)"
"

Here I got what cause misunderstanding. My work is to create ColourBars HDR in distribution format (that is 4:2:2 SDR/HDR PQ/HLG 10bit) but to test if it is correct I need some 'ideal moving pictures display simulator' - So the sinc-upscaling must be performed with (after receiving) 'distribution' data. The GaussResize is member of production side just to condition transients of the typical awful output of ColorBars(HD). In the actual work I use not gauss and sinc but userdefined2resizemt and sinclin2resizemt but close results may be shown with built-in avisynth resizers and do not introduce more bugs from external to avisynth plugins.

So the workflow to test:

Production:ColorBarsHD->(conditioning_in_linear_domain and generate output size)->convert_to_HDR_PQ(primaries,transfer,matrix)->standard(sharp_cut-off_chromaLPF)_subsample_to_4:2:2->Bitdepth_reduction_to10->Release.

Quality control: Release->Convert_to_4:4:4(monitorLPF_slow_roll-off for UV)->Bitdepth_upto_16->Convert_to_linear(transfer,primaries(?),matrix(?))->4:4:4 Sinc upscale (to some larger size like 4x)->Put_to_display(convert to sRGB/709 (primaries,transfer,matrix) for standard pixel display feed).

Used standard 4:4:4 to 4:2:2 conversion is simple and equal for all versions with sharp sincresize cut-off in UV:
Code:
Function ConvertTo422p16(clip c)
{
  yuv444=ConvertToYUV444(c)
  uc=UToY(yuv444)
  vc=VToY(yuv444)
  uc=SincLin2ResizeMT(uc,src_left=0, uc.width/2, uc.height, taps=16)
  vc=SincLin2ResizeMT(vc,src_left=0, vc.width/2, vc.height, taps=16)
  return CombinePlanes(yuv444, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV422P16")
}

My current 'quality_control' functions for 709/HDR_PQ/HDR_HLG is
Code:
Function Convert422ToRGB24mon_lin_x4_709(clip c)
{
  uc=UToY(c)
  vc=VToY(c)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=95, c=-10)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=95, c=-10)
  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=125, c=18)
  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=125, c=18)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=80, c=-20)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=80, c=-20)
  uc=SincLin2ResizeMT(uc, src_left=0, uc.width*2, uc.height, taps=16)
  vc=SincLin2ResizeMT(vc, src_left=0, vc.width*2, vc.height, taps=16)
  yuv444=CombinePlanes(c, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV444P10")
  rgb_lin=ConvertYUVToLinearRGB(yuv444,Color=2)
  rgb_lin=SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  yuv_out=ConvertLinearRGBToYUV(rgb_lin,Color=2)
  return yuv_out.ConvertToRGB24(matrix="PC.709")  
}

Function Convert422ToRGB24mon_lin_x4_PQ(clip c)
{
  uc=UToY(c)
  vc=VToY(c)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=95, c=-10)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=95, c=-10)
  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=125, c=18)
  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=125, c=18)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=80, c=-20)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=80, c=-20)
  uc=SincLin2ResizeMT(uc, src_left=0, uc.width*2, uc.height, taps=16)
  vc=SincLin2ResizeMT(vc, src_left=0, vc.width*2, vc.height, taps=16)
  yuv444=CombinePlanes(c, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV444P10")
  rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  yuv_out=ConvertLinearRGBToYUV(rgb_lin,Color=2)
return yuv_out.ConvertToRGB24(matrix="PC.709")  
}

Function Convert422ToRGB24mon_lin_x4_HLG(clip c)
{
  uc=UToY(c)
  vc=VToY(c)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=95, c=-10)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=95, c=-10)
  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=125, c=18)
  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=125, c=18)
#  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=80, c=-20)
#  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=80, c=-20)
  uc=SincLin2ResizeMT(uc, src_left=0, uc.width*2, uc.height, taps=16)
  vc=SincLin2ResizeMT(vc, src_left=0, vc.width*2, vc.height, taps=16)
  yuv444=CombinePlanes(c, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV444P10")
  rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:f")
  rgb_lin=SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
  yuv_out=ConvertLinearRGBToYUV(rgb_lin,Color=2)
return yuv_out.ConvertToRGB24(matrix="PC.709")  
}
The task of pereserving exact RGB codevalues is not main. The main task is to check presence and possible quality of required colour-difference anti-ringing filter in the control monitor (being feed with 'Release' dataset). So some errors in exact code values of colour patches are acceptable. More important is relative position of Y and UV to make colour transients as great as possible. Currently I not very happy with some black gaps around blue patch for example. My current idea - the build-in ColorBars(HD) datasource produces not very perfect data and may be it is good to start from RGB manually designed in MS-Paint and it allow to adjust position of RGB levels transients in each RGB channel down to 0.1 in-between_samples step independently when using /10 downsampling at production. May it can help to get more perfect transients.

"note "red" is RGB is 188,0,0, "mg" is 187,0,189 - ie. everything a bit off - but absense of large ringing artifacts"

It is just testing of the quality of transients of source at 'Production' side - to proof if it generate good conditioned dataset and we have a chance to got some close quality at end of chain side after we create 4:2:2 HDR PQ or HLG 10bit release dataset (compressed in bitdepth and spatially in colour-difference).

"AVS, Sinc scaling step in YUV444P16
colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l"
Same, but ringing artifacts
"

Here it not only YUV444P16 but also transfer converted from linear to PQ-domain, so it losts conditioning (prepared in linear domain) and starts to ring with sinc-resize.

"VPY, avs input, sinc scaling step in YUV444P16"

It is also not only YUV444P16 but transfer converted to PQ-domain (in the end of avisynth script with z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")) so sinc-resize in PQ-domain will ring. To simulate correct display-side scaling transform back to linear domain required.

Last edited by DTL; 16th September 2021 at 19:31.
DTL is offline   Reply With Quote
Old 16th September 2021, 19:17   #10  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by DTL View Post
The task of pereserving exact RGB codevalues is not main. The main task is to check presence and possible quality of required colour-difference anti-ringing filter in the control monitor (being feed with 'Release' dataset). So some errors in exact code values of colour patches are acceptable. More important is relative position of Y and UV to make colour transients as great as possible. Currently I not very happy with some black gaps around blue patch for example. My current idea - the build-in ColorBars(HD) datasource produces not very perfect data and may be it is good to start from RGB manually designed in MS-Paint and it allow to adjust position of RGB levels transients in each RGB channel down to 0.1 in-between_samples step independently when using /10 downsampling at production. May it can help to get more perfect transients.
ok... but when the code values are +/- 30 off, it's a big deal (!!!) . +/- 3 in 8bit is acceptable tolerance. The main reason for colorbars in the first place are for... color accuracy

The primaries are not adjusted in ConvertLinearRGBToYUV, I think that is the main reason for the large color errors in post #4. But how do you know the errors that you see are not a cascade of errors effect ? eg. The primaries omission was the cause for the 1st error in the 1st post

The above post demonstrates 16tap sinc works ok in YUV444P16 in vapoursynth fmtc and resize.Shader without the large ringing seen avisynth. So maybe it's an avisynth sinc bug? Or maybe the sinc implmentation is not the same

I point out differences in zimg, because how do you know those errors upstream are not cascading and producing larger errors in avisynth downstream when you use sinc or anything else ? It's the same library, but the same operation (e.g. omit all scaling) produces different results
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 20:06   #11  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
"The main reason for colorbars in the first place are for... color accuracy"

In the target testing the colour accuracy is handled with separate large number of colour full-screen patches. This colorbar bar is added only to test scaling (in colour mode, not only black and white). The shape of colorbars pattern is just something containing colour transients with high amplitude colour-difference data walking and so if no required filtering applied and depends on the monitor scaler it may shows (small) ringing. Unfortunately at the awfull HDR epoch and with freeware tools it still better in compare with nothing. If there were any table of the HDR colorbars (of some exact type) in PQ and HLG (of some exact white/black and other variables) it can be compared in the release 4:2:2 10bit files for total code value error. But I think non of official tables still exist. Having perfect decoded levels accuracy is sort of 'nice to have' feature for now I think.

"maybe the sinc implmentation is not the same"

Yes - there at least 2 ways possible:
1. anti-ringing pre-filtering before sinc (not clear sinc so it will show fading of frequency responce at highest valid frequencies and a bit less sharpness)
2. it really do not scale in the input HDR-transfer but perform converting to linear inside. Though it need to know the current transfer-domain of the input content to perform converting to linear and back.

" without the large ringing seen avisynth."

Avisynth sinc mostly probably do not perform non-requested conversions to linear domain (and back) and not perform pre-filtering so if input content is not anti-ringing conditioned it will expose full ringing.

Last edited by DTL; 16th September 2021 at 20:35.
DTL is offline   Reply With Quote
Old 16th September 2021, 21:51   #12  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by DTL View Post
If there were any table of the HDR colorbars (of some exact type) in PQ and HLG (of some exact white/black and other variables) it can be compared in the release 4:2:2 10bit files for total code value error. But I think non of official tables still exist. Having perfect decoded levels accuracy is sort of 'nice to have' feature for now I think.
Vapoursynth colorbars are exact, because they are synthetically generated according to SMPTE RP 219-1, 219-2 and ITU-R BT.2111-0 . The diagram and exact 10,12 bit code values are listed in the later for PQ narrow/full and HLG

https://github.com/ifb/vapoursynth-colorbars

You can import vpy scripts in to avs for many pixel types with VSImport("script.vpy")

Quote:

"maybe the sinc implmentation is not the same"

Yes - there at least 2 ways possible:
1. anti-ringing pre-filtering before sinc (not clear sinc so it will show fading of frequency responce at highest valid frequencies and a bit less sharpness)
2. it really do not scale in the input HDR-transfer but perform converting to linear inside. Though it need to know the current transfer-domain of the input content to perform converting to linear and back.

" without the large ringing seen avisynth."

Avisynth sinc mostly probably do not perform non-requested conversions to linear domain (and back) and not perform pre-filtering so if input content is not anti-ringing conditioned it will expose full ringing.

It does not look like they do, taking a quick look at the code. But I might have missed something

If you replace internal SincResize with

fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

it does not exhibit the large ringing patterns either


avs internal code I think is this

https://github.com/AviSynth/AviSynth..._functions.cpp

Code:
]
/***********************
 *** Sinc filter ***
 ***********************/
SincFilter::SincFilter(int _taps) {
   taps = (double)clamp(_taps, 1, 20);
}

double SincFilter::f(double value) {
   value = fabs(value);

  if (value > 0.000001) {
    value *= M_PI;
    return sin(value)/value;
  } else {
    return 1.0;
  }
}
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 22:46   #13  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
"ITU-R BT.2111-0 . The diagram and exact 10,12 bit code values are listed in the later for PQ narrow/full and HLG"

Well - it is good to see. But the tables listed only R'G'B' values - not YUV in actual distribution formats. For manual checking in hex-editor in the exported 4:2:2 raw files.
Also to be mentioned - in the more and more degrading world the https://www.itu.int/dms_pubrec/itu-r...2-I!!PDF-E.pdf looks like finally missed completely description about transients between levels. The old standards for HD and SD colorbars at least put some words about it. If I have time I will try to make R'G'B' to YUV table to check for levels in the 4:2:2 YUV files.
Also tables list only sustained code values and can not cover the transients because of too much numbers - so anyway producing of real test patterns require transients conditioning with software-processing and it need to be done in RGB-linear (not in R'G'B' even) and later subsampled to typical 4:2:2.

"If you replace internal SincResize with
fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)
it does not exhibit the large ringing patterns either
"

To check quality of sincing I use 1-sample pattern (like 235 code value at 16 field or 200 at 30 or like this to see result at any sRGB-full monitor without additional black-raising).
PNG file is

Code:
LoadPlugin("fmtcavs.dll")

ImageReader("1-sample.png", pixel_type="RGB24")

#SincResize(width*4, height*4, taps=16)
ConvertToPlanarRGB()
fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)
fmtc_bitdepth(bits=8)

ConvertToRGB24()

Levels(0,1,60,0,255)
I do not see any significant difference in kernel response between avisynth built-in sincresize and fmtc_convert(sinc)


They looks like match close to bitexact:
Code:
avs=SincResize(width*4, height*4, taps=16)
fmtc=ConvertToPlanarRGB()
fmtc=fmtc_resample(fmtc,w=last.width*4, h=last.height*4, kernel="sinc", taps=16)
fmtc=fmtc_bitdepth(fmtc,bits=8)

fmtc=ConvertToRGB24(fmtc)
Subtract(avs,fmtc)
Produces flat grey field.
So may be some other magic work somewhere. Though I made black and white Y-data test. May be with colour data results are different.

Last edited by DTL; 16th September 2021 at 22:56.
DTL is offline   Reply With Quote
Old 16th September 2021, 23:01   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by DTL View Post

"If you replace internal SincResize with
fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)
it does not exhibit the large ringing patterns either
"

They looks like match close to bitexact:
That comment was referring to the earlier example in YUV444P16. Check internal SincResize vs. FMTC. Why such a big difference ?


Code:
ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

#SincResize(last.width*4, last.height*4, taps=16)
fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

#8bit sRGB Preview
z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:2020:l=>rgb:srgb:709:f")
poisondeathray is offline   Reply With Quote
Old 16th September 2021, 23:27   #15  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
" Check internal SincResize vs. FMTC. Why such a big difference ?"

The provided script do not work for me to control in VirtualDub for some reason. But I made return after sinc proc
Code:
LoadPlugin("avsresize.dll")
LoadPlugin("fmtcavs.dll")

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(300,300,300,300) # here enlarge from 100 to 300.

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

SincResize(last.width*4, last.height*4, taps=16)
#fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

return ConvertToRGB24(matrix="PC.709")
and see the difference between sincresize and fmtc at the top/bottom borders and frame left and right ringing. I think it is because of different edge-workarounds in different resamplers.
Current black borders are 100/10=10 and taps=16 > width of border.

When I enlarge border to AddBorders(300,300,300,300) the SincResize gives also non-ringing result. It may be reported as sort of issue to avisynth development - may be as another fix to resampler engine. Though it may allow additional resources and slow-down processing. Typically users do not look at the very edge of frame and center of frame processed well.

As for the lowest size of required black border: A.5 of ARIB-STD-B28 Transient
The number of samples to be used for the transient shall be 6 to 9, in the case of 1920
horizontal samples, although it may depend upon the scale of hardware, process performance
and the so-called “make up”. (It was about 20 years old when degradation of industry was lower in compare with todays)

So I hope 10 is enough to have up to 9 samples symmetrical to useful data.

Last edited by DTL; 16th September 2021 at 23:50.
DTL is offline   Reply With Quote
Old 17th September 2021, 09:32   #16  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 301
Quote:
Originally Posted by poisondeathray View Post
e.g. 8bit sRGB converted "red" should 191,0,0 . AVS version is 188,0,0. VPY version is 191,0,0

Maybe StvG can look into it because he can compile both
I cannot reproduce the difference.

Are you sure you used the very same conversion for both AVS and VS? You have destination transfer "srgb" in the final step for AVS but for VS it's "709".


Quote:
Originally Posted by DTL View Post
When I enlarge border to AddBorders(300,300,300,300) the SincResize gives also non-ringing result. It may be reported as sort of issue to avisynth development - may be as another fix to resampler engine. Though it may allow additional resources and slow-down processing. Typically users do not look at the very edge of frame and center of frame processed well.
The visible difference is not only at the very edge of the frame.

Internal avs sinc:


fmtc_resample(kernel="sinc")


Click for the full size images. Both examples are from 16-bit processing.
StvG is offline   Reply With Quote
Old 17th September 2021, 09:58   #17  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Quote:
Originally Posted by poisondeathray View Post
That comment was referring to the earlier example in YUV444P16. Check internal SincResize vs. FMTC. Why such a big difference ?


Code:
ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(100,100,100,100)

z_convertformat(pixel_type="rgbps", colorspace_op="709:709:709:l=>rgb:linear:2020:l") 

GaussResize(width/10,height/10, p=10)

#SincResize(last.width*4, last.height*4, taps=16)
fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

#8bit sRGB Preview
z_convertformat(pixel_type="rgbp", colorspace_op="2020ncl:st2084:2020:l=>rgb:srgb:709:f")
At the morning I see the error - the first z_convert if from 709 to linear but last z_convert is from st2084 to srgb. So it produce black or grey field.
After correction I see the SincResize really produced additional ringing from the edges even with 300 initial borders and it is colored. FMTC is clear. Will try to report this issue but I unsure if the resampler of avisynth can be fixed quickly - it is very complex and highly asm-optimized. And FMTC may use slower simple float logic.
The conversion to linear and back only shows the bug better.
It also may be seen without converting to linear and back:
Code:
LoadPlugin("fmtcavs.dll")

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(300,300,300,300)

GaussResize(width/10,height/10, p=10)

SincResize(last.width*4, last.height*4, taps=16)
#fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

ConvertBits(8)

Levels(0,1,20,0,255).Crop(1000,0,width-1000, height).ConvertToRGB24(matrix="PC.709")

Last edited by DTL; 17th September 2021 at 10:12.
DTL is offline   Reply With Quote
Old 17th September 2021, 14:26   #18  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by StvG View Post
You have destination transfer "srgb" in the final step for AVS but for VS it's "709".
Thanks, nice catch - they match
poisondeathray is offline   Reply With Quote
Old 17th September 2021, 14:33   #19  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,984
Quote:
Originally Posted by DTL View Post
At the morning I see the error - the first z_convert if from 709 to linear but last z_convert is from st2084 to srgb.
Sorry, I copied /pasted incorrectly


There are other differences in the result using placebo shader implementation of sinc also

What is the expected result in other programs, say matlab ?
poisondeathray is offline   Reply With Quote
Old 17th September 2021, 22:38   #20  |  Link
DTL
Registered User
 
Join Date: Jul 2018
Posts: 551
Because of still existing bugs in avisynth resampler I made temporal workaround function SafeSincResize() . It auto-adds padding and crops after resize. But only works with sources with already having some black borders.

Now it looks output of both fmtc_resample(sinc) and SafeSincResize is identical as I see after Subtract()

Code:
LoadPlugin("avsresize.dll")
LoadPlugin("fmtcavs.dll")

Function SafeSincResize(clip c, int width, int height, int taps)
{
  xratio = width/c.width
  yratio = height/c.height
  tt=2*taps
  c=AddBorders(c,tt,tt,tt,tt)
  c=SincResize(c, c.width*xratio, c.height*yratio, taps=taps)
  return Crop(c,tt*xratio,tt*yratio,width,height)
}

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(200,200,200,200)

z_convertformat(pixel_type="rgbp", colorspace_op="709:709:709:l=>rgb:linear:2020:l")

GaussResize(width/10,height/10, p=10)

z_convertformat(pixel_type="yuv444p16", colorspace_op="rgb:linear:2020:l=>2020ncl:st2084:2020:l")
#PQ HDR here

  rgb_lin=z_convertformat(pixel_type="rgbps", colorspace_op="2020ncl:st2084:2020:l=>rgb:linear:2020:l")
  rgb_lin_ss=SafeSincResize(rgb_lin, rgb_lin.width*4, rgb_lin.height*4, 16)
  rgb_lin_fmtc=fmtc_resample(rgb_lin,w=last.width*4, h=last.height*4, kernel="sinc", taps=16)
  out_ss=z_convertformat(rgb_lin_ss, pixel_type="YUV444P16", colorspace_op="rgb:linear:2020:f=>709:709:709:l")
  out_fmtc=z_convertformat(rgb_lin_fmtc, pixel_type="YUV444P16", colorspace_op="rgb:linear:2020:f=>709:709:709:l")

Subtract(out_ss, out_fmtc)

return ConvertToRGB24(matrix="PC.709")

"There are other differences in the result using placebo shader implementation of sinc also"

Can you show example of that shader output ? I do not have vapoursynth.

Also I found more execution non-stability of SincResize at different CPUs ans OS version - at about Core2 6400 CPU and Win 7 https://i3.imageban.ru/out/2021/09/1...d58ca4e007.png and at i5-9600 and Win10 https://i4.imageban.ru/out/2021/09/1...6c2093b402.png .

"the expected result in other programs, say matlab ?"

In theory sinc interpolation is very simple sum of weighted by input samples sincs and have to match between different programs with equal size of kernel. May be some difference because of different order of V and H passes at 2D simulated processing as V+H 1D+1D for speed instead of 2D one pass. But 2D one pass will produce significant different results (assuming sinc_2d=sinc(radius)) and will be slower in speed.

"The visible difference is not only at the very edge of the frame."

It looks like poor of high frequences source. Can you process some test pattern like hyperbolic zoneplate ? It linear raw calculator is https://github.com/DTL2020/hpzp or converted to 709 transfer h264 encoding is https://cloud.mail.ru/public/h1kR/o56qALaK5

Last edited by DTL; 17th September 2021 at 23:01.
DTL is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.