AviSynth+ thread Vol.2

[DTL]

Based on note in https://tech.ebu.ch/docs/techreview/...rec601_bbc.pdf
More satisfactory was the consideration of chrominance filtering which recognized the need for
sharp-cut filters at all conversions except the last one. This allowed the bandwidth needed for
chroma-key and other processing to be retained through the system; however, the inclusion of a
slow roll-off in the composite coder for analogue broadcasts or in the picture monitor for direct
component signals avoided the dreadful chrominance ringing that would otherwise occur.

It looks current ConvertToRGB() and all other chroma-upsampling Convert() like from 4:2:0 and 4:2:2 to 4:4:4 need to be supplemented with additional option like 'in-between' conversion or 'last/display' conversion. Or may be at least Convert() filter documentation need to be supplemented with this feature of digital sampled moving pictures systems design (from rec.601 and may be up to latest HDR).
I sometime use ConvertToRGB24() before checking output result in VirtualDub and it creates some distorted output because for 'last' conversion the UV planes need to be filtered with some low-pass slow-roll-off filter to fix posible colour-difference source of ringing.
Or special function need to be used for 'monitor conversion to RGB' like (pseudo-functions)

u_plane=ExtractU()
v_plane=ExtractV()
u_plane=Blur(u_plane,some_param) or GeneralConvolution(u_plane,some_param) or special ColourDifferenceLastMonitorProc(u_plane)
v_plane=Blur(v_plane,some_param) or GeneralConvolution(v_plane,some_param) or special ColourDifferenceLastMonitorProc(v_plane)
yuv_back=CombinePlanes(last,u_plane,v_plane)
ConvertToRGB(yuv_back)

[LigH]

Do you mean something like the "coring" parameter of some AviSynth functions (disabling the clamping to TV scale in YUV modes)?

[DTL]

Originally Posted by LigH

Do you mean something like the "coring" parameter of some AviSynth functions (disabling the clamping to TV scale in YUV modes)?

I made example of working conversion functions to show the difference in the post https://forum.doom9.org/showthread.p...73#post1952273 .

Also the post https://forum.doom9.org/showthread.p...13#post1952213 have link to updated hand-drawing schematic of different sub-sampled to 4:4:4 decoding - for final/control displaying and for intermediate processing.

Because the exact 'anti-ringing' filter in display/final transform looks like not standard-defined it also possible to have 'colour-sharpness' adjustment at 422(420)ToRGB transform to make adjustments between 'film-looking' with smooth transients and 'video-look' with peaked/overshooted.

In that example I use UserDefined2ResizeMT() with src_left !=0 to enable filter-processing (convolution with kernel) without actual resizing of UV channels.

[DTL]

Oh - again some (minor) complain for Avisynth resampler:

LoadPlugin("fmtcavs.dll")

ColorBarsHD(9600,1000, pixel_type="YV24")
ConvertBits(16) # YUV 444 16
Crop(0,0,9600,500)
AddBorders(300,300,300,300)

GaussResize(width/10,height/10, p=10)

SincResize(last.width*4, last.height*4, taps=16)
#fmtc_resample(w=last.width*4, h=last.height*4, kernel="sinc", taps=16)

ConvertBits(8)

Levels(0,1,20,0,255).Crop(1000,0,width-1000, height).ConvertToRGB24(matrix="PC.709")

It produces additional (and colored) ringing-like lines (though it constant not-faded amplitude and different frequency) from the very edges of the frame.
Image: https://i3.imageban.ru/out/2021/09/1...d58ca4e007.png

When trying to process in 'linear light' and convert to and from modern highly-non-linear TFs like HDR the bug significally increases in amplitude.

The kernel's response between SincResize and fmtc(sinc) checked and looks like equal. So I assume it is another issue with edge-workarounds in the avisynth's resampler.

The fmtc(sinc) frame edges is clear (there is some minor ringing from the useful data transitions but it correct).

SincLin2ResizeMT in jpsdr's plugin also affected - same resampler core.

LanczosResize(taps=16) output clear.

[pinterf]

Originally Posted by DTL

It produces additional (and colored) ringing-like lines (though it constant not-faded amplitude and different frequency) from the very edges of the frame.
Image: https://i3.imageban.ru/out/2021/09/1...d58ca4e007.png

The kernel's response between SincResize and fmtc(sinc) checked and looks like equal. So I assume it is another issue with edge-workarounds in the avisynth's resampler.

Those lines are independent from the bit depth. Unfortunately I don't really even know what to look for.

[DTL]

The top and bottom buggy lines exist only if top and bottom 'borders' (in the AddBorders()) < 320 (320 few lines, 300 and less full number). If it is 'edge computational bug of pure sinc' it should disappear with moving to SincLin2 weighting of the edge of sinc kernel - but it only shifts buggy lines a bit. Also it absent with fmtc resampler without sinc kernel modification.

With 'borders' 300 it starts (around colour patches) from taps=11, no with taps=12, taps=13 yes, 15 yes - more height, taps=18 - yes, moves to the edges of frame.

With SincResizeMT with possibly extended taps value (to 100..150) it looks dissappear after taps >35..40 (when the edges of kernel fades below some max abs value). Also Lanczos weighting have much more suppression of the wider parts of sinc kernel edges so it do not appear.

So it looks interconnected with 'edges of sinc features' but in fmtc it possibly have workaround (for edges of frame processing) for any taps value and without kernel additional weighting. Though fmtc(sinc) do have 'standard sinc edge issue' in any position inside frame - as shown in https://forum.doom9.org/showthread.p...33#post1950633 .

It looks the buggy lines depends on distance between useful data and buffer border. If useful data is closer about taps*2 from buffer edge - there is a chance of bug (may be also in some range of taps param like 11..30). So as temporal workaround it may be recommended to pad useful buffer with borders >2*taps size. And crop after processing.

Addition: I test at different CPU and Win10 and the script above (borders '300' and taps 16) produces much smaller number of bright buggy lines.
Image: https://i4.imageban.ru/out/2021/09/1...6c2093b402.png
It looks depends on memory or CPU SIMD. Though at my old home CPU (about Core-2 6400) I tried SetMaxCPU('none') and it changes nothing.

I make temporal workaround function for auto-padding and cropping:

Function SafeSincResize(clip c, int width, int height, int taps)
{
  xratio = width/c.width
  yratio = height/c.height
  tt=2*taps
  c=AddBorders(c,tt,tt,tt,tt)
  c=SincResize(c, c.width*xratio, c.height*yratio, taps=taps)
  return Crop(c,tt*xratio,tt*yratio,width,height)
}

Same should work with SincLin2ResizeMT(). Though the source must have some black borders already to not introduce additional stepping with AddBorders(color=black).

[qyot27]

Okay, so I'm going to need those of you that are really invested in the particulars of the color representation in AviSynth+ to test this for me, as I don't have nearly the amount of samples needed for this:
FFmpeg
mpv

This build of FFmpeg has a patch that reads some of the frame properties from AviSynth+ and populates them in what I hope is the correct fashion. The mpv build is provided because it's more comfortable than ffplay, and because unlike ffplay, it doesn't require additional filtering steps to work on the additional information (read: mpv knows the script is outputting HDR and can tonemap it automatically for SDR displays if that's how you have it set up).

Will require one of the test builds pinterf provided from after April, since there had been a bug in 3.7.0 (and probably the earlier versions that were also inside interface version 8) preventing the C interface from accessing frame properties; if you use 3.7.0, it will crash*. You'll also need a source filter that sets frame properties (I used the test build of ffms3000); propSetInt does work to force the properties to a given value, but that's artificial and I want more real-world examples just in case there's a problem with how the equivalencies between the frame props and FFmpeg's color enums were set up.

*this will be resolved in the final version of the patch, since it's going to require bumping to interface version 9 just to make the detection in FFmpeg work correctly (but bumping to version 9 should probably go hand in hand with more new API functions and a way of future-proofing against bugs like this).

What frame properties is it reading?
_FieldBased (the one that doesn't have to do with color; people interested in interlaced video, take note of this one)
_ChromaLocation
_Primaries
_Transfer
_Matrix
_ColorRange

I used the LG New York HDR UHD 4K Demo for verifying that mpv with the patch can pick up and use the information instead of seeing the script as BT.709. But there's obviously a lot more entries under all those different frame properties, and I have no clue if the way I set this up actually can handle all of them the way they're ostensibly supposed to (admittedly, some of that is also up to the source filters setting the correct value and the other filters in the script not screwing it up somehow, but making sure the script->libavformat chain works as intended is the goal here). I also didn't have any interlaced content on hand to really test the field detection on.

[pinterf]

Originally Posted by DTL

The top and bottom buggy lines exist only if top and bottom 'borders' (in the AddBorders()) < 320 (320 few lines, 300 and less full number). If it is 'edge computational bug of pure sinc' it should disappear with moving to SincLin2 weighting of the edge of sinc kernel - but it only shifts buggy lines a bit. Also it absent with fmtc resampler without sinc kernel modification.

With 'borders' 300 it starts (around colour patches) from taps=11, no with taps=12, taps=13 yes, 15 yes - more height, taps=18 - yes, moves to the edges of frame.
...

In last days I spent quite a few hours on the topic and feeling now what the problem is. The relevant part is basically untouched since 2002. It is a new part for me so I need more knowledge gathering on my side.
btw this is what they implemented: http://avisynth.nl/index.php/Resampling

[pinterf]

Meanwhile.
Avisynth+ 3.7.1 test build 17 (20210924)

20210924 WIP
------------
- Expr: allow x.framePropName syntax (Akarin's idea)
  Where x is the usual clip identifier letter, and after the . is the name of the frame property.
  Nonexistent or non-number frame properties return with 0.0 value
  Example (increasing brightness until frame nunmber 255)
    ColorbarsHD()
    ScriptClip("""propset("medi", current_frame)""")
    expr("x.medi","","")

- More checks on array parameters in user defined functions.
  Array-typed parameters with "name" have the value "Undefined" when they are not passed.
  Note: but the value is defined and is a zero-sized array if the parameter is unnamed, like in other Avisynth functions.

Special thanks to real.finder for not allowing me to be too lazy

[DTL]

Originally Posted by pinterf

btw this is what they implemented: http://avisynth.nl/index.php/Resampling

It is described 'common theory' of resampling. Also it is even not very good calls different kernels only as 'different resamplers' - Avisynth have only one resampler engine and it accepts different kernels (named *Resize()) at creation of 'resampling program' for the resampling engine.

Also it have a direct 'bugs' like
"The resampling kernel, sinc(x), is symmetric. That is, sinc(x) = sinc(-x). This means that the samples s(n*T) and s(-n*T) will contribute equally to s(0)."
The main and very important property of sinc(x) that it =1 at sinc(0) and =0 at the all other integer n*pi. That practically means that _samples_ of the data are _independent_ . So no sample of 1D sinc processing do not contribute anything to s(0) (and also to any other sample). But _all_ samples (in theory) contribute to any interpolated (new calculated) samples in-between original input samples. Because in-between sinc(x) !=0.
And all this nice properties of sinc works only for 1D tansform and not work as nice for attempt of 2D processing with rectangular sampling grid because sinc (distance_to_diagonal(angled)_samples*pi) !=0 (and the jinc() is too).

Also it have almost nothing about Gibbs phenomenon and working against ringing. Only mention about sometime ringing occur.

Also that 'common theory' applied to infinite size of buffer (theoretical) and may be not lists the issues with real buffers of non-infinite size at all. Processing these real buffers require special workarounds for resample engines to create less distorted results (at the edges and at some distance from the edge (typical ~=size of kernel, filter support, etc). Part of these workarounds looks like implemented in the 'create resampling program' and part may be in the resampler engine.

"untouched since 2002. "

Now with large dynamic range of HDR and much more curved HDR transfer functions even small bugs at the LSBs of 16bit become highly visible if trying to process HDR in linear and convert back to HDR transfer. With old 8bit and SDR transfers most of small amplitude bugs was about invisible.

[FranceBB]

Originally Posted by qyot27

mpv

Remember the DNX120 green field issue?
Well, I'm glad you compiled a new build of MPV 'cause with your build I could check again and it is actually fixed: https://i.imgur.com/0PGbiB0.png

Cheers

Originally Posted by qyot27

What frame properties is it reading?
_FieldBased (the one that doesn't have to do with color; people interested in interlaced video, take note of this one)
_ChromaLocation
_Primaries
_Transfer
_Matrix
_ColorRange

This is a game changer!! Thank you so much for improving metadata passthrough from Avisynth! I really really really appreciate it!
Testing now but if it works it's gonna make my life so much easier, thank you!!

[Dogway]

pinterf, is it possible to add atan2 to expr. Currently I'm using the convoluted method of ternaries which is very slow, this will come useful for cylindrical color spaces:

atan2

"x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ?"

[Reel.Deel]

Originally Posted by Dogway

pinterf, is it possible to add atan2 to expr.

+1 on that request. I'm currently using Gavino's method:

Originally Posted by Gavino

function atan2(string y, string x) {
# Returns an expression for atan2(y, x) (in range 0-360), corresponding to
#  (x == 0 ? (y > 0 ? 90 : (y < 0 ? 270 : 0)) : atan(y/x)*180/pi + (x < 0 ? 180 : (y < 0 ? 360 : 0)))
# Form of expression chosen to evaluate atan() once only.
  return "("+x+" == 0 ? ("+y+" > 0 ? 90 : ("+y+" < 0 ? 270 : 0)) : " + 
\               "atan(("+y+")/("+x+"))*180/pi + " +
\                   "("+x+" < 0 ? 180 : ("+y+" < 0 ? 360 : 0)))"
}

[pinterf]

Originally Posted by Dogway

pinterf, is it possible to add atan2 to expr. Currently I'm using the convoluted method of ternaries which is very slow, this will come useful for cylindrical color spaces:

atan2

"x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ?"

Probably I could make it work but note that 'atan' will keep being slow because expressions containing trigonometric functions have zero SIMD acceleration, they are executed as plain C interpreted code. Akarin implemented sin and cos acceleration lately for VapourSynth which I have not yet imported into Avisynth.

[tormento]

Originally Posted by pinterf

Probably I could make it work but note that 'atan' will keep being slow because expressions containing trigonometric functions have zero SIMD acceleration, they are executed as plain C interpreted code. Akarin implemented sin and cos acceleration lately for VapourSynth which I have not yet imported into Avisynth.

Can't you use Taylor series? Usually with 5/6 terms you have very good approximation.

[pinterf]

Yes of course this is the way how other functions are implemented. And one should specify the allowed input range and the max error on valid inputs. As it was documented on sin and cos implementations.

[wonkey_monkey]

Originally Posted by Dogway

pinterf, is it possible to add atan2 to expr. Currently I'm using the convoluted method of ternaries which is very slow, this will come useful for cylindrical color spaces:

atan2

"x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ?"

Is that a work in progress? It seems to have a few bugs and oddities.

[tormento]

Originally Posted by pinterf

As it was documented on sin and cos implementations.

And being tan the ratio of them…

[Dogway]

@wonkey_monkey: It's supposed to be final (for 32-bit). I took the atan2() from Wikipedia example.

# Opposing RGB. Like HSV but with opposing warm and cool axis: Luma-RED/GREEN-YELLOW/BLUE
# https://graphics.stanford.edu/~boulos/papers/orgb_sig.pdf
function RGB_to_oRGB (clip RGB, bool fulls) {

    bi = BitsPerComponent(RGB)
    fs = Default (fulls, false)

	# R'G'B' to L'C'C' to
    LCC = [ 0.298967,  0.586421,  0.114612, \
            0.500000,  0.500000, -1.000000, \
            0.866000, -0.866000,  0.000000]

    LCC = MatrixClip(RGB,MatrixTranspose(LCC),"YUV")
    R  = ExtractR(RGB)
    L  = ExtractY(LCC)
    C1 = ExtractU(LCC)
    C2 = ExtractV(LCC)

    G = Expr(C2, C1, R, ex_dlut(Format("x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ? 
                                        O@ pi 0.333333 * < O 1.5 * O pi 0.333333 * >= pi O >= & pi 0.5 * O 0.75 * + 0.785398163 - 0 ? ? O - z * y * "), bi, fs), optSingleMode=false)
    B = Expr(C1, C2, R, ex_dlut(Format("x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ? 
                                        O@ pi 0.333333 * < O 1.5 * O pi 0.333333 * >= pi O >= & pi 0.5 * O 0.75 * + 0.785398163 - 0 ? ? O - z * x * "), bi, fs), optSingleMode=false)
    CombinePlanes(L, G, B, planes="YUV") }


function oRGB_to_RGB (clip oRGB, bool fulls) {

    bi = BitsPerComponent(oRGB)
    fs = Default (fulls, false)

    R = ExtractY(oRGB)
    G = ExtractU(oRGB)
    B = ExtractV(oRGB)

	# L'C'C' to R'G'B'
    RGB = [ 1.0,  0.11461199820041656,  0.7433336973190308, \
            1.0,  0.11461199820041656, -0.4114006757736206, \
            1.0, -0.8853879570960999,   0.16596652567386627]

#	 Evaluating for atan2(C2,C1) since Expr lacks the operator
    G = Expr(B, G, R, ex_dlut(Format("x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ? 
                                     O@ pi 0.5 * < O 0.666666 * O pi 0.5 * >= pi O >= & pi 0.333333 * O 1.333333 * + 2.094395102 - 0 ? ? O + y * z /"), bi, fs), optSingleMode=false)
    B = Expr(G, B, R, ex_dlut(Format("x 0 > y x / atan A@ x 0 < y 0 >= & A pi + x 0 < y 0 < & A pi - x 0 == y 0 > pi 0.5 * y 0 < pi -0.5 * 0 0 ? ? ? ? ? ? 
                                     O@ pi 0.5 * < O 0.666666 * O pi 0.5 * >= pi O >= & pi 0.333333 * O 1.333333 * + 2.094395102 - 0 ? ? O + x * z /"), bi, fs), optSingleMode=false)
	
    MatrixClip(CombinePlanes(R, G, B, planes="RGB"),MatrixTranspose(RGB),"RGB") }

HSV doesn't use atan2() (but other cylindrical models do)

function RGB_to_HSV (clip R, clip G, clip B, bool fulls) {

    bi = BitsPerComponent(R)
    fs = Default (fulls, false)

    V = Expr(R, G, B,    "x y max z max", optSingleMode=false) # Lightness Hexcone
    S = Expr(R, G, B, V, "a 0 == 0 a x y min z min - a / ?",     optSingleMode=true) # Hexagonal Chroma
    H = Expr(R, G, B, V, "a x y min z min - N@
                          0 == 0
                          x a == y z - N / % 6
                          y a == z x - N / 2 +
                          z a == x y - N / 4 + 0 ? ? ? ? 60 360 / *", optSingleMode=true)

    CombinePlanes(H, S, V, planes="RGB") }


function RGB_to_HSV2 (clip R, clip G, clip B, bool fulls) {

    bi = BitsPerComponent(R)
    fs = Default (fulls, false)


    V = Expr(R, G, B,    "x y max z max", optSingleMode=false)
    S = Expr(R, G, B, V, "a x y min z min - a /",     optSingleMode=true)
    H = Expr(R, G, B, V, S, "a x - a x y min z min M@ - / R^
                             a y - a M                - / G^
                             a z - a M                - / B^
                             b 0 == 0
                             x a == Rm@ y M == & 5 B +
                             Rm y M != & 1 G -
                             y a == Gm@ z M == & 1 R +
                             Gm z M != & 3 B -
                             Rm 3 G + 5 R - ? ? ? ? ? ? 60 360 / *", optSingleMode=true)

    CombinePlanes(H, S, V, planes="RGB") }


function HSV_to_RGB (clip H, clip S, clip V, bool fulls) {

    bi = BitsPerComponent(H)
    fs = Default (fulls, false)

    fix = "1 +"
    Hu = " x "+fix+" 360 60 / * "
    Ch = " y z * "
    X  = " 1 H % 2 1 - abs - C * "

    m  = " z C - "  # addition to the end

    R = Expr(H, S, V, Hu+" H@ 0 == 0 H 1 < "+Ch+" C@ H 2 < "+X+" X@ H 4 < 0 H 5 < X C ? ? ? ? ? "+m+" +", optSingleMode=false)
    G = Expr(H, S, V, Hu+" H@ 0 == 0 "+Ch+" C^ H 1 < "+X+" X@ H 3 < C H 4 < X H 6 < 0 0 ? ? ? ? ? "+m+" +", optSingleMode=false)
    B = Expr(H, S, V, Hu+" H@ 2  < 0 "+Ch+" C^ H 3 < "+X+" X@ H 5 < C H 6 < X 0  ? ? ? ? "+m+" +", optSingleMode=false)

    CombinePlanes(R, G, B, planes="RGB") }


function HSV_to_RGB2 (clip H, clip S, clip V, bool fulls) {

    bi = BitsPerComponent(H)
    fs = Default (fulls, false)

#   pr = Expr(H, "x 60 / floor")
#   se = Expr(H, pr, "x y -")
#   a = Expr(S, V, "1 x - y *")
#   b = Expr(S, V, se, "1 x z * - y *")
#   c = Expr(S, V, se, "1 x 1 z - * - y *")

    pr = " x 60 360 / * floor "
    se = " x P - "
    a  = " 1 y - z * "
    b  = " 1 y SE * - z * "
    c  = " 1 y 1 "+se+" SE@ - * - z * "

    R = Expr(H, S, V, pr+" P@ 0 == P 5 == or z P 4 == "+c+" P 1 == "+b+a+" ? ? ? ", optSingleMode=false)
    G = Expr(H, S, V, pr+" P@ 0 == "+c+" P 1 == P 2 == or z P 3 == "+b+a+" ? ? ? ", optSingleMode=false)
    B = Expr(H, S, V, pr+" P@ 0 == P 1 == or "+a+" P 2 == "+c+" P 5 == "+b+" z ? ? ? ", optSingleMode=false)

    CombinePlanes(R, B, G, planes="RGB") }

EDIT: continued here.

[real.finder]

since avs+ has array since 3.6, isn't better to add vs ShufflePlanes? or at least update CombinePlanes to make it support arrays same as ShufflePlanes