Gamma corrected resizing

[colours]

Gamma aware resizers will come when we will add 16 bit per color. It's a bit tricky though, i'm still not sure what an acceptable way is to do it directly in YCbCr (since it's an RGB concept).

The short answer is that the One True Way of doing gamma-corrected resizing is to go to RGB, which isn't really all that expensive compared to the actual resizing anyway. (I didn't actually benchmark this, but the speeds should have equal order of magnitude.)

The long answer is that if you decide to work out the equations, it's probably (!) possible to get a decent first-order approximation (for small Cb and Cr values) that runs faster than a roundtrip YCbCr→RGB→linear RGB→RGB→YCbCr conversion. Or you could stick with the zeroth-order approximation of just applying the inverse gamma to luma, resizing, then applying gamma again, which actually works moderately well because no one cares about chroma.

[feisty2]

The short answer is that the One True Way of doing gamma-corrected resizing is to go to RGB, which isn't really all that expensive compared to the actual resizing anyway. (I didn't actually benchmark this, but the speeds should have equal order of magnitude.)

The long answer is that if you decide to work out the equations, it's probably (!) possible to get a decent first-order approximation (for small Cb and Cr values) that runs faster than a roundtrip YCbCr→RGB→linear RGB→RGB→YCbCr conversion. Or you could stick with the zeroth-order approximation of just applying the inverse gamma to luma, resizing, then applying gamma again, which actually works moderately well because no one cares about chroma.
how about even better, YCbCr→RGB→Linear RGB→CIE 1931→CIE Lab→Resize :devil:
http://www.imagemagick.org/Usage/resize/#resize_lab

[Motenai Yoda]

@feisty2 I don't think CIE Lab is so different from YUV maybe a bit bigger, also imho resizing on RGB will be better, isn't recommended to use a very sharp resizer on those color-opponent colorspaces (coz haloing and ringing will be bad on colorplanes).
so imho easier YCbCr→RGB→Linear RGB→YCbCr→Resize

as colours wrote, no one cares about chroma, and converting to RGB needs a croma upsample pass and a matrix too.

[feisty2]

so imho easier YCbCr→RGB→Linear RGB→YCbCr→Resize

that's chilling


as colours wrote, no one cares about chroma, and converting to RGB needs a croma upsample pass and a matrix too


well, chroma does matter even if you don't wanna care about it, cuz Y in most YUV colorspaces ain't exactly like L in Lab, most YUV images are converted from some gamma compressed RGB colorspace, unlike Lab, derived from XYZ which is linear, some luminance gets kicked into UV planes due to this, that's why filtering under RGB and YUV are so different sometimes, you have to think about chroma unless you are using a 2020cl matrix, which is linear, just like Lab

[DarkSpace]

unlike Lab, derived from XYZ which is linear [...]
unless you are using a 2020cl matrix, which is linear, just like Lab
CIELAB isn't linear, it's perceptual (which means perceived color differences, rather than actual ones, are linear), so I believe it is unsuited for resizing (where you want to compute an actual rather than a perceived weighting of colors.

Just saying (http://en.wikipedia.org/wiki/Lab_color_space#Perceptual_differences), you know, for the record...

[feisty2]

I was saying "XYZ is linear"....
The linear stuff ain't about lab and yuv, it's about their source color spaces (xyz and rgb)
The point is, the source color space gotta be linear so the luminance separated from it would be pure, like lab or ycbcr with 2020cl matrix

[Wilbert]

you have to think about chroma unless you are using a 2020cl matrix, which is linear, just like Lab
Rec2020 has two matrices: linear (Yc'Cbc'Crc') and the traditional non-linear (Y'Cb'Cr'). See page 4 of ITU-R BT.2020-1 (https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2020-1-201406-I!!PDF-E.pdf). I'm always confused by this stuff, is the following understanding correct?

A dvd/blu-ray is encoded in non-linear Y'Cb'Cr'. So you convert to R'G'B', convert that to linear RGB (= undo gamma) and convert back to linear Yc'Cbc'Crc' [except it's Rec601/709 instead of Rec2020, but that's not the point]. Then you can apply the smoothing and resizing, etc ... ??

I'm a bit puzzled by the derivation of Cbc'Crc' though. I expected it would be something like: Cbc' = (B-Yc')/(1-Kb), but it's not.


@StainlessS, where do you see tau mentioned in the docs? I thought i removed that. Will look at your other issue later.

[feisty2]

A dvd/blu-ray is encoded in non-linear Y'Cb'Cr'. So you convert to R'G'B', convert that to linear RGB (= undo gamma) and convert back to linear Yc'Cbc'Crc' [except it's Rec601/709 instead of Rec2020, but that's not the point]. Then you can apply the smoothing and resizing, etc ... ??

Yeah, that's it, but I'm no expert about this, it just feels right to me, better do some tests and see if it gives you what you want.

[feisty2]

source
http://i.imgur.com/zzZraQ8.jpg

regular

dither_convert_rgb_to_yuv (matrix="709",lsb=true,output="yv24",tv_range=False)
dither_resize16 (600,300)
dither_convert_yuv_to_rgb (matrix="709",lsb_in=true,output="rgb24",mode=6,tv_range=False)

http://i.imgur.com/6ynXgCg.png

new method

r=showred ("y8").convert8to16 (false).Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
g=showgreen ("y8").convert8to16 (false).Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
b=showblue ("y8").convert8to16 (false).Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
dither_convert_rgb_to_yuv (r,g,b,matrix="709",lsb=true,output="yv24",tv_range=False)
dither_resize16 (600,300)
dither_convert_yuv_to_rgb (matrix="709",lsb_in=true,output="rgb48y",tv_range=False)
Dither_y_linear_to_gamma (tv_range_in=False, tv_range_out=False, curve="srgb")
round8 (false)
mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))

Function Convert8To16 (clip input, bool "tv_range")
{
tv_range = Default (tv_range, True)

iCceil = (255-128) / (255.5-128) * (65535.5-32768) + 32768
Yexpr = "0-0 ; 255-65535 ;65535-65535 "
Cexpr = "0-0.5;0.5-0.5;128-32768;255-"+String(iCceil)+";65535-"+String(iCceil)
fullrange = StackVertical (input.Dither_gen_null_lsb (), input).
\ SmoothCurve16 (Ycurve=Yexpr, Ucurve=Cexpr, Vcurve=Cexpr, mode=0, interp=0, HQ=True,
\ dither=-1, limiter=False, TVrange=0)

output = tv_range ? input.Dither_convert_8_to_16 ()
\ : fullrange

return output
}

Function Round8 (clip input, bool "tv_range")
{
tv_range = Default (tv_range, True)

iCceil = (255-128) / (255.5-128) * (65535.5-32768) + 32768
Yexpr = "0-0; 65535-255"
Cexpr = "0-0.5;0.5-0.5;32768-128;"+String(iCceil)+"-255;65535-255"
fullrange = input.SmoothCurve16 (Ycurve=Yexpr, Ucurve=Cexpr, Vcurve=Cexpr, mode=0, interp=0, HQ=True,
\ dither=-1, limiter=False, TVrange=0)

output = tv_range ? input.ditherpost (mode=-1, y=3, u=3, v=3)
\ : fullrange.Dither_get_lsb ()

return output
}

http://i.imgur.com/6O8fmCT.png

guess it works :devil:

[colours]

Rec2020 has two matrices: linear (Yc'Cbc'Crc') and the traditional non-linear (Y'Cb'Cr'). See page 4 of ITU-R BT.2020-1 (https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2020-1-201406-I!!PDF-E.pdf). I'm always confused by this stuff, is the following understanding correct?

A dvd/blu-ray is encoded in non-linear Y'Cb'Cr'. So you convert to R'G'B', convert that to linear RGB (= undo gamma) and convert back to linear Yc'Cbc'Crc' [except it's Rec601/709 instead of Rec2020, but that's not the point]. Then you can apply the smoothing and resizing, etc ... ??

I'm a bit puzzled by the derivation of Cbc'Crc' though. I expected it would be something like: Cbc' = (B-Yc')/(1-Kb), but it's not.

I'd always thought 2020 constant-luminance version was linear in RGB, but it's actually linear in neither RGB nor R'G'B', so it can't even be called a matrix. I guess it's a common misunderstanding that "constant luminance" and "linear light" mean the same thing.

Anyway, the only correct way of doing gamma-aware resizing is in a linear-light colourspace (like linear-light RGB). What feisty2 posted works, but it has an unnecessary roundtrip conversion to linear-light YCbCr.

[feisty2]

but it has an unnecessary roundtrip conversion to linear-light YCbCr.

it's necessary, see #13 and #14, you will need ycbcr to separate intensity and colors to avoid clipping artifacts under linear light, I was thinking about CIE Lab, but Motenai Yoda came up with YCbCr, easier and should do the trick also

edit: just did a test on that rose image, and ycbcr fails... turns out Lab can't be simply replaced by ycbcr and Lab resize is almost identical to gamma ignorant resize so... gotta choose between gamma aware and dark halos...

[colours]

it's necessary, see #13 and #14, you will need ycbcr to separate intensity and colors to avoid clipping artifacts under linear light, I was thinking about CIE Lab, but Motenai Yoda came up with YCbCr, easier and should do the trick also

False. This produces an effectively identical result. (The lut16 for range conversion is necessary because in Dither_y_gamma_to_linear/Dither_y_linear_to_gamma, "full range" is 0 to 256×256, while in Dither_convert_yuv_to_rgb, the output range is 0 to 255×256.)

jpegsource("O:\earthlights_big.jpg")
dither_convert_yuv_to_rgb(tv_range=false,cplace="MPEG1",matrix="601",output="rgb48y")
dither_lut16("x 255 / 219 * 4096 +")
dither_y_gamma_to_linear()
dither_resize16(600,300)
dither_y_linear_to_gamma()
dither_lut16("x 4096 - 219 / 255 *")
ditherpost(mode=-1)
mergergb(selectevery(3,0),selectevery(3,1),selectevery(3,2))

What Motenai Yoda suggested would be more like using different resampling kernels for luma and chroma, in which case a conversion to linear-light YCbCr would indeed be pretty much unavoidable, but that's not what your script does.

[DarkSpace]

I was saying "XYZ is linear"....
The linear stuff ain't about lab and yuv, it's about their source color spaces (xyz and rgb)
The point is, the source color space gotta be linear so the luminance separated from it would be pure, like lab or ycbcr with 2020cl matrix
In that case, I apologize.
The first line of my quotes was ambivalent, but the second one seemed to point me to you saying "Lab is linear", which was obviously a misunderstanding. I'm curious, though: Is a perceptual colorspace actually more suited to resizing than a linear one?
And, as long as the colorspace is linear (or perceptual), why should it matter whether the luminance is extracted into a separate channel or mixed into the other channels? Linear (or perceptual) weighting should just take care of that, shouldn't it?

In any case, the bottom line is: I misunderstood, and I'm sorry :o

[feisty2]

why should it matter whether the luminance is extracted into a separate channel or mixed into the other channels? Linear (or perceptual) weighting should just take care of that, shouldn't it?


uh, dunno about that, it just feels so wrong to me, maybe those instincts are stupid and leading me nowhere

[feisty2]

anyways, I made a new sort of gamma aware resizer tryna overcome dark ringings caused by linear light resizing
so, "rose image"
source
http://i.imgur.com/EdRYwZm.png
gamma ignorant resizing

r=showred ("y8")
g=showgreen ("y8")
b=showblue ("y8")
gamma=interleave (r,g,b).convert8to16 (false)
gamma.dither_resize16 (210,138)
round8 (false)
mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))

http://i.imgur.com/0iAp9Sn.png
gamma aware resizing

r=showred ("y8")
g=showgreen ("y8")
b=showblue ("y8")
gamma=interleave (r,g,b).convert8to16 (false)
linear=gamma.Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
linear.dither_resize16 (210,138)
Dither_y_linear_to_gamma (tv_range_in=False, tv_range_out=False, curve="srgb")
round8 (false)
mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))

http://i.imgur.com/0hxwldl.png
new fake gamma aware resizing

r=showred ("y8")
g=showgreen ("y8")
b=showblue ("y8")
gamma=interleave (r,g,b).convert8to16 (false)
linear=gamma.Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
gresize=gamma.dither_resize16nr (210,138).Dither_y_gamma_to_linear (tv_range_in=False, tv_range_out=False, curve="srgb")
#resizing under gamma compressed space gets highlight areas into trouble, highlight ringings and lack of brightness, pick dither_resize16NR to get rid of highlight ringings
lresize=linear.dither_resize16 (210,138)
#real gamma aware resizing, works better on highlight areas, less highlight ringings and better brightness, but makes dark halos
brightdif=dither_sub16 (lresize,gresize,dif=true).dither_lut16 ("x 32768 < 32768 x ?")
#find the differences between pixels, keep brighter difs and discard darker difs (halos)
gresize.dither_add16 (brightdif,dif=true)
Dither_y_linear_to_gamma (tv_range_in=False, tv_range_out=False, curve="srgb")
round8 (false)
mergergb (selectevery (3,0),selectevery (3,1),selectevery (3,2))

http://i.imgur.com/M4P72B2.png

edit: typo in the code

[feisty2]

"map image"
source
http://i.imgur.com/zzZraQ8.jpg
gamma ignorant
http://i.imgur.com/FHTpRQU.png
gamma aware
http://i.imgur.com/C3wtSKK.png
new fake gamma aware
http://i.imgur.com/31ov9zX.png

edit: replaced an image due to the code change at #27

[Lenchik]

I think this resizing discussion should form a new discussion thread, cause it is not only "AviSynth 2.6.0 RC3 [Apr 15th, 2015]" development question.
By the way, are you aware of decade old discussion at avsforum (http://www.avsforum.com/forum/26-home-theater-computers/460922-lanczos-vs-bicubic-comparison-3.html#post4811927) concerning resize? Quote from it:
For best results, scaling should in fact be done with linear-light values, not gamma-corrected values. And of course it should be done in RGB, not Y'CbCr, since Y'CbCr values aren't even gamma-corrected values, they're a linear combination of gamma-corrected values. Mathematically, processing in Y'CbCr is a complete abomination. Practically, it works sorta OK and requires a lot less CPU, so people go ahead and do it.

The best pipeline for scaling images in a Y'CbCr space is:

Y'CbCr->R'G'B'->RGB->[scale]->RGB->R'G'B'->Y'CbCr

Obviously the downsampling and upsampling of the chroma channel for 4:2:0 or 4:2:2 has an effect on the final image, but it can't be helped (other than by not using 4:2:0 or 4:4:4). There's no way to linearize Y', Cb, or Cr.

And also some image for resizing test from http://www.4p8.com/eric.brasseur/gamma.html
Jonas Berlin sent this image. Scale it down 1:2 with your software...
http://www.4p8.com/eric.brasseur/gamma-1.0-or-2.2.png

[feisty2]

yeah, but as you can see, how u gonna slay those dark halos caused by pure linear light resizing (see #27, "rose image")


I think this resizing discussion should form a new discussion thread

right, will do it

edit:
result of new fake gamma aware resizing
http://i.imgur.com/JUFCkDd.png