Chroma scaling

[GeoffreyA]

Happy New Year. When scaling chroma, up or down, what algorithms are recommended? Is bilinear acceptable, or would better quality be got with bicubic, Lanczos, or spline36?

While it has bothered me for some time, what prompted this question was my discovering yesterday that zscale/FFmpeg and zimg/VapourSynth may use an unexpected resampler to scale the chroma when doing only colour-space conversion, because inevitably, scaling happens when going between subsampled YUV and RGB. For example, zscale is configured to use bilinear by default; and in VS, one's choice of resize function, Bilinear(), Bicubic(), etc., dictates that.

I did some minor testing and can't tell the difference; but WinMerge demonstrated that, at a byte level, files were different from start to finish when the resampler was changed. Thanks.

[FranceBB]

My own personal choice for the YUV->RGB->YUV roundtrip when applying a LUT is to use Spline64 as resizer to first upscale and then downscale.
Up until this point I haven't had any issues with it, but I'm also interested in knowing what other people are using to perform those kind of conversions.

For reference, in a normal filterchain I would do something like:

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", fullrange=1, interp=1)

#From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

[Z2697]

I have some simialr question posted in other thread and nobody seems to give it attention.
I assume if you are not doing any process in RGB, output YUV, and are not changing the final YUV subsampling, the "PointResize" is a good choice.

But other than that special case, I think the "general rule" of the quality of resize kernels (if it exists) applies.

Let's go back to original track a bit.

But even we're not going a roundtrip to RGB, that still means it needs to somehow make the subsampled planes "unsubsampled", right?
I don't know how this filter works, but I assume the simplest way is to use the equivalent of PointResize.
Either it creates a upsampled copy in memory, or just reference the same pixel 4 times, the effect should be identical, right?

Now here comes the second question: when you do this roundtrip to RGB or 4:4:4, what resampler to use?
I again assume PointResize might be a good choice because you avoid "actual" interpolation, if not doing any other operation, PointResize shoule be just perfectly reversible by itself.
But is the "blocky" chroma detrimental to the matrix conversion?

Since we are talking about AI: most image / video processing NNs nowadays take RGB as input. For the roundtrip, what resampler to use then? (we probably should take different types into account: super resolution, restoration, frame interpolation, etc.)

[GeoffreyA]

My own personal choice for the YUV->RGB->YUV roundtrip when applying a LUT is to use Spline64 as resizer to first upscale and then downscale.
Up until this point I haven't had any issues with it, but I'm also interested in knowing what other people are using to perform those kind of conversions.

For reference, in a normal filterchain I would do something like:

Thanks. Yesterday or the day before, I changed my scripts to use Spline36 when there would be a YUV<>RGB conversion.

Regarding common defaults in the FFmpeg world, I would guess it is Bicubic, since that's the default resampler for swscale. If zscale, bilinear. And if libplacebo, Spline36 for upscaling and Mitchell for downscaling.

I have some simialr question posted in other thread and nobody seems to give it attention.
I assume if you are not doing any process in RGB, output YUV, and are not changing the final YUV subsampling, the "PointResize" is a good choice.

But other than that special case, I think the "general rule" of the quality of resize kernels (if it exists) applies.

Reading online here and elsewhere, I see that many are not content with the basic kernels. There are EWA_Lanczos, NGU, and other more esoteric ones. However, I think it's going into the realm of diminishing returns. There was an interesting discussion on libplacebo's Github page (https://github.com/haasn/libplacebo/issues/241).

Regarding the Point resampler, I think the "blocky chroma" can be visible, no? Remember that issue we had with Jay's example, where the pink letters had jaggedness.

[huhn]

that git didn't work out...

super xBr is mathematically irrelevant it is so easy to do and the difference is eye watering cost is similar to EWA lanczos. but you can not easily use it so it does not matter.

point can not be used savely because the Chroma is usually moved.
i have samples where it is impossible to not notice the difference.

[Z2697]

Remember that it was special case I was talking about: when you do resize.Xxx(matrix_in=x, matrix=y), the final resolution of each plane is not changed, but the filter internally scale up the chroma (for subsampled input, of course)
The PointResize "scaling" of chroma itself is safe: it is reversible and we reversed it.
But will the conversion be affected by the "blocky" (and maybe out of place) chroma? If will, how badly?

For the "real scaling" some tools designed for chroma may be better, for example "KrigBilateral" or "Chroma from Luma", but they are stepping from scaling into the restoration territory IMO.

[huhn]

luma guided are traps for bad image quality.
when chroma matters most there is no luma to guide them.

[Z2697]

luma guided are traps for bad image quality.
when chroma matters most there is no luma to guide them.

Who cares if they work 90% times better but maybe 9% times not as good and 1% times worse.
Just like everything (not really) ;)

[huhn]

no a scaler that is messing up beyond believe is the worst that is there.

it barely ever wins anyway at anything...
so you have scaler that can scale stuff that is horrifically down sampled like a netflix logo good and nothing else... but mess up so hard bilinear would be a better choice.

[Z2697]

I don't think they are worse or terribly mess up in generic scenes
Test is done with mpv - using it's vapoursynth filter to zero out the luma plane
So absolutely no luma can be referenced, which sounds to me like the worst case scenario huhn said
https://slow.pics/c/oicxMJM2
This is only a rough trest, a better test would be have a YUV444 video as reference

Nonetheless, I do think they should be considered as restoration rather than "good old" scaling, just like what I said in previous post

[GeoffreyA]

I don't think they are worse or terribly mess up in generic scenes
Test is done with mpv - using it's vapoursynth filter to zero out the luma plane
So absolutely no luma can be referenced, which sounds to me like the worst case scenario huhn said
https://slow.pics/c/oicxMJM2
This is only a rough trest, a better test would be have a YUV444 video as reference

Nonetheless, I do think they should be considered as restoration rather than "good old" scaling, just like what I said in previous post

I would say it's very hard to tell the difference. (But my monitor is quite dark and I had to turn the brightness to 100 in the Radeon settings.) Mainly with the characters on the top, there's a faint increase in brightness, and anything beyond bilinear is slightly sharper.

[huhn]

just ask me for a chroma scale sample next time.

here a test:
https://slow.pics/c/cE5SnaUT

prediction is worse then bilinear. this is BTW. the first image i tested i didn't need to search for a case it fails utterly.

subsampled this way:
ffmpeg.exe -i gundam5.png -vf scale=out_color_matrix=bt709:out_chroma_loc=left:flags= -pix_fmt yuv420p gundam5.y4m

encoded this way:
x264.exe --crf 0 --colorprim bt709 --transfer bt709 --colormatrix bt709

maybe i should add chromaloc next time. what ever

having access to super XBR should be a human right.

mpv input config:
# Optimized shaders for higher-end GPU
CTRL+1 change-list glsl-shaders set "~~/shaders/CfL_Prediction.glsl";
CTRL+2 change-list glsl-shaders set "~~/shaders/JointBilateral.glsl";
CTRL+3 change-list glsl-shaders set "~~/shaders/ArtCNN_C4F32_Chroma.glsl";
CTRL+4 change-list glsl-shaders set "~~/shaders/superxbr-chroma.hook";
CTRL+5 change-list glsl-shader-opts set "ar_strength=0.0";
CTRL+6 change-list glsl-shader-opts set "ar_strength=1.0";
CTRL+7 change-list glsl-shader-opts set "ar_strength=2.0";
CTRL+8 change-list glsl-shaders set "~~/shaders/FSRCNNX_x2_16-0-4-1.glsl";
CTRL+9 cycle-values cscale "bilinear" "sinc" "jinc" "bicubic" "ewa_lanczos4sharpest" "ewa_robidouxsharp"
CTRL+0 no-osd change-list glsl-shaders clr ""; show-text "GLSL shaders cleared"
PGUP playlist-prev ; show-text "${playlist-pos-1}/${playlist-count}"
PGDWN playlist-next ; show-text "${playlist-pos-1}/${playlist-count}"
d cycle-values vf "lavfi=[dejudder, fps=30000/1001, fieldmatch=mode=pcn_ub:combmatch=full, yadif=deint=interlaced, decimate]" ""
D cycle-values vf "lavfi=[fieldmatch=full]" ""
? cycle video-unscaled

mpv config:
profile=gpu-hq
vo=gpu-next
gpu-api=vulkan
fbo-format=rgba32f

screenshot-high-bit-depth=no
volume=25
loop-file=yes
screenshot-format=png

cscale=bilinear

[Z2697]

just ask me for a chroma scale sample next time.

here a test:
https://slow.pics/c/cE5SnaUT

prediction is worse then bilinear. this is BTW. the first image i tested i didn't need to search for a case it fails utterly.

subsampled this way:
ffmpeg.exe -i gundam5.png -vf scale=out_color_matrix=bt709:out_chroma_loc=left:flags= -pix_fmt yuv420p gundam5.y4m

encoded this way:
x264.exe --crf 0 --colorprim bt709 --transfer bt709 --colormatrix bt709

maybe i should add chromaloc next time. what ever

having access to super XBR should be a human right.

mpv input config:
# Optimized shaders for higher-end GPU
CTRL+1 change-list glsl-shaders set "~~/shaders/CfL_Prediction.glsl";
CTRL+2 change-list glsl-shaders set "~~/shaders/JointBilateral.glsl";
CTRL+3 change-list glsl-shaders set "~~/shaders/ArtCNN_C4F32_Chroma.glsl";
CTRL+4 change-list glsl-shaders set "~~/shaders/superxbr-chroma.hook";
CTRL+5 change-list glsl-shader-opts set "ar_strength=0.0";
CTRL+6 change-list glsl-shader-opts set "ar_strength=1.0";
CTRL+7 change-list glsl-shader-opts set "ar_strength=2.0";
CTRL+8 change-list glsl-shaders set "~~/shaders/FSRCNNX_x2_16-0-4-1.glsl";
CTRL+9 cycle-values cscale "bilinear" "sinc" "jinc" "bicubic" "ewa_lanczos4sharpest" "ewa_robidouxsharp"
CTRL+0 no-osd change-list glsl-shaders clr ""; show-text "GLSL shaders cleared"
PGUP playlist-prev ; show-text "${playlist-pos-1}/${playlist-count}"
PGDWN playlist-next ; show-text "${playlist-pos-1}/${playlist-count}"
d cycle-values vf "lavfi=[dejudder, fps=30000/1001, fieldmatch=mode=pcn_ub:combmatch=full, yadif=deint=interlaced, decimate]" ""
D cycle-values vf "lavfi=[fieldmatch=full]" ""
? cycle video-unscaled

mpv config:
profile=gpu-hq
vo=gpu-next
gpu-api=vulkan
fbo-format=rgba32f

screenshot-high-bit-depth=no
volume=25
loop-file=yes
screenshot-format=png

cscale=bilinear

Something's wrong with the vulkan and that shader LOL

[huhn]

yes that's mpv in a nutshell maybe it works in different API:
still unremarkable: https://i.slow.pics/MIjNbwbx.png

[GeoffreyA]

Have you got the original picture? I'd like to test it with the basic kernels.

[huhn]

the gundam5 is original RGB.

[GeoffreyA]

Ah, all right.

[Balling]

spline36?? We know that spline144 is best, maybe even more. https://forum.doom9.org/showthread.php?t=175187

[Balling]

yes that's mpv in a nutshell maybe it works in different API:
still unremarkable: https://i.slow.pics/MIjNbwbx.png

This was used to show some "bug" in mpv, where I doubt there is any bug. Looks like it was you: https://github.com/mpv-player/mpv/issues/15299

You still did not tell what the actual bug there is.

[GeoffreyA]

spline36?? We know that spline144 is best, maybe even more. https://forum.doom9.org/showthread.php?t=175187

Oh, no, I'll have to go and re-encode everything now :)

[huhn]

This was used to show some "bug" in mpv, where I doubt there is any bug. Looks like it was you: https://github.com/mpv-player/mpv/issues/15299

You still did not tell what the actual bug there is.

the bug was fixed at the time of reporting because someone else also reported based on another report by me there simply was no windows binary from an "official" source at the time it was broken.

Oh, no, I'll have to go and re-encode everything now :)


you should at least test it first.

feel free to fall for the bicubic 150 AR trap too.

[Z2697]

A side quest: what is the Spline in SplinexxResize? Spline seems to be a more generic word, there's bicubic B-spline... or maybe just the fact that I'm bad at math

[huhn]

i don't know the details either.
the result is very very similar to lanczos it's a trade for less ringing vs less sharpness

[Z2697]

This was used to show some "bug" in mpv, where I doubt there is any bug. Looks like it was you: https://github.com/mpv-player/mpv/issues/15299

You still did not tell what the actual bug there is.

Same image is used, but different bug, I guess.
This time the bug is on the vulkan side (both vo=gpu and vo=gpu-next).
Or perhaps not a bug, but definitely "something wrong with the vulkan and that shader" as the way I call it.
Incompatible syntax or something in the shader, or unsupported feature/api, for example.
I have, like zero knowledge in the field of shaders.

[huhn]

most likely fixed my version is a couple month old.

there is always something broken. kinda used to it.
not the first time something broke with vulkan and shader maybe even was that one it works in d3d11 and i shared that image.
i guess the fix is just stop using vulkan on my side...

[GeoffreyA]

you should at least test it first.

feel free to fall for the bicubic 150 AR trap too.

I meant it as a joke, the search for placebo. I am happy with Spline36. :)

[huhn]

well me too bicubic 150 is quite good. but it is still a trap maybe one of the best one in mpv.

spline 144 with a very strong anti ringing may work.... not interested in testing such huge kernels... the processing cost.

[Z2697]

I stand for ewa_lanczos (the reality is I don't care much), seems pretty good, less aliasing than non-ewa kernels.
That was speaking of the "traditional" scaling methods.
My "real" favorite is probably nnedi3 or NGU sharp. (the reason why I don't care much of "traditional" method)

However for downscaling I think ewa_lanczos sucks, and I use Catmull-Rom (bicubic 0,0.5) in mpv. I have 3840x2160 screen so the downscaling don't happen very often.

p.s. I was actually takling about the general use, not specifically chroma scaling. But it's actually the same "list" with addition of previously mentioned luma guided methods.

[GeoffreyA]

I stand for ewa_lanczos (the reality is I don't care much), seems pretty good, less aliasing than non-ewa kernels.
That was speaking of the "traditional" scaling methods.
My "real" favorite is probably nnedi3 or NGU sharp. (the reason why I don't care much of "traditional" method)

However for downscaling I think ewa_lanczos sucks, and I use Catmull-Rom (bicubic 0,0.5) in mpv. I have 3840x2160 screen so the downscaling don't happen very often.

p.s. I was actually takling about the general use, not specifically chroma scaling. But it's actually the same "list" with addition of previously mentioned luma guided methods.

With huhn's RGB picture, I did some chroma testing, and yes, EWA_Lanczos does have less aliasing. Later, I'll test it on ordinary scaling. The reason I shy away from Lanczos, the classic one, are memories from back in the DivX/DVD days of its ringing and fake-looking HD. Perhaps things have changed.

[huhn]

no nothing changed laczos is still lanczos mpv may apply decent anti ringing behidn your back you can over write that.

but if you use something like this for testing: CTRL+9 cycle-values cscale "bilinear" "sinc" "jinc" "bicubic" "ewa_lanczos4sharpest" "ewa_robidouxsharp"
the quality of ewa_lanczos4sharpest is horrible like no AR is applied utterly useless while jinc will do just fine...

NGU sharp is BTW. utterly pointless for chroma NGU AA is so so much better for chroma there is barely any point to ever use NGU sharp.

super XBR is so cheap processing wise it cost about the same as jinc3 AR so barely any point to not use it.

[Z2697]

no nothing changed laczos is still lanczos mpv may apply decent anti ringing behidn your back you can over write that.

but if you use something like this for testing: CTRL+9 cycle-values cscale "bilinear" "sinc" "jinc" "bicubic" "ewa_lanczos4sharpest" "ewa_robidouxsharp"
the quality of ewa_lanczos4sharpest is horrible like no AR is applied utterly useless while jinc will do just fine...

NGU sharp is BTW. utterly pointless for chroma NGU AA is so so much better for chroma there is barely any point to ever use NGU sharp.

super XBR is so cheap processing wise it cost about the same as jinc3 AR so barely any point to not use it.

"jinc" is an alias of "ewa_lanczos"... not sure what you are talking.
(Second thought, you are replying to the Perhaps things have changed., my bad. But my following content also support nothing changed laczos is still lanczos)

The real jinc is a function like sinc, lanczos is sinc windowed sinc.
The "colloquial jinc" as alias of "ewa_lanczos" is sinc windowed jinc (or jinc windowed, don't really remember).
So the name "ewa_lanczos" is given because it's ewa, which I have no idea what it is, my math bad :( , and it's similarity to the lanczos.

So, lanczos is still lanczos because it's definition is simple and clear, and no change has made to it. And ewa_lanczos is not lanczos.

Yeah, I think NGU AA is better for chroma as well, I just wrote that with general upscaling in mind. :o

[GeoffreyA]

"jinc" is an alias of "ewa_lanczos"... not sure what you are talking.
(Second thought, you are replying to the Perhaps things have changed., my bad. But my following content also support nothing changed laczos is still lanczos)

The real jinc is a function like sinc, lanczos is sinc windowed sinc.
The "colloquial jinc" as alias of "ewa_lanczos" is sinc windowed jinc (or jinc windowed, don't really remember).
So the name "ewa_lanczos" is given because it's ewa, which I have no idea what it is, my math bad :( , and it's similarity to the lanczos.

So, lanczos is still lanczos because it's definition is simple and clear, and no change has made to it. And ewa_lanczos is not lanczos.

Yeah, I think NGU AA is better for chroma as well, I just wrote that with general upscaling in mind. :o

That's what I love about encoding: there's always a new riddle to baffle the mind!

[huhn]

"jinc" is an alias of "ewa_lanczos"... not sure what you are talking.
(Second thought, you are replying to the Perhaps things have changed., my bad. But my following content also support nothing changed laczos is still lanczos)

The real jinc is a function like sinc, lanczos is sinc windowed sinc.
The "colloquial jinc" as alias of "ewa_lanczos" is sinc windowed jinc (or jinc windowed, don't really remember).
So the name "ewa_lanczos" is given because it's ewa, which I have no idea what it is, my math bad :( , and it's similarity to the lanczos.

So, lanczos is still lanczos because it's definition is simple and clear, and no change has made to it. And ewa_lanczos is not lanczos.

Yeah, I think NGU AA is better for chroma as well, I just wrote that with general upscaling in mind. :o
yes ewa lanczos should be jinc and all are related to sinc but is EWA lanczos4sdharp is utterly terrible for chroma.

here are madVR jinc3 AR mpv ewa_lanczossharp (what ever that means) and my jinc2m:
https://slow.pics/c/QN3xeFMl

mpv is literally just dirty and has white shadows all over the place. madVR jinc 3 AR is clean and my baby is utterly clean.

so what is mpv ewa lanczos compared to madVR jinc3 is it 4 tabs and has more issues because of that and looks different?

so the name does not matter anymore in this very very specific test mpv fails.

this test is only good to look for kernel issues like the white shadowing some ringing. as said before jinc2m can have issues in real images and only looks clean here because i made it to look clean in this test.

[Z2697]

yes ewa lanczos should be jinc and all are related to sinc but is EWA lanczos4sdharp is utterly terrible for chroma.

here are madVR jinc3 AR mpv ewa_lanczossharp (what ever that means) and my jinc2m:
https://slow.pics/c/QN3xeFMl

mpv is literally just dirty and has white shadows all over the place. madVR jinc 3 AR is clean and my baby is utterly clean.

so what is mpv ewa lanczos compared to madVR jinc3 is it 4 tabs and has more issues because of that and looks different?

so the name does not matter anymore in this very very specific test mpv fails.

this test is only good to look for kernel issues like the white shadowing some ringing. as said before jinc2m can have issues in real images and only looks clean here because i made it to look clean in this test.

errr... probably because it's "sharp".
Which is not the same as "normal" ewa_lanczos (aka "jinc").

[huhn]

it is the "softest" of these 3.
and i have no clue of the number of tab.
what does sharp even mean?

the mpv jinc luma implementation is hands down broken.

here is jinc and ewa_lanczos they are not even close to be the same:
https://slow.pics/c/sBew7MgC

edit: here is GPU i really can't handle mpv anymore this just insane...
https://i.slow.pics/28fIBMfS.png

[Z2697]

it is the "softest" of these 3.
and i have no clue of the number of tab.
what does sharp even mean?

the mpv jinc luma implementation is hands down broken.

here is jinc and ewa_lanczos they are not even close to be the same:
https://slow.pics/c/sBew7MgC

edit: here is GPU i really can't handle mpv anymore this just insane...
https://i.slow.pics/28fIBMfS.png

I don't know why your ewa_lanczos looks bad, it looks nothing like that on my end.
But apparently the "jinc" in mpv (gpu-next at least) is not the same as "ewa_lanczos", it's not EWA. It shows as "ortho" in the frame time info, EWA should be "polar".
So while people often use "jinc" as "ewa_lanczos", in mpv gpu-next it's the real jinc, even without implication that "the math is polar" (honestly I don't know what it means... I'm basically illiterate in this area)

[Z2697]

What is jinc2 and jinc2m by the way, I know jinc2m is in mpcVR, I wanna know the basic "internal structure" of them.

[huhn]

2 tab jinc. m was added after i found the new weights that don't artefact on the BW 36x36 test.

if the jinc in gpu-next is the real jinc what is the the one in madVR which should be much older. both GPU and GPU-next fail teh test.

https://github.com/mightyhuhn/VideoRenderer/blob/master/Shaders/examples/resizer_onepass_jinc2.hlsl

it looks at only at a 4x4 grid to calculate a new pixel which is very little and a major reason it is super fast.

[Z2697]

2 tab jinc. m was added after i found the new weights that don't artefact on the BW 36x36 test.

if the jinc in gpu-next is the real jinc what is the the one in madVR which should be much older. both GPU and GPU-next fail teh test.

https://github.com/mightyhuhn/VideoRenderer/blob/master/Shaders/examples/resizer_onepass_jinc2.hlsl

it looks at only at a 4x4 grid to calculate a new pixel which is very little and a major reason it is super fast.

It looks like the weights are calculated with sin(x * wa) * sin(x * wb) / (x * x).
That's more like sinc (contrary to jinc)... or lanczos (it's windowed) :eek:

[huhn]

you can now use it on bad quality DVD see why it isn't good and go back to chroma scaling.

[GeoffreyA]

I don't know why your ewa_lanczos looks bad, it looks nothing like that on my end.
But apparently the "jinc" in mpv (gpu-next at least) is not the same as "ewa_lanczos", it's not EWA. It shows as "ortho" in the frame time info, EWA should be "polar".
So while people often use "jinc" as "ewa_lanczos", in mpv gpu-next it's the real jinc, even without implication that "the math is polar" (honestly I don't know what it means... I'm basically illiterate in this area)

According to the reference (https://guide.encode.moe/encoding/resampling.html), which is pretty handy by the way, orthogonal resamples horizontally, then vertically; so treating it as rows and columns. Polar, which is the same as EWA, works by using a radius of pixels round a pixel, all contributing to its final value.