fmtconv/Avisynth+: resize, bitdepth and colorspace conversions

[Z2697]

If you use ConvertToFloat(fulls=false, fulld=true) it does the "expansion".
Maybe you should just use the internal functions if you want something behaves exactly like internal functions.

[tormento]

Maybe you should just use the internal functions if you want something behaves exactly like internal functions.
It's not that I want something that behaves like the internal functions, I was just wondering myself why the internal functions behave like that, i.e. differently than any other plugin, keeping the limited range after the floating point conversion.

[pinterf]

Limited range YUV in float is an artificial thing, maybe it's my "invention". Now I'd reinterpret it, and make it to have the very same range (0..1, -0.5..+0.5) like full range, but keep the limited/full property in order to be able to be able to do a to and from integer conversion without losing the range information. If I was sure that no external plugin writer handles this "limited" 32-bit float, then it's only some constant in Aviysynth code which I have to change.

I think, both avsresize (zimg) and fmtconv expects float in one valid format: 0..1, +/-0.5, regardless of the color range flag.

I wrote about it here https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/colorbarsuhd.html (Find section: "fmtconv: Use Full Float Workflow"), since ColorbarsUHD creates limited/narrow range pattern, if I wanted to use float workflow I definitely had to convert the clip to "full-range" float before, using Avisynth's ConvertBits(32, fulld=true) .

[Z2697]

Frankly, it doesn't matter, either way the information is fully intact (minus the potential rounding errors and binary fp oddities), binary32 have enough precision to store all the values, nothing is lost, or added.
If one stick to a same plugin for converting to and from float format (which bears a question of why not), then it's all fine.

And if one knows the difference between plugins, it's just some extra code... and the risk of human error...

[Z2697]

This point is a bit obscure to me.

So, my "correct" question becomes: should I feed video to BM3D by using

fmtc_bitdepth(bits=32, fulls=true, fulld=false)
fmtc_resample(css="444", kernel="spline36")

or

fmtc_bitdepth(bits=32)
fmtc_resample(css="444", kernel="spline36")

and think about getting back to limited after the noise reduction?

The behavior of fmtc is, determin it's (x-16)/(235-16) or x/255 based on the fulls option, so every value in legal range will just fall in 0.0-1.0 without any additional processing; the fulld option in float output only determines the frame property, nothing else.
For zimg this is more or less the same, so the idea of how to make them behave like "limited float" is the same.
(I didn't actually look into the source code, but this is easy to confirm just by using the plugin.)

limited YUV -> float -> full YUV is just a range expansion, which is what the plugin is doing internally when doing limited YUV -> full YUV.
range matters in integer format because there actually less/more code values, but with the "magic" of floating point, it doesn't matter, to some extent.

BM3D would expect values in 0.0-1.0 range so use zimg/fmtc's default behavior.
Just use the same plugin to convert back. zimg and fmtc have same behavior so you can mix them, but why.
The fmtc and avsresize plugin might not support frame property, so it's better to specify the range in function call.
The "trick" is specifically for the compatibility of float to integer conversion with current internal function. Don't ever use that in other places.

[PoeBear]

Is there any benefit of switching from using Avsresize/zimg over to this, for things like simple resizing and colorspate/bit depth conversions? I see it offers a lot of similar features, is it any faster/more accurate? Is it missing any of Avsresize's features? Just wondering if I should change up my workflow

[Z2697]

Practically, they are the same.
fmtc have more dithering methods but most being in the error diffusion category and are bad for lossy encoding;
more resizing kernels but most being not something to use everyday...

[tormento]

Is there any benefit of switching from using Avsresize/zimg over to this
Slight more precision and, AFAIK, libplacebo is the only one to internally work in linear color space instead of ignoring gamma coefficients.

[Z2697]

Slight more precision and, AFAIK, libplacebo is the only one to internally work in linear color space instead of ignoring gamma coefficients.

Slightly different, but I won't say definitively which one is more precise.
libplacebo doesn't always use linear light, and linear light doesn't always make sense. (horrible ringings when upscale in linear)
fmtc and zimg support linear light as well with fmtc supporting sigmoidize in addition. Just some extra function calls.

[tormento]

libplacebo doesn't always use linear light, and linear light doesn't always make sense.
When doesn't it linearize? It's default as it is sigmoidize too.

Personally, I love libplacebo, especially the last iteration from Asd-G, you can do really many things with a single call.

I think it's really complete but for some reason underestimated.

It's the CPU-GPU overhead that makes it sometimes really slow, at least on my very old PC.
fmtc and zimg support linear light
Internally with no multiple calls?

[DTL]

Slightly different, but I won't say definitively which one is more precise.
libplacebo doesn't always use linear light, and linear light doesn't always make sense. (horrible ringings when upscale in linear)


It is just because all rescaling (mostly upscale) needs to be done in the same transfer domain. An image file needs to have (one more) metadata property like 'scaling conditioning domain'. Same needs to be read in the AVS+ and pass as clip/frame property for scaling filters to read and use and change. And 'auto' function software simply needs to read and use this property if the user needs 'auto' operation and does not want to dig in the image file properties or search the proper conditioning for scaling domain by manually selecting all available transfer domains. Though some automation tools may be designed to scan image files and select the best scaling domain with lowest output ringing (this may be useful 'auto scaling domain' feature too to make some software a bit different from other full manual scaling software).

If input image data was prepared (anti-Gibbs and Nyquist conditioned) in the linear transfer domain - it will be upscaled with lowest ringing distortion also in the linear transfer domain. But this may be still rare footage with such properties and also no supplementary metadata typically available so it is very easy to get ringing distortion at upscaling if always attempt to do upscale in the linear transfer domain while I think most digital moving picture files are prepared in system transfer domains. Until we do not have industry standards (regional or international recommendations) about scaling conditioning domain for full digital workflow from light to light and appropriate metadata signalling for scalers.

[rgr]

/ off-topic
I guess you used accurate_rnd for swscale?
/ end off-topic

And this is still needed? Apparently it is currently default enabled.
I haven't checked, I don't use swscale, but zscale (by the way, I recommend caution --> https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23026 )