fmtconv/Avisynth+: resize, bitdepth and colorspace conversions

[cretindesalpes]

I just ported my fmtconv (https://forum.doom9.org/showthread.php?t=166504) Vapoursynth plug-in to Avisynth+.

>>> fmtconv-r31.zip <<< (https://ldesoras.fr/src/vs/fmtconv-r31.zip)

Fmtconv is a format-conversion plug-in for the Vapoursynth and Avisynth+ video processing engines. It does:

Resizing and convolution.
Bitdepth conversion with dithering.
Colorspace conversion (matrix, transfer characteristics and chromatic adaptation).

Supports:

8-–12-, 14- and 16-bit integer, 32-bit float
Colorspaces: RGB, Y, YUV, YCoCg, YDzDx and ICtCp in 4:4:4, 4:2:2, 4:2:0 and 4:1:1 chroma subsampling factors.
Progressive and interlaced content.

Fmtconv is focussed primarily on quality and exactness rather than execution speed. This does not mean it is slow or unoptimized, but fmtconv is clearly not on par with the fastest equivalent 8-bit filters.

The full documentation is included in the zip file.

Source code is also available as Git repository (https://gitlab.com/EleonoreMizo/fmtconv).

Please note:
This plug-in will work only in Avisynth+ (v3.7.0 and above), not in regular Avisynth, because it makes extensive use of the new high-bitdepth colorspaces. Stack-16 format is not supported.
Fmtconv is not internally threaded at the moment (excepted resample that can use AVSTP (https://forum.doom9.org/showthread.php?t=164407)), but could be if there is a real need. All the functions support the MT_NICE_FILTER multi-treaded mode. Enabling multi-threading is currently not automatic, you have to activate it from your scripts with SetFilterMTMode (http://avisynth.nl/index.php/SetFilterMTMode) and use the Prefetch function at the right location. This may change later.
32-bit version is strongly discouraged. I may remove it completely in the future.
The whole thing is complex and testing every parameter combination is impossible. The core functionalities work and have been tested for several years with Vapoursynth, but the Avs+ interface is new and may have related issue. Please report here anything suspicious, I’ll try to fix it as soon as possible.
Documentation lacks examples for Avs+, but this will come later.

[Dogway]

Thanks a lot! Will play with it today. I still have to finish TransformsPack but fmtconv will come handy to double check results.
Finally can use Void and Cluster! :thanks:

[Reel.Deel]

Thank you very much cretindesalpes! :thanks:
Nice to see you back around :)

[real.finder]

thanks for avs+ porting


Fmtconv is not internally threaded at the moment, but could be if there is a real need. All the functions support the MT_NICE_FILTER multi-treaded mode. Enabling multi-threading is currently not automatic, you have to activate it from your scripts with SetFilterMTMode (http://avisynth.nl/index.php/SetFilterMTMode) and use the Prefetch function at the right location. This may change later.


isn't fmtconv used to use avstp?

[kedautinh12]

thanks for avs+ porting



isn't fmtconv used to use avstp?

You will replace dither with fmtconv in your scripts???

[cretindesalpes]

real.finder:

LOL right, I wrote this too quickly!
Indeed resample can use AVSTP if available.

[real.finder]

You will replace dither with fmtconv in your scripts???

yes, I plan to do that for at least nnedi3_resize16 now, I will think about the others later

[real.finder]

real.finder:

LOL right, I wrote this too quickly!
Indeed resample can use AVSTP if available.

I did try

LoadPlugin("C:\a\avstp.dll")
LoadPlugin("C:\b\fmtcavs.dll")
ColorBars(width=640, height=480, pixel_type="yv12")
ConvertBits(16).fmtc_resample(Width(),Height()*2).fmtc_resample(Width(),Height()).ConvertBits(8)


I can't see any multi threaded speed with avstp

[FranceBB]

Nice one.
It's good to see you back. :)

[cretindesalpes]

real.finder:

1. Make sure that avstp.dll is found by the fmtconv. Check the thread list with Process Explorer or any other task manager.

2. resample splits its workload in tiles, which are dependent on the resizing parameters and input clip. The tiles are processed in different threads. If there is only one tile per frame, there won’t be any gain.

Anyway fmtconv has only minimal support for avstp at the moment, because it was primarily developed for Vapoursynth which handles threading very well natively and it doesn’t need avstp at all. If threading is still a performance issue on Avisynth+, I’ll add a better threading support.

[real.finder]

2. resample splits its workload in tiles, which are dependent on the resizing parameters and input clip. The tiles are processed in different threads. If there is only one tile per frame, there won’t be any gain.
yes with fmtc_resample(Width()*2,Height()*2).fmtc_resample(Width(),Height()) I can see the speedup!

[kedautinh12]

Don't sure about problem of avstp or mvtools2. If you have freetime, can you check it??
https://forum.doom9.org/showthread.php?p=1942642#post1942642

[StvG]

Thanks for porting the filter.

It seems i420 is not supported.
FFVideoSource() #yuv 8-bit input
fmtc_bitdepth(16) # error

Edit: Converting yuv < 32-bit to yuv 32-bit: _ColorRange is still 1. Shouldn't be 0?

[cretindesalpes]

It seems i420 is not supported.
Got it. I was naively trying to build a I420 high-bitdepth format which is not supported by Avs+. It is now converted to a regular YUV420Pxx. Here (http://ldesoras.free.fr/src/vs/fmtconv-avs-bin.zip) are some fixed binaries that you can try in the meantime of a new release.

Converting yuv < 32-bit to yuv 32-bit: _ColorRange is still 1. Shouldn't be 0?
The _ColorRange property has no meaning for floating point formats. So I just keep it as a trace of the original format. Should I erase it? I don’t know. What is the common practice in this case?

[StvG]

The _ColorRange property has no meaning for floating point formats. So I just keep it as a trace of the original format. Should I erase it? I donÂ’t know. What is the common practice in this case?

There is still no common practice... I think. I was curious because:
1. If you do yuv < 32-bit to 32-bit (default options) you have limited->full range.
2. If you do yuv < 32-bit + fulls=true (mimic internal conversion functions) to 32-bit you have no range range expand.

You have same _ColorRange for both cases.

Edit: Thanks for the new binaries.

[StvG]

Can you add top left cplace for fmtc_resample?

[cretindesalpes]

Yes, this can be accommodated.

[DTL]

I can suggest next additions:
1. Add also sinc interpolation as option to lmpulse kernel setting - as made in UserDefined2Resize in jpsdr plugins. I like an idea of extending filter support as user-provided setting instead of hardcoded value.
2. Add option to weight to zero at the end some end lobes of sinc resample kernel (for example linear-trapezioid as made in SincLin2Resize). So users can use sinc resize with lower taps value without computational issue at the end of kernel when taps lower about 70.

[cretindesalpes]

I can suggest next additions:
1. Add also sinc interpolation as option to lmpulse kernel setting
You mean replacing the current linear interpolation with a sinc one? This would make the function unnecessarily complex, especially for a niche functionality. Because sinc is infinite and has to be windowed. But which window? How long? It is expected that the user provides a pre-oversampled kernel (64x is more than enough), the linear interpolation is just here to square the aliasing level (-25 dB goes to -50 dB) at almost no cost, computationally (when generating the final kernel) and spatially.

2. Add option to weight to zero at the end some end lobes of sinc resample kernel.
blackman (https://en.wikipedia.org/wiki/Window_function#Blackman_window), blackmanminlobe (https://en.wikipedia.org/wiki/Window_function#Nuttall_window,_continuous_first_derivative) and lanczos (https://en.wikipedia.org/wiki/Window_function#Lanczos_window) are already sinc kernels of adjustable length with a window. But I can add a configurable Tukey (https://en.wikipedia.org/wiki/Window_function#Tukey_window) window; I’m not sure how useful it would be, but this is not much work anyway.

[DTL]

"replacing the current linear interpolation with a sinc one?"

Adding sinc interpolation option and a user-accessible switch to select between old linear and new sinc.

"Because sinc is infinite and has to be windowed. But which window?"

Simple 'natural-box-window' of filter-support that is not require additional computation or programming.
I.e. filter-support is about (number_of_user_provided kernel samples)*2+1.
Also I see you have filter support as user-definable in the field value and the Impulse may be array of any number of values. It greatly expands functionality of UserDefined2 function based on limited ability of passing only 2 kernel members via equivalent of BicubicResize 2-user arguments instead of unlimited list. I do not know how to pass any required number of user-provided kernel samples as an array so currently we have only fixed by 2 kernel samples UserDefined2 (in the plans are UserDefined4 or more) but if Avisynth API allow to pass any required kernel members in user-provided in the field array it is much better. For example for more precise test data generation we need up to 10 kernel members. They are provided in the book by Krivosheev in about 197x or 198x year (when engineers was high level and think about new digital video systems as very precise and high quality not what awful mix that we got today in 202x).

"It is expected that the user provides a pre-oversampled kernel "

Using sinc-interpolation allow to provide very short number of kernel samples down to 2 and it already works very well. For typical user to provide even 2 kernel samples is not very easy task. One of the main idea from that book about impulse responce of ideal band-limited data - if it is designed as a linear sum of weighted sincs it will be band-limited (close to ideally as sinc itself) with any values of weighting samples. And linear interpolation require user to provide highly oversampled kernel array as I see.

"blackman, blackmanminlobe and lanczos are already sinc kernels of adjustable length with a window. But I can add a configurable Tukey window; I’m not sure how useful it would be, but this is not much work anyway."

The most 'classic DSP/math' windows have special purposes in the other domain - like having special form of Fourier transform. And here we just make bug-fixed sinc resize for the very limited number of taps to get enough computation speed. And as the best bugfix it is better to have self-configured bugfix window without user thinking about it. From my quick tests simple trapezoidal starting from taps/2 works about good and not require additional user input and allow to lower non-visible distortions level taps number from about 70 to about 10 that is good speed increase because with typical good conditioned data we rarely need >10 or >20 taps in sinc kernel to good restore highest possible frequences. And set taps >70 for just fix end of kernel calcultaion issue is wasting of time and resources. Ofcourse it is better to search for better possible window to make sinc resize possible with as low taps as possible without that end of kernel issue _but_ still keeping as much as possible 'full-strike/full-blood' sinc lobes untouched for better restoration of highest valid frequencies with lowest possible kernel taps and so lowest possible filter support and highest possible computing speed.

It may be called 'fixed-sinc' without direct naming of the weighting window because it is programmer-side. The requirements for the window:
1. Work without computational errors at the end of kernel at least with 8bit output integer precision (for >8bit may be more taps or adjustments of window required)
2. Restore with higher possible amplitude the highest valid frequencies with equal number of taps with typical windows like Lanczos. Because we can set Lanczos with taps >50 and have about 10 starting lobes of sinc almost untouched but speed will be less in compare with sinc(taps=10).

Here is current state:
For linear in space and in levels test sequence (16), 27,132,235,132,27,(16)
Test pattern file link https://www.sendspace.com/file/ubhv78 . It is not 2D-ideal elementary point but for exact H and V directions from center it about good conditioned against ringing for sinc-based workflow.
Script

LoadPlugin("fmtcavs.dll")
LoadPlugin("ResampleMT.dll")

ImageReader("1_point_132_27_lim_lin.bmp")

ConvertToPlanarRGB()

#fmtc_resample(width*10, height*10, kernel="sinc", taps=10)
SincLin2ResizeMT(width*10, height*10, taps=10)

ConvertToRGB24()

Levels(10,1,50,0,255)

Results:
https://i5.imageban.ru/out/2021/08/26/396c1df1dbbb90b228a58a93f6cd4127.png
https://i6.imageban.ru/out/2021/08/26/6bcdc69aecc51e541c480c8cd871c14a.png
https://i4.imageban.ru/out/2021/08/26/26c22e0466aac8aefffadc9e394da833.png

Also the overall field of fmtc output is unclear with looks like +-1 LSB ringing error - may be something wrong with conversion from float to int like +0.5f rounding shift missed or added or compiler error with float to int conversion or other.

[cretindesalpes]

Also the overall field of fmtc output is unclear with looks like +-1 LSB ringing error - may be something wrong with conversion from float to int like +0.5f rounding shift missed or added or compiler error with float to int conversion or other.
fmtc_resample() outputs at least 16-bit data, so the subsequent ConvertToRGB24() has to convert the bitdepth too, and this is where lies the problem. Use fmtc_bitdepth(bits=8) or ConvertTo8bit(dither=1) in between.

EDIT: there’s also a bug with the fulls and fulld parameters in fmtc_resample, declared as int instead of bool as they should be. I will fix this quickly.

[Dogway]

Does fmtc_resample support jinc function? I'm trying to find out if I can use N.Robidoux ewa_lanczossharp with lobes = 3.2383154841662362 and blur = 0.9812505644269356.

[cretindesalpes]

That’s pretty accurate constants! But no, jinc is not supported at the moment.

[DTL]

"Use fmtc_bitdepth(bits=8)"

Yes - it helps. Though as it plugin for Avisynth - may be made its default output compatible with ConvertToRGB(24) and may be other without special steps. Some users may miss required steps for more quality result and got distortion.

[cretindesalpes]

I’m very likely to remove the planes parameter from fmtc_transfer introduced in r24. I added it because it was almost free, but with the upcomping addition of OOTFs, planes are no longer independant, and the parameter is making the implementation too complicated for my taste. Any objection?

[Quadratic]

"Use fmtc_bitdepth(bits=8)"

Yes - it helps. Though as it plugin for Avisynth - may be made its default output compatible with ConvertToRGB(24) and may be other without special steps. Some users may miss required steps for more quality result and got distortion.
fmtc was originally made for Vapoursynth, asking the developer to go out of their way to consider the behavior of decade-old functions is absurd. Avisynth users need to learn how to control their scripts and stop relying on internal operations.

Levels() may introduce some error when being used at RGB24, it would be better for you to perform that operation when the clip is still of high precision.

[DTL]

"a sinc one?"

I made some simulation with sinc-summing tool and made bc and bcde coefficients table for (almost) non-ringing kernel.
With conversion or bc(de) values to kernel program:

Filter(double _b, double _c) // same for d,e
{
a = 1.0; // 0 sample = 1
b = (double)clamp(_b, -50.0, 250.0); // 1 and -1 sample
c = (double)clamp(_c, -50.0, 250.0); // 2 and -2 sample
b = (b - 16.0) / 219.0;
c = (c - 16.0) / 219.0;
// same for d,e

}

double sinc(double value)
{

if (fabs(value)>0.000001)
{
value *= M_PI;
return sin(value)/value;
}
else return 1.0;
}

double Filter::f(double x) // kernel output
{
x = fabs(x);

if (x<=3)
// - here is rect window of kernel and =support, <=5 for bcde, practically with non-ringing bc(de) params this kernel is self-windowed outside 3..5 around center,
// i.e. it is both real space and Fourier-space constrained, so do not suffer from any significant members cut-off by rect window of enough size and Gibbs-effects
{
return c*sinc(x+2) + b*sinc(x+1) + a*sinc(x) + b*sinc(x-1) + c*sinc(x-2); // for bc
// for bcde
// return e*sinc(x+4) + d*sinc(x+3) + c*sinc(x+2) + b*sinc(x+1) + a*sinc(x) + b*sinc(x-1) + c*sinc(x-2) + d*sinc(x-3) + e*sinc(x-4);

}
else return 0;

}


The required number of user-provided samples very quickly fades and last e-member usually about zero (16 code value). And using bcd instead of bc allows to have lower level of ringing at high overshooting kernels or higher overshooting with less ringing.
https://i4.imageban.ru/out/2021/08/30/23ad291161c5f75384738ceaed2600db.png

[tormento]

I just ported my fmtconv

I am having an AviSynth exception when playing the following AVS with VirtualDub2 or encoding with x256 --crf 22 --preset slow --output-depth 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --input-depth 10, any idea?

Source is 1080p anime.

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Eva01\eva01.dgi")
ConvertBits(16)
SMDegrain(tr=4,thSAD=400,refinemotion=true,contrasharp=false,PreFilter=4,plane=4,chroma=true)
fmtc_bitdepth(bits=10,dmode=8)
Prefetch(6)

It works ok if I set fmtc_bitdepth(bits=8,dmode=8) or use a plain ConvertBits(10) instead.

[cretindesalpes]

tormento: Indeed, it is fixed now.

fmtconv r25 (https://forum.doom9.org/showthread.php?t=183139) is mostly a bug-fix release:
bitdepth/Avisynth+: fixed I420 input that couldn’t be converted to higher bitdepth. Thanks to StvG for the report.
matrix: fixed the output colorspace autodetection which failed with specific matrix combinations.
resample/Avisynth+: fixed fulls and fulld parameters which were wrongly defined as int instead of bool.
resample/Avisynth+: fixed cplaced which was wrongly copied from cplaces. Thanks to TbtBI for the report.
transfer: automatically adapts the output format (16-bit integer) if the input is a low-bitdepth clip and nothing is specified in bits nor flt, instead of emitting an error.
transfer: fixed the BT.2100 HLG curve that was inverted.
transfer: fixed the sRGB curve linear part and extended its positive range. Thanks to poisondeathray for the report.
Avisynth+: fixed a crash occuring when using data bitdepths between 9 and 14. Thanks to tormento for the report.

[tormento]

Indeed, it is fixed now.
Thank you, will try ASAP!

[DTL]

The download link in the first post for r25 do not work for unknown reason and github link only shows r24 build release. Though source code shows update to r25. Why no r25 release at github for download ?

[StainlessS]

r25 first post link works for me [just now], and worked on 19th sept too.

[real.finder]

r25 first post link works for me [just now], and worked on 19th sept too.

same, anyway there are also https://web.archive.org/web/20210919222003/http://ldesoras.free.fr/src/vs/fmtconv-r25.zip

[cretindesalpes]

Yes sorry I forgot to make a “Github release”. This is now done.

[DTL]

Yes - in r25 the HLG transfer looks like works much better.

Can I make LPF (convolution) with such impulse response using 'impulse' kernel setting:
https://i2.imageban.ru/out/2021/09/22/4e627652febaa9e86d1d64b11d2de4ee.png

Or direct 1/2 downsize of chroma planes with this kernel pre-filtering. I too lazy to program another kernel for Avisynth resampler (also known as having some bugs at the edges) to test it and remember fmtc provides some functionality for user-defined kernels. It is for 4:4:4 to 4:2:2 chroma subsampling operation. How to correctly define 'impulse' kernel is possible ? Do I need to interpolate it somehow or direct samples like half or full width (both) sides possible like 0.5;0.315;0;-0.09... ?

[cretindesalpes]

You can downsample the chroma like this:

ColorBars (pixel_type="YV24")
imp = [-0.00643, 0, 0.014234, 0, -0.027172, 0, 0.049515, 0, -0.097162, 0, 0.315252,
\ 0.5, 0.315252, 0, -0.097162, 0, 0.049515, 0, -0.027172, 0, 0.014234, 0, -0.00643]
fmtc_resample (Width (), Height (), css="420", kernel="impulse", impulse=imp, kovrspl=2)

Because your kernel is already a half-band filter, you have to specify that it is 2x oversampled with kovrspl. Custom kernels are expected to be full-band. If you just want to convolve without decimating, remove the css parameter, set kovrspl=1 and add fv=-1, fs=-1 to make sure that the convolution is not optimized out because source and destination windows are identical. However all planes will be processed, so you’ll have to specify planes=[2, 3, 3] to bypass the luma plane.

I realize that I forgot to mention in the doc that "impulse" should be specified as kernel parameter when using custom kernel, otherwise fmtc_resample ignores it.

Edit: oops it seems that planes doesn’t do anything here, another bug to fix…

[real.finder]

You can downsample the chroma like this:

ColorBars (pixel_type="YV24")
imp = [-0.00643, 0, 0.014234, 0, -0.027172, 0, 0.049515, 0, -0.097162, 0, 0.315252,
\ 0.5, 0.315252, 0, -0.097162, 0, 0.049515, 0, -0.027172, 0, 0.014234, 0, -0.00643]
fmtc_resample (Width (), Height (), css="420", kernel="impulse", impulse=imp, kovrspl=2)

Because your kernel is already a half-band filter, you have to specify that it is 2x oversampled with kovrspl. Custom kernels are expected to be full-band. If you just want to convolve without decimating, remove the css parameter, set kovrspl=1 and add fv=-1, fs=-1 to make sure that the convolution is not optimized out because source and destination windows are identical. However all planes will be processed, so you’ll have to specify planes=[2, 3, 3] to bypass the luma plane.

I realize that I forgot to mention in the doc that "impulse" should be specified as kernel parameter when using custom kernel, otherwise fmtc_resample ignores it.

Edit: oops it seems that planes doesn’t do anything here, another bug to fix…

impulse can work with array? it should be array parameter then https://github.com/AviSynth/AviSynthPlus/issues/226#issuecomment-914035294

[DTL]

It works close to as expected with command

fmtc_resample(impulse="-0.00643,0,0.014234,0,-0.027172,0,0.049515,0,-0.097162,0,0.315252,0.5,0.315252,0,-0.097162,0,0.049515,0,-0.027172,0,0.014234,0,-0.00643", fh=-0.5, fv=-0.5, cnorm=true,center=false,fulls=true, fulld=true)

Looks like it is not scaled in that table for input sampling, so fh and fy is -0.5. Also normalization enabled.

"kernel="impulse""

List of 'kernel' param values do not have 'impulse' value in the provided documentation.

" "impulse" should be specified as kernel parameter when using custom kernel, otherwise fmtc_resample ignores it."

It looks like working here with version r25 even without setting kernel="impulse".
The result have significally wider transient field in compare with provided amplitude-frequency response at that drawing and in compare with hard cuf-off lowpass with something like

SincResize(width/2,height/2,taps=16)
SincResize(width*2,height*2,taps=16)


Source:
https://i1.imageban.ru/out/2021/09/23/54d40a2105f86c3fbc88658548c4f11f.png

This impulse kernel process with command above:
https://i7.imageban.ru/out/2021/09/23/519afd0804023b13d86781b7ecbcfeb1.png

SincResize down and up
https://i1.imageban.ru/out/2021/09/23/cae97582abd1dc0445edd01a4ed990be.png

[DTL]

Is it a bug in fmtc_transfer (2084):


LoadPlugin("fmtcavs.dll")

BlankClip(100,100,100,color=$101010, pixel_type="RGB24")
Trim(0,1)
ConvertToRGB24(matrix="PC.709")

ConvertToPlanarRGB()
ConvertBits(16)

fmtc_transfer(transs="2084", transd="linear", fulls=true, fulld=true,bits=32, flt=true)

fmtc_transfer(transs="linear",transd="2084", fulls=true, fulld=true)
fmtc_matrix(mat="2020",fulls=true,fulld=true)
ConvertBits(10)


Output Y'=0x004C.
0x0040 expected (in 709 and hlg it is).

It looks fmtc_transfer(transs="2084", transd="linear", fulls=true, fulld=true,bits=32,flt=true) do not switches to float(32) RGBPS (silently, no error raised), stays in RGBP16 and lost precision.
fmtc_transfer(transs="2084", transd="linear", fulls=true, fulld=true,bits=32) - switches to RGBPS and output OK.
flt is not listed in Avisynth syntax but somehow works and produces bugs in Avisynth. May be raise an error in Avisynth if flt is in arguments with any type of argument ?

[cretindesalpes]

real.finder:

Yes impulse can be an array of float or a string containing printed numbers separated with a space. That’s why it is a '.' variant type.

DTL:

You have to specify kernel="impulse" for custom kernels. Otherwise the default kernel is used (spline36). Your picture shows the spline36 kernel, not the custom one.

You can set fh and fv to -0.5 but IMHO for this case it makes more sense to specify that the kernel is oversampled.

For the HLG curve, indeed adding
ScriptClip ("""Subtitle (String (AverageLuma ()))""") at the end of the script shows 76 with r25, I don’t know exactly why. However my current build displays 64 as expected. I’m doing a lot of modifications to transfer which are not committed yet, so I guess I fixed it while I was about it. Current transfer works always in integer with integer input, because it is just a LUT. But the result should be exact if target format is floating point.

I also fixed the flt and bits combination in resample.

[real.finder]

array with only '.' variant type will not be work in next update of avs+ unless it become '.+'

[cretindesalpes]

This is really unfortunate so I read in the readme for the 3.6.0 release that ".+" and ".*" cannot be followed by other arguments. Does this mean that I can only have one parameter like this per function? I tried to replace every "." with ".+" or ".*", but this crashes Avisynth+. Is there any workaround or should I remove the arrays and keep only the strings?

[kedautinh12]

Avs+ last ver
https://forum.doom9.org/showthread.php?p=1953038#post1953038

[pinterf]

This is really unfortunate so I read in the readme for the 3.6.0 release that ".+" and ".*" cannot be followed by other arguments. Does this mean that I can only have one parameter like this per function? I tried to replace every "." with ".+" or ".*", but this crashes Avisynth+. Is there any workaround or should I remove the arrays and keep only the strings?
Do you have them as named arrays, or unnamed ones?
It is true that unnamed-untyped arrays can appear only as the very last parameter in Avisynth. But when they have names and you pass their values with name specifier, well, this has to work.
(at least in latest 3.7.1 test version, which has some fixes regarding array parameters, mainly because of script array syntax)
If your array can hold only strings/floats then specify the array with the exact type, e.g. "[paramname]s+" or "[coeffs]f+".

[StainlessS]

I know very little (nothing) about these new array things, but I know I can use named optional args after data arrays.


env->AddFunction("RT_WriteFile", "ss.*[Append]b",RT_WriteFile, 0);



RT_WriteFile(String FileName, string format, dat1, ... , datn, bool "Append"=False)


Where (unnamed) data args [dat1 ... datn] can be zero or more of any type (except clip in this case - internally trap/abort on clip args), and if you need to specify "Append" arg, then have to use eg "Append=True" with explicit (optional) name, ie name not optional and must be specified.

I would think this would work (or could be made to work) with new arrays too.

EDIT: Above works in v2.58, v2.57 too I think, and maybe prior to that.

EDIT: works

RT_WriteFile(".\temp.txt","%d # comment=%s",current_frame,"some text",Append=true)

dont work (true interpreted as data arg, function throws error as format string specifies int %d and string %s data args only expected)

RT_WriteFile(".\temp.txt","%d # comment=%s",current_frame,"some text",true)

[cretindesalpes]

pinterf:

Yes, the parameters are named. I didn’t pay attention to this aspect when reading the readme, so finally there isn’t any problem here.

Anyway the ".+" behaviour is slightly different of ".". When reading its value on the plug-in side, the parameter is always set as defined and typed as an array, possibly empty. If the parameter is set by the user to a single value (not an explicit array), this value comes as the first member of the array instead of being the parameter itself. However this is not a big deal, I wrote some code to work around it.

[real.finder]

Anyway the ".+" behaviour is slightly different of ".". When reading its value on the plug-in side, the parameter is always set as defined and typed as an array, possibly empty. If the parameter is set by the user to a single value (not an explicit array), this value comes as the first member of the array instead of being the parameter itself. However this is not a big deal, I wrote some code to work around it.

in last update Avisynth+ 3.7.1 test build 17 (20210924) .+ should not always set as defined, but yes it always array

[tormento]

@cretinedesalpes

What can we use as a "modern" version of GradFun3 after your last releases? I have a badly mastered anime with coarse banding and I would like to try to smooth it out.

[cretindesalpes]

Tools included in the Dither package remain unchanged at the moment. You can still use GradFun3, it works.

[DTL]

Is this an issue with full/limited:

LoadPlugin("fmtcavs.dll")

plY=Blankclip(width = 10, height = 1, pixel_type="Y10").Expr("sx 0 == 64 sx 1 == 64 sx 2 == 64 sx 3 == 64 sx 4 == 64 sx 5 = 64 sx 6 = 64 sx 7 == 64 sx 8 == 64 64 ? ? ? ? ? ? ? ? ?")
plU=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 512 sx 1 == 512 sx 2 == 512 sx 3 == 512 512 ? ? ? ?")
plV=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 512 sx 1 == 512 sx 2 == 512 sx 3 == 512 512 ? ? ? ?")
CombinePlanes(plY, plU, plV, "YUV", "YYY", pixel_type="YUV422P10")

ConvertToYUV444()

PointResize(500,500)
ScriptClip ("""Subtitle (String (AverageLuma ()))""")

rgb_lin=fmtc_matrix(mat="RGB", mats="2020",fulls=false, fulld=false)
rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear",bits=32,fulls=false, fulld=false)

return fmtc_transfer(rgb_lin,transs="linear",transd="709",fulls=false, fulld=false).ConvertBits(8).ConvertToRGB()

return R'G'B' 0,0,0.
expected 16,16,16 (limited black).
I hope input to fmtc part of processing is Y=64, U=512, V=512 (10-bit black limited). ScriptClip(AverageLuma) confirms Y=64.00.

[cretindesalpes]

Fmtconv always uses ”full range” floating point values whatever the specified full* parameter value, because the narrow range is meaningless for float. Avisynth+ converters don’t work like this and use a narrow range for float by default. You can add fulls=true, fulld=false to ConvertBits(8) and get the expected result.

[StvG]

Thanks for adding top_left chroma location.

fmtc_matrix - with csp data type cannot be changed (int->float). It seems the same is valid for bits parameter too (but not mentioned in the readme)?
csp doesn't accept rgbpx values: ColorBars(pixel_type="yv24")
fmtc_matrix(mat="rgb", mats="2020", csp="rgbp10")

[DTL]

Fmtconv always uses ”full range” floating point values whatever the specified full* parameter value"

If software currently can not provide any warning about non-compatible user input with other enviroment (and internal software features)- may be better to fail processing with stop-error (and some useful hint messages why this stop occur) instead of silently ignoring user control param and creating unexpected output.

Also float32 processing can process shifted and scaled "limited" levels encoding without significant losses but may reqiure more operations and will be just slower.

[cretindesalpes]

StvG:

Yes at the moment the data type cannot be changed. This may change later, but for now I’ll update the doc.

DTL:

In my previous message I enclosed full range between double quotes, because range hasn’t any meaning for float. It applies to integer and indicates the values coding 0.0 and 1.0 or -0.5 and +0.5. But the float values are the real values we want to process, the real number scale for the algorithms. There is no need for this kind of full/TV coding, it doesn’t apply. And reflecting the narrow range coding in float with an offset and scale doesn’t make any sense, I would even say it’s an insanity. With floating point data, we have both the headroom and the dynamic range, and even more than with integers. Why make things more complicated than they should be? And issuing an error when using the fulls or fulld with float would most likely break wrapping functions that can process different data formats.

[DTL]

"And reflecting the narrow range coding in float with an offset and scale doesn’t make any sense, I would even say it’s an insanity."

It is good if programmer live in float world only. But in current reality we still use generally integer-coded low bits number values but in the new awful HDR-world need to deep into float to not lost precision in linear HDR (I not sure if using integer 'limited' 32bit per sample available/possible - not tested, though 32bit integer in theory enough to cover 1..10 milion linear range of typical todays HDR PQ). So most of workflows in nowdays HDR reality (in september 2021 EBU Recs r153 and r154 make deprecated SDR transfer at all) are 'integer(transfer_domain compressed)->float(linear,expanded)->integer(transfer_domain compressed)" and if float domain silently shift and scale the range it creates unexpected errors. User assumes dataset already in 'limited' encoding and if create again conversion to 'limited' looks like error situation - twice conversion to 'limited' that in typical life will cause levels error.
It is good to add to documentation about 'fmtc always will shift and scale 'limited' integer encoded input data to 0..1 range so if user need 'transparent' workflow - the output data again need to be converted to 'limited' mode (and before touching integer domain to prevent rounding and limiting errors).

"Why make things more complicated than they should be?"

It is just 'common practice' that most of moving pictures processing world uses not very programmer-friendly 'limited' mode of black and white data encoding (in the integers). Also to use signed integers with black at zero is mean to lost almost half of available integer range. It is again sad shadows of poor past where every codelevel of 8 and 10 bit integers was long debated. So the best result for 8 and 10 bit poor world was 'shifted unsigned integer' encoding.

[StvG]

StvG:

Yes at the moment the data type cannot be changed. This may change later, but for now I’ll update the doc.

Just to be sure you didn't overlooked it:

csp doesn't accept rgbpx values: ColorBars(pixel_type="yv24")
fmtc_matrix(mat="rgb", mats="2020", csp="rgbp10")

This is ok:BlankClip(pixel_type="rgbp")
fmtc_matrix(matd="709", mats="rgb", csp="yuv444p10")

This doesn't work:BlankClip(pixel_type="yv24")
fmtc_matrix(matd="rgb", mats="709", csp="rgbp10")

[cretindesalpes]

Ah sorry I see now, RGBP* values were not interpreted as planar. The bug fix will be available in the next release.

[cretindesalpes]

fmtconv r26 (https://forum.doom9.org/showthread.php?t=183139):
matrix: fixed the output colorspace when using singleout.
matrix: fixed the output colorspace when using a custom matrix. Thanks to vxzms for the report.
matrix/Vapoursynth: _Matrix and _ColorSpace frame properties are actually deleted when a custom matrix is used or the final colorspace is unknown. Thanks to mysteryx93 for the report.
resample: fixed empty custom impulse that could cause a crash.
resample: added top-left chroma location.
resample/Avisynth+: interlacing detection now uses the global stream information when the frame properties are not available.
resample/Avisynth+: fixed the planes parameter.
transfer: Linear light can now be display- or scene-referred. Added sceneref, lw, lws, lwd, lb, ambiant and deprecated blacklvl.
transfer: By default, automatically matches the reference white levels for source and destination transfer curves. This may cause some backward incompatibilities. match parameter added.
transfer: Removed the planes parameter introduced in r24 because planes are no longer independent of each other.
transfer: debug provides information about the transfer operation and levels as a frame property.
transfer: BT.470M characteristic is now a pure power curve, instead of a copy of the sRGB curve.
transfer/Avisynth+: fixed flt + bits combination that was ignored. Thanks to DTL for the report.
transfer/Avisynth+: fixed case sensitivity to transs and transd.
transfer/Vapoursynth: fixed logceis and logceid that were missing from the function registration since their introduction in r23… Thanks to groucho86 for the report.
Fixed compatibility with Avisynth+ 3.7.1.
Avisynth+: fixed "RGBPxx" colorspace values that weren’t correctly interpreted as planar.

[StainlessS]

Whats new pussycat :)

Alert from Firefox about insecure download [due to non https], nothing to worrry about.

Thanks.

[hello_hello]

Would someone be so kind as to clarify my interpretation of the fmtconv help file in relation to the "center" argument?

The help file says the default for center is true.

center
Like the Avisynth standard resizers, this resizer preserves the position of the picture center.

my interpretation:
"This resizer assumes mpeg1 chroma placement by default, as Avisynth's resizers do."

So far so good, I assume to change that behaviour, "center" should be false.

cplace, cplaces, cplaced
Placement of the chroma samples.
The chroma placement is ignored when center is set to False or kernel to "point".

But then if centre=false the chroma placement is ignored?
Should the last line say "when center is set to True...."?

[cretindesalpes]

The help file says the default for center is true.
center
Like the Avisynth standard resizers, this resizer preserves the position of the picture center.
my interpretation:
"This resizer assumes mpeg1 chroma placement by default, as Avisynth's resizers do."
No, this is not really related to chroma placement, the statement works for any plane of any colorspace. It just means that the sampling grid coincides with the center of each (rectangular) displayed pixel, and that resize operations keep the center of the picture at the same relative location. Borders are not cropped nor extended. It’s sounds trivial because this is what users expect in general, but mathematically speaking it isn’t. There are other possible implementations. The most naive one, preserving the sampling position of the top-left corner for example, results in a slight picture shift along the top-left–bottom-right axis. This is what center = False does. PointResize is an exception in Avisynth built-in resizers and does it too; notice the shift between both pictures:
Version ().ConvertToPlanarRGB ().BicubicResize (1920, 720)
Interleave (PointResize (640, 240), GaussResize (640, 240, p=100))
Remarks:
1. GaussResize (p=100) is more or less equivalent to PointResize with a correct center preservation, so it’s easier to compare both.
2. I added ConvertToPlanarRGB because interleaved RGB frames are stored upside down like BMP pictures, and the shift would be vertically inverted.

BTW in fmtconv, default chroma placement is MPEG-2.

cplace, cplaces, cplaced
Placement of the chroma samples.
The chroma placement is ignored when center is set to False or kernel to "point".
But then if centre=false the chroma placement is ignored?
Should the last line say "when center is set to True...."?
The documentation stands correct. center = True is the default behaviour. When center = False, all planes are processed “naively”, without compensation to preserve the center of the picture and the plane’s relative positions to each other.

[hello_hello]

cretindesalpes,
thank you very much for the clarification.

[cretindesalpes]

fmtconv r27 (https://forum.doom9.org/showthread.php?t=183139):
matrix: fixed coefficient errors when using singleout.
primaries: added Sony P22 primaries.
transfer: better selection between linear and logarithmic input for LUTs, LUT size reduced.
transfer: fixed crash with grey colorspace. Thanks to vxzms for the report.

[gispos]

Why is the download blocked by Firefox? Security risk.

[cretindesalpes]

Probably because the download link is from a http site, not https. You can tell Firefox to bypass this security check. The releases are also available on Github (https://github.com/EleonoreMizo/fmtconv/releases) using https if http gives you shivers.

[gispos]

Probably because this is a http site, not https. You can tell Firefox to bypass this security check. The releases are also available on Github (https://github.com/EleonoreMizo/fmtconv/releases) using https if http gives you shivers.
OK, thanks

[tormento]

Probably because this is a http site
Actually, I open doom9 by https. Did you mean the file link?

[cretindesalpes]

fmtconv r28 (https://forum.doom9.org/showthread.php?t=183139):
bitdepth: Faster void and cluster pattern generation. Larger patterns are allowed.
matrix: Added an alias for "fcc"
primaries: New presets: Free Scale-gamut, DaVinci Wide Gamut, DRAGONcolor, DRAGONcolor2, REDcolor, REDcolor2, REDcolor3, REDcolor4 and REDWideGamutRGB.
primaries: Fixed Sony P22 primaries.
primaries: Removed the deprecated "dcip3" alias for "p3d65".
transfer: Added DaVinci Intermediate, RED Log3G10, REDlog, Cineon and Panalog.
Vapoursynth and Avisynth+ plug-ins are now contained in the same binary file.

[StainlessS]

Why is the download blocked by Firefox? Security risk.
Actually, I open doom9 by https. Did you mean the file link?
Firefox still blocks download, but you can open "Downloads", right click the RED marked download and "Allow Download",
its just as cretindesalpes said
You can tell Firefox to bypass this security check.

Thanks for the update C.

[Dogway]

Thank you!

[Dogway]

@cretindesalpes: Can you have a look at the graph (https://www.desmos.com/calculator/fddyni7beb) and check my weights are correct? I compared for example BlackmanMinLobe (Nuttall in the graph) and my version is softer:

fmtc_resample(round(w*1.5),round(h*1.5),kernel="impulse",kovrspl=2,cnorm=true,center=true,impulse=[-0.00024,0,0.01005,0,-0.09096,0,0.5812,1.0,0.5812,0,-0.09096,0,0.01005,0,-0.00024])

This is 4 taps or supposed to be. Also is it possible to use Jinc (as in EWA) weights?

[cretindesalpes]

I haven’t checked the coefficient details, but the kernel oversampling looks way too low. Currently custom kernels are linearly interpolated (this could change in the future with spline or sinc interpolation) so with 2× you’ll get some triangular shapes between zero crossings—far from the function you could expect. You need something like 16× or more to get decent results.

Jinc is a different beast because the kernel is not separable in horizontal and vertical passes. It requires a single 2D pass; the current code is not designed to handle this. It may be a future addition (https://github.com/EleonoreMizo/fmtconv/issues/4) too.

[Dogway]

I see, I thought it was spline interpolated. With kovrspl=16 I get grey output or scanline type artifacts (depends if normalization is true).
The way I compute coefficients in the graph is by taking the y-intercept every 0.5 units in x. This looked similar to DTL's impulse values (https://forum.doom9.org/showthread.php?p=1952985#post1952985)

imp=[-0.00024*16,0,0.01005*16,0,-0.09096*16,0,0.5812*16,1.0*16,0.5812*16,0,-0.09096*16,0,0.01005*16,0,-0.00024*16]
imp=[-0.00024,0,0.01005,0,-0.09096,0,0.5812,1.0,0.5812,0,-0.09096,0,0.01005,0,-0.00024]
fmtc_resample(1280*2,634*2, kovrspl=16, cnorm=true, center=true, kernel="impulse",impulse=imp)

[cretindesalpes]

Ah no actually the 16× factor applies to the kernel sampling interval. So you have to fill the array with the kernel values for each 1/16 unit in [-4 ; 4], instead of each 1/2 unit with kovrspl=2. Don’t multiply the values by 16.

[Dogway]

Wow thanks, that's gonna be a long array, thank you!

EDIT: Maybe I can do something with Spline (http://avisynth.nl/index.php/Internal_functions#Spline).

[Dogway]

Finally went with 64 (32 each side). But I had to set kovrspl to 8 and was about to ask why. Well I guess I found out, oversampling by 8 in unit interval, so "32 / kovrspl = taps".

Here's my call, 'nuttall' matches 'blackmanminlobe'. I guess kovrspl doesn't affect performance. Thanks again!

nuttall = [1,0.968978694959837,0.8800877014862263,0.7450789501548785,0.5811536099884803,0.4079380255054445,0.24421357419883338,0.10506827778662947,0,-0.0677506795383086,-0.1006542288117901,-0.10540908368482169,-0.09096328276239886,-0.06652252976938802,-0.0399481722582298,-0.016788722421354926,0,0.0097560148734007, \
0.013447841939631484,0.012890350335053615,0.010046723030434222,0.0065452031473278455,0.0034500048853390547,0.0012517909430134226,0,-0.0005100722653928198,-0.000565616944011163,-0.00041987662546901855,-0.00023981418211867437,-0.00010446859341761584,-0.00003063039438713025,-0.000003718533785179404]

a=fmtc_resample(1280*2,634*2, kovrspl=8, cnorm=true, center=true, kernel="impulse",impulse=ArraySym(nuttall))
b=fmtc_resample(1280*2,634*2, cnorm=true, center=true, kernel="blackmanminlobe")
ex_makediff(a,b, metric="none", aug=true, dif=true, show=2)

[DTL]

Jinc can be calculated in 1D as usual function. So it can be provided as impulse kernel to exist resampler of 1D. It is simply replaced sin(x)/x to bessel_J1(x)/x. Also typically as Lanczos is sinc weighted by sinc so real working low taps jinc is jinc weighted by jinc (or any other weighting). Though as jinc(x) fades significantly faster in compare with sinc(x) with increasing of x it require less weighting.
https://github.com/Asd-g/AviSynth-JincResize/blob/b7fbf5d680a2950dff65b907134e6719efd11916/src/JincResize.cpp#L915

Also _j1(x) is standard lib C function and not need complex calculating each time in each handcrafted software. https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/bessel-functions-j0-j1-jn-y0-y1-yn?view=msvc-170

The JincResize is 2D processing 1-pass. But any other kernel may be calculated as 2D kernel f_2D(x,y) = f_1D(raduis_vector_length(x,y)) and used in 2D 1-pass processing. The difference is only in the processing engine - 2D single pass or 1D+1D 2 passes.

Main difference between sinc and jinc - the bessel_J1 is not = 0 at the regular offset like sinc(x) (neither 1D offsets nor 2D offsets). So when doing jinc-interpolation (upsizing) - the input samples do not keeps its values in the output. It is somehow viloates the idea of upsize interpolation as calculating only intermediate samples and keep input unchanged. Same for sub-sample shifts. So it looks all nice math of sinc in 1D do not work in 2D and with jinc too.

"I thought it was spline interpolated."

May be it is better to provide more 'real' kernel samples instead of rely on interpolation inside. Current input is not limited with number of provided kernel samples ?

"You need something like 16× or more to get decent results."

May be it is possible to somehow allow user to provide analytical math equation for kernel (with weighting) ? So it will save user from providing lots of kernel samples for non-integer resampling ratios or some filtering.
Like via some math interpreter. It may slow down processing very because of lots of kernel samples required for each output sample. But for some science projects it may make development faster. Or provide some software tool (plugin to some math CAD ?) to convert analytical kernel formula into sampled vector of any required precision (sampling interval) and generate text string to copy-paste into AVS script.

[Dogway]

Playing with some fmtc_resample() I noticed that "fmtc_resample(css="41")" doesn't work, since fmtc_resample() always output 16-bit and YUV411 HBD is not supported in AVS+.
Also checked "fmtc_resample(cplaced="tl")" which doesn't write frame properties.
And:
fmtc_resample(1920,1034,invks=true,kernel="bicubic",fh=2.6,fv=2.6,a1=-0.6,a2=0.4)
produces a crash

[cretindesalpes]

fmtconv r29 (https://forum.doom9.org/showthread.php?t=183139):
Vapoursynth: switched to API v4. Fmtconv now requires Vapoursynth r55 or later, API v3 is not supported anymore.
resample: Fixed degnerated cases with invks causing a crash. Thanks to Dogway for the report.
resample/Avisynth+: fixed frame property writing when only cplaced is defined, thanks to Dogway for the report.
transfer: Added sigmoid curve as a transfer function.
transfer: Fixed the float to integer path.

[hello_hello]

cretindesalpes,
Using something similar to one of the examples in your help file (Avisynth+ 3.7.2 and fmtconv r29), propShow() says for the output video _ColorRange=0=full, when for a YV12 source I would've expected it to be either limited or not set at all.

fmtc_bitdepth(bits=16)
fmtc_resample(css="444")
fmtc_matrix(mat="709", bits=16)
fmtc_transfer(transs="709", transd="linear")
fmtc_primaries(prims="709", primd="601-625")
fmtc_transfer(transs="linear", transd="601")
fmtc_matrix(mat="601", bits=16)
fmtc_resample(css="420")
fmtc_bitdepth(bits=8)
propShow()

Best as I can tell the transfer function is setting _ColorRange=0 (why is that when the input is RGB?) but the second matrix function isn't updating _ColorRange when converting back to YUV, and fmtc_matrix(mat="601", bits=16, fulld=false) is required to correct it.

The output looks the same with or without fulld=false, so I assume it's just the _ColorRange property that's incorrect and the video is still being converted to limited range.

Is that expected behaviour?

Cheers.

[cretindesalpes]

hello_hello:

Yes it is the expected behaviour. From the matrix doc:
The _ColorRange frame property is set if the fulld parameter has been explicitely defined.
And if fulld is not defined, the property is just copied from the source frame. But this doesn’t make sense when using a matrix preset because the target range is always known. So I might change this behaviour in the next release.

[tormento]

Can I use fmtconv to replace the lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

and

z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

?

I'd like to try if there is any speed improvement over z.

[Dogway]

I think there might some error in the docs, for bobbing an interlaced source it shows interlaced(d) args as boolean, but in the function description as int type.

Also I'm trying to understand how do you do a chromatic adaptation with fmtconv. I tried the next but the result looks noop.

fmtc_resample(css="444")
fmtc_matrix(mat="709", bits=16)
fmtc_transfer(transs="709", transd="linear")
fmtc_primaries(prims="709", primd="709", ws=[0.3127,0.3290], wd=[0.33242, 0.34743]) # From D65 to D55
fmtc_transfer(transs="linear", transd="709")
fmtc_matrix(mat="709", bits=16)

[tormento]

I just ported fmtconv Vapoursynth plug-in to Avisynth+.
We need you. :)

[cretindesalpes]

tormento:

Yes you can. I’m pretty sure z is faster, but you can try.

Dogway:

Thanks, indeed the examples haven’t been checked for long, and function prototypes slightly evolved since. I just fixed the doc.

Your chromatic adaptation is right. How do you check it? Results may be subtle. Or maybe what you’re looking for is complete conversion, for example to make a picture displayed on a D55 monitor look exactly like the same picture displayed on a D65 monitor placed side by side?

[Dogway]

Yes I was expecting a warmer output, maybe I should try with D50. Eyes adapt to the white point specially at matching environment temperatures as there's not a single "white" in psychovisually terms. I was double checking as I had difficulties on color space conversions on non-standard illuminants.

[tormento]

tormento:Yes you can. I’m pretty sure z is faster, but you can try.
Can you please translate them for me? The AVS porting doc is a bit obscure and there are missing parts too.

[Dogway]

Can I use fmtconv to replace the lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")


Maybe something like this?

fmtc_bitdepth (bits=16)
fmtc_resample (css="444",kernel="spline64")
fmtc_matrix (mats="2020", matd="rgb", fulls=false, fulld=true)
fmtc_bitdepth (bits=16, dmode=6)

EDIT: Correction, fmtconv does dither from float to 16-bit int, but the effect is minimal.

[StvG]

I would say this is the equivalent of that z_ line:
fmtc_bitdepth(bits=32)
fmtc_resample(css="444",kernel="spline64")
fmtc_matrix(mats="2020", matd="rgb")
fmtc_bitdepth(bits=16, dmode=6)

[tormento]

I would say this is the equivalent of that z_ line:
fmtc_bitdepth(bits=32)
fmtc_resample(css="444",kernel="spline64")
fmtc_matrix(mats="2020", matd="rgb")
fmtc_bitdepth(bits=16, dmode=6)
I guess 4 calls are slower than a single z one.

[DTL]

It may also cause massive memory overload with lots of cached frames with massive multithreading at todays AVS+ at todays massive multicore end-user CPUs. So single filter processing may be much more RAM friendly.

[tormento]

I was getting crazy, trying to find why BM3D_CUDA had some slight tint change and I found out that it is the combination of Convertbits and fmtconv that makes strange things.

Look at this comparison (https://slow.pics/c/LkQ6sKsA).

Reality is that BM3D_CUDA needs RGB to work and... am I doing something silly or there is something terribly wrong?

Image 0:

original

Image 1: OK

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
ConvertBits(8)
Prefetch(6)

Image 2: WRONG!!!

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
fmtc_bitdepth (bits=8,dmode=8)
Prefetch(6)[

Image 3: WRONG!!!

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
fmtc_bitdepth (bits=32,dmode=8)
ConvertBits(8)
Prefetch(6)

Image 4: OK

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
fmtc_bitdepth (bits=32,dmode=8)
fmtc_bitdepth (bits=8,dmode=8)
Prefetch(6)

[StvG]

I was getting crazy, trying to find why BM3D_CUDA had some slight tint change and I found out that it is the combination of Convertbits and fmtconv that makes strange things.

Look at this comparison (https://slow.pics/c/LkQ6sKsA).

Reality is that BM3D_CUDA needs RGB to work and... am I doing something silly or there is something terribly wrong?

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).
For Image 3 change ConvertBits(8) to ConvertBits(8, fulls=true, fulld=false)

[tormento]

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).
For Image 3 change ConvertBits(8) to ConvertBits(8, fulls=true, fulld=false)
Please explain me why and why it's not "transparent" to fmtconv but they work only with the same function properly.

[tormento]

Ok, here things get tricky.

I need to convert to 32, then planar RGB and back to 8.

I am getting some tint changes (https://slow.pics/c/403yxn91) again.

00

original

https://i2.lensdump.com/i/1ELwFZ.md.png (https://lensdump.com/i/1ELwFZ)

01

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
ConvertToPlanarRGB()
ConvertToYUV420()
ConvertBits(8)

https://i3.lensdump.com/i/1ELy0P.md.png (https://lensdump.com/i/1ELy0P)

02

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32, fulls=false, fulld=true)
ConvertToPlanarRGB()
ConvertToYUV420()
fmtc_bitdepth (bits=8,dmode=8)

https://i.lensdump.com/i/1EL5Bm.md.png (https://lensdump.com/i/1EL5Bm)

Even a

ConvertToPlanarRGB()
ConvertToYUV420()
fmtc_bitdepth (bits=8,dmode=8)

Gives slight wrong tint again.

https://i1.lensdump.com/i/1ELGoi.md.png (https://lensdump.com/i/1ELGoi)

It happens even with:

ConvertToPlanarRGB()
ConvertToYUV420()

I am getting crazy with conversions.

EDIT: it seems that ConvertToYUV420(matrix="Rec709") fixed things. WTF.

[tormento]

I talked too early (https://slow.pics/c/9GiDGmGp)...

0

Original

1

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
DeBicubicResizeMT(target_width=1280,target_height=720,threads=1)
ConvertBits(32)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")
DeBicubicResizeMT(target_width=1920,target_height=1080,threads=1)
ConvertBits(8)

2

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
DeBicubicResizeMT(target_width=1280,target_height=720,threads=1)
ConvertBits(32, fulls=false, fulld=true)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")
DeBicubicResizeMT(target_width=1920,target_height=1080,threads=1)
fmtc_bitdepth (bits=8,dmode=8)

I smashed my head against the wall enough, need your help.

[DTL]

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).


Do ConvertBits(32) really process fulld argument anyway ? As I start to learn in the world of AVS and plugins programmers the 32bit float always treated as 0..1.0 nominal range with black at 0.0f and always internally converted into this range. It is very important to know that any going into float format around AVS and its plugins always require to convert back to narrow range in integer.

"ConvertBits(32)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")"

It looks in 32bit float matrix=Rec709 is no-op for range change ? It may be hidden feature of 32bit float AVS format.

"ConvertBits(8) "

It looks after any return back from float to integer you need to convert to narrow range (as usual integer moving picture data representation), so correct may be
ConvertBits(8, fulls=true, fulld=false)
or shorter
ConvertBits(8, fulld=false) because fulls for 32bit float input will be auto-detected as true (?).

AVS defaults looks like source of error if going into float: http://avisynth.nl/index.php/ConvertBits
bool fulls = (auto)

Conversion from and to float is always full-scale.

bool fulld = fulls

So as default 32bit float is full range and default fulld = fulls the simple
ConvertBits(32)
ConvertBits(8)
will cause range convert from narrow to full. The only safe and transparent ConvertBits is 8<->16 integer.

[tormento]

Conversion from and to float is always full-scale.
Ok but why the "fix" on Convertbits(32) worked and now, that I have a planar to YUV, not anymore?

[FranceBB]

Conversion from and to float is always full-scale.

bool fulld = fulls

So as default 32bit float is full range and default fulld = fulls the simple
ConvertBits(32)
ConvertBits(8)
will cause range convert from narrow to full.

Well... that's what I thought too, but actually it's not.
When you go to 32bit float, but stay in YUV, looks like it's still limited tv range, in fact even a simple:


ColorBars(848, 480, pixel_type="YV12")
ConvertBits(32)
TurnRight.Histogram.TurnLeft


shows that levels are still limited tv range:

https://i.imgur.com/0fG17aB.png


The conversion to Full Range happens when Tormento calls "ConverttoPlanarRGB()" 'cause RGB by standard is full range.
My "theory" was like yours, namely that when he called back ConverttoYUV420(matrix="Rec709") it was going to clip values rather than converting them back to limited, but no, I was wrong, in fact by doing all the operations he has in the script (except for Waifu) we get a perfectly identical image in terms of levels:

#Limited tv range source
ColorBars(848, 480, pixel_type="YV12")
original=last

#We go to 32bit float limited tv range
ConvertBits(32)

#We go from Limited TV Range YUV to Full Range RGB 32bit float
ConvertToPlanarRGB()

#We go back to Limited TV Range YUV 32bit float
ConverttoYUV420(matrix="Rec709")

#We go back to Limited TV Range YUV 8bit planar
ConvertBits(8)
converted=last

#We compare the two images
StackHorizontal(converted, original)

https://i.imgur.com/41Fyapw.png


So... it must be Waifu screwing it up, there's no other way...

[tormento]

So... it must be Waifu screwing it up, there's no other way...
So both convertbits and fmtc (as last command) should give the same output, thing that doesn't happen.

[cretindesalpes]

tormento:

In [2], replace the last line with:

fmtc_bitdepth (bits=8, dmode=8, fulld=true)
propSet ("_ColorRange", 1) # Limited

The reason:

AviSynth+ uses full and limited ranges with float, using 16/256 and 235/256 [Edit: /255, not /256] for black and white in limited range, same with chroma data. Avs+ fmtconv is derived from its original Vapoursynth version, where nominal range for float is always [0 ; 1] and [-0.5 ; +0.5] for chroma information, and full/limited concept doesn’t apply (however range information can be carried as frame property).

So, just before the bitdepth conversion, we have a YUV clip in “limited” float. fmtc_bitdepth cannot handle such a range: converting gives a double-limited range. However it is possible to restore the single-limited range at the function output by using the inverse factor, therefore converting to full range with fulld=true. This is a dirty hack, it requires to overwrite the frame property afterwards, but it should work.

[tormento]

replace the last line with
Thus doing, can I use standard convertbits(32) or have I still to use the fulls fulld switches?

[cretindesalpes]

You can do whatever you want, both configurations should be handled gracefully by the subsequent ConvertToPlanarRGB(), giving the same output.

[DTL]

"#We go to 32bit float limited tv range
ConvertBits(32)
"

It looks the 'float32' format in AVS may be very different between authors ? So not compatible between different filters/plugins/AVS-internal.


"BlankClip(pixel_type="YV12", color_yuv=$100000)

ScriptClip(Last, """
luma = AverageLuma ## gives the average luma of the current frame
Subtitle("luma=" + String(luma), align=2)
""")


return luma=16.0000 as expected for integer 10Hex = 16 decimal black


BlankClip(pixel_type="YV12", color_yuv=$100000)

ConvertBits(32)


ScriptClip(Last, """
luma = AverageLuma ## gives the average luma of the current frame
Subtitle("luma=" + String(luma), align=2)
""")

ConvertBits(8)


returns luma=0.062745 . It is magic number really not zero black expected in full range scaled. It is 255*0.062745=15.9999 (16). So it looks AVS internal 32bit float is mapping in 'float-narrow' of 0..255 integer data range to 0.0f..1.0f float range. It is really not logical engineering mapping of nominal video range to 0..1.0 data range as I think. The other plugins may not use this mapping so produce errors in processing ?

So the comment in the AVS docs about 'float is always full range' may be correct for 0..255 (and 0..65535) to 0.0f..1.0f mapping - but may be not what is expected by other plugins developers (using also external libraries and so on) ?

So AVS sequence
ConvertBits(32)
ConvertBits(8)
is transparent without range convert because it uses the single AVS-rule of range mapping to float32. Other plugins/filters may be use real zero black (and some other for nominal white) and things going wrong.

"#We go from Limited TV Range YUV to Full Range RGB 32bit float
ConvertToPlanarRGB()"

It looks not change range:

BlankClip(pixel_type="RGB24", color=$101010)
ConvertToYV12(matrix="PC.709") # to finally have correct zero black UVs

ConvertBits(32)

ConvertToPlanarRGB()

ScriptClip(Last, """
# avg = AverageLuma ## gives the average luma of the current frame
avg = AverageR ## gives the average luma of the current frame
Subtitle("avg=" + String(avg), align=2)
""")

ConvertBits(8)
ConvertToRGB24()


return also avg=0.062745 . So
ConvertBits(32)
ConvertToPlanarRGB()
still keeps range in 'AVS-float-narrow'. Not convert black to zero 0.0f.

"AviSynth+ uses full and limited ranges with float, using 16/256 and 235/256 for black and white in limited range, same with chroma data. Avs+ fmtconv is derived from its original Vapoursynth version, where nominal range for float is always [0 ; 1] and [-0.5 ; +0.5] for chroma information, and full/limited concept doesn’t apply (however range information can be carried as frame property)."

This confirms the idea of different range mapping to float in different environments/plugins. One more point of failure in AVS scripting.

"So, just before the bitdepth conversion, we have a YUV clip in “limited” float. fmtc_bitdepth cannot handle such a range: converting gives a double-limited range. However it is possible to restore the single-limited range at the function output by using the inverse factor, therefore converting to full range with fulld=true. This is a dirty hack, it requires to overwrite the frame property afterwards, but it should work."

It turns that going into float do not makes things easy and clear and border-less and virtually infinite precision as expected - it again adds lots of range-mapping nightmare.

[StvG]

"#We go to 32bit float limited tv range
ConvertBits(32)
"

It looks the 'float32' format in AVS may very different between authors ? So not compatible between different filters/plugins/AVS-internal.

This was discussed several times in the past. Here (https://forum.doom9.org/showthread.php?p=1926448#post1926448) (read from this post) an example (https://github.com/sekrit-twc/zimg/issues/134 https://github.com/AviSynth/AviSynthPlus/issues/189)

[DTL]

May be it is good to supplement ConvertBits() description at wiki http://avisynth.nl/index.php/ConvertBits with description of internal AVS-float format (range mapping) and some warning because it may be not directly compatible with plugins. With HDR times people like to go into float and mix plugins so faces this issues more frequently ?

As a 'natural and logical progress' in digital precision enhancement (8->10...16 bit) I expect also different range mapping of integer to float as 16..235 to 16.0f..235.0f for example. But things going differently in different environments/plugins.

Also may be AVS developers may add one more frame/clip property to auto-handle this issue like _FloatRangeMapping of different possible ways.

The
ConvertBits(32, fulls=false, fulld=true)
really makes range expansion so narrow range black of 16 integer finally maps to 0.0f of float (ConvertToPlanarRGB keeps it too). So it may be way to convert into some 'Vapoursynth' plugins float format.

[cretindesalpes]

fmtconv r30 (https://forum.doom9.org/showthread.php?t=183139):
matrix: The _ColorRange frame property is now set when a matrix preset is used.
transfer: Added ACEScct transfer function.
primaries: Added DCI P3+ and Cinema Gamut presets.
primaries: Added wconv parameter for full conversion.
Changed the configure options to compile with Clang.
Updated datatypes in the examples.

[Dogway]

Thanks!
I was having an issue tweaking fmtc_transfer() lws and lwd, they didn't seem to do anything, am I doing it wrong?
ConvertBits(32)
fmtc_resample (css="444")
fmtc_matrix (mats="2020", matd="rgb")
fmtc_transfer (transs="PQ",transd="linear",lws=4000.0,lwd=100.0,sceneref=false) # Also tested with '1886' transd

[cretindesalpes]

I should document better all the parameter effects in the documentation.

1. If you want to match the peak luminance of the transfer functions (and not the reference white), use match=2

However this won’t work here:

2. You can specify lws or lwd only when the component scale is not specified by the EOTF.

PQ uses an explicit value of 10000 cd/m² for component F’ = 1.0, and this cannot be overridden. Same with the 1886 transfer curve, which is in the generic SDR category (100 cd/m² for F’ = 1.0). lws and lwd are mainly meant for HLG operations, or transfer functions with unspecified luminance scale. Anyway you can use the cont parameter to force a scale change, and possibly use match=0 to get a full control on the transfer scales.

Check fmtc_transfer (/* parameters, */ debug=1) propShow () to know a bit more what’s happening under the hood.

Maybe should I relax some constraints to allow more flexibility?

[Dogway]

Check fmtc_transfer (/* parameters, */ debug=1) propShow () to know a bit more what’s happening under the hood.

Maybe should I relax some constraints to allow more flexibility?

Thanks! No I don't think so, maybe only adding some clarifications, like using a tonemapper or EETF for proper HDR scaling. I positively see fmtconv as strongly abiding to the standard so that's fine if the feature fits in 2446 or an EETF better.
I ported 2446 (method C only) here (https://forum.doom9.org/showthread.php?p=1973845#post1973845) and combining it with fmtconv gave me pleasant results, specially on highlights. It has some milky shadows and didn't implement achromatic highlights but I will be adding more TM and shapers.

[tormento]

I should document better all the parameter effects in the documentation.
Is possible to use fmtc_resample to invert a bicubic interpolation?

Getnative gives me (from a 1080p source):

Resize Kernel: Bicubic b 0.33 c 0.33
Native resolution: 720p

I just would like to see if I can get better results than DeBicubicResizeMT(target_width=1280,target_height=720,threads=1), as it gives me some aliasing.

What parameters would you suggest me?

[cretindesalpes]

tormento:

fmtc_resample (1280, 720, kernel="bicubic", a1=0.33, a2=0.33, invks=True)

is probably what you’re looking for. Please note that invks works differently of DeBicubic. It performs the inversion in the spectral domain instead of solving an equation system to find the kernel coefficients, so the result may be slightly softer. You can increase the invkstaps parameter (default: 4) but this can introduce more ringing.

[tormento]

is probably what you’re looking for
Thanks!

About a1(=b ?) and a2(=c ?) parameters, should they be the same of the getnative utility or is there a better way to find their optimal values?

As they form a ∞² space, it's not trivial to have a proper domain of search. Any help is welcome.

[cretindesalpes]

Yes a1 = b and a2 = c, and they are the same as the one found by getnative. I don’t know if there are better ways to find these values.

[tormento]

Yes a1 = b and a2 = c, and they are the same as the one found by getnative. I don’t know if there are better ways to find these values.
I have tried

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Ishuzoku reviewers\01.dgi")
fmtc_resample (1490, 838, kernel="bicubic", a1=0.3, a2=0.3, invks=True)
Prefetch(x)

on a YV12 video and it gives me YUV420P16 back.

Is it wanted?

Plus, there is almost no advantage in using a Prefetch >=2 (fps 60ish), while something else such as DeBicubicResizeMT accelerates almost linearly (fps 200ish).

Why?

[cretindesalpes]

on a YV12 video and it gives me YUV420P16 back.
Is it wanted?
Yes.

Plus, there is almost no advantage in using a Prefetch >=2 (fps 60ish), while something else such as DeBicubicResizeMT accelerates almost linearly (fps 200ish).
Why?
fmtc_resample is already multithreaded via AVSTP. Use avstp_set_threads(1) to disable internal multithreading.

[tormento]

is already multithreaded via AVSTP
I don't use avstp dll.

[hello_hello]

Just a question to clear up a little ambiguity in the FMTConv (and DitherTools) help file.

The fmtc.resample section shows the default for taps as 4.

arrayi taps (4),
arrayi tapsh (taps),
arrayi tapsv (taps),

Under the kernel section, it says Lanczos is the same as LanczosResize.
The default for LanczosResize is 3 taps though, so I'm just wondering which it is.

Cheers.

[tormento]

Yes.
Just saw that you moved the repo to GitLab (https://gitlab.com/EleonoreMizo/fmtconv/)and there are some changes. When do you plan to release a new build?

[tormento]

I'd like to replace the two lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

with fmtconv equivalents, as it's usually faster on my pc.

Tried to read the manual and I have lost my mind.

Can someone help me?

[StvG]

I guess there will be some additional filtering between the two conversion.

I'd like to replace the two lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

fmtc_bitdepth(32)
fmtc_resample(css="444", kernel="spline64")
fmtc_matrix (mat="2020", col_fam="RGB")
fmtc_bitdepth(16, dmode=6)

z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="Spline64", dither_type="error_diffusion", use_props=0)

fmtc_bitdepth(32)
fmtc_matrix (mat="2020", col_fam="YUV")
fmtc_resample (css="420", kernel="spline64")
fmtc_bitdepth(10, dmode=6)

[tormento]

I guess there will be some additional filtering between the two conversion.
Indeed, there is:

DGCube("2a_PQ4000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", in="full", lut="full", out="full")

Are those fmtconv lines still valid?

:thanks:

[StvG]

Are those fmtconv lines still valid?

Yes, they are valid.

[tormento]

Yes, they are valid.


Have I to convert32 even the 2nd time?

[StvG]

Have I to convert32 even the 2nd time?

Yes, if you want to mimic as close as possible the zimg calls.

[DTL]

Looks like found an issue with 'forcing' :

Documentation says the fh param is both for kernel additional scaling and also to force convolution if size not changed if fh<0. But too low negative fh values like -0.1 causes significant kernel distortion (though working) and too close to zero negatives like -0.000001 to keep kernel as unchanged as possible causes freeze (forever ?).

Test script:

LoadPlugin("fmtconv.dll")

Function AddBordersHF(clip c, int left, int right, int flt_rad)
{
unflt=AddBorders(c, left, 0, right, 0)
flt=GaussResize(unflt, unflt.width, unflt.height, p=10, b=2.71828, s=0, force=1)
uf_internal=Crop(c, flt_rad, 0, c.width-flt_rad*2, c.height)
return Overlay(flt, uf_internal, x=left+flt_rad, y=0)
}

Function AddBordersHF_FMTC(clip c, int left, int right, int flt_rad)
{
unflt=AddBorders(c, left, 0, right, 0)
# flt=fmtc_resample(unflt, w=unflt.width, h=unflt.height, kernel="gauss", taps=10, a1=10, fh=-0.000001).ConvertBits(8)
flt=fmtc_resample(unflt, w=unflt.width, h=unflt.height, kernel="gauss", taps=10, a1=10, fh=-0.1).ConvertBits(8)
uf_internal=Crop(c, flt_rad, 0, c.width-flt_rad*2, c.height)
return Overlay(flt, uf_internal, x=left+flt_rad, y=0)
}

ColorBarsHD(2000,2000)
UserDefined2Resize(width/10, height/10)

r1=2

left=20

std=AddBorders(left, 0, 0, 0, r=r1).SubTitle("AVS 3.7.4", align=5)

#return std

s1=AddBordersHF(last, left, 0, r1).SubTitle("AVS s1", align=5)

s2=AddBordersHF_FMTC(last, left, 0, r1).SubTitle("FMTC p=10", align=5)

StackVertical(std, s1, s2)
#StackVertical(std, s1)


The line
flt=fmtc_resample(unflt, w=unflt.width, h=unflt.height, kernel="gauss", taps=10, a1=10, fh=-0.1).ConvertBits(8)
is working, but too significant kernel distortion (too wide and soft)

The line
flt=fmtc_resample(unflt, w=unflt.width, h=unflt.height, kernel="gauss", taps=10, a1=10, fh=-0.000001).ConvertBits(8)
cause CPU load high enough but no result returned in many seconds on 200x200 frame size at E7500 CPU.

Tried with latest version r30 (and AVS+ core r4246).

Is it possible to force processing in other way ? May be setting of sx != 0 will work too as with AVS resampler ?

Addition: The sx !=0 is also working for forcing.
flt=fmtc_resample(unflt, w=unflt.width, h=unflt.height, kernel="gauss", taps=10, a1=10, sx=0.000001).ConvertBits(8)

[tormento]

I just ported my Vapoursynth plug-in to Avisynth+.
Please, two requests:
update the project link to the new one (https://gitlab.com/EleonoreMizo/fmtconv/)
I saw that the last update to the project was 7 months ago; would you please provide us a new build for avs+?

[tormento]

There is a probable "bug" about frame properties.

The problem is fmtc_bitdepth(bits=32) has frame properties _ColorRange=1 while in reality is _ColorRange=0.

Have a look here (https://github.com/Asd-g/libplacebo_Render/issues/4#issuecomment-4104255351).

[cretindesalpes]

DTL:
You should use fh = -1 and fv = -1. 1 means the kernel is used unscaled (default), and negative sign forces the processing. Very small values will create a gigantic kernel, probably not what you want.

tormento:
1. This is not really a bug. Maybe should I explicitly write the property as “full range” (0) in case of 32-bit float output whatever the fulld parameter value?
2. Thank you for the report, link update done.
3. See below.

[cretindesalpes]

fmtconv r31 (https://forum.doom9.org/showthread.php?t=183139):
bitdepth/Vapoursynth: Can work on clips of variable frame size. The function was already designed to handle this feature, but the parameter check was more strict than necessary. Thanks to Vardë for the report.
matrix: Added the SMPTE ST 2128 IPT-PQ-C2 transform to and from L’M’S’.
resample: Fixed 14 to 16 bit AVX2 conversion path, thanks to NSQY for the report.
resample: Fixed chroma placement: U and V vertical positions for interlaced PAL-DV were swapped and vertical subsampling > 2 was not handled at all. Thanks to Chortos-2 for the report.
transfer: Added L* transfer function.
transfer: Added Arri LogC4 transfer function.
transfer: Rescaled the S-Log3 curve to be used with limited-range code values, like the other S-Log curves.
primaries: Added Arri Wide Gamut 4 colorspace.
primaries: Fixed white XYZ coordinates for the NTSC-J colorspace (exact D93 location), thanks to Chortos-2 for the report.
Program path without x86 SIMD: fixed wrong conversions affecting a lot of functions (noticed on ARM/Apple). Thanks to SaltyChiang for the fix.
On Windows/x64, the Vapoursynth plug-in can be installed with the command line pip install vapoursynth-fmtconv. Requires Vapoursynth r74 or above.

[tormento]

1. This is not really a bug. Maybe should I explicitly write the property as “full range” (0) in case of 32-bit float output whatever the fulld parameter value?
I will leave the last word to the more experienced users here.

My issue is that I have to invoke libplacebo_render with src_levels="full" or it outputs the wrong range. Once I knew, I could easily fix it.

Thanks for your new build.

[tormento]

This is not really a bug. Maybe should I explicitly write the property as “full range” (0) in case of 32-bit float output whatever the fulld parameter value?
Why:

DGSource("M:\In\La collina dei papavery ~Lucky Red\collina.dgi", ct=24, cb=24, cl=0, cr=0)
propSet("_Matrix", 1)
propSet("_Transfer", 1)
propSet("_Primaries", 1)
propSet("_ColorRange", 1)
fmtc_bitdepth(bits=32, fulls=false, fulld=false)

expands the video to full range and doesn't leave it as it is?

A

fmtc_bitdepth(bits=16, fulls=false, fulld=false)

correctly leaves limited as limited.

Original image:

https://i.ibb.co/PZDRZgbW/image.png (https://ibb.co/0jskjrW9)

Script image:

https://i.ibb.co/nqnxRH6G/image.png (https://ibb.co/p6RYntb7)

[Z2697]

Because it's floating point. Values should always be in the range of 0.0-1.0, or -0.5-+0.5
Who knows. :rolleyes:

You see, by definition the full or limited range is the use of 0-255 or 16-235 (scale for other bitdepths and channels in your mind) to represent 0.0-1.0/-0.5-+0.5

[StvG]

0. YUV limited => resulted to YUV float [0, 1]
1. YUV limited => resulted to YUV float [16/255, 235/255]

_ColorRange does show which of the cases we have.

@tormento:
fmtc_bitdepth(bits=32, fulls=false, fulld=false)

expands the video to full range and doesn't leave it as it is?


You probably want fmtc_bitdepth(bits=32, fulls=true)

[Z2697]

The purpose of the limited range is to preserve the over/undershoot.
The IEEE 754 floating point number inherently "just have that attribute".
It is reasonable to have a legal range of 0.0–1.0 (-0.5 to +0.5) for everything.

However the color range frame property may be used to indicate the source's range.
So you would actually expect some over/undershoot value beyond 0.0-1.0 if the source is limited range.
Full range source will have those values clipped out.

Not that it really matters... if you work in float32 format, there will be over/undershoots introduced...

[StvG]

The purpose of the limited range is to preserve the over/undershoot.
The IEEE 754 floating point number inherently "just have that attribute".
It is reasonable to have a legal range of 0.0–1.0 (-0.5 to +0.5) for everything.

However the color range frame property may be used to indicate the source's range.
So you would actually expect some over/undershoot value beyond 0.0-1.0 if the source is limited range.
Full range source will have those values clipped out.

Not that it really matters... if you work in float32 format, there will be over/undershoots introduced...

I think we are not in the same page.

The practical matter is that in the Avisynth YUV 32-bit (float) world you can have [16/255, 235/255] when [0, 1] is expected - range mismatch (double compression, double expansion and what not). Examples:
- https://github.com/AviSynth/AviSynthPlus/issues/189
- https://forum.doom9.org/showthread.php?p=1926770#post1926770
- ColorBars(pixel_type="yuv420ps") / BlankClip(pixel_type="yuv420ps")

So in practice scripts and plugins read _ColorRange even for float and can treat it as "limited".
You have example of such script few posts above (Tormento, PixelScope) - that reads _ColorRange and based on the value it decide how to do YUV=>RGB - YUV [16/255, 235/255] => RGB [0, 1] or YUV [0, 1] => RGB [0,1]. fmtconv has YUV [0, 1] values but in some cases it has _ColorRange 0 (if fulld explicitly set) and in other cases for the same values it has _ColorRange 1.

[Z2697]

What I found more interesting is that libplacebo is doing 16/256-255/256 :)
It seems to convert both range to limited first. (so 255 in full range become 235/256, in limited range become 255/256...)

And swscale is doing something more quirky.
From limited range 235 is converted to 0.917983 which may suggest 235/256 with some error, but 255 is 0.999985... (update: I used 255 for all 3 channels by accident)
I had to use grayf32le as dest because there's no float YUV format in FFmpeg.
When input is YUV, it clamps the value to 16-235 then divides it by 256 (with low accuracy). When input is GRAY, it divides the value by 255 without clamping.
From full range there's no clamping or range conversion, just divide by 256 or 255 depending on the input...

Both libplacebo and swscale don't have their output value range depend on the output range setting.

Looks like everyone except zimg and fmtc have their own mind.
I think zimg and fmtc are clearly better.

[StvG]

/ off-topic
libplacebo with Vulkan processing? If so, it is in own area due to how GPUs handle image data.

I guess you used accurate_rnd for swscale?
/ end off-topic

[Z2697]

/ off-topic
libplacebo with Vulkan processing? If so, it is in own area due to how GPUs handle image data.

I guess you used accurate_rnd for swscale?
/ end off-topic

It's still floating point (fp16, probably) so shouldn't be very different.
But maybe because divide by 256 is faster than 255, just decrese the exponent.

I've tried both with and without accurate_rnd, it doesn't change the result (at least for a solid color).

[tormento]

If the default for fp is to normalize values, i.e. expand the range from limited to full, why AVS+ internal functions keep them in limited range?

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("M:\In\[CR] Chainsaw Man - Il Film La storia di Reze.dgi",ct=140,cb=140,cl=0,cr=0)
propSet("_Matrix", 1)
propSet("_Transfer", 1)
propSet("_Primaries", 1)
propSet("_ColorRange", 1)
ConvertToYUV444()
ConvertBits(32)
VideoTek(Mode="SDR", Type="nits", Detailed=true)
Prefetch(4)

https://i.ibb.co/RT3VtyyX/image.png (https://ibb.co/60HfGBBK)

You probably want fmtc_bitdepth(bits=32, fulls=true)

That one expands range too:

fmtc_bitdepth(bits=32)
fmtc_resample(css="444", kernel="spline36", fulls=true)

https://i.ibb.co/wr4hq0R0/image.png (https://ibb.co/cXQh9g1g)

There is no way I can get limited range YUV444PS with fmtc_conv. I tried any combination of fulls & fulld.

libplacebo_Render(src_csp="709", out_fmt="YUV444PS", src_levels="limited", dst_levels="limited")

works as intended:

https://i.ibb.co/QvnqWRVN/image.png (https://ibb.co/s9QDTG4w)

With z_ConvertFormat I had to fake the origin color range:

z_ConvertFormat(resample_filter="Spline64", pixel_type="YUV444PS", colorspace_op="709:709:709:f=>709:709:709:f")

https://i.ibb.co/gZRqFtS9/image.png (https://ibb.co/23kVYcvh)

In general, as I am converting to YUV444PS to feed BM3D_CUDA, is it better to apply noise reduction to the original, uninterpolated, limited range or to the expanded, interpolate, one?

What are those "black lines" that I can see in the expanded luma range?

[Z2697]

That's wrong.

fmtc_bitdepth(bits=32, fulls=true)

fmtc_bitdepth(bits=32)
fmtc_resample(css="444", kernel="spline36", fulls=true)

Can't you see the difference between the two?
The idea is the same as you faking input flag for zimg.

What you see as "black lines" is the result from not converting it back to it's original range.
The range in floating point doesn't matter. Binary32 have enough precision to hold either set of the values.
There's no interpolation being done.
uint8/235*255 creates some gap, not surprising.
x/235*235 or x/255*255, gives you exactly same thing (almost).

BM3D is designed for "full range".
The paper discuss firstly RGB and then "opponent" colorspace which is pretty much like some quirky YCgCo.
Both are most likely in the "full range".

(I put quotes on the "full range" because fp32 have much much wider actual range.)

[real.finder]

There is no way I can get limited range YUV444PS with fmtc_conv. I tried any combination of fulls & fulld.


I think this should be a bug, maybe report it here https://gitlab.com/EleonoreMizo/fmtconv/-/boards

[tormento]

I think this should be a bug, maybe report it here https://gitlab.com/EleonoreMizo/fmtconv/-/boards
Could you try too and let me know your results?

I'd prefer not to submit a false issue.

[Z2697]

You need to confirm the range not by converting it back to integer format (https://github.com/FranceBB/VideoTek/blob/6012d45f255e0ea12385d0c3f6e9ae54dd0b7e04/VideoTek.avsi#L97).

[tormento]

You need to confirm the range not by converting it back to integer format.
Such as?

What's wrong with my script?

[Z2697]

What you want really is just fmtc_bitdepth(bits=32, fulls=true, fulld=false)

[tormento]

What you want really is just fmtc_bitdepth(bits=32, fulls=true, fulld=false)


Does it work for you?

I get range expansion anyway.

[StvG]

Does it work for you?

I get range expansion anyway.

Yes, it does work. The following results to YUV "limited".

source 8-16 bit
propSet("_Matrix", 1)
propSet("_Transfer", 1)
propSet("_Primaries", 1)
propSet("_ColorRange", 1)
fmtc_bitdepth(bits=32, fulls=true, fulld=false)

Share your exact script.

[tormento]

Yes, it does work. The following results to YUV "limited".

Now I see where the problem is.

It works when you use fulls=true, fulld=false in fmtc_bitdepth but not in fmtc_resample. WTF.

Thanks.

P.S: Why doesn't the internal AVS+ conversion plugins "expand" to full range?

[tormento]

Can't you see the difference between the two? The idea is the same as you faking input flag for zimg. What you see as "black lines" is the result from not converting it back to it's original range.
This point is a bit obscure to me.
BM3D is designed for "full range".
So, my "correct" question becomes: should I feed video to BM3D by using

fmtc_bitdepth(bits=32, fulls=true, fulld=false)
fmtc_resample(css="444", kernel="spline36")

or

fmtc_bitdepth(bits=32)
fmtc_resample(css="444", kernel="spline36")

and think about getting back to limited after the noise reduction?

[Z2697]

If you use ConvertToFloat(fulls=false, fulld=true) it does the "expansion".
Maybe you should just use the internal functions if you want something behaves exactly like internal functions.

[tormento]

Maybe you should just use the internal functions if you want something behaves exactly like internal functions.
It's not that I want something that behaves like the internal functions, I was just wondering myself why the internal functions behave like that, i.e. differently than any other plugin, keeping the limited range after the floating point conversion.

[pinterf]

Limited range YUV in float is an artificial thing, maybe it's my "invention". Now I'd reinterpret it, and make it to have the very same range (0..1, -0.5..+0.5) like full range, but keep the limited/full property in order to be able to be able to do a to and from integer conversion without losing the range information. If I was sure that no external plugin writer handles this "limited" 32-bit float, then it's only some constant in Aviysynth code which I have to change.

I think, both avsresize (zimg) and fmtconv expects float in one valid format: 0..1, +/-0.5, regardless of the color range flag.

I wrote about it here https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/colorbarsuhd.html (Find section: "fmtconv: Use Full Float Workflow"), since ColorbarsUHD creates limited/narrow range pattern, if I wanted to use float workflow I definitely had to convert the clip to "full-range" float before, using Avisynth's ConvertBits(32, fulld=true) .

[Z2697]

Frankly, it doesn't matter, either way the information is fully intact (minus the potential rounding errors and binary fp oddities), binary32 have enough precision to store all the values, nothing is lost, or added.
If one stick to a same plugin for converting to and from float format (which bears a question of why not), then it's all fine.

And if one knows the difference between plugins, it's just some extra code... and the risk of human error...

[Z2697]

This point is a bit obscure to me.

So, my "correct" question becomes: should I feed video to BM3D by using

fmtc_bitdepth(bits=32, fulls=true, fulld=false)
fmtc_resample(css="444", kernel="spline36")

or

fmtc_bitdepth(bits=32)
fmtc_resample(css="444", kernel="spline36")

and think about getting back to limited after the noise reduction?

The behavior of fmtc is, determin it's (x-16)/(235-16) or x/255 based on the fulls option, so every value in legal range will just fall in 0.0-1.0 without any additional processing; the fulld option in float output only determines the frame property, nothing else.
For zimg this is more or less the same, so the idea of how to make them behave like "limited float" is the same.
(I didn't actually look into the source code, but this is easy to confirm just by using the plugin.)

limited YUV -> float -> full YUV is just a range expansion, which is what the plugin is doing internally when doing limited YUV -> full YUV.
range matters in integer format because there actually less/more code values, but with the "magic" of floating point, it doesn't matter, to some extent.

BM3D would expect values in 0.0-1.0 range so use zimg/fmtc's default behavior.
Just use the same plugin to convert back. zimg and fmtc have same behavior so you can mix them, but why.
The fmtc and avsresize plugin might not support frame property, so it's better to specify the range in function call.
The "trick" is specifically for the compatibility of float to integer conversion with current internal function. Don't ever use that in other places.

[PoeBear]

Is there any benefit of switching from using Avsresize/zimg over to this, for things like simple resizing and colorspate/bit depth conversions? I see it offers a lot of similar features, is it any faster/more accurate? Is it missing any of Avsresize's features? Just wondering if I should change up my workflow

[Z2697]

Practically, they are the same.
fmtc have more dithering methods but most being in the error diffusion category and are bad for lossy encoding;
more resizing kernels but most being not something to use everyday...

[tormento]

Is there any benefit of switching from using Avsresize/zimg over to this
Slight more precision and, AFAIK, libplacebo is the only one to internally work in linear color space instead of ignoring gamma coefficients.

[Z2697]

Slight more precision and, AFAIK, libplacebo is the only one to internally work in linear color space instead of ignoring gamma coefficients.

Slightly different, but I won't say definitively which one is more precise.
libplacebo doesn't always use linear light, and linear light doesn't always make sense. (horrible ringings when upscale in linear)
fmtc and zimg support linear light as well with fmtc supporting sigmoidize in addition. Just some extra function calls.

[tormento]

libplacebo doesn't always use linear light, and linear light doesn't always make sense.
When doesn't it linearize? It's default as it is sigmoidize too.

Personally, I love libplacebo, especially the last iteration from Asd-G, you can do really many things with a single call.

I think it's really complete but for some reason underestimated.

It's the CPU-GPU overhead that makes it sometimes really slow, at least on my very old PC.
fmtc and zimg support linear light
Internally with no multiple calls?

[DTL]

Slightly different, but I won't say definitively which one is more precise.
libplacebo doesn't always use linear light, and linear light doesn't always make sense. (horrible ringings when upscale in linear)


It is just because all rescaling (mostly upscale) needs to be done in the same transfer domain. An image file needs to have (one more) metadata property like 'scaling conditioning domain'. Same needs to be read in the AVS+ and pass as clip/frame property for scaling filters to read and use and change. And 'auto' function software simply needs to read and use this property if the user needs 'auto' operation and does not want to dig in the image file properties or search the proper conditioning for scaling domain by manually selecting all available transfer domains. Though some automation tools may be designed to scan image files and select the best scaling domain with lowest output ringing (this may be useful 'auto scaling domain' feature too to make some software a bit different from other full manual scaling software).

If input image data was prepared (anti-Gibbs and Nyquist conditioned) in the linear transfer domain - it will be upscaled with lowest ringing distortion also in the linear transfer domain. But this may be still rare footage with such properties and also no supplementary metadata typically available so it is very easy to get ringing distortion at upscaling if always attempt to do upscale in the linear transfer domain while I think most digital moving picture files are prepared in system transfer domains. Until we do not have industry standards (regional or international recommendations) about scaling conditioning domain for full digital workflow from light to light and appropriate metadata signalling for scalers.

[rgr]

/ off-topic
I guess you used accurate_rnd for swscale?
/ end off-topic

And this is still needed? Apparently it is currently default enabled.
I haven't checked, I don't use swscale, but zscale (by the way, I recommend caution --> https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23026 )