fmtconv/Avisynth+: resize, bitdepth and colorspace conversions

[cretindesalpes]

hello_hello:

Yes it is the expected behaviour. From the matrix doc:

The _ColorRange frame property is set if the fulld parameter has been explicitely defined.

And if fulld is not defined, the property is just copied from the source frame. But this doesn’t make sense when using a matrix preset because the target range is always known. So I might change this behaviour in the next release.

[tormento]

Can I use fmtconv to replace the lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

and

z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

?

I'd like to try if there is any speed improvement over z.

[Dogway]

I think there might some error in the docs, for bobbing an interlaced source it shows interlaced(d) args as boolean, but in the function description as int type.

Also I'm trying to understand how do you do a chromatic adaptation with fmtconv. I tried the next but the result looks noop.

fmtc_resample(css="444")
fmtc_matrix(mat="709", bits=16)
fmtc_transfer(transs="709", transd="linear")
fmtc_primaries(prims="709", primd="709", ws=[0.3127,0.3290], wd=[0.33242, 0.34743]) # From D65 to D55
fmtc_transfer(transs="linear", transd="709")
fmtc_matrix(mat="709", bits=16)

[tormento]

Originally Posted by cretindesalpes

I just ported fmtconv Vapoursynth plug-in to Avisynth+.

We need you.

[cretindesalpes]

tormento:

Yes you can. I’m pretty sure z is faster, but you can try.

Dogway:

Thanks, indeed the examples haven’t been checked for long, and function prototypes slightly evolved since. I just fixed the doc.

Your chromatic adaptation is right. How do you check it? Results may be subtle. Or maybe what you’re looking for is complete conversion, for example to make a picture displayed on a D55 monitor look exactly like the same picture displayed on a D65 monitor placed side by side?

[Dogway]

Yes I was expecting a warmer output, maybe I should try with D50. Eyes adapt to the white point specially at matching environment temperatures as there's not a single "white" in psychovisually terms. I was double checking as I had difficulties on color space conversions on non-standard illuminants.

[tormento]

Originally Posted by cretindesalpes

tormento:Yes you can. I’m pretty sure z is faster, but you can try.

Can you please translate them for me? The AVS porting doc is a bit obscure and there are missing parts too.

[Dogway]

Originally Posted by tormento

Can I use fmtconv to replace the lines:

z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

Maybe something like this?

	fmtc_bitdepth (bits=16)
	fmtc_resample (css="444",kernel="spline64")
	fmtc_matrix   (mats="2020", matd="rgb", fulls=false, fulld=true)
	fmtc_bitdepth (bits=16, dmode=6)

EDIT: Correction, fmtconv does dither from float to 16-bit int, but the effect is minimal.

[StvG]

I would say this is the equivalent of that z_ line:

fmtc_bitdepth(bits=32)
fmtc_resample(css="444",kernel="spline64")
fmtc_matrix(mats="2020", matd="rgb")
fmtc_bitdepth(bits=16, dmode=6)

[tormento]

Originally Posted by StvG

I would say this is the equivalent of that z_ line:

fmtc_bitdepth(bits=32)
fmtc_resample(css="444",kernel="spline64")
fmtc_matrix(mats="2020", matd="rgb")
fmtc_bitdepth(bits=16, dmode=6)

I guess 4 calls are slower than a single z one.

[DTL]

It may also cause massive memory overload with lots of cached frames with massive multithreading at todays AVS+ at todays massive multicore end-user CPUs. So single filter processing may be much more RAM friendly.

[tormento]

I was getting crazy, trying to find why BM3D_CUDA had some slight tint change and I found out that it is the combination of Convertbits and fmtconv that makes strange things.

Look at this comparison.

Reality is that BM3D_CUDA needs RGB to work and... am I doing something silly or there is something terribly wrong?

Image 0:

original

Image 1: OK

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
ConvertBits(8)
Prefetch(6)

Image 2: WRONG!!!

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
fmtc_bitdepth (bits=8,dmode=8)
Prefetch(6)[

Image 3: WRONG!!!

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
fmtc_bitdepth (bits=32,dmode=8)
ConvertBits(8)
Prefetch(6)

Image 4: OK

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
fmtc_bitdepth (bits=32,dmode=8)
fmtc_bitdepth (bits=8,dmode=8)
Prefetch(6)

[StvG]

Originally Posted by tormento

I was getting crazy, trying to find why BM3D_CUDA had some slight tint change and I found out that it is the combination of Convertbits and fmtconv that makes strange things.

Look at this comparison.

Reality is that BM3D_CUDA needs RGB to work and... am I doing something silly or there is something terribly wrong?

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).
For Image 3 change ConvertBits(8) to ConvertBits(8, fulls=true, fulld=false)

[tormento]

Originally Posted by StvG

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).
For Image 3 change ConvertBits(8) to ConvertBits(8, fulls=true, fulld=false)

Please explain me why and why it's not "transparent" to fmtconv but they work only with the same function properly.

[tormento]

Ok, here things get tricky.

I need to convert to 32, then planar RGB and back to 8.

I am getting some tint changes again.

00

original



01

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
ConvertToPlanarRGB()
ConvertToYUV420()
ConvertBits(8)



02

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32, fulls=false, fulld=true)
ConvertToPlanarRGB()
ConvertToYUV420()
fmtc_bitdepth (bits=8,dmode=8)



Even a

ConvertToPlanarRGB()
ConvertToYUV420()
fmtc_bitdepth (bits=8,dmode=8)

Gives slight wrong tint again.



It happens even with:

ConvertToPlanarRGB()
ConvertToYUV420()

I am getting crazy with conversions.

EDIT: it seems that ConvertToYUV420(matrix="Rec709") fixed things. WTF.

[tormento]

I talked too early...

0

Original

1

PHP Code:

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
DeBicubicResizeMT(target_width=1280,target_height=720,threads=1)
ConvertBits(32)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")
DeBicubicResizeMT(target_width=1920,target_height=1080,threads=1)
ConvertBits(8) 


2

PHP Code:

SetMemoryMax()
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
DeBicubicResizeMT(target_width=1280,target_height=720,threads=1)
ConvertBits(32, fulls=false, fulld=true)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")
DeBicubicResizeMT(target_width=1920,target_height=1080,threads=1)
fmtc_bitdepth (bits=8,dmode=8) 


I smashed my head against the wall enough, need your help.

[DTL]

Originally Posted by StvG

For Image 2 change ConvertBits(32) to ConvertBits(32, fulls=false, fulld=true).

Do ConvertBits(32) really process fulld argument anyway ? As I start to learn in the world of AVS and plugins programmers the 32bit float always treated as 0..1.0 nominal range with black at 0.0f and always internally converted into this range. It is very important to know that any going into float format around AVS and its plugins always require to convert back to narrow range in integer.

"ConvertBits(32)
ConvertToPlanarRGB()
w2xncnnvk(noise=0,model=0,gpu_thread=2)
ConvertToYUV420(matrix="Rec709")"

It looks in 32bit float matrix=Rec709 is no-op for range change ? It may be hidden feature of 32bit float AVS format.

"ConvertBits(8) "

It looks after any return back from float to integer you need to convert to narrow range (as usual integer moving picture data representation), so correct may be
ConvertBits(8, fulls=true, fulld=false)
or shorter
ConvertBits(8, fulld=false) because fulls for 32bit float input will be auto-detected as true (?).

AVS defaults looks like source of error if going into float: http://avisynth.nl/index.php/ConvertBits
bool fulls = (auto)

Conversion from and to float is always full-scale.

bool fulld = fulls

So as default 32bit float is full range and default fulld = fulls the simple
ConvertBits(32)
ConvertBits(8)
will cause range convert from narrow to full. The only safe and transparent ConvertBits is 8<->16 integer.

[tormento]

Originally Posted by DTL

Conversion from and to float is always full-scale.

Ok but why the "fix" on Convertbits(32) worked and now, that I have a planar to YUV, not anymore?

[FranceBB]

Originally Posted by DTL

Conversion from and to float is always full-scale.

bool fulld = fulls

So as default 32bit float is full range and default fulld = fulls the simple
ConvertBits(32)
ConvertBits(8)
will cause range convert from narrow to full.

Well... that's what I thought too, but actually it's not.
When you go to 32bit float, but stay in YUV, looks like it's still limited tv range, in fact even a simple:

ColorBars(848, 480, pixel_type="YV12")
ConvertBits(32)
TurnRight.Histogram.TurnLeft

shows that levels are still limited tv range:




The conversion to Full Range happens when Tormento calls "ConverttoPlanarRGB()" 'cause RGB by standard is full range.
My "theory" was like yours, namely that when he called back ConverttoYUV420(matrix="Rec709") it was going to clip values rather than converting them back to limited, but no, I was wrong, in fact by doing all the operations he has in the script (except for Waifu) we get a perfectly identical image in terms of levels:

#Limited tv range source
ColorBars(848, 480, pixel_type="YV12")
original=last

#We go to 32bit float limited tv range
ConvertBits(32)

#We go from Limited TV Range YUV to Full Range RGB 32bit float
ConvertToPlanarRGB()

#We go back to Limited TV Range YUV 32bit float
ConverttoYUV420(matrix="Rec709")

#We go back to Limited TV Range YUV 8bit planar
ConvertBits(8)
converted=last

#We compare the two images
StackHorizontal(converted, original)

So... it must be Waifu screwing it up, there's no other way...

[tormento]

Originally Posted by FranceBB

So... it must be Waifu screwing it up, there's no other way...

So both convertbits and fmtc (as last command) should give the same output, thing that doesn't happen.