Log in

View Full Version : z.lib resizers for AviSynth+


Pages : 1 2 3 [4] 5 6 7 8

zambelli
29th October 2020, 06:09
zimg always assumes that float input is in the range 0.0-1.0 regardless of whether you tell it that the input is limited range or not, and I believe likewise always converts float output into that range. [...] There is no point in choosing a different arbitrary convention.
I'm OK with that, it makes sense now, but I'm surprised that Avisynth might not be aligned with that same convention.

I'm still curious: what happens to superblack and superwhite values from a limited Y integer range when it's converted to float? Do they end up mapping to <0 and >1?

pinterf
29th October 2020, 09:10
I'm OK with that, it makes sense now, but I'm surprised that Avisynth might not be aligned with that same convention.

I'm still curious: what happens to superblack and superwhite values from a limited Y integer range when it's converted to float? Do they end up mapping to <0 and >1?
Yes, they are mapped then outside the 0..1 range.
At the time I was developing the high bit depth additions I was not aware of any conventions. I suppose this convention was not even a strong one. And it had two phases, even putting the float UV chroma range to zero center was an extra project for me years ago, mainly because I wanted it to be compatible with zimg (and VS). And because it is a reasonable convention. Now it needs a third step: handle 32 bit float format uniformly across avs+ core and plugins. With additions attention to the implemented auto-conversions in Expr and masktools lut.

hello_hello
29th October 2020, 14:17
To clarify....

z_ConvertFormat(720,480,colorspace_op="709:709:709:limited=>170m:601:170m:limited")

If the source is some flavour of YUV and the input to z_ConvertFormat is 32 bit float, should "limited" be changed to "full", or is the specified range ignored for float?

StvG
29th October 2020, 21:04
... and I believe likewise always converts float output into that range.
If input is float and output is float - color_range source/destination doesn't have effect; range is always assumed full.
If input is int and output is float - color_range source set to full keeps the values same and returns x/255 (8-bit) within 0..1 range; color_range source set to limited expands the range limited to full; color_range destination doesn't have effect.
If input is float (full range) and output is int - color_range source doesn't have effect; color_range destination set to limited does full to limited; color_range destination set to full keeps the full range.

To clarify....

z_ConvertFormat(720,480,colorspace_op="709:709:709:limited=>170m:601:170m:limited")

If the source is some flavour of YUV and the input to z_ConvertFormat is 32 bit float, should "limited" be changed to "full", or is the specified range ignored for float?
Using the same value for source and destination means no conversion. It doesn't matter if you use limited for source and destination or full for source and destination, or omit color_range at all.

StvG
30th October 2020, 11:12
avsresize_r5 (https://cloud.owncube.com/s/Zq6iPjyYGcTLAT8): use chromaloc frame property (when available) for the legacy resizers.

kedautinh12
15th May 2021, 14:42
I seen zimg have new ver
https://github.com/sekrit-twc/zimg/blob/master/ChangeLog

StvG
30th May 2021, 15:09
avsresize_r6 (https://cloud.owncube.com/s/KfAcEpqkMPSdagr): - registered as MT_NICE_FILTER;
- read frame properties from every frame (previously only from the first frame);
- zimg@8d0b839.

kedautinh12
30th May 2021, 15:55
Thanks

StvG
31st May 2021, 08:34
If anyone downloaded avsresize_r6, download it again (fixed regression: approximate_gamma (r5 is ok)).

tormento
31st May 2021, 09:46
If anyone downloaded avsresize_r6, download it again (fixed regression: approximate_gamma (r5 is ok)).

Unable to download.

Il server non è stato in grado di completare la tua richiesta.

Se ciò si verifca nuovamente, invia i seguenti dettagli tecnici all'amministratore del server.

Ulteriori dettagli sono disponibili nel log del server.

Dettagli tecnici
Indirizzo remoto: 84.221.196.138
ID richiesta: WXU1VipibSaEEFhe3TpD

Edit: now it works.

FranceBB
21st June 2021, 16:47
It was working, I updated Avisynth on my production server to 3.7.0, now it's throwing this error:

https://i.imgur.com/UxWQT6G.png

Why?

Edit: Tested on Avisynth 3.7.1 Beta 6, same result, same error.

Edit 2: I'm temporarily using ConverttoYUV422(matrix="PC2020") as a workaround. Technically, it should leave the levels untouched from the Studio RGB...

StvG
23rd June 2021, 02:12
avsresize_r7 (https://ppp.woelkli.com/s/8ZoEqE7o3GFfREc) (pass: hIaK6dSB69uK): fixed frame properties reading.

kedautinh12
23rd June 2021, 03:09
Thanks

FranceBB
23rd June 2021, 06:42
I updated but now I get: RGB Color family cannot have YUV matrix coefficients

https://i.imgur.com/ije1YiC.png

if I remove the last conversion and leave it as RGB it works:

https://i.imgur.com/1isEVxY.png


Version: Avisynth 3.7.1 Beta 6 x64
OS: Windows 10 Enterprise x64

StvG
23rd June 2021, 07:23
Check the last line from the first screenshot: pixel_type=rgb but destination matrix 2020 (yuv).

Dogway
23rd June 2021, 08:44
Check the last line from the first screenshot: pixel_type=rgb but destination matrix 2020 (yuv).

This is one of the many reasons why I started Transforms Pack, no-sense limitations and restrictions in avsresize. YUV 2020 can be converted to RGB 2020, it's even a standard conversion (CL and NCL).

FranceBB
23rd June 2021, 09:15
Check the last line from the first screenshot: pixel_type=rgb but destination matrix 2020 (yuv).

Daaaamn, this is why I shouldn't test things at 6.42 in the morning before having my breakfast...

Version 7 works like a charm now, thanks! ;)
I'm rolling it on all my production server. :D
Now I've got a few MJPEG2000 12bit PQ masterfiles to encode in HLG eheheheh

StvG
29th June 2021, 09:28
avsresize_r8 (https://ppp.woelkli.com/s/nCNwep672C4CzYs) (pass: Zz299Qct9i4P): changed MT mode to MT_MULTI_INSTANCE.

kedautinh12
29th June 2021, 10:31
Thanks

Audionut
30th June 2021, 14:05
Hi, I'm seeing an issues with levels that begun in r7.

This is what the frames should look like.

And this is the output of ComparisonGen.avsi after updating to r7.

Still same issue in r8.

StvG
30th June 2021, 18:01
avsresize_r9 (https://ppp.woelkli.com/s/GMieecQBi2wrMnp) (pass: au7cG6ztmrW4): do not use the same color range for the destination when frame property available and source/destination color family is different;
- set the correct color range frame property value for destination YUV 32-bit.

kedautinh12
1st July 2021, 03:05
thanks

FranceBB
1st July 2021, 12:31
Thanks for the update!
Time to test and then replace it in our production servers. :)

Boulder
23rd July 2021, 12:18
Out of interest, should the value for nominal_luminance be adjusted based on source metadata (=maximum content light level) in case of HDR?

StvG
24th July 2021, 02:31
Out of interest, should the value for nominal_luminance be adjusted based on source metadata (=maximum content light level) in case of HDR?

Check the first two comments from this issue (https://github.com/sekrit-twc/zimg/issues/73).
Ideally you want to set it to the frame peak luminance.

Boulder
24th July 2021, 08:57
Check the first two comments from this issue (https://github.com/sekrit-twc/zimg/issues/73).
Ideally you want to set it to the frame peak luminance.

Thanks, looks like it's not worth it and best just to leave it at the default if unknown. There probably is no tool to find the value by scanning the whole video.

Dogway
24th July 2021, 16:27
@Boulder: runtime yplanemax maybe? Maybe do a first pass analysis.

FranceBB
26th July 2021, 20:00
Thanks, looks like it's not worth it and best just to leave it at the default if unknown. There probably is no tool to find the value by scanning the whole video.

PQStat (https://github.com/wswartzendruber/hlg-tools/tree/main/pqstat) (written by William Swartzendruber, who is also a user of this forum) can do it for you: it's free, open source and written in Rust.
It will scan the whole video and report the right values in nits.
Please note, though, that if you're not scanning a lossless masterfile made of 12bit tiff but rather a consumer relatively low bitrate HEVC .ts file from a broadcasting channel or a consumer UHD-BD HEVC .m2ts file (which also has a low bitrate compared to the master), there are gonna be compression overshooting (i.e outliers) which might affect the overall detection.
(Just as a reference, a ProRes 4K mezzanine file, which is NOT lossless, has 1 Gbit/s as bitrate and it's still losing quality and potentially creating overshooting, so those ARE DEFINITELY GONNA HAPPEN in a "low bitrate" H.265 HEVC file...)

Boulder
26th July 2021, 20:36
PQStat (https://github.com/wswartzendruber/hlg-tools/tree/main/pqstat) (written by William Swartzendruber, who is also a user of this forum) can do it for you: it's free, open source and written in Rust.
It will scan the whole video and report the right values in nits.
Please note, though, that if you're not scanning a lossless masterfile made of 12bit tiff but rather a consumer relatively low bitrate HEVC .ts file from a broadcasting channel or a consumer UHD-BD HEVC .m2ts file (which also has a low bitrate compared to the master), there are gonna be compression overshooting (i.e outliers) which might affect the overall detection.
(Just as a reference, a ProRes 4K mezzanine file, which is NOT lossless, has 1 Gbit/s as bitrate and it's still losing quality and potentially creating overshooting, so those ARE DEFINITELY GONNA HAPPEN in a "low bitrate" H.265 HEVC file...)

Thanks, I'll check that one out.

Wouldn't it actually be better to use the value, overshooting or not, since it's from the source that the decoder gets anyway?

FranceBB
27th July 2021, 10:31
Wouldn't it actually be better to use the value, overshooting or not, since it's from the source that the decoder gets anyway?

Well no.
Simply put: whenever you have a file and it has a waveform and it has a certain set of values and you re-encode it with some kind of compression, those values might not be respected.
What happens is this kind of overshooting, but here's the thing: the overshoot is an outlier.
Simply put: let's suppose that you're looking for the peak brightness in nits in a PQ file and the official metadata say it's 2500 nits. You analyze the file and you find out it's actually 2842 nits. At this point, you set the conversion to respect that and everything looks dark, way darker than it should have been. Why? Easy: the 2842 pixel was indeed the peak, but it was an outlier, which means that although the average was like closer to the right value, every now and then there are few overshoots occurred due to the compression and those will be much higher than the real value. Of course, if you take into account a simple average, what you're gonna get is a skewed result as the outliers are gonna affect the final value of the average brightness, so what you need is to avoid to take those outliers into account, however it would be hard to write a logic around that 'cause you would have to decide how far away a pixel has to go to be considered an outlier, which can be tough. Luckily for us, the standard deviation comes in rescue:

https://www.gstatic.com/education/formulas2/355397047/en/population_standard_deviation.svg

which tells us how much a set of values is disperse/various.
Anyway, we're digressing.
The point is that if you do end up using PQStat or similar tools on compressed files, you're gonna have to take the results with a pinch of salt, so always remember to look at the waveform and to trust your eyes as well. :)

DTL
27th July 2021, 17:50
Small size over and undershoots can be removed with low-pass filtering or blurring. Like gauss-blur with enough radius. And the medium and large size details levels can be analyzed for min/max/average .

StvG
12th August 2021, 22:01
avsresize_r10 (https://ppp.woelkli.com/s/iw6jdxokxfBsdWX) (pass: 7fgQwBn9zreN): do not set matrix frame property when source matrix frame property is undef and color family is not changed;
- zimg@bf73dbe.

FranceBB
13th August 2021, 12:53
Thanks for the new build and most importantly thanks for making the plugin in the first place.
I wouldn't have been able to encode some Studio RGB 12bit PQ masterfiles by reliably applying my non-public LUTs without messing up with the levels without this one.
You saved me in a certain sense. ;)

StvG
18th November 2021, 01:35
avsresize_r11 (https://ppp.woelkli.com/s/Pndi2kQnJz6AiLB) (pass: 2bzWmpDAZbk8): added parameter use_props (whether to read and set frame properties);
- zimg 3.0.3.

Boulder
18th November 2021, 06:26
Thank you for the new version :)

Would it be possible to add an option to use properties from a different clip than the source for the colorspace_op (and maybe chromaloc_op)? My use case is a simple "original clip" -> "convert to linear RGB and downscale" -> "convert back to original colorspace".

DTL
18th November 2021, 08:27
use case is a simple "original clip" -> "convert to linear RGB and downscale" -> "convert back to original colorspace".'

If you use typical chroma subsampled input and output to this 'simple' - it is really very complex. Because chroma subsampling introduce distortions that can not be completely fixed in 'linear processing'. To better fix the both luma and chroma distortions of nonlinear systems the 'non-linear' that is content-adaptive processing required. Also to build the better processing the standartization of chroma antialiasing filter in display device required (or standartization of other display subsampled-decompressor if it finally going out of 'linear processing' at all).
So the playing 'simple' games around chroma-loc and 'going to linear' that is not perfectly possible from chroma-subsampled ugly old lossy compression still not allow to get best possible results. The distortions raises with saturation of colours and sharpness of colour transients.
So if the developer of z.lib reads here - may be try to add some content-adaptive non-linear resizer methods too to this library of functions.

FranceBB
18th November 2021, 22:55
avsresize_r11 (https://ppp.woelkli.com/s/Pndi2kQnJz6AiLB) (pass: 2bzWmpDAZbk8): added parameter use_props (whether to read and set frame properties);
- zimg 3.0.3.

It's always nice to see developers beginning to use the new Avisynth+ 3.7.1 frame properties. Well done and thanks! :)

StvG
19th November 2021, 07:37
Thank you for the new version :)

Would it be possible to add an option to use properties from a different clip than the source for the colorspace_op (and maybe chromaloc_op)? My use case is a simple "original clip" -> "convert to linear RGB and downscale" -> "convert back to original colorspace".

Can you elaborate why would you use different frame properties than ones of the source clip?

It's always nice to see developers beginning to use the new Avisynth+ 3.7.1 frame properties. Well done and thanks! :)

After r3 frame properties are always read and set when the avs+ version supports them.
When every option of colorspace_op is specified (matrix, transfer, primaries, range) and different than "auto", use_props=false can be used for higher fps.
But now I'm seeing that use_props=false doesn't pass frame properties (removes the frame properties). Another version is needed to fix that.

Boulder
19th November 2021, 08:34
Can you elaborate why would you use different frame properties than ones of the source clip?

In that phase, the source clip is in RGB. It's basically YUV420P16 (BT709 or BT2020) -> linear RGB, downscale -> YUV420P16 (original format) -> feed to x265 for encoding.

StvG
19th November 2021, 09:22
In that phase, the source clip is in RGB. It's basically YUV420P16 (BT709 or BT2020) -> linear RGB, downscale -> YUV420P16 (original format) -> feed to x265 for encoding.

You want to use the frame properties of "YUV420P16 (BT709 or BT2020)" as target matrix/transfer/primaries... when doing "linear RGB, downscale -> YUV420P16" ?

Boulder
19th November 2021, 09:33
You want to use the frame properties of "YUV420P16 (BT709 or BT2020)" as target matrix/transfer/primaries... when doing "linear RGB, downscale -> YUV420P16" ?

Yes, that's the idea.

StvG
21st November 2021, 11:06
avsresize_r12 (https://ppp.woelkli.com/s/dTceZSPdLf4YDjQ) (pass: oPrkZ7IH132A): changed use_props type from bool to int.
use_props (default -1) - whether to read and set frame properties:
-1: If frame properties are supported - if every option of colorspace_op and chromaloc_op are specified and different than "auto", 0, otherwise 1.
If frame properties are not supported - 0.
0: If frame properties are supported - only set frame properties.
1: Read and set frame properties.
2: Save properties of the source clip as frame properties: z_ChromaLocation, z_ColorRange, z_Matrix, z_Transfer, z_Primaries.
Every option of colorspace_op and chromaloc_op must be specified and different than "auto".
3: Save properties of the source clip as frame properties: z_ChromaLocation, z_ColorRange, z_Matrix, z_Transfer, z_Primaries.
Frame properties _ChromaLocation, _ColorRange, _Matrix, _Transfer, _Primaries must exist.
Frame properties _Matrix, _Transfer, _Primaries must have values different than 2 (unspec).
4: Use z_xxx frame properties as target values.
z_xxx frame propeties are removed after the colorspace conversion.
use_props=0 is faster than use_props=1
use_props=2 is faster than use_props=3

Example use_props=2 and use_props=4
z_ConvertFormat(pixel_type="rgbp", colorspace_op="709:709:709:l=>rgb:linear:709:f", chromaloc_op="left=>left", use_props=2)
z_ConvertFormat(pixel_type="yv12", use_props=4) # convert rgb to yuv with the yuv initial properties
Example use_props=3 and use_props=4
z_ConvertFormat(pixel_type="rgbp", colorspace_op="709:709=>rgb:linear", use_props=3)
z_ConvertFormat(pixel_type="yv12", use_props=4) # convert rgb to yuv with the yuv initial properties

FranceBB
22nd November 2021, 10:29
Well, looks like it works! :D


#Indexing the HDR10 PQ BT2100 4:2:2 10bit planar source
video=FFVideoSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov")
ch1=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=1)
ch2=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=2)
ch3=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=3)
ch4=FFAudioSource("\\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=4)
ch5=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=5)
ch6=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=6)
ch7=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=7)
ch8=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=8)
ch9=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=9)
ch10=FFAudioSource("\\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=10)
ch11=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=11)
ch12=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=12)
ch13=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=13)
ch14=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=14)
ch15=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=15)
ch16=FFAudioSource("\\mibcisilonsc\avisynth\Scambio\FILM\Harry Potter and The Chamber of Secrets\HARRYPOTTERANDTHECHAMBEROFSECRETS_FEA_HDR_REC2020PQ_444F_PR444_2160P23-20211024145222.mov", track=16)
audio=MergeChannels(ch1, ch2, ch3, ch4, ch5, ch6, ch7, ch8, ch9, ch10, ch11, ch12, ch13, ch14, ch15, ch16)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("\\mibcisilonsc\avisynth\Server\encoder\encoder\Processors\avs_plugins\LUTs\WarnerBros_PQToHLG_MaxCLL_2508.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:limited=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

#Clipping
Limiter(min_luma=64, max_luma=940, min_chroma=64, max_chroma=960)

Prefetch(8)


correctly reports std-b67 which stands for arib-std-b67 which is HLG in the frame properties! :D

https://i.imgur.com/tjBEizV.png

ErazorTT
7th January 2022, 01:12
I would like to report that the dithering methods ordered, random, error_diffusion of the z_ConvertFormat function return slightly different averaged values.
So when having big single color patches like in test patterns, the averages over all pixels are slightly different for the methods.

I admit that the difference is small (9 in the full range of 0-32768 which equals 0.07 for the range of 0-255). But still for 16 bit clips this would be measurable.

I put all the screenshots in the folder and also the saved pictures from the clip, which can be opened an checked:

https://drive.google.com/drive/folders/1kbs9ndCz4cI_NLx5Z1dX_gpi_aJVCEpn?usp=sharing

If this is expected, then which method is nearer to the true average?

kedautinh12
7th January 2022, 03:00
I would like to report that the dithering methods ordered, random, error_diffusion of the z_ConvertFormat function return slightly different averaged values.
So when having big single color patches like in test patterns, the averages over all pixels are slightly different for the methods.

I admit that the difference is small (9 in the full range of 0-32768 which equals 0.07 for the range of 0-255). But still for 16 bit clips this would be measurable.

I put all the screenshots in the folder and also the saved pictures from the clip, which can be opened an checked:

https://drive.google.com/drive/folders/1kbs9ndCz4cI_NLx5Z1dX_gpi_aJVCEpn?usp=sharing

If this is expected, then which method is nearer to the true average?

I think it's faster than convertbits. Maybe can replace convertbits in TemporalDegrain2???

StvG
8th January 2022, 01:12
I would like to report that the dithering methods ordered, random, error_diffusion of the z_ConvertFormat function return slightly different averaged values.
So when having big single color patches like in test patterns, the averages over all pixels are slightly different for the methods.

I admit that the difference is small (9 in the full range of 0-32768 which equals 0.07 for the range of 0-255). But still for 16 bit clips this would be measurable.

I put all the screenshots in the folder and also the saved pictures from the clip, which can be opened an checked:

https://drive.google.com/drive/folders/1kbs9ndCz4cI_NLx5Z1dX_gpi_aJVCEpn?usp=sharing

If this is expected, then which method is nearer to the true average?

It's expected. About the choice of the dither algorithm - you can check this thread (https://forum.doom9.org/showthread.php?t=172266).
For info:
- colorspace_op conversions are done in float and then dither to the target bit depth (if different than float);
- resize operations are done in 16-bit (if source/target are not float) and then dither to the target bit depth (if target lower than source and lower than 16/32-bit).

ErazorTT
10th January 2022, 01:21
I think it's faster than convertbits. Maybe can replace convertbits in TemporalDegrain2???

Really? I’ve never benchmarked that. I would assume that both are orders of magnitudes faster than anything TemporalDegrain2 is doing.

tormento
11th January 2022, 10:56
Really? I’ve never benchmarked that. I would assume that both are orders of magnitudes faster than anything TemporalDegrain2 is doing.
fmtc_bitdepth works really fine too.

Boulder
25th January 2022, 06:08
It's expected. About the choice of the dither algorithm - you can check this thread (https://forum.doom9.org/showthread.php?t=172266).
For info:
- colorspace_op conversions are done in float and then dither to the target bit depth (if different than float);
- resize operations are done in 16-bit (if source/target are not float) and then dither to the target bit depth (if target lower than source and lower than 16/32-bit).

If the process chain is denoise -> downscale -> deband, is it better to leave dithering out of the downscaling phase, since debanding also does some sort of dithering (f.ex. f3kdb)?

StvG
25th January 2022, 23:31
If the process chain is denoise -> downscale -> deband, is it better to leave dithering out of the downscaling phase, since debanding also does some sort of dithering (f.ex. f3kdb)?

It depends. If your source is 10-bit (it will be valid for any bit depth < 16-bit):
a) if denoise in source bit depth (10-bit), downscale w/o bit depth change (10-bit->16-bit->10-bit - how internally avsresize will scale), deband w/o bit depth change (assume f3kdb: 10->16-bit->10-bit - how internally f3kdb will deband)
b) convert to 16-bit, denoise, downscale, deband, convert to source bit depth (10-bit)

For a) it's better to apply dither after the downscaling.
For b) 16-bit->10-bit is done only once at the end - dither.

If the speed penalty of b) isn't the bottleneck, it should be always the preferred option imo.