Dogway's Filters Packs

[MysteryX]

At first I get confused or turned-off when I see too many dependencies like Dogway where you need the whole bundle just to use a function.

Then I'm thinking. What you're developing is a framework kind of like jQuery or TypeScript; that allow us to write modern full-featured JavaScript on the same-old JavaScript. That gets useful when it's well-documented and standardized. It gets a pain when you need the whole framework for a few functions, and another framework for other features, and the libraries conflict with each other.

It would particularly be useful if it would provide more unified syntax/features between Avisynth and VapourSynth. Like I see you use CombinePlanes (ShufflePlanes) extensively in Avisynth.

[Dogway]

The dependency amount is something relative as I try to reduce dependencies to the max, for example yesterday I was debating whether to use nmod() in SMDegrain or not, and decided not to, because it requires ResizersPack and SMDegrain is a widely used filter, even when I think ResizersPack should be in everyone's toolbox.

For example, nmod() tries to standardize the mod functions everyone includes in their scripts, m4(), m8(), m4_sh(), modx(). The at-least-16 (x<16?16:...) that many include is wrong because the mod(n) can output a value below 16. PadBorders() tries to replace Padding(), Addborders(), PadMirror(), sh_Padding(), _Padding(), BorderMirror(), _MirrorBorders(), etc.

I'm constantly trying to reduce dependencies so I included ex_vinverse() in QTGMC, ex_retinex, ex_minblur, ex_luma_rebuild in SMDegrain, and so on, following the concept of Packs. ex_median() replaces a handful of plugins as well. I try to shift the dependencies from plugins or individual scripts to Packs, and then to 'mode' arguments. Some people have been doing this already like manyPlus, plugins_JPSDR, Utils-r4x, jdl_xxx, Zs_RF_Shared, RT_Stats, avsresize, fmtconv.

[Boulder]

Avisynth's test21 build probably broke some similarity metrics. I just ran ex_makediff and I can see the difference with aug=true, but the metric values for PlaneSSIM, PlaneGMSD and FrameMDSI are 1.0, 0.0 and 0.0 respectively. I'm not entirely sure but I think I did check those also with test20 earlier and didn't notice anything.

[pinterf]

Originally Posted by Boulder

Avisynth's test21 build probably broke some similarity metrics. I just ran ex_makediff and I can see the difference with aug=true, but the metric values for PlaneSSIM, PlaneGMSD and FrameMDSI are 1.0, 0.0 and 0.0 respectively. I'm not entirely sure but I think I did check those also with test20 earlier and didn't notice anything.

Try please with test22, there was a glitch with some property handling script functions.

[Dogway]

@Boulder: I'm updating SimilarityMetrics and ex_makediff() today, big refactor for the later. Give me some minutes.

[Dogway]

Updated ExTools and SimilarityMetrics.
Added mode arg to ex_blur, it accepts "binomial", "gaussian" and now "butterworth", this one is a mix between mean and gaussian controlled by a bias arg included as 'n', still need to default it to 6dB/oct slope.
ex_median() now allows 'thres' for all modes.
Now ex_makediff(aug=true) shows more metrics, check below (Requires Butteraugli plugin, and TransformsPack), and added a 'show' argument to decide to what clip add the metric frameprops. I didn't include VMAF because it doesn't write frameprops, I want to add other modes like 4-SSIM, MS-SSIM, DEitp, VQM, MOVIE, BRISQUE, etc.



I've been parametrizing the blur in SMDegrain to allow frequency inputs for the LFR argument. I think it's correct now but want to do a few more checks.

[tormento]

Originally Posted by Dogway

I've been parametrizing the blur in SMDegrain to allow frequency inputs for the LFR argument. I think it's correct now but want to do a few more checks.

I find lfr really great. How does it work now?

[Dogway]

I haven't updated it yet but since the frequency sample rate depends on the resolution, instead of an absolute frequency cutoff it's now relative to the resolution.

For example by setting LFR=300, you are setting the cutoff to 300Hz from the frequency sample rate, for a 1080p clip setting the value to 1920 means no lowpass/blur. Defaults account for input sample rate. It's designed so at the cutoff the attenuation is 3dB. There's another filter I designed today the "butterworth" which typically shows a better lowpass shape, default being a bandpass + lowpass with a 6dB/oct decay/slope. I still have to find how to design this decay and see if it works as good for images as it does for audio.
There's also the "brickwall" filter, it's called the "ideal filter" but it doesn't seem to work fine over images, it should correspond to a "mean" blur kernel. Haven't had the time to test each.

[Boulder]

I've now updated a number of packs, and SPresso gives me a new error message. "Expr: Not enough elements on stack to perform operation", points to extools.avsi, line 3801.

[tormento]

EDIT: ignore following... Windows 11 latest insider decided to f*ck up the Nvidia Driver. Reinstalled the latest DCH and now it is working.

Originally Posted by Dogway

I haven't updated it

Latest everything.

Script Error

KNLMeansCL: fatal error!
(clCreateContext: OCL_UTILS_UNKNOWN_ERROR)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 892)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 915)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 197)

[tormento]

Another one arose:

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\2_24 007 21 - Casino Royale\21casino.dgi",ct=144,cb=144,cl=0,cr=0)
CompTest24(1)
ConvertBits(16)
SMDegrain (tr=4, thSAD=400, refinemotion=true, contrasharp=false, PreFilter=4, plane=4, chroma=true, lfr=1)
fmtc_bitdepth (bits=8,dmode=8)

Script Error

Script error: There is no function named 'nmod'.
(D:/Programmi/Media/AviSynth+/plugins64/ExTools-6.2_22~Dogway.avsi, line 1330)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 378)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 380)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 380)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 385)
(D:/Programmi/Media/AviSynth+/plugins64/SMDegrain-3.3.4d~Dogway.avsi, line 560)

It works on Ex_tools 6.1.

[Dogway]

Yes, nmod() is in ResizersPack, it replaces sh_m4() from Zs_RF_Shared. You can copy/paste it anywhere else.

Originally Posted by Boulder

I've now updated a number of packs, and SPresso gives me a new error message. "Expr: Not enough elements on stack to perform operation", points to extools.avsi, line 3801.

Shit sorry, a stupid typo. Updated.

[Boulder]

Originally Posted by Dogway

Shit sorry, a stupid typo. Updated.

Thanks, it works now

One more I found out, CASm with a 1080p source cropped by 210 pixels off left and 272 pixels off right, I get an error "CombinePlanes: source and target plane dimensions are different".

[Dogway]

Fixed now in ExTools, keep'em coming!! : D

[tormento]

Originally Posted by Dogway

it replaces sh_m4() from Zs_RF_Shared

So I can get rid of ZS?

[Dogway]

It's still required for sh_GetUserGlobalIMTint() but I will try to remove that dependency in SMDegrain for the next version.

EDIT: Does anybody have a build of KPassFilterCL? I need it to double check some frequency related functions. I have GCC but I can't build avs plugins with this compiler.

Visual examples for lowpass filters:

Mean (brickwall)


Butterworth (with n: 50, 10 and 2)




Gaussian




Butterworth 'n' setting animated from 300 to 0:

[poisondeathray]

Originally Posted by Dogway

EDIT: Does anybody have a build of KPassFilterCL? I need it to double check some frequency related functions. I have GCC but I can't build avs plugins with this compiler.

There is a link mirror on the top right corner of this page to v0.1.0
http://avisynth.nl/index.php/KPassFilterCL

[tormento]

Originally Posted by Dogway

I have GCC but I can't build avs plugins with this compiler.

You can easily get MSVC or Intel really easily.

[Dogway]

I don't have space in C: and 4Gb (at the very least) for building once every half a year...

I tested the plugin but didn't work for me... It would really be great to have vcmohan F1QLimit in avisynth to preview frequency domain in 1D.

By the way I built a sharpener ex_sharp() that by chance matched unsharp(), but I added a twist by also adding 20% of the sharpness from a lower frequency cutoff as reinforcement. It's frequency based so the varY option of unsharp() is now Fc and you can use 1920 for pixel accuracy, 1920*2 for subpixel accuracy and 1920*0.5, or simply 960 to sharpen low frequency. I also finished designing the butterworth slope, first order is actually the 6dB/oct slope, very gaussian like.

[Dogway]

Just updated SMDegrain with a battery of changes. You can read the log in the commit description.

To expand, I repurposed the LFR argument so now it describes frequency. The higher the more higher frequencies you restore, notice the frequency is bound to clip width (typically) resolution so 300 is not the same for a 1080p clip than for a 2160p one. By default it's to set to 300Hz for 1080p, 600Hz for 1080p and so on.
I reworked the LFR code in non-MDegrain modes to use the one in MDegrain which is sensibly faster.

I included a new setting called DCTFlicker, this is a problem I described some weeks ago which is low frequency noise so I wanted to do something with it. It happened I stumbled over this Didée's post to tackle this very same issue. For that I had to port FluxSmoothT to Expr() which was a pending task in order to expand temporal radius and include a soft threshold. FluxSmoothT is nice because it's the minimum change of a temporal average and a temporal median, along with the soft threshold it makes for a good low frequency calmer. It's not perfect, but there's no perfect solution either (reference). That's why a multistep solution although slower might be desired. Check below with the great sample clip from tormento.

raw

DNR..........................................................................DNR+LFR
............
DNR+LFR+DCTR........................................................DNR+LFR+DCTR (tailored: multi-step)
............

Call for the multi-step solution

pre=smdegrain(tr=1,mode="temporalsoften",blksize=32,thSAD=900,LFR=400,prefilter=5,DCTFlicker=true,contrasharp=false,refinemotion=true)
smdegrain(tr=2,mode="MDegrain",blksize=32,prefilter=pre,thSAD=400,LFR=false,contrasharp=true,refinemotion=true)