Dogway's Filters Packs

[Adub]

Hello Dogway, thank you so much for your work on ExTools, etc.

I'm currently reading though some of your blurring functions, and I have a few questions just to ensure that I'm understanding things properly.

Looking at your ex_median("smart"), it looks like you're processing a 3x3 grid. You take the average of the surrounding 8 pixels, and then use a sorting network on the same 8 pixels.

What I'm not totally clear on is what's being stored in the "E^ D^ H^" (https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi#L4268-L4270) variables near the end.

I see that you are storing some part of the sorting network into these variables and then popping them off the stack before you conduct some squared differences using what's left on the stack and then what's stored in H.

So in short, what do the E, D, and H variables contain, and what's left on the stack that's being used for the first squared difference with the average?

Edit: I *think* D and E hold the 4th smallest/largest pixel values in the 3x3 grid, and the stack and H hold the 3rd smallest/largest pixels, respectively. Do I have that right?

[Dogway]

What I'm not totally clear on is what's being stored in the "E^ D^ H^" (https://github.com/Dogway/Avisynth-Scripts/blob/master/ExTools.avsi#L4268-L4270) variables near the end.

I didn't remember but reading through it a bit, if I'm not mistaken, E and D are the "max" and "min" values, so the middle values of the ranking order. There are still two values in the stack, the same middle values but unordered, so H pops out one (doesn't matter which, it's for the variance), and the left one is immediately consumed by the first substraction "leftover-mean-on-the-stack W - dup *"

So the end of the sorting network is like this:

4 3 (unordered)
3 4 (ordered)

Pop 4 (max as E), 3 (min as D), and 3 (unordered, as H), consume 4 for the substraction, then 3 (H now) for the second substraction.

Check davidhorman tool here. https://horman.net/expr_sort.php. Select 8 inputs, then checkboxes for 3 and 4.

EDIT: I see you are making ZSmooth, one the missing pieces of my project was implementing mt_hysteresis (it's very slow), because that needs to be done on a robust language, maybe you can give it a try.

[Adub]

Ahhh got it, I was close but a little off the mark. Thank you for the thorough explanation.

The one last thing I'm not sure I understand is the squared difference logic. I understand the order of operations, but it's more of the "why" that I'm lacking depth on. It seems to be a key part of the the "variance" calculation, but I haven't quite groked what its doing yet.

Put another way, what is the motivation for taking the square root of the squared differences and multiplying by 13?

Re: Zsmooth - yup, I'm currently implementing spatial/temporal smoothing/denoising functions there. I haven't seen a need for a hysterisis implementation just yet, as none of the algorithms (so far) have utilized a mask.

Did you mean that a hysterisis implementation would be nice to have in general, or that it would specifically benefit some of your "smoothing" functions in some way?

[Dogway]

Its not a strict variance although it's based on it. From the formula it's basically an euclidean distance. The reason I didn't divide by the count, or that I multiplied by 13 was too normalize or saturate the output for a consistent thresholding with the rest of modes. Basically I did the full implementation with variance and from there started optimizing the ops for performance.
The problem with hysteresis is that there exists only one implementation. It's in masktools and it's very slow, pairing that with its huge benefits for refining masks (acts as a smart gap filler) makes it in my view a great candidate for revision. I don't know how the scenario is in vapoursynth but I guess it's much faster there.

[Adub]

Got it. Thanks again for the clear explanation.

Yeah, Vapoursynth has at least two different implementations of Hysteresis that I'm aware of:
1. https://github.com/vapoursynth/vs-miscfilters-obsolete/blob/master/docs/misc.rst
2. https://github.com/sgt0/vapoursynth-hysteresis

I haven't personally done any performance tests though. If I had to guess, the later is likely the most performant. Might be worth asking if they'll create an Avisynth port at the very least.

[Adub]

Actually, I think I may have found a bug with the smart median implementations. After implementing a similar approach in Zsmooth, I noticed that dark pixels were getting filtered before light pixels, even on smooth/flat textures.

Looking at the expression, I figured out that taking the difference between the "variance" and the source pixel was the root of the issue. Using grayscale as an example, if the source pixel is brighter, then the difference between the bright pixel and the variance will often exceed the provided threshold.

I simply removed the difference and the filter started smoothing like I expected.

So I think that removing "x - abs" from here is all that's really necessary: https://github.com/Dogway/Avisynth-Scripts/blob/c6a837107afbf2aeffecea182d021862e9c2fc36/ExTools.avsi#L4270

[Lucky38]

@Dogway


You can also add some masks to protect details...
dt=ex_edge("kayyali",lo=40)
dtm=dt.ex_expand(2,mode="disk")
em=ex_edge("kroon")
gib=em.ex_expand(2).ex_inflate()


ConvertBits(16)
pre=ex_FluxSmoothST().ex_minblur(1)
msk=ex_logic(dtm,em,"max").ConvertBits(16)
pre=ex_merge(pre,last,msk)
SMDegrain(2, 200, mode="RMDegrain", prefilter=pre,RefineMotion=true)
ConvertBits(8,dither=1)[/QUOTE]


can this mask to protect details be used in general, or it is for this specific case...?

[Tlen]

Hi Dogway,

thank you for your work.

I'm trying to make a bit of temporal degraining and

SMDegrain(tr=1,mode="TemporalSoften")

gives me quite a satisfactionary result.

But it eats a little bit too much grain.

There is a way to loosen it up a little to retain a bit more (like a treshold or something)?

Thank you very much

EDIT:
Found it myself thSAD parameter

[hello_hello]

Dogway,

if you're still around, I've had a couple of issues with QTGMC+ not de-interlacing anywhere near as well as QTGMC on "problematic" video. Unfortunately at the time I didn't get around to uploading a sample, but today I came across a problematic progressive video that QTGMC+ doesn't cope with well at all (the result is similar to problematic interlaced video). QTGMC cleans it up reasonably well. The zip file I've linked to below contains a small section of the source video (1080p) along with a sample encode with QTGMC and another using QTGMC+. As you can see, the difference is massive. The following is the only line in the script, aside from the FFMS2 source filter.

Either
QTGMC(InputType=1, EzDenoise=1.5, Preset="very slow")
or
QTGMCp(InputType=1, EzDenoise=1.5, Preset="very slow", gpuid=-1)

QTGMC v QTGMCp.zip (https://files.videohelp.com/u/210984/QTGMC%20v%20QTGMCp.zip) (39.6 MiB)

Cheers.

[junah]

Perhaps try without gpuid=-1 to make comparable examples.

[rgr]

I noticed a long time ago and mentioned here that QTGMC+ simply works much worse than QTGMC.

[hello_hello]

Perhaps try without gpuid=-1 to make comparable examples.

I'm pretty sure they're comparable the way they are.

QTGMC+

# Core plugins:
# ResizersPack (for nnedi3=true)
# NNEDI3CL (for nnedi3=true and gpuid=0) (requires at least AVS+ v3.7.3)
# nnedi3 (for nnedi3=true and gpuid=-1)

I noticed a long time ago and mentioned here that QTGMC+ simply works much worse than QTGMC.

It's odd because for fairly good quality video (interlaced or progressive) they seem fairly similar, but I think I'll go back to using the original QTGMC for the moment, just to be safe.

[rgr]

It's odd because for fairly good quality video (interlaced or progressive) they seem fairly similar, but I think I'll go back to using the original QTGMC for the moment, just to be safe.

I tested it once and found that the + was faster, but too light. It seemed like its filters were much weaker. It's a pity, because at 10 bits it was significantly faster.

[hello_hello]

The sample I uploaded contains a lot of shimmering, although I'm not sure why (the video is only like that in a few places), so I assume it's QTGMC's shimmer repair doing most of the work stabilizing it, and maybe that isn't working correctly in the QTGMC+ version, or something....

[WAusJackBauer]

Hello all. I've used this SMDegrain for a while but after updating it, it seems to have changed a bit. I'm looking for recommendations (hopefully from Dogway) on what settings to use as a one size fits all thing even though I know it's better to fine tune settings depending on the source video.

My main priority is minimal ghosting/smearing/artifacts and my second priority is removal of most noise but a slight amount remaining isn't a big problem. The content I encode is raw bluray movies, so the videos are 1920x1080 and they're live action. My settings currently look like this:

SMDegrain(tr=3, thSAD=400, RefineMotion=true, contrasharp=true, plane=4, prefilter=6, chroma=true)

I'm wondering if anyone has any suggestions on different settings to use that would give better results as I don't have a lot of knowledge on the parameters.

[DTL]

thSAD setting depends only on the relative noise level in your content. It is generally only protection setting from too bad blends and smoothing. The main adjustment of linear temporal nosie reduction is tr-value. To get lowest level of distortions you typically need first select your working tr-value (balance between performance and degrain level) and slowly increase thSAD (from very low values where it is not degrain at all) to adjust balance between degrain and smoothing. If you lucky you have some range of thSAD where degrain already enough and details smoothing is not yet started. If not - you need to adjust thSAD between either better degrain or better details saving.

Some time ago I found there is a simple API call in AVS+ allow to print debug data on output frame - it is very highly wanted feature for adjusting lots of params for mvtools (MAnalyse + MDegrain). But still not used.

[TR-9970X]

Hello all. I've used this SMDegrain for a while but after updating it, it seems to have changed a bit.

SMDegrain(tr=3, thSAD=400, RefineMotion=true, contrasharp=true, plane=4, prefilter=6, chroma=true)

I'm wondering if anyone has any suggestions on different settings to use that would give better results as I don't have a lot of knowledge on the parameters.

I have one specific question - I recently looked into thSAD and was thinking of changing it to 500 because it removes slightly more noise than 400 which I feel is nicer for some extra noisy scenes but I'm worried that it will cause ghosting/smearing. Is thSAD 500 fine to use for my type of content?

Here's one that use a LOT, but does get changed depending on the footage quality.

video=SMDegrain(video,tr=1,thSAD=100,thSADC=50,contrasharp=true,prefilter="MinBlur3",str=5.0,refinemotion=true)

video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter="MinBlur3",str=5.0,refinemotion=true)

etc, etc

I use RipBot264, but for most other app's, you'd remove the video from the call.

[Emulgator]

And may I add:
To whom prefilter=6 (KNLMeansCL) is too smeary (to me it was) I can suggest prefilter=8 (BM3D).
Well worth the effort.

[TR-9970X]

And may I add:
To whom prefilter=6 (KNLMeansCL) is too smeary (to me it was) I can suggest prefilter=8 (BM3D).
Well worth the effort.

I kind of agree. I had forgotten what #6 was.

But doesn't BM3D work best with a hi end nVidia GPU ??

I tried it years ago, and I wasn't impressed, but I think I know why.

[Emulgator]

Yes, BM3D is resource hungry, RTX3080 here.
BM3D became my choice for the least harming treatment, keeping a filmic impression, subtly improving compressibilty.

[tormento]

But doesn't BM3D work best with a hi end nVidia GPU ??
1660 Super here. ;)

Remember to disable BM3D fast parameter when used in combination with cpu filter, i.e. almost always.

[WAusJackBauer]

And may I add:
To whom prefilter=6 (KNLMeansCL) is too smeary (to me it was) I can suggest prefilter=8 (BM3D).
Well worth the effort.

Yeah I heard prefilter 8 was a good one but I have an AMD GPU unfortunately.

I mainly used prefilter 6 because it apparently utilizes a GPU a little bit so it was slightly faster than the others. But if you believe a different prefilter would be better for my content, please let me know :)

[TR-9970X]

Yeah I heard prefilter 8 was a good one but I have an AMD GPU unfortunately.

I mainly used prefilter 6 because it apparently utilizes a GPU a little bit so it was slightly faster than the others. But if you believe a different prefilter would be better for my content, please let me know :)

That's unfortunate :(

You can use BM3D with a CPU, but what's the point in that.

So what app are you using ??

Have you tried my script suggestion ?

[LeXXuz]

Yeah I heard prefilter 8 was a good one but I have an AMD GPU unfortunately.


No shame in that. KNLMeansCL is a great denoiser if set up correctly.
I'd pick it over BM3D any day. ;)
And it runs much better on AMD GPUs than on comparable Nvidia cards.

Keep in mind those prefilter presets are merely defaults for different kinds of noise. Those are for people who can barely create their own Avisynth script.
If you want to get the optimum out of SMDegrain, nothing beats your very own prefilter setup.

Just my 2 cents.

[tormento]

KNLMeansCL is a great denoiser if set up correctly. I'd pick it over BM3D any day. ;)
Let’s agree to disagree ;)

Show me a source and your scripts to prove me that BM3D, especially in temporal+spatial flavor, is worse than klnmeanscl. :)

[LeXXuz]

Let’s agree to disagree ;)

Show me a source and your scripts to prove me that BM3D, especially in temporal+spatial flavor, is worse than klnmeanscl. :)

Prove? There's nothing to prove here. I said I'd pick NLM over BM3D any day for my personal taste and needs.

Why do you want to do inferior temporal denoising on a clip you'll feed to an excellent temporal denoiser afterwards?

As a standalone filter, I agree with you because KNLMCLs temporal denoising is utter garbage. And that's why I'd never use prefilter-6 setting in SMDegrain.

Let SMDegrain do the temporal denoising and give it a little help with a custom spatial denoiser, if needed.

And here I pick NLM over BM3D because it does a good job in the spatial domain, it is fast and needs way less CPU resources and much less VRAM on AMD.

Which brings me back to my last post where I said it is no shame to have an AMD GPU. OpenCL apps run faster on AMD and need way less VRAM because Nvidia is too dumb to code proper memory management for OpenCL, especially on RTX GPUs.
People complain in NV forums for a long time, but they don't care because they want devs to use CUDA of course.

[pwnsweet]

Remember to disable BM3D fast parameter when used in combination with cpu filter, i.e. almost always.

What is the reasoning behind this?

[tormento]

What is the reasoning behind this?
On the majority of computers, the overhead will kill the performance gain, above all when other filters are used too.

[Tlen]

I read on the manual regarding planes

Keep in mind that plane=4 (2 and 3 as well) can sometimes create chroma smearing. In such case I recommend denoising chroma planes separately in the spatial domain.

And I saw a post on a blog comparing denoiser where the blogger show a clear example of the problem.

Is this still valid today?
Which are the risky conditions?

Actually I've always used to call 3 times on separate planes, but now I'm in a situation with an extreme grain source, when raising the tr the script doesn't even start being too much articulated with the 3 separate planes.

How can i handle this?
Thanks

[Arx1meD]

ex_bilateral does not work with different radius and radiusV values.

ex_bilateral(radius=3, radiusV=1)

AviSynth gives an error:
Expr: keyword or variable not found: 'B'
(D:\AviSynth\AviSynth plugins\ExTools.avsi, line 1623)

Does anyone know how to fix it?

[Emulgator]

I wouldn't attempt to separate chroma planes U,V before denoising.
I am expecting the result to obtain non-matching U,V planes, so more prominent faults.

[real.finder]

I noticed a long time ago and mentioned here that QTGMC+ simply works much worse than QTGMC.

btw, there are also M_QTGMC https://forum.doom9.org/showthread.php?p=2018905#post2018905 if you have the time and videos to test

[Tempter57]

btw, there are also M_QTGMC https://forum.doom9.org/showthread.php?p=2018905#post2018905 if you have the time and videos to test

You have specified in the link where DTL uses your version QTGMC.avsi, mvtools2.dll by pinterf and M_QTGMC_old_mvt.avsi on their basis.
Besides it at DTL there is a version QTGMC2.avsi, M_QTGMC.avsi and mvtoos2.dll https://github.com/DTL2020/QTGMC/releases/tag/m_0.2
It will be valid to enter for comparing a script on this basis too.
The result will turn out qualitative, but handling process very slow.

[LeXXuz]

I just gave QTGMC+ another shot. It still seems broken for me, causing massive artefacts. :(

The old QTGMC is working fine but smoothes the picture way too much for my taste. Tried to set NoiceProcessing=0 but it looks like this has little to no effect.

QTGMC+ lookes much crisper but sadly said artefacts make it unusable.

[DTL]

"The old QTGMC is working fine but smoothes the picture way too much for my taste."

If you use 'placebo' preset as base it is recommended to manually switch Search to some of 2,3,4 modes (or may be some DCT-based). In my tests search mode 5 causes significant general frame blurring and I do not know why it is enabled for possibly 'quality' preset placebo. The only idea is to compensate for significant performance drop from other processing steps. But this also causes quality degradation and may destroy the total idea of the 'placebo' preset.

In QTGMC+ I also see all Search modes are 0,2,4 and placebo is 4 too - https://github.com/Dogway/Avisynth-Scripts/blob/c6a837107afbf2aeffecea182d021862e9c2fc36/MIX%20mods/QTGMC%2B.avsi#L260 .

Also with latest updates of QTGMC the new quality options like RefineMotionAll and trymany added and dual refining stages possible (with blksize->blksize/2->blksize/4 spatial refining) - this makes quality of single QTGMC call closer to the old ideas about gathering details from several calls in M_QTGMC script.

[Lucky38]

Hello
Do you know how to find for each hdr/dovi source white value to be used in below call?

pre = original.DGHDRtoSDR(mode="pq", white=849, gamma=1/2.4, tm=1.0)

SMDegrain(tr=3, thSAD=280, thSCD2=100, limit=140, limitC=120, thSADC=240, prefilter=pre, contrasharp=false)