Dogway's Filters Packs

[tormento]

Originally Posted by anton_foy

I am maybe wrong but does not knlmeanscl use some kind of motion compensation already? Try it with 'd=12' and see how it acts. BM3D is already using block matching.

You are absolutely right for BM3D, I don't know nlmeans enough to answer you.

My request was originated by the large amount of time and processing required by BM3D for the block matching. I am just curious to know if results could be better or not.

[LouieChuckyMerry]

Originally Posted by Dogway

I tested and Deblock_QED works fine in HBD.
The Deblock dependency has nothing to do with BlockDetect, its Deblock, here's the avisynth entry page. It's basically a deblock filter, Deblock_QED limits the deblocking to the block edges.

Maybe the issue is on the other filters?

Thanks for your patience, Dogway . I've Deblock, DCTFilter, ExTools, and ResizersPack, all the latest x64 versions, but the simple call Deblock_QED results in the error message:

Avisynth open failure: Script error: There is no function named 'Deblock_deblock'... DeblockPack.avsi, line 148.

Any thoughts?

[Dogway]

Originally Posted by LouieChuckyMerry

I've Deblock, DCTFilter, ExTools, and ResizersPack, all the latest x64 versions, but the simple call Deblock_QED results in the error message:

Any thoughts?

You should test separately to see why does it happen.

Here's the reasoning about the basename for the call.


Basically there are a few Deblock functions, so the basename was to determine from which plugin it came.

Basically load a blocky source and try with this:

Deblock_deblock(quant=24,aOffset=1,bOffset=2)

[tormento]

Originally Posted by Dogway

You should test separately to see why does it happen.

Please, when you have time, have a look here.

[LouieChuckyMerry]

Originally Posted by Dogway

You need to download Deblock plugin.

The plugin prefix Deblock_ is to not be confused with other Deblock called functions, it was a change in 2014 by GMJCZP.
It shouldn't trigger an error, it doesn't for me. As Reel.Deel stated the plugin should be called Deblock.dll (case insensitive).

Thank you . I always name my .dlls and .avsis with the version number and architecture (like Neo_FFT3D__64bit.dll or Neo_F3KDB9_0__64bit.dll) because it helps me keep track of them (and I need all the help I can get, ha ha) and have never had a problem until now. Renaming the Deblock .dll from "Deblock1_3__64bit" to just "Deblock" fixed the problem. Thank you again, be well.

[salvo00786]

Hi... I have a problem with qtgmc+...

This is my script:
QTGMCp(Preset="Placebo", TR0=2, TR1=2, TR2=0, Rep0=7, Rep1=7, Rep2=7, DCT=5, SourceMatch=3, Lossless=1, MatchPreset="Placebo", MatchPreset2="Placebo", MatchEnhance=1.0, MatchTR2=2, EZKeepGrain=1.0, Sharpness=0.2, Sbb=3, ThSCD1=250, ThSCD2=95, NoiseProcess=0, SMode=2, SLMode=4, Subpel=4, Search=5, PelSearch=7, SearchParam=7, Overlap=8, EdiMode="EEDI3CL+NNEDI3CL", Ediqual=2, ChromaMotion=True, TrueMotion=true, Precise=True, NNeurons=4)

When I use the same script on the old QTGMC with EdiMode="EEDI3+NNEDI3CL" (mix cpu and gpu) and Edithreads=16, I have a speed of 25fps. When I use the complete script on qtgmc+ with EdiMode="EEDI3CL+NNEDI3CL" (total gpu) but without edithreads=16 (because in qtgmc+ edithreads is not accepted), I have 2fps encoding speed.

Yesterday I removed one by one all the setting in my script and I found that this occur only when I use Sourcematch... If delete Sourcematch the speed go at 30fps... Please can you tell me why this occur and can you give me the latest version of qtgmc+ with edithreads? Or do you have some other solution for me?

Also, what are the maximum values for rep0 rep1 and rep2?

[Selur]

can you give me the latest version of qtgmc+ with edithreads?

https://github.com/Dogway/Avisynth-S.../QTGMC%2B.avsi, there is no edithreads, it's called Threads here.

Also, what are the maximum values for rep0 rep1 and rep2?

https://github.com/Dogway/Avisynth-S...%2B.avsi#L1599

[salvo00786]

[QUOTE=Selur;2017056]https://github.com/Dogway/Avisynth-S.../QTGMC%2B.avsi, there is no edithreads, it's called Threads here.

Yes, I know, but I am searching for an older version of qtgmc+ after 4.0, like 4.0p with edithreads (if exist)

[Selur]

you can get the older versions on the github page too,....

[Dogway]

@salvo00786, can you check the speed in qtgmc+ when using full cpu "EEDI3+NNEDI3"?, Try also setting 'Threads' to 16, and then without the setting (defaults).

If it goes slow, then play with the 'Threads' value in this line in ResizersPack.
https://github.com/Dogway/Avisynth-S...Pack.avsi#L997

GPU versions ignore 'Threads' as usual. From what I recall mixing cpu and gpu in heavy scripts create some race conditions that harm performance. I typically run GPU when they are like 2 (maybe 3 max) simple filters in line.

[Adub]

Hello Dogway, thank you so much for your work on ExTools, etc.

I'm currently reading though some of your blurring functions, and I have a few questions just to ensure that I'm understanding things properly.

Looking at your ex_median("smart"), it looks like you're processing a 3x3 grid. You take the average of the surrounding 8 pixels, and then use a sorting network on the same 8 pixels.

What I'm not totally clear on is what's being stored in the "E^ D^ H^" variables near the end.

I see that you are storing some part of the sorting network into these variables and then popping them off the stack before you conduct some squared differences using what's left on the stack and then what's stored in H.

So in short, what do the E, D, and H variables contain, and what's left on the stack that's being used for the first squared difference with the average?

Edit: I *think* D and E hold the 4th smallest/largest pixel values in the 3x3 grid, and the stack and H hold the 3rd smallest/largest pixels, respectively. Do I have that right?

[Dogway]

Originally Posted by Adub

What I'm not totally clear on is what's being stored in the "E^ D^ H^" variables near the end.

I didn't remember but reading through it a bit, if I'm not mistaken, E and D are the "max" and "min" values, so the middle values of the ranking order. There are still two values in the stack, the same middle values but unordered, so H pops out one (doesn't matter which, it's for the variance), and the left one is immediately consumed by the first substraction "leftover-mean-on-the-stack W - dup *"

So the end of the sorting network is like this:

4 3 (unordered)
3 4 (ordered)

Pop 4 (max as E), 3 (min as D), and 3 (unordered, as H), consume 4 for the substraction, then 3 (H now) for the second substraction.

Check davidhorman tool here. https://horman.net/expr_sort.php. Select 8 inputs, then checkboxes for 3 and 4.

EDIT: I see you are making ZSmooth, one the missing pieces of my project was implementing mt_hysteresis (it's very slow), because that needs to be done on a robust language, maybe you can give it a try.

[Adub]

Ahhh got it, I was close but a little off the mark. Thank you for the thorough explanation.

The one last thing I'm not sure I understand is the squared difference logic. I understand the order of operations, but it's more of the "why" that I'm lacking depth on. It seems to be a key part of the the "variance" calculation, but I haven't quite groked what its doing yet.

Put another way, what is the motivation for taking the square root of the squared differences and multiplying by 13?

Re: Zsmooth - yup, I'm currently implementing spatial/temporal smoothing/denoising functions there. I haven't seen a need for a hysterisis implementation just yet, as none of the algorithms (so far) have utilized a mask.

Did you mean that a hysterisis implementation would be nice to have in general, or that it would specifically benefit some of your "smoothing" functions in some way?

[Dogway]

Its not a strict variance although it's based on it. From the formula it's basically an euclidean distance. The reason I didn't divide by the count, or that I multiplied by 13 was too normalize or saturate the output for a consistent thresholding with the rest of modes. Basically I did the full implementation with variance and from there started optimizing the ops for performance.
The problem with hysteresis is that there exists only one implementation. It's in masktools and it's very slow, pairing that with its huge benefits for refining masks (acts as a smart gap filler) makes it in my view a great candidate for revision. I don't know how the scenario is in vapoursynth but I guess it's much faster there.

[Adub]

Got it. Thanks again for the clear explanation.

Yeah, Vapoursynth has at least two different implementations of Hysteresis that I'm aware of:
1. https://github.com/vapoursynth/vs-mi.../docs/misc.rst
2. https://github.com/sgt0/vapoursynth-hysteresis

I haven't personally done any performance tests though. If I had to guess, the later is likely the most performant. Might be worth asking if they'll create an Avisynth port at the very least.

[Adub]

Actually, I think I may have found a bug with the smart median implementations. After implementing a similar approach in Zsmooth, I noticed that dark pixels were getting filtered before light pixels, even on smooth/flat textures.

Looking at the expression, I figured out that taking the difference between the "variance" and the source pixel was the root of the issue. Using grayscale as an example, if the source pixel is brighter, then the difference between the bright pixel and the variance will often exceed the provided threshold.

I simply removed the difference and the filter started smoothing like I expected.

So I think that removing "x - abs" from here is all that's really necessary: https://github.com/Dogway/Avisynth-S...ols.avsi#L4270

[Lucky38]

@Dogway


You can also add some masks to protect details...

dt=ex_edge("kayyali",lo=40)
dtm=dt.ex_expand(2,mode="disk")
em=ex_edge("kroon")
gib=em.ex_expand(2).ex_inflate()


ConvertBits(16)
pre=ex_FluxSmoothST().ex_minblur(1)
msk=ex_logic(dtm,em,"max").ConvertBits(16)
pre=ex_merge(pre,last,msk)
SMDegrain(2, 200, mode="RMDegrain", prefilter=pre,RefineMotion=true)
ConvertBits(8,dither=1)

[/QUOTE]


can this mask to protect details be used in general, or it is for this specific case...?

[Tlen]

Hi Dogway,

thank you for your work.

I'm trying to make a bit of temporal degraining and

SMDegrain(tr=1,mode="TemporalSoften")

gives me quite a satisfactionary result.

But it eats a little bit too much grain.

There is a way to loosen it up a little to retain a bit more (like a treshold or something)?

Thank you very much

EDIT:
Found it myself thSAD parameter

[hello_hello]

Dogway,

if you're still around, I've had a couple of issues with QTGMC+ not de-interlacing anywhere near as well as QTGMC on "problematic" video. Unfortunately at the time I didn't get around to uploading a sample, but today I came across a problematic progressive video that QTGMC+ doesn't cope with well at all (the result is similar to problematic interlaced video). QTGMC cleans it up reasonably well. The zip file I've linked to below contains a small section of the source video (1080p) along with a sample encode with QTGMC and another using QTGMC+. As you can see, the difference is massive. The following is the only line in the script, aside from the FFMS2 source filter.

Either
QTGMC(InputType=1, EzDenoise=1.5, Preset="very slow")
or
QTGMCp(InputType=1, EzDenoise=1.5, Preset="very slow", gpuid=-1)

QTGMC v QTGMCp.zip (39.6 MiB)

Cheers.

[junah]

Perhaps try without gpuid=-1 to make comparable examples.