Dogway's Filters Packs

[kedautinh12]

New LSFmod
https://github.com/Dogway/Avisynth-S...mod.v2.9x.avsi
BM3D
http://avisynth.nl/index.php/BM3DCUDA
Edit: oh DG faster in info of BM3D

Originally Posted by videoh

https://forum.doom9.org/showthread.p...highlight=BM3D

Read the whole thread. test3 supports AVS+.

Originally Posted by kedautinh12

New LSFmod
https://github.com/Dogway/Avisynth-S...mod.v2.9x.avsi
BM3D
http://avisynth.nl/index.php/BM3DCUDA
Edit: oh DG faster in info of BM3D

[Dogway]

Just updated SMDegrain to v3.2.5d. Among the mentioned, about updating LSFmod argument I also added an option to disable the previous hard-coded "UHDhalf", for faster UHD processing, by default it's set to true though.
The main updates are moving ex_contrasharpening() to SharpenersPack, optimization of ex_sbr() and big overhaul of ex_MinBlur().

About ex_MinBlur() I did heavy edits for two reasons, faster processing (between 50 and 100% more) and better quality, by replacing removegrain(4) with ex_median("edgeCL") and MedianBlur(2) with ex_median("EMF2"), a custom made median filter that I'm yet to upload with ExTools, probably tomorrow.
Since these replacements protect edges quite well I added some post-blur so lines are not dirty and aiding to denoise for a practical match, therefore respecting the original intention while actually improving on it. I added an argument 'old' to go back to old method.

[StainlessS]

Had a bit of a look at ExTools a bit earlier, outstanding!
Respect Bro.
[wish I understood some of it].

And thanks from here too

EDIT: And I learnt a few things just from this

##################
## MORPHOLOGICAL #
##################
##
## Opening: To clean small blotches
##   ex_inpand(n).ex_expand(n)
##
## Closing: To close gaps
##   ex_expand(n).ex_inpand(n)
##
## Smooth: Opening + Closing, effectively cleaning binary images
##   ex_inpand(n).ex_expand(n+n).ex_inpand(n)
##
## Outline: To reveal the outer edge of a shape
##   ex_makediff(ex_expand(n), a, dif=false)
##
## Inline: To reveal the inner edge of a shape
##   ex_makediff(a, ex_inpand(n), dif=false)
##
## Gradient: To reveal the boundary of elements in a binary image
##   ex_makediff(ex_expand(n), ex_inpand(n), dif=false)
##   or
##   ex_hitormiss(mode="boundary") # thinner
##
## Top Hat:   (a - opening) To reveal bright elements over dark backgrounds on a greyscale image
##   ex_makediff(a, ex_inpand(n).ex_expand(n), dif=false)
##
## Black Hat: (closing - a) To reveal dark elements over bright backgrounds on a greyscale image
##   ex_makediff(ex_expand(n).ex_inpand(n), a, dif=false)
##

All that masking stuff is mostly a mystery to me.

EDIT: I've never known what term to look for as reference for masktools type operations [inpand/expand etc], just found it,
Google "morphological operations"

Originally Posted by Dogway

Maybe if you are doing contrasharpening with values you need my mod of LSFmod, or if you are using BM3D prefiltering you might need Utils-r41.
I noticed that in latest LSFmod I changed 'defaults' arg to 'preset', try changing that in line 337 until I update SMDegrain.

Well, I've been "fiddling" most of the day, and I'm just confused...

There are too many SMDegrain builds out there, yesterday it was 3.2.4d, today it's 3.2.5d, there's different variants of LSFMod...the ex & mx "series".

It would be great to know ALL the dependencies required for these.

And I have found that depending on the prefilter option I choose, to what SMDegrain error gets thrown up.

I was trying to get BM3D to work, and that seems to be prefilter #5, but that shows an error with Line 158 (I think) inputP, then you change the prefilter to another number, and then there's an error with Line 448 which is just 2 of these {.

There's Utils r41, that doesn't help...

And I couldn't find this :-

I noticed that in latest LSFmod I changed 'defaults' arg to 'preset', try changing that in line 337 until I update SMDegrain.

Anyway, most of this it WAY over my head, so maybe I just need to stick with what works, and leave it at that.

[Dogway]

@Pauly Dunne: It works for me so I think the scripts are fine. You have got suggestions from others to update to latest avisynth+ and scripts. About ex and mix mods, it's described in OP. If you want raw speed go to mix mods, ex mods usually just remove the dependency of RGTools and masktools2 at the cost of some speed.

For my scripts the dependencies almost go to the scripts in OP; ExTools, Transforms Pack and Resizers Pack, and externally Zs_Shared and Utils-r41 (will get rid of this dependency soon with TransformsPack). Then you have all the plugins which really are not many, just the typical; RGTools, masktools2, mvtools2, GRunT, and then depending on each script (ie. DFTTest, KNLMeansCL, BM3D, AddgrainC...). That's the purpose of this project, remove plugin dependencies to the max.

I can only vouch for my scripts like SMDegrain or mods like EX/MIX mods. I came back here 3 months ago after being away for a few years so it's normal you see some changes. When I finish everything in a few months you won't see changes anymore. Bug reports aside, ExTools is one version off of being finished, SMDegrain is finished today. I won't work on these anytime soon and will focus on TransformsPack only.

I just updated ExTools and SMDegrain. Try it and if you find an error report back with the error and filter call.

[Dogway]

@StainlessS: Thanks! The sentiment is mutual lol. Actually I learn as I go. The most intriguing for me was the hit-or-miss functions, really I spent days trying to figure out how to translate that to mach... Expr code. Was kind of a feat for me. Anyway I couldn't port Hysteresis -__- but ex_luts is shaping up nicely, it's in a working state for some modes. With some tests it shows between double or three times faster than mt_luts(). Pixel Addressing was a hidden gem, hope it gets some optimization love further down the line.

I updated the morphological guide with one extra addition, pseudomedian:

##################
## MORPHOLOGICAL #
##################
##
## Opening: To clean small blotches
##   ex_inpand(n).ex_expand(n)
##
## Closing: To close gaps
##   ex_expand(n).ex_inpand(n)
##
## Smooth: Opening & Closing, effectively cleaning binary images
##   ex_inpand(n).ex_expand(n+n).ex_inpand(n)
##
## Outline: To reveal the outer edge of a shape
##   ex_makediff(ex_expand(n), a, dif=false)
##   or
##   ex_hitormiss(mode="outline") # faster
##
## Inline: To reveal the inner edge of a shape
##   ex_makediff(a, ex_inpand(n), dif=false)
##   or
##   ex_hitormiss(mode="inline") # faster
##
## Gradient Magnitude: To reveal the boundary of elements in a binary image
##   ex_makediff(ex_expand(n), ex_inpand(n), dif=false)
##   or
##   ex_edge("min/max")          # faster
##   or
##   ex_hitormiss(mode="boundary")
##
## Top Hat:   (a - opening) To reveal bright elements over dark backgrounds on a greyscale image
##   ex_makediff(a, ex_inpand(n).ex_expand(n), dif=false)
##
## Black Hat: (closing - a) To reveal dark elements over bright backgrounds on a greyscale image
##   ex_makediff(ex_expand(n).ex_inpand(n), a, dif=false)
##
## Pseudomedian: (opening + closing)/2  Approximation to a simple median filter with morphological structuring elements
##   Merge(ex_inpand(n).ex_expand(n), ex_expand(n).ex_inpand(n))
##

[kedautinh12]

@Dogway you have plane to port mt_xxpand_multi.avsi to ex_ family???

[Dogway]

Just do ex_xxpand(3), it's faster.

[kedautinh12]

But ex_xxpand don't have mode="ellipse"

[Dogway]

Well, ex_xxpand don't have mode="ring"

'ellipse' is being worked out in ex_shapes() (forgot about it before declaring the function 'final') so you can do something like:

ex_luts(mode="max",pixels=ex_shapes(2,1,mode="disk"))

I won't add anisotropic/anamorphic shapes to ex_xxpand()

[kedautinh12]

If i'll change mt_lutxy, it can be same with ex_lutxy??
https://github.com/realfinder/AVS-St...DB_s.avsi#L236

[Dogway]

It isn't using an anamorphic radius, you can use ex_xxpand(), here the function:

function F3KDB_3_masking(clip c, float "mask_thr", int "mask_radius", bool "fulls") {

    rgb  = isRGB(c)
    bi   = BitsPerComponent(c)

    mthr = Default(mask_thr, 2)
    mrad = Default(mask_radius, 2)
    fs   = Default(fulls, rgb)

    Assert(mrad > 0, "F3KDB_3_masking: mask_radius must be more than 0")

    thr_lo = ex_bs(max(mthr * 0.75, 1) - 0.0001, 8, bi, fulls=fs)
    thr_hi = ex_bs(max(mthr,        1) + 0.0001, 8, bi, fulls=fs)

    c
    mrad > 1 ? ex_lutxy (ex_expand(mrad,mode="circle"), ex_inpand(mrad,mode="circle"), "x y -",clamp_float=true,fulls=fs)  : mt_edge (mode="min/max", thY1=0, thY2=255)
               ex_lut (Format("x {thr_lo} - {thr_hi} {thr_lo} - / 0 1 clip range_max *"), fulls=fs)
               removegrain (22, -1)
    mrad > 1 ? removegrain (12, -1) : last
    mrad > 2 ? removegrain (20, -1) : last }

EDIT: By the way, did you notice? That line is Gradient Magnitude from my morphological guide above, you can optimize that further by using ex_luts(), wait a min.

EDIT: Optimized further (58% gain) (still some issues for ex_luts() when using high radii)

function F3KDB_3_masking(clip c, float "mask_thr", int "mask_radius", bool "fulls") {

    rgb  = isRGB(c)
    bi   = BitsPerComponent(c)

    mthr = Default(mask_thr, 2)
    mrad = Default(mask_radius, 2)
    fs   = Default(fulls, rgb)

    Assert(mrad > 0, "F3KDB_3_masking: mask_radius must be more than 0")

    thr_lo = ex_bs(max(mthr * 0.75, 1) - 0.0001, 8, bi, fulls=fs)
    thr_hi = ex_bs(max(mthr,        1) + 0.0001, 8, bi, fulls=fs)

    c
    mrad > 1 ? ex_luts(mode="range",pixels=ex_shape(mrad,shape="circle"),clamp_float=true,fulls=fs)  : \
               mt_edge (mode="min/max", thY1=0, thY2=255)
               ex_lut (Format("x {thr_lo} - {thr_hi} {thr_lo} - / 0 1 clip range_max *"), fulls=fs)
               removegrain (22, -1)
    mrad > 1 ? removegrain (12, -1) : last
    mrad > 2 ? removegrain (20, -1) : last }

Originally Posted by Dogway

@Pauly Dunne: It works for me so I think the scripts are fine. You have got suggestions from others to update to latest avisynth+ and scripts. About ex and mix mods, it's described in OP. If you want raw speed go to mix mods, ex mods usually just remove the dependency of RGTools and masktools2 at the cost of some speed.

For my scripts the dependencies almost go to the scripts in OP; ExTools, Transforms Pack and Resizers Pack, and externally Zs_Shared and Utils-r41 (will get rid of this dependency soon with TransformsPack). Then you have all the plugins which really are not many, just the typical; RGTools, masktools2, mvtools2, GRunT, and then depending on each script (ie. DFTTest, KNLMeansCL, BM3D, AddgrainC...). That's the purpose of this project, remove plugin dependencies to the max.

I can only vouch for my scripts like SMDegrain or mods like EX/MIX mods. I came back here 3 months ago after being away for a few years so it's normal you see some changes. When I finish everything in a few months you won't see changes anymore. Bug reports aside, ExTools is one version off of being finished, SMDegrain is finished today. I won't work on these anytime soon and will focus on TransformsPack only.

I just updated ExTools and SMDegrain. Try it and if you find an error report back with the error and filter call.

Hi Dogway, thanks for the great explanation, but like I said this is all way over my head...

Not sure what you use all these with to test & use, and I am trying to incorporate more up to date filters to RipBot264, which is a pretty closed app, for the most part.

I have had a fair bit of success, and the first builds of Extools, and earlier builds of SMDegrain worked very well, I think SMDegrain does a lot better job than MDegrain...but the last few builds have just messed the whole thing up

Maybe I just need to back track, and use what use to work, and stick with that.

I will try the newer ExTools & SMDegrain, and comment back

[kedautinh12]

Try avisynth info tool to check your error
https://forum.doom9.org/showthread.php?t=176079

Originally Posted by kedautinh12

Try avisynth info tool to check your error
https://forum.doom9.org/showthread.php?t=176079

Thanks, I will give that a shot....I've been using AVSMeter to show what line the error is, but that doesn't always makes sense.

Yeah, I can't make any sense of the info this tool provides, it only seems to check what is on the C:/ drive, and installed within the Avisynth folder...

Originally Posted by Pauly Dunne

Hi Dogway, thanks for the great explanation, but like I said this is all way over my head...

Not sure what you use all these with to test & use, and I am trying to incorporate more up to date filters to RipBot264, which is a pretty closed app, for the most part.

I have had a fair bit of success, and the first builds of Extools, and earlier builds of SMDegrain worked very well, I think SMDegrain does a lot better job than MDegrain...but the last few builds have just messed the whole thing up

Maybe I just need to back track, and use what use to work, and stick with that.

I will try the newer ExTools & SMDegrain, and comment back

OK, after some interesting testing, this is what I've got:-

This is how the scripts are, in RipBot264 !!!

With this script:-

#Custom
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools52.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain324d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
video=SMDegrain(video,tr=6,thSAD=600,contrasharp=true,refinemotion=true)

Seems to work well, and if I change to SMDegrain 3.2.5d, I get an error @ line 449 in AVSMeter.

Adding LSFmod doesn't make any difference. (meaning no errors)

And this script:-

#Custom
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod29ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools52.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain325d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
video=SMDegrain(video,tr=6,thSAD=600,contrasharp=30,prefilter=2,str=1.2,refinemotion=true)

If I change this back to SMDegrain 3.2.4d, I get an error @ line 446. in AVSMeter.

If I remove LSFmod, I then get an error @ line 449, in 3.2.5d.

And just for good measure:-

#Custom
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod29ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools52.avs")
LoadPlugin("%AVISYNTHPLUGINS%\KNLMeansCL\KNLMeansCL.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain325d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
video=SMDegrain(video,tr=6,thSAD=600,contrasharp=30,prefilter=4,str=1.2,refinemotion=true)

Works OK too, until..(same as above)

So it would be good to know why the first script doesn't like 3.2.5d, and I haven't got a clue how to fix that

[kedautinh12]

Maybe you don't have enough memory, how much gb in your rams??
https://github.com/Dogway/Avisynth-S...2.5d.avsi#L446

Originally Posted by kedautinh12

Maybe you don't have enough memory, how much gb in your rams??
https://github.com/Dogway/Avisynth-S...2.5d.avsi#L446

Wow, I don't understand why that could be an issue.....

I've only got 32Gb, 3600Mhz DDR4.