Dogway's Filters Packs

[guest]

You can try TemporalDegrain2 (postFFT=4), that option have KNLMEANSCL :D

Hi ked,

But why remove it altogether...just add another pre filter (DGDecNV).

I'm having enough issues (see post 783), that I haven't really had time to "fix", and then sort of unannounced there is a significant change to the pre-filters in SMDegrain :(

[Dogway]

Hey Dogway, and the updates just keep on coming...

So I just noticed in the latest iteration of SMDegrain, that KNLMeansCL (pre filter=5) has "disappeared" !!!!

In it's place, has appeared DGDecNV !!!! How does that work.

Can "we" have KNLMeansCL back ???

I haven't done anymore testing BM3D since my post a few days ago (been busy with other things), I will post my script & call's soon, and I would appreciate your help with that.

Have the dependencies for BM3D changed recently ??

Don't worry the updates are almost done, I will be working mainly on Zopti (SMDegrain, QTGMC, FRC, etc) and SceneStats for the following weeks.

Do you have something to prove KNLMeansCL should be back? In my tests it showed poor performance, see the following images (very bad motion estimation).

KNLMeans Spatio-Temporal (bug so edited out)
KNLMeans Spatial (bug so edited out)
DGDenoise
http://i.imgur.com/0Wqxc1Km.png (https://i.imgur.com/0Wqxc1K.png)
DGDenoise has sharp edges so a bit of blurring afterwards is fine, I will add it.

As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug.

[ChaosKing]

Results are a bit strange. Isn't DGDenoise also an implementation of nlmeans algo? (but without any temporal filtering)
I mean they should look more similar...

Is your source filter frame accurate?

[Dogway]

I'm using DGDecNV r239 so I it should be frame accurate.
I'm not sure about DGDenoise algorithm, there are several edge aware denoisers like nlmeans, bilateral and variations like smart blur, surface blur and those from the papers; guided blur, etc. To me DGDenoise looks like a bilateral filter but videoh has to confirm.
Temporal GPU denoisers in AviSynth is a chimera, both KNLMeansCL and BM3D have issues BM3D CPU is fine though.

[ChaosKing]

From DGDecodeNVManual.html
DGDenoise() is a fast stand-alone CUDA filter that implements NLM denoising. The input clip must be in the YV12 or YUV420P16 format. DGSource() delivers YUV420P16 when the source stream has a 10/12/16-bit depth.

I never had such blending artifacts in knlm temporal. But I remember that some graphics card had some issues with this plugin. Do you get the same output in vapoursynth too?

[guest]

Don't worry the updates are almost done, I will be working mainly on Zopti (SMDegrain, QTGMC, FRC, etc) and SceneStats for the following weeks.

Do you have something to prove KNLMeansCL should be back? In my tests it showed poor performance, see the following images.

Very bad motion estimation. If converted to spatial only, not as good flat denoiser as DGDenoise and edge blurring in excess. DGDenoise has sharp edges so a bit of blurring afterwards is fine, I will add it.

As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug.

I have no "good" reason to want KNLMeansCL back, I guess it's just that when you set up certain scripts to use a particular pre-filter, and then it changes without warning, that's the only reason, I guess,
if you think that DG's stuff works better, then so be it.

You also hinted that there are some issues with BM3D (GPU), but with BM3D (CPU) it's ONLY good for AVX2 CPU's, unless the dev of that updates that function.

as for this As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug. not sure how to do that.

All I can say about that is, that it used to work, but since updating last week sometime, the error showed up, but I can't figure out what versions I changed, that brought it back.

I need to back date, and start almost from scratch :(

BTW, where can I get older builds ??? is there an archive somewhere ??

[Dogway]

From DGDecodeNVManual.html

I never had such blending artifacts in knlm temporal. But I remember that some graphics card had some issues with this plugin. Do you get the same output in vapoursynth too?

Oh :scared:! ::facepalm:: shameful bug. Will update later today and reimplement ex_KNLMeansCL (happy TDS)

@TDS: Wait for v3.4.6 tonight or something and report back if it still doesn't work.

[guest]

Oh :scared:! ::facepalm:: shameful bug. Will update later today and reimplement ex_KNLMeansCL (happy TDS)

@TDS: Wait for v3.4.6 tonight or something and report back if it still doesn't work.

Well it looks as though you've left all the KNLMeansCL stuff in SMDegrain, so why not leave that as pre filter 5, and put in the new one as either 6 or 7, well probably better at 7, and leave bm3d as 6.

[kedautinh12]

Well it looks as though you've left all the KNLMeansCL stuff in SMDegrain, so why not leave that as pre filter 5, and put in the new one as either 6 or 7, well probably better at 7, and leave bm3d as 6.

Doggy updated latest ver :D

[guest]

Doggy updated latest ver :D

Thnx ked :)

@Dogway, nice work, I see what you've done there....will "test" as soon as I can, and report back.

Cheers

---------------------------------------------------------------------------

So, reporting back, and have had limited success, it works on a stand alone PC, but using the Distributed Encoding feature of RipBot264, the Client encoding servers will not encode :(, with this error y4m [error]: bad sequence header magic
x264 [error]: could not open input file `-' OR y4m [error]: bad sequence header magic
x265 [error]: could not open input file `-'

I have no idea wtf has gone wrong, 'cause I'm sure it was working a couple of weeks ago before I "blindly" updated everything :(

---------------------------------------------------------------------------

Another update, I have managed to get a "set" of scripts from and old build of RipBot264, which has most of your "goodies", but rather old builds, BUT those errors have gone :)

So now I need to update one @ a time, and see where the issue is / was. I'm kinda thinkin' it might be something to do with BM3D, but KNL did the same thing, suspecting some OpenCL problem, but wtfk !

[tormento]

both KNLMeansCL and BM3D have issues BM3D CPU is fine though.
I dunno.

I have always used both with a bias for the second because of more detail retaining while doing a good job at helping removing noise.

I have never found any artifact in the output video from SMDegrain and I will keep on using BM3D on CUDA, as my CPU is more ancient than simply old. :)

DGDenoise is a nice add, I still have to evaluate detail retaining and differences with KNLMeansCL.

[Dogway]

With BM3D you have to be too careful on many aspects. First don't crop with DGSource(). And second, we lack "blocking_step" arg, so the maximum block-free settings are sigma=4, radius=1, which is almost useless for prefiltering. I think it also has issues with FFMS2 loader. With CUDA=false I didn't see any of these bugs.

DGDenoise() is fine for Anime, as it's only spatial filtering. I haven't run benchmarks though, it would be very interesting to compare DGDenoise() and KNLMeansCL().

[tormento]

With BM3D you have to be too careful on many aspects. Fist don't crop with DGSource().
I always do. How could it interfere? Please provide some examples.
it would be very interesting to compare DGDenoise() and KNLMeansCL().
I haven't done any serious comparison yet but SMDegrain is way faster when using DGDenoise instead of KNLMeansCL. About denoising, using DGDenoise as prefilter, gives bitrates that are between KNLMeansCL (the lowest ones but with detail loss) and BM3D, that AFAIK was better preserving details, mostly moving ones.

[Dogway]

I posted the issue here (https://forum.doom9.org/showthread.php?p=1963046#post1963046). Confirmed by other users.

DGDenoise is purely spatial, you would need to compare with spatial KNLMeansCL as it's spatio-temporal in SMDegrain, and do some matching with strengths. For visual comparison you can use show=true or directly compare the filters outside SMDegrain.

EDIT: I made a benchmark between both.

ex_DGDenoise(str=0.10,UV=3,LFR=0) # P(4) 83
ex_KNLMeansCL(a=2,s=2,d=0,h=6.0,wmode=1,chroma=true,device_type="GPU",LFR=0) # P(4) 80

DGDenoise is very similar to KNLMeansCL when used spatially with wmode=1 which retains a bit more of the lineart. But suprisingly KNLMeansCL retains more low frequency detail than DGDenoise, this might be good or bad as it will also show faint blotches of grain.

[Dogway]

Hi Dogway,
the 'Highlighter' filter is now called 'ex_Highlighter' and is 6 - 8 times faster with almost identical results. So full success.

And therefore (if you feel like it) you may try the opposite function. 'HighlightLimiter' and 'HighlightLimiterMod'
The filter reduces too bright areas. The mod version was changed by me (long time ago).

Check this:

function HighlightLimiter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method") {

bi = BitsPerComponent(v)
fs = propNumElements (v,"_ColorRange") > 0 ? \
propGetInt (v,"_ColorRange") == 0 : false

grad = Default (gradient,true)
gblur = Default (gblur, grad ? 100 : 5)
thres = Default (threshold,160) #The lower the value, the more sensitive the filter will be.
twopass = Default (twopass,false)
amount = Default (amount,20)
method = Default (method, 0) # 0 or 1

amount = ex_bs(abs(amount), 8, bi, true)

darken = v.ex_binarize(thres,hi=-1)
blurred = grad ? darken.ex_gaussianblur(0.073*gblur+17.78,UV=1) : \
darken.ex_gaussianblur(0.064*gblur+ 3.75,UV=1)

multiply = (method==0) ? v.ex_lut("x dup * range_max /", fulls=fs) : v.ex_lut(Format("x {amount} + ")+(!fs?"ymax min":""), fulls=fs)
multiply = !twopass ? multiply : \
(method==0) ? multiply.ex_lutxy(v,"x y * range_max /", fulls=fs) : \
multiply.ex_contrast(0.15, 1, 163, tv_range=!fs)
return mt_merge(v,multiply,blurred) }


function HighlightLimiterMod(clip v, int "threshold", int "dark", float "rad",int "mode", float "gamma1_2") {

w = Width (v)
h = Height(v)
bi = BitsPerComponent(v)
fs = propNumElements (v,"_ColorRange") > 0 ? \
propGetInt (v,"_ColorRange") == 0 : false

thres = Default (threshold,160)
dark = Default (dark, 255)
rad = Default (rad, 64)
mode = Default (mode, 0) # 0,1,2
gamma1_2= Default (gamma1_2, 0.6) # for mode 1,2

m = min(w,h)
dark = ex_bs(dark, 8, bi, true)
rad = round(m/rad/4) < 1 ? max(round(m/4.0),16) : rad

darken = v.ex_binarize(thres,hi=-1)
blurred = darken.ex_GaussianBlur(rad*0.83825-0.0631,pad=true,UV=1)

multiply = (mode==0) ? v.ex_lut(Format("x dup * {dark} /"), fulls=fs) : \
(mode==1) ? v.ex_levels(gamma=gamma1_2,tv_range=!fs) : \
v.ex_lut(Format("x dup * {dark} /"), fulls=fs).ex_levels(gamma=gamma1_2,tv_range=!fs)

return mt_merge(v,multiply,blurred) }

[guest]

Hey Dogway,

I've had a major win (several win's, actually).

So I started from an old "set" of your scripts, and worked thru until they worked.

I even got the DGDecNV pre-filter to work :)

So this is my script for KNLMeansCL, is there any "fine tuning" that could be done to it, as I noticed you changed the instructions a bit in SMDegrain 3.4.6d

LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\MedianBlur2\MedianBlur2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\KNLMeansCL\KNLMeansCL.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools74.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod50ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Resizers Pack78.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Sharpeners Pack34.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain346d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
pre=ex_KNLMeansCL(video,a=2,s=4,d=1,h=4,wmode=0)
video=SMDegrain(video,tr=4,thSAD=400,thSADC=200,thSCD2=80,contrasharp=true,prefilter=pre,refinemotion=true)

And this is my DGDecNV script, again, any "fine tuning" for this :-

LoadPlugin("%AVISYNTHPLUGINS%\dgdecnv\DGDecodeNV.dll")
LoadPlugin("%AVISYNTHPLUGINS%\dfttest\dfttest.dll")
LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\MedianBlur2\MedianBlur2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools74.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod50ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Sharpeners Pack34.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain346d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Transform Packv1.0 RC36.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
pre=ex_DGDenoise(video,str=0.10)
video=SMDegrain(video,tr=2,thSAD=400,thSADC=100,thSCD2=80,contrasharp=true,prefilter=pre,refinemotion=true)

HOWEVER, can you please change SMDegrain back to it's BM3D default (earlier builds) being CUDA, you've done something and now it's BM3D CPU by default, and I need CUDA for some of my older servers that don't support BM3D CPU builds.

It was fairly easy to change the CUDA settings from "True" to "False", but with this latest build, that doesn't seem to apply. :(

Thanks

[kedautinh12]

Hey Dogway, i added some ex_ family into f3kgrain. Can you check it's complete replace ex_ into f3kgrain or not?? Thanks
https://github.com/kedaitinh12/AVSPlus-Scripts/blob/master/f3kgrain_v0.4MOD.avsi

[Dogway]

Hey Dogway, i added some ex_ family into f3kgrain. Can you check it's complete replace ex_ into f3kgrain or not?? Thanks
https://github.com/kedaitinh12/AVSPlus-Scripts/blob/master/f3kgrain_v0.4MOD.avsi

I reply you there.

@TDS: You should be able to use CUDA by setting it to true here (https://github.com/Dogway/Avisynth-Scripts/blob/9b84fe4957f5f748d85309b6ef664d634d4ca6f9/SMDegrain%20v3.4.6d/SMDegrain%20v3.4.6d.avsi#L248).

For the KNLMeansCL prefilter, test with wmode=1, depending on content you might like it more. By default with 'd' > 0, LFR is used, you can try disabling it with LFR=0, but it will smear quite a bit. ex_DGDenoise() seems also to smear quite a bit, so for next version I will also add LFR argument.
CUDA in BM3D is broken really, you can try comparing them CUDA vs CPU outside SMDegrain, I posted this in the BM3D thread.

[ChaosKing]

And second, we lack "blocking_step" arg, so the maximum block-free settings are sigma=4, radius=1, which is almost useless for prefiltering.

Not true, there is a blocking_step paramater in the cuda version. I just set mine to 2 so I don't have to worry about evil blocking :devil:

[Dogway]

Oh shii... the AVS filter is so undocumented it looks like shady business haha

block_step, that is: https://github.com/WolframRhodium/VapourSynth-BM3DCUDA/blob/94e1380cab621b9a9226824e7ba6876b8b502c78/source/kernel.cu#L49

[anton_foy]

Oh shii... the AVS filter is so undocumented it looks like shady business haha

block_step, that is: https://github.com/WolframRhodium/VapourSynth-BM3DCUDA/blob/94e1380cab621b9a9226824e7ba6876b8b502c78/source/kernel.cu#L49

Tried out BM3DCUDA again two days ago and also lowered block_step to 2 which rid the blocking.
I usually go here (http://avisynth.nl/index.php/BM3DCUDA)

[ChaosKing]

Undocumented? It's in the paramaters list here https://github.com/WolframRhodium/VapourSynth-BM3DCUDA#parameters
It is mentioned 3 times in the read me ;) (ok, note2 says blocking_step instead of block_step)

[Dogway]

There's also bm_range and ps_range so I made a preset matrix to conform to original profiles. For preset "fast" and "normal" I will lower the array by 1, I had to make an ArrayOp() function, just edited the old MatrixMul() in TransformsPack so now it's multipurpose.

[gispos]

Check this:

function HighlightLimiter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method") {

bi = BitsPerComponent(v)
fs = propNumElements (v,"_ColorRange") > 0 ? \
propGetInt (v,"_ColorRange") == 0 : false

grad = Default (gradient,true)
gblur = Default (gblur, grad ? 100 : 5)
thres = Default (threshold,160) #The lower the value, the more sensitive the filter will be.
twopass = Default (twopass,false)
amount = Default (amount,20)
method = Default (method, 0) # 0 or 1

amount = ex_bs(abs(amount), 8, bi, true)

darken = v.ex_binarize(thres,hi=-1)
blurred = grad ? darken.ex_gaussianblur(0.073*gblur+17.78,UV=1) : \
darken.ex_gaussianblur(0.064*gblur+ 3.75,UV=1)

multiply = (method==0) ? v.ex_lut("x dup * range_max /", fulls=fs) : v.ex_lut(Format("x {amount} + ")+(!fs?"ymax min":""), fulls=fs)
multiply = !twopass ? multiply : \
(method==0) ? multiply.ex_lutxy(v,"x y * range_max /", fulls=fs) : \
multiply.ex_contrast(0.15, 1, 163, tv_range=!fs)
return mt_merge(v,multiply,blurred) }


function HighlightLimiterMod(clip v, int "threshold", int "dark", float "rad",int "mode", float "gamma1_2") {

w = Width (v)
h = Height(v)
bi = BitsPerComponent(v)
fs = propNumElements (v,"_ColorRange") > 0 ? \
propGetInt (v,"_ColorRange") == 0 : false

thres = Default (threshold,160)
dark = Default (dark, 255)
rad = Default (rad, 64)
mode = Default (mode, 0) # 0,1,2
gamma1_2= Default (gamma1_2, 0.6) # for mode 1,2

m = min(w,h)
dark = ex_bs(dark, 8, bi, true)
rad = round(m/rad/4) < 1 ? max(round(m/4.0),16) : rad

darken = v.ex_binarize(thres,hi=-1)
blurred = darken.ex_GaussianBlur(rad*0.83825-0.0631,pad=true,UV=1)

multiply = (mode==0) ? v.ex_lut(Format("x dup * {dark} /"), fulls=fs) : \
(mode==1) ? v.ex_levels(gamma=gamma1_2) : \
v.ex_lut(Format("x dup * {dark} /"), fulls=fs).ex_levels(gamma=gamma1_2)

return mt_merge(v,multiply,blurred) }
Thanks Dogway. Results:
For HighlightLimiter, your version 'ex_HighlightLimiter' wins. Almost the same result but 10 times faster.

I can't decide about HighlightLimiterMod.
Your version is 8% faster but there are small differences. Sometimes darker areas are also slightly darkened.
Which parameters could I play around with?

On these?
blurred = darken.ex_GaussianBlur(rad*0.83825-0.0631,pad=true,UV=1)

Edit: Everything Ok. I just have to make the radius a little smaller to get almost the same result.

[Dogway]

8% or 8 times :rolleyes: ?

Yes, mapping bicubic blur to gaussian blur is not exact but the later gives better results gaussian like, I couldn't decide if this or (rad-1) so tweak visually.


EDIT: By the way, I made a bandpass replacement filter to replace mid frequencies from one clip to another, please have a look if you find something odd. I think gaussian is not ideal here (might need mean blur) but anyway here it is:


# Bandpass replacement
# Replace Mid Frequencies of first clip from the second clip
#
# Bandpass workflow diagram (shaded '/' means deleted by lowpassing):
#
# A A A A A
# _______ _______ _______ _______ _______
# | | |/ / / /| | | |/ / / /| | |
# | | | / / / | | | | / / / | | |
# | | |-------|-> HI ------- |/ / / /| -------
# | | | | | / / / |
# | | - | | = + |/ / / /| =
# | | | | |-------|-> LO _______
# | | | | | | | |
# | | | | | | | |
# ------- ------- ------- -------
# A
# B B B / A+B
# _______ _______ / _______
# |/ / / /| |/ / / /| /\ + | |
# | / / / | | / / / | / \ | A |
# |-------|-> HI |/ / / /| _______ / \ |-------|
# | | | / / / | | | B -----> | |
# | | - |/ / / /| = | | | B |
# | | |-------|-> LO ------- |-------|
# | | | | | |
# | | | | | A |
# ------- ------- -------
#
# Example:
# ex_MFR(a,b,1700,700) # Replace mid frequencies from 700 to 1700 from 'b' to 'a'
#
#
function ex_MFR (clip c, clip ref, float "hi", float "lo", int "UV") {

rgb = isRGB (c)
is42 = is420 (c)
w = width (c)
h = height(c)
mx = max(w,h)
bi = BitsPerComponent(c)
fulls= propNumElements (c,"_ColorRange") > 0 ? \
propGetInt (c,"_ColorRange") == 0 : rgb

HI = Default( HI, 1200*(w/1920.)) # Default is 1200 for 1080p
LO = Default( LO, HI/2. ) # Range of Mid Frequency to restore
UV = Default( UV, 3) # Set UV to 4 to copy the chroma from the 'ref' clip
UV2 = UV==4?2:UV

LO = max(LO, 4)

Fs = max(w,h) * 2 # Frequency sample rate is resolution * 2 (for Nyquist)
k = sqrt(log(2)/2) * HI # Constant for -3dB
HI = Fs / ( k * 2 * pi ) # Frequency Cutoff for Gaussian Sigma

k = sqrt(log(2)/2) * LO
LO = Fs / ( k * 2 * pi )

LOUV = c.is422() || is42 ? LO/2. : c.isYV411() ? LO/4. : LO
LOUVv= is42 ? LO/2. : LO
HIUV = c.is422() || is42 ? HI/2. : c.isYV411() ? HI/4. : HI
HIUVv= is42 ? HI/2. : HI

mpc = c.vsTCanny(HI, HIUV, HIUVv, mode=-1, u=UV2, v=UV2)
mpc2 = c.vsTCanny(LO, LOUV, LOUVv, mode=-1, u=UV2, v=UV2)
mpr = ref.vsTCanny(HI, HIUV, HIUVv, mode=-1, u=UV2, v=UV2)
mpr2 = ref.vsTCanny(LO, LOUV, LOUVv, mode=-1, u=UV2, v=UV2)

Expr(c, mpc, mpc2, mpr, mpr2, "x y - z + a b - +", isy(c) ? Undefined() : ex_UVexpr("x y - z + a b - +", UV2, bi, rgb, fulls, "none") ) }

[gispos]

8% or 8 times :rolleyes: ?
Not a typo, 8 to 10% is already correct.

The normal version used:
blurred=darken.gaussianblur(gblur).gaussianblur(gblur+100).gaussianblur(gblur+200)

and that is slower than my Mod version:
blurred=darken.bicubicresize(round(clp.width()/rad/4)*4, round(clp.height()/rad/4)*4).bicubicresize(clp.width(),clp.height(),1,0)

therefore the difference to your variant is not so big:
blurred = darken.ex_GaussianBlur(rad*0.83825-0.0631,pad=true,UV=1)


EDIT: By the way, I made a bandpass replacement filter to replace mid frequencies from one clip to another, please have a look if you find something odd.
Um, I do not know what to do with it? There are no reference clips for my clips. Surely this must be the same content?

Edit:
I think the topic of lightening dark areas or reducing light areas interests many users here. One could prepare with it here many a joy.:)

Edit2:
The normal version you changed 'ex_HighlightLimiter' is the best, same results but 8 to 10 times faster. Thanks again.

[Dogway]

Yes I noticed, you were already using bicubic for blurring which is inherently fast, you can also try vsTCanny for blurring, for mid sigmas sometimes it's faster than ex_gaussianblur().
I edited the scripts because I missed 'tv_range' in ex_levels. Well 10% is already very good in my book. These filters look to me like mini Retinex filters, the concept works fine, I will have to compare to ex_retinex() for speed and quality benchmarks.

No, ex_MFR() is totally unrelated, I was testing on improving UVRecon in TransformsPack finally used ex_LFR though.

Also for anyone interested, I'm making a window kernel list in Desmos (https://www.desmos.com/calculator/qs2j9oo7ku) for study (edited from here (https://www.reddit.com/r/DSP/comments/ojlae6/i_made_a_desmos_graph_that_demonstrates_various/)) and include in TransformsPack, still investigating on how to derive 'taps'.

[gispos]

Dogway, one more question:
You use the properties for TV_Range, I don't really like that. At least I still use filters that don't pass properties.
I have added tv_range as parameter and and also changed the 'fs' assignments in the code. Do you see any disadvantages?

function ex_Highlighter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method", bool "tv_range") {

bi = BitsPerComponent(v)
fs = default(tv_range, True)

grad = Default (gradient,true)
gblur = Default (gblur, grad ? 100 : 5)
thres = Default (threshold,100) #The lower the value, the more sensitive the filter will be.
twopass = Default (twopass,false)
amount = Default (amount,10)
method = Default (method, 1) # 0, 1 or 2

amount = ex_bs(abs(amount), 8, bi, true)
thres = ex_bs(thres, 8, bi, true)

lighten = v.ex_lut(Format("x {thres} > 0 x ?"), UV=128)
blurred = grad ? lighten.ex_gaussianblur(0.073*gblur+17.78,UV=1) : \
lighten.ex_gaussianblur(0.064*gblur+ 3.75,UV=1)

photoshop_overlay = (method==1) ? ex_lutxy(v,blurred,"x range_half > range_max y - Y@ range_max x - X@ range_half / * range_max + Y x range_half / * ? ", fulls=!fs) : \
(method==2) ? ex_lutxy(v,blurred,"range_max range_max x - y * range_max / -",fulls=!fs) : v.ex_lut(Format("x {amount} + ")+(fs?"ymax min":""), fulls=!fs)

photoshop_overlay = twopass ? photoshop_overlay.ex_contrast(-0.8,1,167,tv_range=fs) : photoshop_overlay
return mt_merge(v,photoshop_overlay,blurred)
}

[Dogway]

'fulls' should have the opposite sign of 'tv_range' so I think it would make better sense to write "fs = !Default(tv_range, True)", then change according in following calls (whether the filter expects 'fulls' or 'tv_range').
But what I normally do is "tv = Default(tv_range, True)" and use 'tv' or '!tv' upon requirements.

Yes, filters that don't pass frameprops is a problem, maybe we should make a list of them. I have been thinking for some time as a header filter to define initial frameprops aside those assigned by the loader, so you can add _ColorRange, or override other frameprops. I have also been thinking on a color space detection but I will leave that for the future.

[gispos]

Hello Dogway,
I just downloaded your latest version.

Very nice that you have already added the AvsPmod function definetion to many functions.
Also the histogram curves are very nice (especially with 'Preview Filter :)).

A wish from me to ex_contrast:
Is it possible to add another pivot slider? Or something like a 'soft limiter'.
So that you can delimit dark, middle and light areas more.

What I noticed is that the contrast value in the middle range does not allow any 0.01 changes, only from 0.1 can visible changes of 0.01 be made.
This is probably due to mathematical facts and cannot be improved?

I just don't like setting parameters by reading the frame properties. This should not be done in hidden functions
Why not set 'tv_range' manually? But this is only my opinion.

[Dogway]

Yes, with the preview histograms and your preview filters this is starting to look like a decent alternative to more modern packages. Just tested Preview Filters and it's awesome, one problem though, I have to add 'last' to my script after '**/' or AvsPmod fails to render output (in case my preview filter is the last one in the script).

I didn't understand your question. You can play with 'pivot' to have dark or light areas affect more, for middle you have gamma in ex_levels (not exactly unbiased but mostly).

ex_contrast function definition is "by 0.1", does it work if you change it to "0.01"? I see changes in the histogram curve going from 0.0 to 0.01 and all the way up to 0.1.

I had a few wishes for function definition, one is max stepping for int types is '4', I don't know why. The other is string types doesn't understand anything that is not alphanumeric, so "string preset="[fast]" ("[slow]" / "[fast]") " fails.

As for frameprops, in my functions they are only read as a fallback (Defaults) if you explicitly don't define the argument, so no problem with that.

[gispos]

I didn't understand your question. You can play with 'pivot' to have dark or light areas affect more, for middle you have gamma in ex_levels (not exactly unbiased but mostly).

ex_contrast function definition is "by 0.1", does it work if you change it to "0.01"? I see changes in the histogram curve going from 0.0 to 0.01 and all the way up to 0.1.

There is a difference whether I can adjust the curve with only one or two controllers and thus have, for example, a control value that can be adjusted from the dark to only the middle areas. SmoothLevels, for example, has extra controls for dark and light areas and the mode parameter can be used to adjust the behavior.

It depends on how you specify the slider values (I do not use your examples). Values from 0.01 are set visible, but only after the value has at least 0.1
From 0.00 to 0.01 there is a jump as if 0.1 was set.


I had a few wishes for function definition, one is max stepping for int types is '4', I don't know why.
I don't know that either. But it should depend on the maximum value of the slider. Must see if I can find that.


The other is string types doesn't understand anything that is not alphanumeric, so "string preset="[fast]" ("[slow]" / "[fast]") " fails.

I think it's because there are valid rules for self-created sliders. And [ ] is for example responsible as a character for slider start and end.
At that time there were e.g. also no arrays in avisynth. The whole mesh around the slider is far-reaching and requires time, effort and desire to tackle.
Edit: In the function definitions [ ] are a flag for hidden parameters. Have a look at the filterdb.dat. The whole thing must also be readable and implementable, so there are certain rules.


As for frameprops, in my functions they are only read as a fallback (Defaults) if you explicitly don't define the argument, so no problem with that.
Ok.

AvsPmod fails to render output (in case my preview filter is the last one in the script).
Sometimes I also have to use a return last in the script. These are the requirements avisynth has for its syntax.
I can't check with every parameter change if a return last is present and also at a valid position. You will manage that!

[gispos]

I had a few wishes for function definition, one is max stepping for int types is '4', I don't know why.

It works for me. max and min and default values must of course be divisible by e.g. 5 or must be 0

There is even a warning if you do not meet these requirements.
I just defined a slider with the values: min 0 default 100 max 200 step by 50

[ENunn]

Does anyone know why Avisynth has been more unstable since I installed the scripts? I'm trying to use QTGMC and the moment I try to load the preview Avspmod insta-crashes. Ffmpeg does nothing and crashes as well. Before I kept getting access violation errors with ZS_RF_Shared.avsi, either that or it would load fine when it feels like it. Now it's not loading at all. I have the latest version of the script and the dependencies so I'm not sure what's up.

[gispos]

I try to load the preview Avspmod insta-crashes.
What version of Avisynth are you using?

[ENunn]

What version of Avisynth are you using?

3.7.1.

I cleared out my plugins folder and installed them all back in, looks like neo_fft3d.dll might be the culprit for the crashing, which kinda sucks because I use tempdegrain2 a lot. But now I'm getting random access violations with ExTools.avsi.

[guest]

So my BM3D woe's continue.

I have several older PC's that aren't AVX2, BUT they have reasonable nVidia GPU's for CUDA / OpenCL support.

Naturally BM3D CPU doesn't work, BUT I just can't get BM3D CUDA to work on them. I really thought I got it to work before...but can't get it to, anymore :(

KNLMeansCL & DGDenoise work fine.

So do I just forget about using BM3D pre-filters ???, what's the next best thing ??

I'm tearing my hair out (and I haven't got much of that anyway) !!

Help...

[Dogway]

@TDS: I think your issue would probably be one of resources distribution through the server. Maybe CUDA instructions are not passed, you can try first outside avisynth to check if it's a server or an AVS problem, for example with a GPU benchmark tool. As a fallback KNLMeansCL is a nice alternative as it's also temporal.

@ENunn: Try updating to 3.7.2 test3, some critical bugs were fixed there, just in case to rule out other possibilities.

[kedautinh12]

@TDS: you can try change radius=1 to preset="low" here
https://github.com/Dogway/Avisynth-Scripts/blob/b4b430d9f0f35e5ebaf7546d769f88a5b9fddeed/SMDegrain%20v3.4.7d/SMDegrain%20v3.4.7d.avsi#L248

[guest]

@TDS: you can try change radius=1 to preset="low" here
https://github.com/Dogway/Avisynth-Scripts/blob/b4b430d9f0f35e5ebaf7546d769f88a5b9fddeed/SMDegrain%20v3.4.7d/SMDegrain%20v3.4.7d.avsi#L248

Hi,

Is this what you mean :-

(prefilter== 6) ? pref.ex_BM3D(sigma=10,preset="low",CUDA=true,UV=Chr,tv_range=tv)

[kedautinh12]

Yes, if you want faster, try preset="fast"

[guest]

Yes, if you want faster, try preset="fast"

Thnx, initially I thought it had fixed it, but it hasn't :(

I still keep getting this error, on the older PC's:-

x265 [error]: could not open input file `-'

[guest]

@TDS: I think your issue would probably be one of resources distribution through the server. Maybe CUDA instructions are not passed, you can try first outside avisynth to check if it's a server or an AVS problem, for example with a GPU benchmark tool. As a fallback KNLMeansCL is a nice alternative as it's also temporal.

@ENunn: Try updating to 3.7.2 test3, some critical bugs were fixed there, just in case to rule out other possibilities.

Curious that you're suggesting KNLMeansCL, and just a short time ago, you removed it....hmmmm. :scared:

[kedautinh12]

Thnx, initially I thought it had fixed it, but it hasn't :(

I still keep getting this error, on the older PC's:-

x265 [error]: could not open input file `-'

The problem from x265 not SMDegrain

[WolframRhodium]

So my BM3D woe's continue.

I have several older PC's that aren't AVX2, BUT they have reasonable nVidia GPU's for CUDA / OpenCL support.

Naturally BM3D CPU doesn't work, BUT I just can't get BM3D CUDA to work on them. I really thought I got it to work before...but can't get it to, anymore :(

KNLMeansCL & DGDenoise work fine.

So do I just forget about using BM3D pre-filters ???, what's the next best thing ??

I'm tearing my hair out (and I haven't got much of that anyway) !!

Help...

Is there any useful error message? What GPU and driver are you using?

[guest]

The problem from x265 not SMDegrain

Maybe so, but how come it works on some PC's, and not others ???

That's the biggest problem, it's random, or should I say, particular. :(

And I also get this when using x264 :-

y4m [error]: bad sequence header magic
x264 [error]: could not open input file `-'

[guest]

Is there any useful error message? What GPU and driver are you using?

Hello,

The only errors I see displayed are in the previous post(s), (842 & 846)...that's the ONLY indication I get with the app I'm using, which no one here uses or know's much about.

I have several PC's with nVidia GPU's ranging from a GTX 660, 760, 770, 780ti, and a 980ti hybrid, I used to have an MSI 1080 Gamer X, but it developed errors, and is basically un usable.

They are all (except the 980ti) using the latest available driver, 472.12.

Maybe I need to install the CUDA development tools or something.

Now several of these PC's have older Xeon's that don't support AVX2, and that's why I need CUDA to work, other wise I will have to use other non Cuda filters...if only you could take the time to mod BM3D avs, to do that :)

Anyway, that's about all I can tell you.

[Boulder]

You have to open the script for example in VirtualDub, which will tell you the error message.

[guest]

You have to open the script for example in VirtualDub, which will tell you the error message.

Yes, I tried that when you suggested it a few days ago..

Is virtualdub2 the same ??

Anyway, if I try to open any .avs, it fails.

I think I might have figured out how to open avs, it has to be a video file..

Anyway, the scripts I tested were ALL faulty, so that's not right.

[Boulder]

Yes, I tried that when you suggested it a few days ago..

Is virtualdub2 the same ??

Anyway, if I try to open any .avs, it fails.

VDub2 is the recommended one. Make sure you use the correct version (32-bit or 64-bit) depending on your system and Avisynth setup. If it fails, it should give you a meaningful error message as well. We cannot see what you see unless you tell us.