Dogway's Filters Packs

[guest]

I found BM3D to be a bit soft on edges, I think because it does some spatial filtering (3D). For video I normally avoid denoising directly in spatial domain unless it's something very specific or masked. For guiding I think it's good, in my brief comparisons motion estimation was better than KNLMeansCL, but it usually hanged when trying higher radius.

@TDS: Post the error message so I can have a look, otherwise can't help. If you want a cleanup tip, just keep core filters and move out old ones or used sparsely, then try to update them to latest version, including AviSynth.

That's the plan....and I will post errors msg's too... it's just they vary from script to script.

Thanks

[guest]

TDS, you need this [in the 1st post, TextArea Cache ].:- https://forum.doom9.org/showthread.php?t=177066

Strange thing was, I had posted several replies to another user, but the one to Dogway failed :(

[guest]

@TDS: Post the error message so I can have a look, otherwise can't help. If you want a cleanup tip, just keep core filters and move out old ones or used sparsely, then try to update them to latest version, including AviSynth.

Hey Dogway, can you please reply to this post :-

https://forum.doom9.org/showpost.php?p=1954281&postcount=491

It was delayed a day or so for some reason, and it's now been allowed :confused:

And another thing, now that MysteryX has mentioned multiple pre-filters, could that be done with your scripts ??

eg:- video=SMDegrain(video,tr=6,thSAD=600,contrasharp=30,str=1.2,refinemotion=true,prefilter=4,prefilter=5)

[kedautinh12]

Oh, hi oldman, TDS is your new account? :D

[kedautinh12]

SMDegrain only support single prefilter. You need wait MysteryX port xclean to avs but you sure will complain about speed again :D

[Dogway]

OK, now I see. Yes there's no 's' argument in ex_KNLMeansCL() I added it now (also wmode) and defaulted to be the same as 'a', and it gains some sharpness back. Anyway this doesn't trigger any error whatsoever...

You can use any prefilter on your own, the 'prefilter' argument is just a commodity, it's not an end-it-all solution, for advanced prefiltering you define it yourself like:
pre=fluxsmootht(3).removegrain(11)
SMDegrain(tr=6,thSAD=500,contrasharp=40,prefilter=pre,str=1.2,refinemotion=true)

The documentation is old but it's still useful for most of the settings, so have a look at it.
Also observe the presence of 'Str' the prefilter clip is still passed the normalization stage with ex_luma_rebuild() in this case.

[guest]

Oh, hi oldman, TDS is your new account? :D

Hmmm, close, we both use RipBot264, and "we" have been helping each other trying to improve RipBot as the dev seems to have lost his interest of late.

When he mentioned that he was going to take a break, I created an account to carry on.

I'm sure he will appear every so often.

You helped him a lot, so hopefully you can continue, for me ?

But it can be a bit tedious (TDS) ;)

[kedautinh12]

Hmmm, close, we both use RipBot264, and "we" have been helping each other trying to improve RipBot as the dev seems to have lost his interest of late.

When he mentioned that he was going to take a break, I created an account to carry on.

I'm sure he will appear every so often.

You helped him a lot, so hopefully you can continue, for me ?

But it can be a bit tedious (TDS) ;)

Ok, i will help you in my possibility :D

[guest]

OK, now I see. Yes there's no 's' argument in ex_KNLMeansCL() I added it now (also wmode) and defaulted to be the same as 'a', and it gains some sharpness back. Anyway this doesn't trigger any error whatsoever...

You can use any prefilter on your own, the 'prefilter' argument is just a commodity, it's not an end-it-all solution, for advanced prefiltering you define it yourself like:
pre=fluxsmootht(3).removegrain(11)
SMDegrain(tr=6,thSAD=500,contrasharp=40,prefilter=pre,str=1.2,refinemotion=true)

The documentation is old but it's still useful for most of the settings, so have a look at it.
Also observe the presence of 'Str' the prefilter clip is still passed the normalization stage with ex_luma_rebuild() in this case.

Thanks, but I really don't understand most of this stuff. :stupid:


HOWEVER:-

#change contrasharp=true to contrasharp=(int) it errors on line 538 of SMDegrain333d

#adding lsb=true or false, wont even start Avsmeter

These didn't work for me.

#pre=fluxsmootht(3).removegrain(11)
#SMDegrain(tr=6,thSAD=500,contrasharp=30,prefilter=pre,str=1.2,refinemotion=true,lsb=true)

So, so far, so good, on the next script :)

[Dogway]

lsb settings are deprecated. They are not in SMDegrain anymore, the error message should tell you.

Also:
#change contrasharp=true to contrasharp=(int) it errors on line 538 of SMDegrain333d

Is not an error message it's the telephone game. If I had to guess you explicitly wrote "(int)" into the argument?

By the way I'm going to update SMDegrain with a fair amount of changes, nothing critical all bugfixes and improvements.
mfilter is also fixed for defined CClip inputs. Recursion now works again for medianT when tr<=2, and added a few Asserts for it. The 's' argument is added to ex_KNLMeansCL(), and passed to the prefilter which has been tuned for a bit more sharpness. ex_retinex() blacks now have been clipped instead of scaled, this respects the original intent better, also it's faster. New setting which I wanted to add since 7 years ago, 'LFR', it will restore low frequency details, specially useful for TemporalSoften, check the images below.

smdegrain(tr=2,mode="temporalsoften",blksize=32,thSAD=900,LFR=5,contrasharp=false,refinemotion=true)
smdegrain(tr=2,mode="temporalsoften",blksize=32,thSAD=300,LFR=false,mfilter=ex_boxblur(5,mode="weighted",UV=3),contrasharp=true,refinemotion=true)

source
http://i.imgur.com/DGUVkQWt.png (https://i.imgur.com/DGUVkQW.png)
before
http://i.imgur.com/fC2YWxTt.png (https://i.imgur.com/fC2YWxT.png)
after low frequency restore
http://i.imgur.com/nirP1ygt.png (https://i.imgur.com/nirP1yg.png)

[guest]

lsb settings are deprecated. They are not in SMDegrain anymore, the error message should tell you

Is not an error message it's the telephone game. If I had to guess you explicitly wrote "(int)" into the argument?

By the way I'm going to update SMDegrain with a fair amount of changes, nothing critical all bugfixes and improvements.


So sorry 'bout that, I was actually going to write "a number" but thought int would describe that.
Is not an error message it's the telephone game. If I had to guess you explicitly wrote "(int)" into the argument?

#change contrasharp=true to contrasharp=(a number) it errors on line 538 of SMDegrain333d

smdegrain(tr=2,mode="temporalsoften",blksize=32,thSAD=900,LFR=5,contrasharp=false,refinemotion=true)
smdegrain(tr=2,mode="temporalsoften",blksize=32,thSAD=300,LFR=false,mfilter=ex_boxblur(5,mode="weighted",UV=3),contrasharp=true,refinemotion=true)

I very much doubt that I will be able to use 2 SMDegrain lines like that.

[Dogway]

Why don't you post the error message or the SMDegrain call? I can't interpret what you interpret of those. Post concise information so I can give you precise answers.
SMDegrain v3.3.4d is released now. Currently LFR requires ExTools v6.1 which is unreleased because I want to solve the optimized median of 48 inputs. I might release it without it if I get stuck.

[guest]

Why don't you post the error message or the SMDegrain call? I can't interpret what you interpret of those. Post concise information so I can give you precise answers.
SMDegrain v3.3.4d is released now. Currently LFR requires ExTools v6.1 which is unreleased because I want to solve the optimized median of 48 inputs. I might release it without it if I get stuck.

OK, I thought I was explaining it as best I could, but in the future I will post the "call", then try to explain the error....not sure if I can copy the error msg.

Will get latest SMDegrain, and see how it goes.

Currently I have everything working sweet, after the "clean up" so that was worth it.

One other question:-

Can the values in red, be easily changed, or does it need to be done somewhere else ??

(prefilter== 4) ? inputP.ex_KNLMeansCL(a=2,s=2,d=1,h=7.0,wmode=0,chroma=chroma,device_type="GPU",device_id=device_id) : \
(prefilter== 5) ? inputP.ex_BM3D(sigma=5,radius=1,CUDA=true,UV=Chr,fulls=fs)

[Dogway]

You can do like this:
pre=ex_KNLMeansCL(a=2,s=2,d=1,h=7.0,wmode=0)
SMDegrain(tr=3,thSAD=300,contrasharp=30,prefilter=pre,refinemotion=true)

[guest]

You can do like this:
pre=ex_KNLMeansCL(a=2,s=2,d=1,h=7.0,wmode=0)
SMDegrain(tr=3,thSAD=300,contrasharp=30,prefilter=pre,refinemotion=true)

I'm pretty sure I cannot use a script like that...see here for an example of what my scripts need to look like (ignore the build numbers, etc)

https://forum.doom9.org/showpost.php?p=1953024&postcount=398

[Dogway]

Why not? elaborate please.

[guest]

Why not? elaborate please.

Several reasons...

The way that scripts are written for/in RipBot264.

My lack of experience & understanding of how to write / adapt scripts.

I have tried a couple of multi line scripts like that, and it just doesn't work :(

[Dogway]

Write in the RipBot264 thread on how you can adapt the above call to be read by RipBot264... I don't use any interface personally.

[Boulder]

That LFR might be useful as a generic function - do you have plans to put it in ExTools? There are often cases where a high thsad is a must but it smooths the flat, low-frequency surfaces a bit too much.

[guest]

Write in the RipBot264 thread on how you can adapt the above call to be read by RipBot264... I don't use any interface personally.

I will give that a try, but I don't expect much.

That is why I'm here.

So how do you test your scripts ?

[StainlessS]

So how do you test your scripts ?
You could try VirtualDub2, AvsPMod, PotPlayer, MPC-HC, MPC-BE, or other players [not VLC].
Timing, AvsMeter.

[Dogway]

Ah yes sorry, I use avspmod, but for encoding I simply use a custom bat file.
That LFR might be useful as a generic function - do you have plans to put it in ExTools? There are often cases where a high thsad is a must but it smooths the flat, low-frequency surfaces a bit too much.
I don't see it to fit much... if any maybe ex_repair() but it's just a lowpass comparison, for MDegrain I used an alternative (cheaper) because vectors are involved. More interesting would be to have a fast filter where you can convert to frequency domain, filter with any script and convert back. I will post the function here tomorrow (football day today : D)

[kedautinh12]

Ah yes sorry, I use avspmod, but for encoding I simply use a custom bat file.

I don't see it to fit much... if any maybe ex_repair() but it's just a lowpass comparison, for MDegrain I used an alternative (cheaper) because vectors are involved. More interesting would be to have a fast filter where you can convert to frequency domain, filter with any script and convert back. I will post the function here tomorrow (football day today : D)

Oh, i know Spain has final match in Nation League today :D

[anton_foy]

Testing the latest SMDegrain (6.0) SMDegrain(tr=3,thSAD=300,contrasharp=30,refinemotion=true) gives me this error:
Script error: Convert: Unknown chromaplacement. Line 211. Yet no problem in 5.8 version.

[guest]

Testing the latest SMDegrain (6.0) SMDegrain(tr=3,thSAD=300,contrasharp=30,refinemotion=true) gives me this error:
Script error: Convert: Unknown chromaplacement. Line 211. Yet no problem in 5.8 version.

Not sure about those version numbers...3.3.4 is the latest.

Try contrasharp=true. (or false)

[anton_foy]

Not sure about those version numbers...3.3.4 is the latest.

Try contrasharp=true. (or false)

Sorry I meant the latest version v3.3.4d but nope still the same error without contrasharp.

[Dogway]

@anton_foy: Maybe you need to update avisynth. top_left alignment was added recently.

@TDS: I think what the problem is. RipBot264 doesn't understand 'contrasharp' val type.You should use a maintained program like avspmod, etc, so at least you know that the problem might be elsewhere.

[guest]

@TDS: I think what the problem is. RipBot264 doesn't understand 'contrasharp' val type.You should use a maintained program like avspmod, etc, so at least you know that the problem might be elsewhere.

You might be right, RipBot264 is pretty "closed up" as far as it's internal coding is concerned, but I have learned recently how to add more filters & scripts.

I posted that script you sent me on the RipBot forum, and the dev (Atak) actually replied with a modified script:-

pre=ex_KNLMeansCL(video,a=2,s=2,d=1,h=7.0,wmode=0)
video=SMDegrain(video,tr=3,thSAD=300,contrasharp=30,prefilter=pre,refinemotion=true)

Contrasharp bool (True/False, default False)
Contrasharp int (0-100, default -)
Contrasharpening is a technique that compares the differences between the clip before blurring (original) and after blurring (filtered), and sharpens locally with consequent strength. By default the "sharp" clip is the input, the "after" clip is the denoised clip. Alternately, a "before" clip can be specified with CClip (See Advanced).
True: use Didée's Contrasharpening() function (ContraHD() for HD) which "Sharpens the denoised clip, but doesn't add more to any pixel than what was removed previously." In the practice you will get a slightly sharper result than the source, which is welcome.
An integer: LSFmod() will be used instead. It will be much slower, but maybe better for certain sources. Its value will serve as a contrasharpening multiplier, use one around 50 for similar strength as Contrasharpening(). LSFmod is less-suitable for HD sources.

So this could be done to ALL the pre filters, especially BM3D ??

[FranceBB]

Oh, i know Spain has final match in Nation League today :D

And... they've lost to France with a probably offside goal xD
(But yeah, I was watching the game too, so I guess Doom9 stopped for a bit while everyone was watching it xD)

[Dogway]

@TDS: Yes, you can use contrasharp to any filtering that does some blur. probably the issue you are facing is because RipBot parses all the arguments and only accepts one type per argument, in this case bool. This is something you have to discuss with them, try first with another filter that uses "val" type.

Yep, lost to Mbappe offside goal, it reminds me of Henry's hand, or PSG's financial "fair play", anyway Spanish squad is very young many playing for first time. My favourite was Italy, best national team by far in my opinion.

By the way I might upload updated ExTools later today, with a fair amount of improvements.

[guest]

@TDS: Yes, you can use contrasharp to any filtering that does some blur. probably the issue you are facing is because RipBot parses all the arguments and only accepts one type per argument, in this case bool. This is something you have to discuss with them, try first with another filter that uses "val" type.


I don't think I need to do anymore about Contrasharp, I'm happy with "true" or "false". I was actually shocked that I got a reply on the RipBot forum.

So can BM3D be setup like you did with KNLMeansCL ??

pre=ex_BM3D(video,???????)
video=SMDegrain(video,tr=3,thSAD=300,contrasharp=30,prefilter=pre,refinemotion=true)

I tried a couple of different things, but couldn't get it to work.

[Boulder]

Tried QTGMC with inputtype=2, got error "I don't know what 'fs' means.", pointing to extools.avsi at line 1191 and qtgmc.avsi at line 594.

EDIT: Fixed by adding ', bool "fs"' to ex_GaussianBlur's parameter list, but got the next error: "MergeLuma: Images must have same width and height!" pointing to lines 1189 (extools) and 882 (QTGMC).

[Dogway]

Just updated ExTools to v6.1 ans SimilarityMetrics to v1.2.

@Boulder: Try the new version as the issue was addressed a week ago but hadn't released the update. I will also have a look for the MergeLuma thing. EDIT: fixed now.
@TDS: should be working, at least works here with defaults "ex_BM3D()"

[guest]

Just updated ExTools to v6.1
@TDS: should be working, at least works here with defaults "ex_BM3D()"

I tried:- ex_BM3D(sigma=5,radius=1) from Line 195.

Didn't work :(

[Dogway]

It works for me. It requires Utils-r41 if you are processing chroma. I might remove the dependency soon. Also be sure to have the latest BM3D version.

By the way, here is the feature rich ex_makediff() when using 'aug=true', requires SimilarityMetrics which I also updated today:

https://i.imgur.com/IQ4rWmh.png

[tormento]

New setting which I wanted to add since 7 years ago, 'LFR', it will restore low frequency details, specially useful for TemporalSoften, check the images below.
Is it automatic according to tr or thsad or is there a rule of thumb to enable it?
By the way, here is the feature rich ex_makediff() when using 'aug=true', requires SimilarityMetrics which I also updated today
Could you post a sample script? It would be nice to make it accept more than one input clip to see differences with different colors. I often do compressibility checks changing tr and thsad and it would be nice how many details they steal.

[Dogway]

No, LFR is not automatic since it makes the filter slower (haven't benchamrked though). I would use it always with TemporalSoften, and also with MDegrain if you are using thSAD>500 or tr>4 but these are loose estimations. I would run tests to see the before and after (I use ex_makediff to see better) and check if it recovers enough to be worth it.

ex_makediff() already accepts two clips. Simply call like "ex_makediff(a,b,aug=true,UV=128)".
The clip output corresponds to MAE only, it's not an indication of the numbers, only to know where to look for differences. Although I haven't tested with chroma, only with luma plane, might need to rework a few things.

[guest]

It works for me. It requires Utils-r41 if you are processing chroma. I might remove the dependency soon. Also be sure to have the latest BM3D version.


OK, well, I'll have to check if I have Utils-r41 in the list, and I have trouble finding the latest builds of BM3D are...hint, hint.

Cheers

[tormento]

ex_makediff() already accepts two clips.
I was asking for more than two :)

I usually launch smdegrain with tr=3,4,5 and thsad=300,400,500 accordingly. Sometimes 6, 600 too. I'd like to look at the differences at the same time.

[guest]

It works for me. It requires Utils-r41 if you are processing chroma. I might remove the dependency soon. Also be sure to have the latest BM3D version.


I double checked, I DID have Utils-r41, but I obviously forgot something simple, as it is now working :)

However, when I went to try the CPU "version", it showed an error on line 838, of SMDegrain334d, BM3D_CPU (sigma=[s,ch?s/2:0,ch?s/2:0], radius=r, chroma=ch) which lead me to line 829, cd = Default(CUDA, false) and I changed the (CUDA,true, to false


So originally you said to change Line 195 (prefilter== 5) ? inputP.ex_BM3D(sigma=5,radius=1,CUDA=false,UV=Chr,fulls=fs)

I hope you can follow that...

[tormento]

No, LFR is not automatic since it makes the filter slower (haven't benchamrked though). I would use it always with TemporalSoften, and also with MDegrain if you are using thSAD>500 or tr>4 but these are loose estimations. I would run tests to see the before and after (I use ex_makediff to see better) and check if it recovers enough to be worth it.
I have briefly tested in some scenes and the result with fog is awful, i.e. random blocks of pure noise (read: mess :)). When my PC will finish the encoding queue, I will test more.

[Dogway]

@TDS: Nice. I tested "ex_BM3D(sigma=5,radius=1,CUDA=false,UV=3,fulls=false)" and it worked here. Do you have the latest version?

@tormento: Great! share the clip (and SMDegrain call) so I can have a look. ExTools and SMDegrain are supposed to be finished except for bugs.

[guest]

@TDS: Nice. I tested "ex_BM3D(sigma=5,radius=1,CUDA=false,UV=3,fulls=false)" and it worked here. Do you have the latest version?


Latest version of what ???

If you're referring to BM3D, I think I have "test 3"...is there newer ??

[Dogway]

test 3 is not latest, it's this (https://forum.doom9.org/showthread.php?p=1951477#post1951477).

[guest]

test 3 is not latest, it's this (https://forum.doom9.org/showthread.php?p=1951477#post1951477).

OK, thanks for that, but I think that might be the one I have...I'll check.

[kedautinh12]

No, this is last
https://github.com/WolframRhodium/VapourSynth-BM3DCUDA/actions/runs/1215772307

[Dogway]

Thanks, that's it.

[guest]

Thanks, that's it.

Were you able to get this ??

It's timestamped @ 09-09-21, older build was 04-09-21.

[kedautinh12]

My link is ver 09-09-21

[guest]

My link is ver 09-09-21

Hi, yes, but it took me a while to get to the downloadable file...