Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th February 2022, 10:09   #801  |  Link
guest
Guest
 
Posts: n/a
Quote:
Originally Posted by kedautinh12 View Post
You can try TemporalDegrain2 (postFFT=4), that option have KNLMEANSCL
Hi ked,

But why remove it altogether...just add another pre filter (DGDecNV).

I'm having enough issues (see post 783), that I haven't really had time to "fix", and then sort of unannounced there is a significant change to the pre-filters in SMDegrain

Last edited by TDS; 8th February 2022 at 10:18.
  Reply With Quote
Old 8th February 2022, 11:12   #802  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
Quote:
Originally Posted by TDS View Post
Hey Dogway, and the updates just keep on coming...

So I just noticed in the latest iteration of SMDegrain, that KNLMeansCL (pre filter=5) has "disappeared" !!!!

In it's place, has appeared DGDecNV !!!! How does that work.

Can "we" have KNLMeansCL back ???

I haven't done anymore testing BM3D since my post a few days ago (been busy with other things), I will post my script & call's soon, and I would appreciate your help with that.

Have the dependencies for BM3D changed recently ??
Don't worry the updates are almost done, I will be working mainly on Zopti (SMDegrain, QTGMC, FRC, etc) and SceneStats for the following weeks.

Do you have something to prove KNLMeansCL should be back? In my tests it showed poor performance, see the following images (very bad motion estimation).

KNLMeans Spatio-Temporal (bug so edited out)
KNLMeans Spatial (bug so edited out)
DGDenoise

DGDenoise has sharp edges so a bit of blurring afterwards is fine, I will add it.

As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 8th February 2022 at 13:51.
Dogway is offline   Reply With Quote
Old 8th February 2022, 11:50   #803  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,822
Results are a bit strange. Isn't DGDenoise also an implementation of nlmeans algo? (but without any temporal filtering)
I mean they should look more similar...

Is your source filter frame accurate?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 8th February 2022, 12:12   #804  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
I'm using DGDecNV r239 so I it should be frame accurate.
I'm not sure about DGDenoise algorithm, there are several edge aware denoisers like nlmeans, bilateral and variations like smart blur, surface blur and those from the papers; guided blur, etc. To me DGDenoise looks like a bilateral filter but videoh has to confirm.
Temporal GPU denoisers in AviSynth is a chimera, both KNLMeansCL and BM3D have issues BM3D CPU is fine though.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 8th February 2022 at 12:14.
Dogway is offline   Reply With Quote
Old 8th February 2022, 13:00   #805  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,822
From DGDecodeNVManual.html
Quote:
DGDenoise() is a fast stand-alone CUDA filter that implements NLM denoising. The input clip must be in the YV12 or YUV420P16 format. DGSource() delivers YUV420P16 when the source stream has a 10/12/16-bit depth.
I never had such blending artifacts in knlm temporal. But I remember that some graphics card had some issues with this plugin. Do you get the same output in vapoursynth too?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 8th February 2022, 13:01   #806  |  Link
guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Dogway View Post
Don't worry the updates are almost done, I will be working mainly on Zopti (SMDegrain, QTGMC, FRC, etc) and SceneStats for the following weeks.

Do you have something to prove KNLMeansCL should be back? In my tests it showed poor performance, see the following images.

Very bad motion estimation. If converted to spatial only, not as good flat denoiser as DGDenoise and edge blurring in excess. DGDenoise has sharp edges so a bit of blurring afterwards is fine, I will add it.

As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug.
I have no "good" reason to want KNLMeansCL back, I guess it's just that when you set up certain scripts to use a particular pre-filter, and then it changes without warning, that's the only reason, I guess,
if you think that DG's stuff works better, then so be it.

You also hinted that there are some issues with BM3D (GPU), but with BM3D (CPU) it's ONLY good for AVX2 CPU's, unless the dev of that updates that function.

as for this
Quote:
As for the BM3D issues I would first check BM3D works fine on its own so it's not an ex_BM3D() bug.
not sure how to do that.

All I can say about that is, that it used to work, but since updating last week sometime, the error showed up, but I can't figure out what versions I changed, that brought it back.

I need to back date, and start almost from scratch

BTW, where can I get older builds ??? is there an archive somewhere ??

Last edited by TDS; 8th February 2022 at 13:37.
  Reply With Quote
Old 8th February 2022, 13:48   #807  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
Quote:
Originally Posted by ChaosKing View Post
From DGDecodeNVManual.html

I never had such blending artifacts in knlm temporal. But I remember that some graphics card had some issues with this plugin. Do you get the same output in vapoursynth too?
Oh ! ::facepalm:: shameful bug. Will update later today and reimplement ex_KNLMeansCL (happy TDS)

@TDS: Wait for v3.4.6 tonight or something and report back if it still doesn't work.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 8th February 2022, 14:50   #808  |  Link
guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Dogway View Post
Oh ! ::facepalm:: shameful bug. Will update later today and reimplement ex_KNLMeansCL (happy TDS)

@TDS: Wait for v3.4.6 tonight or something and report back if it still doesn't work.
Well it looks as though you've left all the KNLMeansCL stuff in SMDegrain, so why not leave that as pre filter 5, and put in the new one as either 6 or 7, well probably better at 7, and leave bm3d as 6.
  Reply With Quote
Old 9th February 2022, 01:27   #809  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,169
Quote:
Originally Posted by TDS View Post
Well it looks as though you've left all the KNLMeansCL stuff in SMDegrain, so why not leave that as pre filter 5, and put in the new one as either 6 or 7, well probably better at 7, and leave bm3d as 6.
Doggy updated latest ver
kedautinh12 is offline   Reply With Quote
Old 9th February 2022, 01:57   #810  |  Link
guest
Guest
 
Posts: n/a
Quote:
Originally Posted by kedautinh12 View Post
Doggy updated latest ver
Thnx ked

@Dogway, nice work, I see what you've done there....will "test" as soon as I can, and report back.

Cheers

---------------------------------------------------------------------------

So, reporting back, and have had limited success, it works on a stand alone PC, but using the Distributed Encoding feature of RipBot264, the Client encoding servers will not encode , with this error
Quote:
y4m [error]: bad sequence header magic
x264 [error]: could not open input file `-'
OR
Quote:
y4m [error]: bad sequence header magic
x265 [error]: could not open input file `-'


I have no idea wtf has gone wrong, 'cause I'm sure it was working a couple of weeks ago before I "blindly" updated everything

---------------------------------------------------------------------------

Another update, I have managed to get a "set" of scripts from and old build of RipBot264, which has most of your "goodies", but rather old builds, BUT those errors have gone

So now I need to update one @ a time, and see where the issue is / was. I'm kinda thinkin' it might be something to do with BM3D, but KNL did the same thing, suspecting some OpenCL problem, but wtfk !

Last edited by TDS; 9th February 2022 at 07:47.
  Reply With Quote
Old 9th February 2022, 20:01   #811  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,867
Quote:
Originally Posted by Dogway View Post
both KNLMeansCL and BM3D have issues BM3D CPU is fine though.
I dunno.

I have always used both with a bias for the second because of more detail retaining while doing a good job at helping removing noise.

I have never found any artifact in the output video from SMDegrain and I will keep on using BM3D on CUDA, as my CPU is more ancient than simply old.

DGDenoise is a nice add, I still have to evaluate detail retaining and differences with KNLMeansCL.
__________________
@turment on Telegram

Last edited by tormento; 9th February 2022 at 20:03.
tormento is offline   Reply With Quote
Old 9th February 2022, 22:34   #812  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
With BM3D you have to be too careful on many aspects. First don't crop with DGSource(). And second, we lack "blocking_step" arg, so the maximum block-free settings are sigma=4, radius=1, which is almost useless for prefiltering. I think it also has issues with FFMS2 loader. With CUDA=false I didn't see any of these bugs.

DGDenoise() is fine for Anime, as it's only spatial filtering. I haven't run benchmarks though, it would be very interesting to compare DGDenoise() and KNLMeansCL().
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 9th February 2022 at 23:40.
Dogway is offline   Reply With Quote
Old 9th February 2022, 23:39   #813  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,867
Quote:
Originally Posted by Dogway View Post
With BM3D you have to be too careful on many aspects. Fist don't crop with DGSource().
I always do. How could it interfere? Please provide some examples.
Quote:
Originally Posted by Dogway View Post
it would be very interesting to compare DGDenoise() and KNLMeansCL().
I haven't done any serious comparison yet but SMDegrain is way faster when using DGDenoise instead of KNLMeansCL. About denoising, using DGDenoise as prefilter, gives bitrates that are between KNLMeansCL (the lowest ones but with detail loss) and BM3D, that AFAIK was better preserving details, mostly moving ones.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 10th February 2022, 00:46   #814  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
I posted the issue here. Confirmed by other users.

DGDenoise is purely spatial, you would need to compare with spatial KNLMeansCL as it's spatio-temporal in SMDegrain, and do some matching with strengths. For visual comparison you can use show=true or directly compare the filters outside SMDegrain.

EDIT: I made a benchmark between both.

Code:
ex_DGDenoise(str=0.10,UV=3,LFR=0) # P(4) 83
ex_KNLMeansCL(a=2,s=2,d=0,h=6.0,wmode=1,chroma=true,device_type="GPU",LFR=0) # P(4) 80
DGDenoise is very similar to KNLMeansCL when used spatially with wmode=1 which retains a bit more of the lineart. But suprisingly KNLMeansCL retains more low frequency detail than DGDenoise, this might be good or bad as it will also show faint blotches of grain.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 10th February 2022 at 02:37.
Dogway is offline   Reply With Quote
Old 10th February 2022, 01:35   #815  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
Quote:
Originally Posted by gispos View Post
Hi Dogway,
the 'Highlighter' filter is now called 'ex_Highlighter' and is 6 - 8 times faster with almost identical results. So full success.

And therefore (if you feel like it) you may try the opposite function. 'HighlightLimiter' and 'HighlightLimiterMod'
The filter reduces too bright areas. The mod version was changed by me (long time ago).
Check this:

Code:
function HighlightLimiter(clip v, float "gblur", bool "gradient", int "threshold", bool "twopass", int "amount", int "method") {

    bi  = BitsPerComponent(v)
    fs  = propNumElements (v,"_ColorRange")  > 0 ? \
          propGetInt      (v,"_ColorRange") == 0 : false

    grad    = Default (gradient,true)
    gblur   = Default (gblur, grad ? 100 : 5)
    thres   = Default (threshold,160)          #The lower the value, the more sensitive the filter will be.
    twopass = Default (twopass,false)
    amount  = Default (amount,20)
    method  = Default (method, 0) # 0 or 1

    amount  = ex_bs(abs(amount), 8, bi, true)

    darken  = v.ex_binarize(thres,hi=-1)
    blurred = grad ? darken.ex_gaussianblur(0.073*gblur+17.78,UV=1) : \
                     darken.ex_gaussianblur(0.064*gblur+ 3.75,UV=1)

    multiply = (method==0) ? v.ex_lut("x dup * range_max /", fulls=fs) : v.ex_lut(Format("x {amount} + ")+(!fs?"ymax min":""), fulls=fs)
    multiply = !twopass    ? multiply : \
               (method==0) ? multiply.ex_lutxy(v,"x y * range_max /", fulls=fs) : \
                             multiply.ex_contrast(0.15, 1, 163,   tv_range=!fs)
    return mt_merge(v,multiply,blurred) }


function HighlightLimiterMod(clip v, int "threshold", int "dark", float "rad",int "mode", float "gamma1_2") {

    w   = Width (v)
    h   = Height(v)
    bi  = BitsPerComponent(v)
    fs  = propNumElements (v,"_ColorRange")  > 0 ? \
          propGetInt      (v,"_ColorRange") == 0 : false

    thres   = Default (threshold,160)
    dark    = Default (dark, 255)
    rad     = Default (rad,    64)
    mode    = Default (mode, 0)       # 0,1,2
    gamma1_2= Default (gamma1_2, 0.6) # for mode 1,2

    m    = min(w,h)
    dark = ex_bs(dark, 8, bi, true)
    rad  = round(m/rad/4) < 1 ? max(round(m/4.0),16) : rad

    darken  = v.ex_binarize(thres,hi=-1)
    blurred = darken.ex_GaussianBlur(rad*0.83825-0.0631,pad=true,UV=1)

    multiply = (mode==0) ? v.ex_lut(Format("x dup * {dark} /"), fulls=fs) : \
               (mode==1) ? v.ex_levels(gamma=gamma1_2,tv_range=!fs)       : \
                           v.ex_lut(Format("x dup * {dark} /"), fulls=fs).ex_levels(gamma=gamma1_2,tv_range=!fs)

    return mt_merge(v,multiply,blurred) }
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 12th February 2022 at 03:03.
Dogway is offline   Reply With Quote
Old 10th February 2022, 04:09   #816  |  Link
guest
Guest
 
Posts: n/a
Success :)

Hey Dogway,

I've had a major win (several win's, actually).

So I started from an old "set" of your scripts, and worked thru until they worked.

I even got the DGDecNV pre-filter to work

So this is my script for KNLMeansCL, is there any "fine tuning" that could be done to it, as I noticed you changed the instructions a bit in SMDegrain 3.4.6d

Code:
LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\MedianBlur2\MedianBlur2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\KNLMeansCL\KNLMeansCL.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools74.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod50ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Resizers Pack78.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Sharpeners Pack34.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain346d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
pre=ex_KNLMeansCL(video,a=2,s=4,d=1,h=4,wmode=0)
video=SMDegrain(video,tr=4,thSAD=400,thSADC=200,thSCD2=80,contrasharp=true,prefilter=pre,refinemotion=true)
And this is my DGDecNV script, again, any "fine tuning" for this :-

Code:
LoadPlugin("%AVISYNTHPLUGINS%\dgdecnv\DGDecodeNV.dll")
LoadPlugin("%AVISYNTHPLUGINS%\dfttest\dfttest.dll")
LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\MedianBlur2\MedianBlur2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
Import("%AVISYNTHPLUGINS%\scripts\extras\ExTools74.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\LSFmod50ex.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Sharpeners Pack34.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\SMDegrain346d.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Transform Packv1.0 RC36.avs")
Import("%AVISYNTHPLUGINS%\scripts\extras\Zs_RF_Shared.avs")
pre=ex_DGDenoise(video,str=0.10)
video=SMDegrain(video,tr=2,thSAD=400,thSADC=100,thSCD2=80,contrasharp=true,prefilter=pre,refinemotion=true)
HOWEVER, can you please change SMDegrain back to it's BM3D default (earlier builds) being CUDA, you've done something and now it's BM3D CPU by default, and I need CUDA for some of my older servers that don't support BM3D CPU builds.

It was fairly easy to change the CUDA settings from "True" to "False", but with this latest build, that doesn't seem to apply.

Thanks

Last edited by TDS; 10th February 2022 at 06:27.
  Reply With Quote
Old 10th February 2022, 04:29   #817  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,169
Hey Dogway, i added some ex_ family into f3kgrain. Can you check it's complete replace ex_ into f3kgrain or not?? Thanks
https://github.com/kedaitinh12/AVSPl...n_v0.4MOD.avsi
kedautinh12 is offline   Reply With Quote
Old 10th February 2022, 12:00   #818  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
Quote:
Originally Posted by kedautinh12 View Post
Hey Dogway, i added some ex_ family into f3kgrain. Can you check it's complete replace ex_ into f3kgrain or not?? Thanks
https://github.com/kedaitinh12/AVSPl...n_v0.4MOD.avsi
I reply you there.

@TDS: You should be able to use CUDA by setting it to true here.

For the KNLMeansCL prefilter, test with wmode=1, depending on content you might like it more. By default with 'd' > 0, LFR is used, you can try disabling it with LFR=0, but it will smear quite a bit. ex_DGDenoise() seems also to smear quite a bit, so for next version I will also add LFR argument.
CUDA in BM3D is broken really, you can try comparing them CUDA vs CPU outside SMDegrain, I posted this in the BM3D thread.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 10th February 2022 at 12:15.
Dogway is offline   Reply With Quote
Old 10th February 2022, 12:37   #819  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,822
Quote:
Originally Posted by Dogway View Post
And second, we lack "blocking_step" arg, so the maximum block-free settings are sigma=4, radius=1, which is almost useless for prefiltering.
Not true, there is a blocking_step paramater in the cuda version. I just set mine to 2 so I don't have to worry about evil blocking
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 10th February 2022, 12:49   #820  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,372
Oh shii... the AVS filter is so undocumented it looks like shady business haha

block_step, that is: https://github.com/WolframRhodium/Va.../kernel.cu#L49
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 10th February 2022 at 13:05.
Dogway is offline   Reply With Quote
Reply

Tags
avisynth, dogway, filters, hbd, packs

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2025, vBulletin Solutions Inc.