Dogway's Filters Packs

[tormento]

If the blurring was too extreme I would recommend the model HurrDeblur
With what AVS+ plugin?

[poisondeathray]

If the blurring was too extreme I would recommend the model HurrDeblur

With what AVS+ plugin?


There is only 1 plugin that can be used directly in avs+ for HurrDeblur - avs-mlrt

https://github.com/Asd-g/avs-mlrt

[tormento]

There is only 1 plugin that can be used directly in avs+ for HurrDeblur - avs-mlrt
:thanks:

My poor 1660 Super will die on fire. :D

[simple_simon]

I've been using removegrain(mode=5) or vsdegrainmedian(modeY=3) to help with compressability without changing the overall look of the picture too much. What's the best ex_median mode to use as an equivalent?

[Lucky38]

hi,

I'm looking option to denoise with smdegrain enitre frame exept area where is the motion. How could i achive that?

[tormento]

@Dogway

Is there a way to pass parameters to prefilters but to use it as external prefilter? I am interested mostly in 6,7,8.

[DTL]

hi,

I'm looking option to denoise with smdegrain enitre frame exept area where is the motion. How could i achive that?

If SMDegrain exports MVs clip (as some named global ?) - you can feed it to MMask() and use output mask with AVS filters like Overlay() or Layer().
If not - you need to calculate MVs again (MSuper+MAnalyse).


After looking into ex_median - it is a great 64 different median (median-like ?) modes ! Now I am in the process of testing some ways of averaging MVs gathered from 'area' searches (with some offset from block's center position in tessellation grid - it sort of very dense overlap with possible steps on 1 sample dx,dy) to make more stable MV for degraded by noise sources. Which of the 64 different median modes can be tested for this process ?

The gathered MVs from AreaMode search can be treated either as 1D array or as 2D array. Currently, the implemented averaging modes process as 1D array and not use possible 2D information gathered (like distance/radius from block's center of the search position). The offset distance from block's center makes positives and negatives:
1. Positives - the more offset distance from block's center - the more new (not tested with original block's position) samples used and gather more new information about motion. It is sort of an increasing block size.
2. Negatives - the more offset distance from block's center - the more image motion may be changed and the offsetted MVs may be really somehow different from the original block's position.

So with 1 and 2 I am still not sure if we can apply some '2D spatial weighting' to the averaging process. Anyway AreaMode gathers lots of new valuable data to process about motion in some close area about block's center (not only MVs but also SAD (or other supported dissimilarity metric)) it is 2D array of 3 values in each (of size 2x2 + center or 3x3 + center minimum with AreaMode 'radius' of 1 sample and many more with > 1). So lots of different new processing modes may be tested to find the best way of refining MVs using new gathered data.

2D view of the question:
https://i.ibb.co/nQ3wbDg/image-2024-04-17-170508492.png

For gathered MVs set (2D array of scanned to 1D) of MV(0,0) and some of the MV(dx, dy) members (where dx and dy are +-integer offsets in the units of current search level of MAnalyse: +-1 sample at the full-pel level for example) which of the ex_median() averaging algorithms may be recommended to test to get more stable output MV ?

Each MV() is a structure of 3 members (dx, dy, sad). Where dx and dy are relative motion vector X and Y components). So for analysys of 2 completely equal frames MV(0,0) = MV(dx,dy)= (dx=0,dy=0,sad=0).
But if frames have some noise added: MV(0,0) may be not equal to any of others MV(dx, dy). Also MV(0,0) may have non-zero dx, dy also.

[Lucky38]

If SMDegrain exports MVs clip (as some named global ?) - you can feed it to MMask() and use output mask with AVS filters like Overlay() or Layer().
If not - you need to calculate MVs again (MSuper+MAnalyse).


@DTL,

Have you got some example how should look like proper call with SMDegrain?

Many Thanks :)

[DTL]

Documentation https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain/SMDegrain.html
says
Motion Vectors Globals Input/Output: Reuse motion vectors globals for faster processing, or just use SMDegrain() as a shortcut for creating nice quality motion vectors.

Example

SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors

Super = MSuper(levels=1) # Add this line just before if you have some processing between Globals Output and Input.
MDegrain3(Super, bVec1, fVec1, bVec2, fVec2, bVec3, fVec3, thSAD=400)


So you can try

denoised=SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors
mask_clip=MMask(fVec1)
Overlay(last, denoised, mask=mask_clip)


I can not check because SMDegrain too complex for me and I can not gather all required dependencies and install as required. You need to check mask polarity (Use Invert() or arguments replacement in reverse order in Overlay() if result is inverted) and set mask 'sensitivity' to motion adjusting ml param at MMask(). Check mask_clip view to see how it masks motion areas,

[Lucky38]

Documentation https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain/SMDegrain.html
says
Motion Vectors Globals Input/Output: Reuse motion vectors globals for faster processing, or just use SMDegrain() as a shortcut for creating nice quality motion vectors.
:thanks:
Example

SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors

Super = MSuper(levels=1) # Add this line just before if you have some processing between Globals Output and Input.
MDegrain3(Super, bVec1, fVec1, bVec2, fVec2, bVec3, fVec3, thSAD=400)


So you can try

denoised=SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors
mask_clip=MMask(fVec1)
Overlay(last, denoised, mask=mask_clip)


I can not check because SMDegrain too complex for me and I can not gather all required dependencies and install as required. You need to check mask polarity (Use Invert() or arguments replacement in reverse order in Overlay() if result is inverted) and set mask 'sensitivity' to motion adjusting ml param at MMask(). Check mask_clip view to see how it masks motion areas,

I need to test this..

:thanks:

[Lucky38]

Documentation https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain/SMDegrain.html
says
Motion Vectors Globals Input/Output: Reuse motion vectors globals for faster processing, or just use SMDegrain() as a shortcut for creating nice quality motion vectors.

Example

SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors

Super = MSuper(levels=1) # Add this line just before if you have some processing between Globals Output and Input.
MDegrain3(Super, bVec1, fVec1, bVec2, fVec2, bVec3, fVec3, thSAD=400)


So you can try

denoised=SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors
mask_clip=MMask(fVec1)
Overlay(last, denoised, mask=mask_clip)


I can not check because SMDegrain too complex for me and I can not gather all required dependencies and install as required. You need to check mask polarity (Use Invert() or arguments replacement in reverse order in Overlay() if result is inverted) and set mask 'sensitivity' to motion adjusting ml param at MMask(). Check mask_clip view to see how it masks motion areas,

Ihave tried this

SMDegrain(tr=3,thSAD=400,globals=3) # Outputs vectors
mask_clip=MMask(fVec1)
denoised = SMDegrain(last ,tr=4, thSAD=480, thSCD2=100, prefilter=-1)
Overlay(last, denoised, mask=mask_clip)

but this is not work to my eyes. keep more grain on flat area than default SMDegrain call:
SMDegrain(tr=4, thSAD=480, thSCD2=100, prefilter=-1)

[DTL]

To debug the script you may start from checking the mask first - add

return mask_clip

(to the end of script or after mask_clip generation)

And look how it good or not tracks the static and motion areas. Depending on lots of settings (like MVs length and pel and many more) you may need to adjust mask with ml-param of MMask and/or with later any levels adjustment filters (Levels, ColorYUV, Tweak or other) to prepare best mask matching your source motion type and preference. And only next step is apply mask to Overlay() to get best result.

May be better to stack mask and real image in some way to see both mask and real frame at the same time like

return StackVertical(last, mask_clip)

But it may need more conversions to make colour formats equal of mask_clip and 'last' clip so Stack* can combine it in the single frame.

Overlay documentation: clip mask = (full opacity)
Optional transparency mask. Must be the same size as overlay. Where mask is darker, overlay will be more transparent. Must be 0-255 range (in general: 0 - 2^N-1 range for N bits format).

overlay clip is second param of Overlay() so where mask is black (no motion) it will output 'last' (denoised) clip.

MMask kind=0 (default) - Motion
Creates motion mask from the motion vectors length. It builds a better mask than MotionMask function of MaskTools plugin because motion vectors are a lot more reliable than the algorithm of MotionMask. Mask value 0 means no motion at all (the length of the motion vector is null). The longer vector length, the larger mask value (saturated to 255), the scale is defined by ml.

ml float default = 100. So if your motion is at low speed you may try to set higher ml param like

mask_clip=MMask(fVec1, ml=500)

Check if white levels at mask_clip is high enough at the moving areas/objects. Better about 100% white. If because of noise false MVs the MMask starts to output too high 'black level' at the static areas with big ml-value - you can add 'contrast' to mask_clip for better separation of slow motion or noised MVs from moving with Tweak(cont > 1.0) or may be Levels (params to increase conrast).

[simple_simon]

@Dogway

I noticed in the deep_resize script notes that it says linear sspace is best for downscaling. Do I need to convert to linear first or is this all that's needed:

convertbits(16) #(or 32?)
propSet("_FieldBased",0).deep_resize(1728,720,grain=0,qual=2,sspace="linear")
convertbits(8,dither=1)

[Dogway]

Yes, that's fine. You can omit grain and qual args in this case. I think 16bit is enough but check your downscaling kernel! By default it might be DPDIR or SSIM2, in such cases linear is not necessary

[simple_simon]

Yes, that's fine. You can omit grain and qual args in this case. I think 16bit is enough but check your downscaling kernel! By default it might be DPDIR or SSIM2, in such cases linear is not necessary

Looks like all three scalers are using "Didee" by default. I think SSIM2 used to be the default for most small downscaling. Would one of the 2 you mentioned be better?

Also, it probably got lost in the mix but did you happen to see my question here about Gradfun3plus?
https://forum.doom9.org/showpost.php?p=1998135&postcount=2852

[Dogway]

The kernel adapts to the downscaling ratio. In case of Didee the linear space might help , but it also depends on content due to risks on ringing. On live content it can be useful, on clean digital animations sources it might not.

As for adding grain option to Gradfun3plus let me check. I thought it was an option? (Writing from mobile)

[Guest]

Why would this script max out my 32Gb of RAM ?? (which sometimes crashes the PC)

SetCacheMode()
SetMemoryMax(16384)
LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\DGHDRtoSDR\DGHDRtoSDR.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\MedianBlur2\MedianBlur2.dll")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\GRADE-PACK\Grade Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\LSF-PLUS\LSFplus.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\RESIZERS-PACK\Resizers Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\SHARPENERS-PACK\Sharpeners Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\SMDEGRAIN\SMDegrain.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack-Models.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\TRANSFORM-PACK\Transforms Pack-Transfers.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\Zs_RF_Shared\Zs_RF_Shared.avs")
video=ConvertBits(video,16)
pre=DGHDRtoSDR(video,mode="pq",white=2000,gamma=1/2.4,tm=1.0).ex_Median(mode="IQMST",thres=255)
video=STTWM(video,sw=50,tw=50,aw=100,sthres=5,tthres=5)
video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,UHDHalf=false,limit=255,contrasharp=true,prefilter=pre,str=5.0,refinemotion=true)

[DTL]

If RAM usage grows constantly at processing - it may mean memory leak exist in some plugin. Try to isolate it by making script shorter and if found memory leak - report to developers.

[Guest]

If RAM usage grows constantly at processing - it may mean memory leak exist in some plugin. Try to isolate it by making script shorter and if found memory leak - report to developers.

Hi DTL,

Now that you've suggested this, the ONLY plugin that is different to the majority of my scripts, is DGHDRtoSDR.

It actually doesn't "grow", it's pretty much immediately :(

I've only just started to use this one, and I have been having some very unusual problems.

:thanks:

[Dogway]

Also try to use UHDHalf=true, mvtools2 might not handle 16-bit UHD resolution nicely.

[Guest]

Also try to use UHDHalf=true, mvtools2 might not handle 16-bit UHD resolution nicely.

:thanks:

Will be interested to hear what DTL might have say about statement, as he has been busy working on mvtools2..

[DTL]

Not nicely means performance is not great (also with blocksize of 8x8 even lower).

AVS cache size estimation for 4K 16bit (even in 4:4:4) and largest 'super' frame size with pel=2 (about 4x larger) - 8 Msamples of 6bytes x4 = 192 MB per frame. With 16threads and 10 frames in the cache per each thread - 25 GB in cache. Really big.

So it may be better to reduce the threads number for mvtools (AVS+ threads number of frames-based MT controlled by Prefetch(x,y) ) and if possible - number of frames in the AVS cache. And to keep more performance - enable internal MT via AVSTP.

I think the number of frames in the cache is expected to be controlled by Prefetch(M,N) where N is the number of frames. But I do not see if it really works.

In the sources it really has second param of frames -
https://github.com/pinterf/AviSynthPlus/blob/fa98d04b1386e5477154f6fc83cfef1a8155d5bb/avs_core/core/parser/script.cpp#L264
{ "Prefetch", BUILTIN_FUNC_PREFIX, "c[threads]i[frames]i", Prefetcher::Create },


So you can try to test different Prefetch(threads, frames) settings too. With frames from 1. If it is not working - it is subject to report to AVS core developers too.

You can try to see memory usage with AVSmeter report at testing your total script. If different 'frames' Prefetch() settings the amount of RAM usage does not change - it looks like a bug to be fixed.

[Boulder]

I think the number of frames in the cache is expected to be controlled by Prefetch(M,N) where N is the number of frames. But I do not see if it really works.

It's been working since day one. The problem with current processors is that the default amount of prefetched frames is double the amount of threads, which then easily leads to excessive memory usage with UHD sources. The amount of prefetched frames is much less crucial than the amount of threads what comes to encoding performance, unless you work on near realtime encodes.

[Guest]

@ DTL & Boulder,

Could either of you show me what you would change in script, please.

I'm running with Prefetch(12), but it doesn't need to be in the script, with RipBot.

Running 7950X, 5950X etc.

Or is it pointless ?

[DTL]

"I'm running with Prefetch(12)"

Try to start with Prefetch(12,1) . And with RipBot you need to find where it sends Prefetch(x,(y?)) line to AVS (if it really sends) and also edit its second param to 1 or something low as 1,2,3,etc. If second param not provided it may be up to double of threads number ? So your Prefetch(12) may be internally Prefetch(12,24) and use too much RAM for cache.

Or try to ask RipBot developers how to set prefetch frames number if it sends Prefetch() line to AVS after user's script.

As some hack you can try to add

return video.Prefetch(12,1)

as last line to the script. But it may broke RipBot if it uses special name of clip to return (video ?) or other.

Also as some experiment you can try to add .Prefetch(12,1) to each filter call (except source) so it may force AVS to set these prefetch params at least for these filters. Like
video=ConvertBits(video,16).Prefetch(12,1)
pre=DGHDRtoSDR(video,mode="pq",white=2000,gamma=1/2.4,tm=1.0).ex_Median(mode="IQMST",thres=255).Prefetch(12,1)
video=STTWM(video,sw=50,tw=50,aw=100,sthres=5,tthres=5).Prefetch(12,1)
video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,UHDHalf=false,limit=255,contrasharp=true,prefilter=pre,str=5.0,refinemotion=true).Prefetch(12,1)

[Guest]

"I'm running with Prefetch(12)"

Try to start with Prefetch(12,1) . And with RipBot you need to find where it sends Prefetch(x,(y?)) line to AVS (if it really sends) and also edit its second param to 1 or something low as 1,2,3,etc. If second param not provided it may be up to double of threads number ? So your Prefetch(12) may be internally Prefetch(12,24) and use too much RAM for cache.

Or try to ask RipBot developers how to set prefetch frames number if it sends Prefetch() line to AVS after user's script.

As some hack you can try to add

return video.Prefetch(12,1)



as last line to the script. But it may broke RipBot if it uses special name of clip to return (video ?) or other.

Also as some experiment you can try to add .Prefetch(12,1) to each filter call (except source) so it may force AVS to set these prefetch params at least for these filters. Like
video=ConvertBits(video,16).Prefetch(12,1)
pre=DGHDRtoSDR(video,mode="pq",white=2000,gamma=1/2.4,tm=1.0).ex_Median(mode="IQMST",thres=255).Prefetch(12,1)
video=STTWM(video,sw=50,tw=50,aw=100,sthres=5,tthres=5).Prefetch(12,1)
video=SMDegrain(video,tr=2,thSAD=200,thSADC=100,UHDHalf=false,limit=255,contrasharp=true,prefilter=pre,str=5.0,refinemotion=true).Prefetch(12,1)


Sadly, none of this made any difference to the RAM usage, even the RipBot dev suggested something, but again, that made no difference.

So it's either not using this script, OR buying more RAM.

One thing I should mention is, as RipBot can use Distributed Encoding, you can "load up" the CPU until it basically slows the whole process down, and I generally process 2 chunks at a time, which maxes out the RAM, IF I only process 1 chunk at a time, the RAM usage is more than acceptable.

I can simply run more encoding servers to "pick up the slack"....

So anyone else using this script would not have a RAM usage issue, except maybe with StaxRip, which offers a form of chunk encoding, but only on the same PC.

:thanks:

[DTL]

" none of this made any difference to the RAM usage"

It is really a sad sign of possible more bugs in the prefetcher exists. I will try to test at my use cases and if it also not works as expected - it is a subject to new issue at AVS+ development.

[Boulder]

I would first test the script (and post it here) with different Prefetch parameters in AVSMeter64 before spending too much time faffing about.

[DTL]

It looks like subject of long testing:

With simple internal script

ColorBars()

TurnRight()
#Prefetch(2,1)
Prefetch(2)


RAM usage difference is big between Prefetch[2] and Prefetch[2,1] host CPU is 2 cores E7500. About 20 vs 13 MB. But with my typical test script with lots of mvtools filters calls it close to several % with frame size 800x480 - about 640 and 630 MB. So Prefetct[2,1] is really working to make RAM usage a bit lower - but not as low as expected. Need to test with UHD frame sizes and pel=2 later at host with may RAM and many cores CPU. Also it is better to test with real SMDegrain script running. I typically can not install all required plugins to test it. So if someone have all environment configured to run SMDegrain it is faster to test at that host.

[Boulder]

I tested a UHD source with my denoiser script with some MVTools stuff, outputting 16-bit data in and out. 32 threads in use.
If I put my resizer there, the usage would be much higher I think.

frames 1: 4560 MB
frames 4: 6196 MB
frames 8: 8580 MB
frames 16: 13097 MB
frames 32: 19188 MB
default, frames 64: 22014 MB (hitting some limit? Max is set to 30000 MB)

[LouieChuckyMerry]

Hi Dogway, I hope you're well (and am glad you're back) :) . After transitioning from your original SMDegrain Mod to real.finder's continuation, I finally have your latest SMDegrain Mod (v4.6.0d) working but can't figure out the equivalent of "LSB_In=True" and "LSB_Out=True" ("n16=True" and "n16_Out=True" with real.finder's versions). My typical SMDegrain, well, NotSMDegrain, call is

SMDegrain(TR=1~5,ThSAD=100~900,RefineMotion=True,Plane=0,Chroma=False,n16=True,n16_Out=True)

Thanks for any help.

[DTL]

Some hacks around 16bit were required for old versions of mvtools. With 2.7.45 (46) you can try simply feed 10..16 bit input without any additional params and expect it is processed unchanged and output the same.

[LouieChuckyMerry]

DTL: Thanks for your reply. I'm just a hobbyist, so my understanding of AviSynth is quite limited and my use of it's very simple. Typically, I'll take either an 8-bit or 10-bit source and process it with the script

SetFilterMTMode("Default_MT_Mode",2)
SMDegrain(TR=1~5,ThSAD=200~900,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
PreFetch(X,depending on which computer)

outputting to 10bit x264. Sometimes I'll use Spotless() as a prefilter, but that's about as complex as I get. So, to achieve processing the above in 16-bit would I add "ConvertBits(Bits=16)" to the script here

SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(Bits=16)
SMDegrain(TR=1~5,ThSAD=200~900,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
PreFetch(X,depending on which computer)

Thanks for your help.

[Boulder]

So, to achieve processing the above in 16-bit would I add "ConvertBits(Bits=16)" to the script here

SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(Bits=16)
SMDegrain(TR=1~5,ThSAD=200~900,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
PreFetch(X,depending on which computer)

That would most likely be enough. You can doublecheck the output video format by adding Info() after the SMDegrain line and opening the script in VDub for example.

[Lucky38]

Hi Dogway, I hope you're well (and am glad you're back) :) . After transitioning from your original SMDegrain Mod to real.finder's continuation, I finally have your latest SMDegrain Mod (v4.6.0d) working but can't figure out the equivalent of "LSB_In=True" and "LSB_Out=True" ("n16=True" and "n16_Out=True" with real.finder's versions). My typical SMDegrain, well, NotSMDegrain, call is

SMDegrain(TR=1~5,ThSAD=100~900,RefineMotion=True,Plane=0,Chroma=False,n16=True,n16_Out=True)

Thanks for any help.


Hi,

could someone explain how SMDegrain will known when use TR=1 or TR=5 and the same for ThSAD?

i did not see in documentation any information that these parameters can have range value....

[Guest]

Hi,

could someone explain how SMDegrain will known when use TR=1 or TR=5 and the same for ThSAD?

i did not see in documentation any information that these parameters can have range value....

Here's an example...

https://forum.doom9.org/showthread.php?p=2001092#post2001092 post #2917

[StvG]

So, to achieve processing the above in 16-bit would I add "ConvertBits(Bits=16)" to the script here

SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(Bits=16)
SMDegrain(TR=1~5,ThSAD=200~900,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
PreFetch(X,depending on which computer)

That would most likely be enough. You can doublecheck the output video format by adding Info() after the SMDegrain line and opening the script in VDub for example.

F3KDB doesn't support native 16-bit (only stacked, interleaved). Repalce it with neo_f3kdb.

[Lucky38]

Here's an example...

https://forum.doom9.org/showthread.php?p=2001092#post2001092 post #2917


hi FTLOY,

thank you for your replay but this is not the explanation i was hope to get...

I use SMDegrain but i havn't seen that TR=1~5 and ThSAD=200~900 can work with range value...

That is why i was asked how does SMDegrain will known when use TR=1 or TR=2 ... or TR=5 ?

[Boulder]

hi FTLOY,

thank you for your replay but this is not the explanation i was hope to get...

I use SMDegrain but i havn't seen that TR=1~5 and ThSAD=200~900 can work with range value...

That is why i was asked how does SMDegrain will known when use TR=1 or TR=2 ... or TR=5 ?

That was just to mark an example of range. You need to set a fixed TR and ThSAD to your liking.

[Lucky38]

That was just to mark an example of range. You need to set a fixed TR and ThSAD to your liking.

thanks for clarification. :thanks:

[LouieChuckyMerry]

That would most likely be enough. You can doublecheck the output video format by adding Info() after the SMDegrain line and opening the script in VDub for example.

Thanks for your reply. It does seem to be enough; unfortunately, adding Info() after the SMDegrain line doesn't show any info upon opening the script in MeGUI or VirtualDub2 :( .

EDIT: Adding Info() after the SMDegrain line works on my second computer and shows that it's processing in 16bits. Thanks for the new trick.

F3KDB doesn't support native 16-bit (only stacked, interleaved). Repalce it with neo_f3kdb.

Good catch, StvG :thanks: . That explains quite a bit...

Lucky38: Sorry for the confusion.

[DTL]

Some idea about different prefilters usage (including no-prefiter): At testing QTGMC it was found single prefilter mode can not make all areas better in details. With prefilter-ON some areas get better and others not. With prefilter-OFF it is changed. So I make some function to gather most possible details from several filter runs. With deinterlace processing of QTGMC it also collect most of noise from different filter runs also. But it may be tested with SMDegrain too with different prefilters at single processing. The function GetSharpest() included into M_QTGMC listed at https://forum.doom9.org/showthread.php?p=2001388#post2001388 and it uses new plugin MostDiffVal.dll to run. Though I think maybe it can be implemented with all internals of AVS like Layer() instead of Average and Expr() instead of MostDiffVal().

So to run SMDegrain with a set of prefilters (any set may be combined in a chain of GetSharpest(clip1, clip2):


pref1=SMDegrain(prefilter=0)
pref2=SMDegrain(prefilter=1)

GetSharpest(pref1, pref2)

where GetSharpest() is

Function GetSharpest(clip c1, clip c2)
{
avg=Average(c1.GaussResize(c1.width, c1.height, src_left=0.001, src_top=0.001, p=2), 0.5, c2.GaussResize(c2.width, c2.height, src_left=0.001, src_top=0.001, p=2), 0.5)

return MostDiffVal(avg, c1, c2)
}

[gispos]

Some idea about different prefilters usage (including no-prefiter): At testing QTGMC it was found single prefilter mode can not make all areas better in details. With prefilter-ON some areas get better and others not. With prefilter-OFF it is changed. So I make some function to gather most possible details from several filter runs. With deinterlace processing of QTGMC it also collect most of noise from different filter runs also. But it may be tested with SMDegrain too with different prefilters at single processing. The function GetSharpest() included into M_QTGMC listed at https://forum.doom9.org/showthread.php?p=2001388#post2001388 and it uses new plugin MostDiffVal.dll to run. Though I think maybe it can be implemented with all internals of AVS like Layer() instead of Average and Expr() instead of MostDiffVal().

So to run SMDegrain with a set of prefilters (any set may be combined in a chain of GetSharpest(clip1, clip2):


pref1=SMDegrain(prefilter=0)
pref2=SMDegrain(prefilter=1)

GetSharpest(pref1, pref2)

where GetSharpest() is

Function GetSharpest(clip c1, clip c2)
{
avg=Average(c1.GaussResize(c1.width, c1.height, src_left=0.001, src_top=0.001, p=2), 0.5, c2.GaussResize(c2.width, c2.height, src_left=0.001, src_top=0.001, p=2), 0.5)

return MostDiffVal(avg, c1, c2)
}


Why isn't it working as expected? It always shows 'c2'

c1=Sharpen(0.80).subtitle("c1")
c2=Sharpen(0.10).subtitle("c2")
GetSharpest(c1, c2)


Edit: The sharper one is returned, but not the subtitle. This confuses me, please clarify.

Edit2: Oh, a combination of both clips will probably be returned. When I use 'a' and 'f' as a subtitle I see that 'a' and 'f' are overlaid.

[DTL]

It works on sample-level granularity from both input clips. So Subtitle text will be mixed (and typically not very well processed as extreme 0 and max values). Better results on natural narrow/limited range.

For test you need to mask-sharpen c1 clip at one area and mask-sharpen c2 clip at different area and process to see if returned both sharpened areas from c1 and c2.

More sad thing about MAnalyse/MDegrain/MCompensate is with different areas grouped MVs errors it also cause different areas global shift (not only fine sharpness lost) so the GetSharpest() will also somehow mix these areas. And with different pre-filters these grouped MVs areas errors also different.

I typically check it as


c1=Process(params1).Subtitle("params1")
c2=Process(params2).Subtitle("params2", align=..some2, not default and not 4)

sh=GetSharpest(c1, c2).Subtitle("gs", align=4)

Interleave(c1, sh, c2)


And check different areas looking in c1 vs sh and c2 vs sh with frame step in VirtualDub.

[DTL]

Finally made replacement of all functions with AVS internal Expr() and ex_Tools for GetSharpest():


Function GetSharpest(clip c1, clip c2)
{
avg=Layer(c1.ex_GaussianBlur(1.8, UV=3), c2.ex_GaussianBlur(1.8, UV=3), "fast")

return Expr(avg, c1, c2, "x y - abs x z - abs >= y z ?")

#or some faster
#return MostDiffVal(avg, c1, c2)
}


The sigma param for GaussianBlur is subject to check for best results (or best for current user/footage/filters/ etc). Currently 1.8 is about equal to GaussResize(p=2) blur. Also compiled plugin MostDiffVal updated to v0.2 with AVX2 functions and opt SIMD manual control (optional or auto). With 8bit it runs about 2 times faster in comparison with Expr() - looks like because Expr() perform internal calculations with float32 and have penalty of convert from integer to float and back.

ex_GaussianBlur() runs about 1.5 times faster in comparison with GaussResize() for blurring. Also Layer("fast") is somehow faster in comparison with Average(0.5,0.5).

About twice faster with not yet tested quality change is single blur of averaged input clips:

Function GetSharpest(clip c1, clip c2)
{
avg=ex_GaussianBlur(Layer(c1, c2, "fast"),1.8, UV=3)

return Expr(avg, c1, c2, "x y - abs x z - abs >= y z ?")

#or some faster
#return MostDiffVal(avg, c1, c2)
}


Edit: With full chroma planes processing (UV=3 for ex_GaussianBlur) the GaussResize still some faster (looks like integer 2 pass V+H AVS resize kernel is faster, at least for not very large radius blur):


Function GetSharpest(clip c1, clip c2)
{
avg=GaussResize(Layer(c1, c2, "fast"), c1.width, c1.height, src_left=0.001, src_top=0.001, p=2)

return Expr(avg, c1, c2, "x y - abs x z - abs >= y z ?")

#or some faster
#return MostDiffVal(avg, c1, c2)
}

[LouieChuckyMerry]

I very recently transitioned from real.finder's last version of NotSMDegrain (3.1.2.116s) to Dogway's most recent version of SMDegrain (4.6.0d), running on Windows 7 with 64-bit AviSynth+ 3.73. It seems to work fine, but I'm trying to understand why the speed is basically the same (within ~5%) but the resulting video bitrate is radically different (roughly a full CRF). To wit, the scripts

SetFilterMTMode("Default_MT_Mode",2)
SMDegrain(TR=4,ThSAD=700,RefineMotion=True,Plane=0,Chroma=False,n16=True,n16_Out=True)
FastLineDarkenMod4()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
SelectRangeEvery(1000,66)
PreFetch(5)

and

SetFilterMTMode("Default_MT_Mode",2)
SMDegrain(Prefilter=SpotLess(),TR=3,ThSAD=500,RefineMotion=True,Plane=0,Chroma=False,n16=True,n16_Out=True)
FastLineDarkenMod4()
F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
SelectRangeEvery(1000,66)
PreFetch(5)

with the filters-plugins

F3KDB__64bit
FastLineDarkenMTMod1_48
LinearTransformation
MaskTools2_2_26_0MTDualSignatureMod_pinterf__64bit
MedianBlur2__1_1__64bit
MVTools2_7_41_0MTMod_pinterf__64bit
SharedFunctions1_153
SMDegrainMod_rf_3_1_2_116s
SpotLess1_07

results in bitrates of 3471 kb/s and 4209 kb/s, respectively, while the scripts

SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(Bits=16)
SMDegrain(TR=4,ThSAD=700,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
Neo_F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
SelectRangeEvery(1000,66)
PreFetch(5)

and

SetFilterMTMode("Default_MT_Mode",2)
ConvertBits(Bits=16)
SMDegrain(Prefilter=SpotLess(),TR=3,ThSAD=500,RefineMotion=True,Plane=0,Chroma=False)
FastLineDarkenPlus()
Neo_F3KDB(Y=100,Cb=100,Cr=100,GrainY=0,GrainC=0)
SelectRangeEvery(1000,66)
PreFetch(5)

with the filters-plugins

ExTools10_1
FastLineDarkenPlus
LinearTransformation
MaskTools2_2_30_0MTDualSignatureMod_pinterf__64bit
MedianBlur2__1_1__64bit
MVTools2_7_46_0MTMod_pinterf__64bit
Neo_F3KDB9_0__64bit
SMDegrain4_6_0d
SpotLess1_07

results in bitrates of 4165 kb/s and 5173 kb/s, respectively. The sources are the same. Any help understanding why the bitrates are so different would be greatly appreciated.

[Emulgator]

Less entropy, less changing high frequency details, and encoder can save bits.
It may not be visible at first, only pixel-peeping and framestepping will tell what those plugins were able to smoothe.
To go through all plugin's parameter's dependencies would make up a matrix which one should like to solve on his side.
From my side of experience I can tell that Dogway's algos give nice details which got to be honoured by bits...
and of course... many thanks, at least from my side.

[kedautinh12]

You can try with High bit depths to see more clearly about speed

[LouieChuckyMerry]

Less entropy, less changing high frequency details, and encoder can save bits.
It may not be visible at first, only pixel-peeping and framestepping will tell what those plugins were able to smoothe.
To go through all plugin's parameter's dependencies would make up a matrix which one should like to solve on his side.
From my side of experience I can tell that Dogway's algos give nice details which got to be honoured by bits...
and of course... many thanks, at least from my side.


Thank you for a very philosophical answer :) . And I also appreciate Dogway's work very much.


You can try with High bit depths to see more clearly about speed

Please, I don't understand what you mean.

[kedautinh12]

Lol, please search google about High bit depths video and try add converttobit(16) or converttobit(32) and compare both of them again. SMDegrain build by Dogway for speed with High bit depths video not for 8 bit video only