Dogway's Filters Packs

[DTL]

Oh good. On some sources with very coarse grain/noise a higher blocksize does a way better job and gives a more homogeneous look than thSAD and tr could achieve. Thanks DTL :thanks:

Mostly probably the large blocksize is good only for primary search and need refine (recalculate) to lower blocksize in second (other) pass/passes because large blocksize can not track complex motion good. thSAD and tr are tuning params for selected search/degrain architecture.
So usage of large blocksize is typically require 'refinemotion' in SMDegrain or other plugins. May be even not single refine but several more stages like blocksize 64->32->16->8. It may allow to work with lower thSAD params and get less detail smoothing.

[kedautinh12]

Hi anton_foy,

could you advice where i made mistake ?

in StaxRip i put this:
Import("E:\4k_Encode\UnBand.avsi")
Import("E:\4k_Encode\Avisynth-Scripts-master\ExTools.avsi")
Loadplugin("E:\4k_Encode\mvtools-2.7.45-with-depans20210608\x64\mvtools2.dll")
Loadplugin("E:\4k_Encode\ApplyEvery031_32_64\ApplyEvery64.dll")
LoadPlugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\Dual\f3kdb Neo\neo-f3kdb.dll")
loadplugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\AVS\FFT3DFilter\fft3dfilter.dll")
Uband(loC=3, grainY=23, mt=false)

but i got this error:
Script error: There is no function named 'si_physicalcores'

and is this change something even if i use f3kdb Neo?

You call lack some plugins and script, you can check from dependencies here
https://github.com/Dogway/Avisynth-Scripts/blob/57d4d0a9f3bb39753fef5314440767b848c27027/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.avsi#L30

[DTL]

hi,

how can this be used in StaxRip with SMDegrain?

Could you please provide some example for test purpose?

Try to set blksize above default and RefineMotion=true.

Docs at avisynth.nl need to be updated to correct link - current download link https://github.com/realfinder/AVS-Stuff/raw/master/avs%202.5%20and%20up/SMDegrain.avsi is not work more - return 404 error.

Current version of SMDegrain at Dogway github lists
[int "blksize"=16 (4 to 64 by 4)],
Assert(blksize==4 || blksize==8 || blksize==16 || blksize==24 || blksize==32 || blksize==48 || blksize==64,"SMDegrain: MAnalyse: Block's size must be 4x4, 8x8, 16x16, 24x24, 32x32, 48x48 or 64x64")

So it looks not limited to 16 as lists at http://avisynth.nl/index.php/SMDegrain but can be only 4x4, 8x8, 16x16, 24x24, 32x32, 48x48 or 64x64 for SMDegrain.

[anton_foy]

Hi anton_foy,

could you advice where i made mistake ?

in StaxRip i put this:
Import("E:\4k_Encode\UnBand.avsi")
Import("E:\4k_Encode\Avisynth-Scripts-master\ExTools.avsi")
Loadplugin("E:\4k_Encode\mvtools-2.7.45-with-depans20210608\x64\mvtools2.dll")
Loadplugin("E:\4k_Encode\ApplyEvery031_32_64\ApplyEvery64.dll")
LoadPlugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\Dual\f3kdb Neo\neo-f3kdb.dll")
loadplugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\AVS\FFT3DFilter\fft3dfilter.dll")
Uband(loC=3, grainY=23, mt=false)

but i got this error:
Script error: There is no function named 'si_physicalcores'

and is this change something even if i use f3kdb Neo?

For Uband you are e missing sys_info.dll (sysinfo plugin) but I can also update the uband function for you this evening for a better new version without the sysinfo plugin.

[LeXXuz]

So usage of large blocksize is typically require 'refinemotion' in SMDegrain or other plugins. May be even not single refine but several more stages like blocksize 64->32->16->8. It may allow to work with lower thSAD params and get less detail smoothing.

Yes I always use motion refinement. Otherwise flat texture areas could start to 'wobble' with too high blocksizes. Especially when there is some pumping involved in the noise.

[DTL]

Yes I always use motion refinement. Otherwise flat texture areas could start to 'wobble' with too high blocksizes. Especially when there is some pumping involved in the noise.

It looks 'interfiltering' of MVs is really good to make as separate mvtools filter so it can be included into any search filterchain (like as intermediate between large and refined blocksize search) and not only inside MDegrainN as final processig of MVs before usage.

[Dogway]

Just wanted to chime in to show how I deal with SMDegrain.
I downloaded a video from a streaming service and H264 artifacts are all over the place.
So my approach is to denoise and then regrain. It's not perfect but it was a fast cobble up to improve the perceived quality.

Original - Restored
http://i.imgur.com/qMBEiYCm.png (https://i.imgur.com/qMBEiYC.png)......http://i.imgur.com/gVwe95Dm.png (https://i.imgur.com/gVwe95D.png)
Line by line:
ConvertBits(16)
CCD(15) # Remove big chroma "blocks"

# Remove blocking and smooth out jagged edges (only applies where motion match fails)
mb=Deblock_QED(quant1=30, quant2=40,UV=1)
mb=mb.ex_smooth(1, mode="SG", limit=true, sharp=true)

# Prefiltering with radius 3 for more coherence. I found preset "normal" to clean edges better.
pre=ex_BM3D(sigma=10,preset="normal",radius=3,UV=1,gpuid=0,tv_range=true)

# Large radius for more coherence. LFR to recover some smearing. limits=false to don't sharpen motion areas
SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
# Debanding to reduce posterization in walls/defocused backgrounds
GradFun3plus(thr=0.25, radius=10, mask=2, smode=0,UV=1)

# Fine grain
str=0.7
size=0.8
GrainFactory3mod(g1str=6.0*str,g2str=8.0*str,g3str=5.5*str,g1size=1.20*size,g2size=1.50*size,g3size=1.40*size,g1cstr=0.5,g2cstr=0.3,g3cstr=0.1,temp_avg=1)

# Good quality dithering
fmtc_bitdepth(bits=8, dmode=7)

As you can see I don't use internal prefiltering. Internal prefiltering is simply a convenience, but you are supposed to build your own ones.
I also need to remove the TV bugs with inpaintdelogo.

By the way, SMDegrain up-to-date documentation is now here (https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.html). It's still work in progress though, but about 70% done.

[kedautinh12]

Why you use gradfun3flus replace f3kdb in debanding??

[tormento]

Line by line
Thanks :)
By the way, SMDegrain up-to-date documentation
❤️

[Dogway]

Why you use gradfun3flus replace f3kdb in debanding??

Looks worse. Simply try:

ex_GaussianBlur(10.00,pad=true,mblur=true,UV=3)
ex_posterize(bits=6, mode=-1)
ConvertBits(16)
# GradFun3plus(thr=0.6, radius=6, mask=1, smode=0)
neo_f3kdb(y=100,cb=48,cr=48,range=10)

ConvertBits(8,dither=1)
PseudoColor(mode="isoparam", iso=1.0)

[kedautinh12]

Looks worse. Simply try:

ex_GaussianBlur(10.00,pad=true,mblur=true,UV=3)
ex_posterize(bits=6, mode=-1)
ConvertBits(16)
# GradFun3plus(thr=0.6, radius=6, mask=1, smode=0)
neo_f3kdb(y=100,cb=48,cr=48,range=10)

ConvertBits(8,dither=1)
PseudoColor(mode="isoparam", iso=1.0)

F3KDB_3 can better than neo_f3kdb
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/F3KDB_s.avsi

[LeXXuz]

Just wanted to chime in to show how I deal with SMDegrain.

# Prefiltering with radius 3 for more coherence. I found preset "normal" to clean edges better.
pre=ex_BM3D(sigma=10,preset="normal",radius=3,UV=1,gpuid=0,tv_range=true)


What performance do you get with this?

I can't use radius >1 on 1080p or above with the normal preset. Even my RTX 3060ti slows down so much that it becomes the bottleneck. Not to mention my RTX 2060 and GTX 1050ti. I use avs_test09 version of BM3D. It's CUDA code surely could use some more speed optimization.
Right now I stick with DGDenoise(). It's lightning fast compared to BM3D, although it's a little unfair to compare block matching with non-local means algo.

[Dogway]

Now that you say probably very slow. I only tested on avspmod. I only got a 1070 (looking forward a 4060 Ti, prolly too expensive :( ) so probably should lower it to 2 or maybe 1.
My intention was to use 2 passes, encode lossless (or transparent) 16-bit after SMDegrain. Then final deband, regrain, and encode. And in 2 chunks (movie is 172 mins).

[Reel.Deel]

By the way, SMDegrain up-to-date documentation is now here (https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.html). It's still work in progress though, but about 70% done.

Nice! I went ahead and added the link to the wiki page since the docs there are outdated.

[guest]

Hi anton_foy,

could you advice where i made mistake ?

in StaxRip i put this:
Import("E:\4k_Encode\UnBand.avsi")
Import("E:\4k_Encode\Avisynth-Scripts-master\ExTools.avsi")
Loadplugin("E:\4k_Encode\mvtools-2.7.45-with-depans20210608\x64\mvtools2.dll")
Loadplugin("E:\4k_Encode\ApplyEvery031_32_64\ApplyEvery64.dll")
LoadPlugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\Dual\f3kdb Neo\neo-f3kdb.dll")
loadplugin("E:\4k_Encode\apps\StaxRip-v2.12.0-x64\Apps\Plugins\AVS\FFT3DFilter\fft3dfilter.dll")
Uband(loC=3, grainY=23, mt=false)

but i got this error:
Script error: There is no function named 'si_physicalcores'

and is this change something even if i use f3kdb Neo?

Hi madey83,

Now this isn't a Staxrip script, but I think you will soon see what the problem is.

This works for me (on most videos) :-

Loadplugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\ApplyEvery\ApplyEvery64.dll") works without this, too !!!
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\neo-f3kdb\neo-f3kdb.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\FFT3dFilter\fft3dfilter.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\SysInfo\SysInfo64.dll")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\UBand\Uband.avs")
video=Uband(video, loC=3, grainY=23, mt=false)


See here :- https://forum.doom9.org/showthread.php?p=1975087#post1975087

[madey83]

Hi madey83,

Now this isn't a Staxrip script, but I think you will soon see what the problem is.

This works for me (on most videos) :-

Loadplugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\ApplyEvery\ApplyEvery64.dll") works without this, too !!!
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\neo-f3kdb\neo-f3kdb.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\FFT3dFilter\fft3dfilter.dll")
LoadPlugin("%AVISYNTHPLUGINS%\PD_TOOLS\SysInfo\SysInfo64.dll")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\UBand\Uband.avs")
video=Uband(video, loC=3, grainY=23, mt=false)


See here :- https://forum.doom9.org/showthread.php?p=1975087#post1975087


thank you :)

[madey83]

Just wanted to chime in to show how I deal with SMDegrain.
I downloaded a video from a streaming service and H264 artifacts are all over the place.
So my approach is to denoise and then regrain. It's not perfect but it was a fast cobble up to improve the perceived quality.

Original - Restored
http://i.imgur.com/qMBEiYCm.png (https://i.imgur.com/qMBEiYC.png)......http://i.imgur.com/gVwe95Dm.png (https://i.imgur.com/gVwe95D.png)
Line by line:
ConvertBits(16)
CCD(15) # Remove big chroma "blocks"

# Remove blocking and smooth out jagged edges (only applies where motion match fails)
mb=Deblock_QED(quant1=30, quant2=40,UV=1)
mb=mb.ex_smooth(1, mode="SG", limit=true, sharp=true)

# Prefiltering with radius 3 for more coherence. I found preset "normal" to clean edges better.
pre=ex_BM3D(sigma=10,preset="normal",radius=3,UV=1,gpuid=0,tv_range=true)

# Large radius for more coherence. LFR to recover some smearing. limits=false to don't sharpen motion areas
SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
# Debanding to reduce posterization in walls/defocused backgrounds
GradFun3plus(thr=0.25, radius=10, mask=2, smode=0,UV=1)

# Fine grain
str=0.7
size=0.8
GrainFactory3mod(g1str=6.0*str,g2str=8.0*str,g3str=5.5*str,g1size=1.20*size,g2size=1.50*size,g3size=1.40*size,g1cstr=0.5,g2cstr=0.3,g3cstr=0.1,temp_avg=1)

# Good quality dithering
fmtc_bitdepth(bits=8, dmode=7)

As you can see I don't use internal prefiltering. Internal prefiltering is simply a convenience, but you are supposed to build your own ones.
I also need to remove the TV bugs with inpaintdelogo.

By the way, SMDegrain up-to-date documentation is now here (https://raw.githack.com/Dogway/Avisynth-Scripts/master/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.html). It's still work in progress though, but about 70% done.

@Dogway,

thank you so much for this.
i did few test with GradFun3plus and neo_f3kdb and i see that GradFun3plus does better job.

prefilter for SMDegrain is also good so this is my for now base for upcoming encodes.

i did not test this though yet:

CCD(15) # Remove big chroma "blocks"

# Remove blocking and smooth out jagged edges (only applies where motion match fails)
mb=Deblock_QED(quant1=30, quant2=40,UV=1)
mb=mb.ex_smooth(1, mode="SG", limit=true, sharp=true)

due to fact that these are new for me and i do not really know what is advanteges to call it before SMDegrain.

BTW: Thx for documentation update :thanks:

[Dogway]

neo_f3kdb seems to simply add noise instead of dithering so output is noisier and consumes more bitrate. Yes it's a few times faster but for me quality is first.

The lines you mention are source specific. In this case it's a heavily compressed clip so you want to get rid of blocking. CCD removes chroma macroblocks, while Deblock_QED removes luma blocking. Unfortunately Deblock_QED is not perfect so some high contrast lines (quant1 arg) are left jagged. I run ex_smooth() on top for that, it's a bit destructive but since it's passed through mfilter it will only be applied on motion areas.


####################

By the way, wanted to share some experiment I have been working on these days. Per-shot grain adjustment.

First you output some scene stats (use SceneStats (https://github.com/Dogway/Avisynth-Scripts/blob/master/ScenesPack.avsi), updated today)
# First Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

SceneStats(mode="Range+Motion",path="Q:\Range+Motion.log")

Prefetch(1)


Now we are going to detect grain level an a per-shot basis. One assumes that the difference between source and a degrained/denoised version estimates the amount of grain level, so that's what we do. First degrain, then compare source with degrained for a diff mask, and subtract edges from it. Export new stats (we want average diff mask values per-shot).

# Second Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

ReadStats("Q:\Range+Motion.log")
ExtractY()
dirt = last
# If this is too slow use a faster spatio-temporal denoiser
SMDegrain(1, 400, ContraSharp=false, RefineMotion=false, LFR=400)

gm = ex_edge (cln,mode="frei-chen", scale=1.0, invert=true).ex_inpand(2,mode="disk")
ex_lutxyz(dirt,gm,"x y - abs z range_max / *") # dirt mask
SceneStats(mode="Stats", path="Q:\Stats.log") # per-shot dirt mask average

Finally we load both stat files and do some parametrization. I intentionally lower grain level slightly when scene has higher motion.

# Third Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

ConvertBits(16)
CCD(15)
mb = Deblock_QED(quant1=30, quant2=40,UV=1)
mb = mb.ex_smooth(1, mode="SG", limit=true, sharp=true)
pre = ex_BM3D(sigma=10,preset="normal",radius=2,UV=1,gpuid=0,tv_range=true)

SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
GradFun3plus(thr=0.25, radius=10, mask=2, smode=0,UV=1)

ReadStats("Q:\Range+Motion.log")
ReadStats("Q:\Stats.log")

# per-shot modulated grain
ScriptClip(function [] () {

avg = propGetAsArray("_SceneStats")[4]
mot = propGetFloat ("_SceneMotion")
navg = avg / (mot + 1)
size = 2*navg + 0.4 # between 0.7 and 1.1 mostly
str = navg + 0.5 # between 0.6 and 0.8 mostly

GrainFactory3mod(g1str=6*str,g2str=8*str,g3str=7*str,g1size=1.20*size,g2size=1.50*size,g3size=1.40*size,g1cstr=0.5,g2cstr=0.3,g3cstr=0.1,temp_avg=1)
} )

ConvertBits(8, dither=1)

[tormento]

F3KDB_3 can better than neo_f3kdb
And what about speed?

[kedautinh12]

And what about speed?

Like Dogway said, prefer quality :D

[kedautinh12]

Here, little-bit faster ver
https://github.com/kedaitinh12/AVSPlus-Scripts/blob/master/F3KDB_sMOD.avsi

[LeXXuz]

mb = Deblock_QED(quant1=30, quant2=40,UV=1)
mb = mb.ex_smooth(1, mode="SG", limit=true, sharp=true)
SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)


That kinda interests me. I usually just use a simple blur for mfilter on sources with little blocking in high-motion areas . I'm talking about some Blu-ray sources. Not that really nasty stuff from broadcasting/streaming that very often craves for way more bitrate. ;)

I'd be interested in a higher quality solution than just blur. Could you give some tips on ex_smooth usage?

[Dogway]

That kinda interests me. I usually just use a simple blur for mfilter on sources with little blocking in high-motion areas . I'm talking about some Blu-ray sources. Not that really nasty stuff from broadcasting/streaming that very often craves for way more bitrate. ;)

I'd be interested in a higher quality solution than just blur. Could you give some tips on ex_smooth usage?

It depends on what your goal is. Maybe Deblock_QED is less destructive than a simple blur but this example is very case specific with a highly compressed source. For blurays maybe you can get away with some faint blur which helps for extra compression.

In my case I wanted the opposite, reconstruct the detail, so Deblock_QED for flat areas and ex_smooth for edge areas. ex_smooth is like a high quality blur, technically it's a polynomial, in practice it performs between a blur and an antialiaser. Helps also to smooth out binary masks, etc.

[DTL]

Well, here is something I'm really scratching my head on.

I'm still struggling with a bluring effect with SMDegrain in moving areas which really disturbs the overall look of the picture.


It is example with updated MDegrainN - https://send.cm/d/FAb2

With simple usage without prefiltering now -

ConvertToYUV420(matrix="Rec709")
super = MSuper(mt=false, pel=2)
tr=12
multi_vec=MAnalyse(super, blksize=8, multi=true, search=3, temporal=false, trymany=false, searchparam=2, chroma=true, delta = tr, truemotion=false, pzero=0, pnew=0, levels=2, mt=false, overlap=0)
MDegrainN(last,super, multi_vec, tr, thSAD=110, thSAD2=100, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.4, adjSADcohmv=0.7, thCohMV=2, MVLPFGauss=1.1,
thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3, MPBthSub=10, MPBthAdd=20, MPBNumIt=2, MPB_SPC=1.5)


Was lazy to wait for full 4x overlap at my very old home CPU so used lower quality interpolated.
Version of mvtools from release - https://forum.doom9.org/showthread.php?p=1975256#post1975256 . x264 with settings in metadata in mp4 file makes average speed about sub 7 MBit/s. Though x264 with crf=18 also delete some fine low-contrast details.

[LeXXuz]

It is example with updated MDegrainN - https://send.cm/d/FAb2

With simple usage without prefiltering now -


Impressive result. :eek: Thanks for sharing that script. Will try it out. :thanks:

[LeXXuz]

It depends on what your goal is. Maybe Deblock_QED is less destructive than a simple blur but this example is very case specific with a highly compressed source. For blurays maybe you can get away with some faint blur which helps for extra compression.

In my case I wanted the opposite, reconstruct the detail, so Deblock_QED for flat areas and ex_smooth for edge areas. ex_smooth is like a high quality blur, technically it's a polynomial, in practice it performs between a blur and an antialiaser. Helps also to smooth out binary masks, etc.

Is the threshold when the mfilter is used adjustable somehow?

I added a simple Greyscale() to the mfilter call and was quite surprised about all the greyed out areas in scenes I really didn't expect this to happen. :eek:

[Boulder]

Is the threshold when the mfilter is used adjustable somehow?

I added a simple Greyscale() to the mfilter call and was quite surprised about all the greyed out areas in scenes I really didn't expect this to happen. :eek:

From the docs: "This clip will be passed through those areas where smdegrain() couldn't find a matching block."

That points to thSAD - the higher thSAD is, the less non-matching blocks.

[madey83]

It is example with updated MDegrainN - https://send.cm/d/FAb2

With simple usage without prefiltering now -

ConvertToYUV420(matrix="Rec709")
super = MSuper(mt=false, pel=2)
tr=12
multi_vec=MAnalyse(super, blksize=8, multi=true, search=3, temporal=false, trymany=false, searchparam=2, chroma=true, delta = tr, truemotion=false, pzero=0, pnew=0, levels=2, mt=false, overlap=0)
MDegrainN(last,super, multi_vec, tr, thSAD=110, thSAD2=100, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.4, adjSADcohmv=0.7, thCohMV=2, MVLPFGauss=1.1,
thMVLPFCorr=50, adjSADLPFedmv=0.9, IntOvlp=3, MPBthSub=10, MPBthAdd=20, MPBNumIt=2, MPB_SPC=1.5)


Was lazy to wait for full 4x overlap at my very old home CPU so used lower quality interpolated.
Version of mvtools from release - https://forum.doom9.org/showthread.php?p=1975256#post1975256 . x264 with settings in metadata in mp4 file makes average speed about sub 7 MBit/s. Though x264 with crf=18 also delete some fine low-contrast details.


@DTL,

thank you for this, but could you please share what would be the best call, if the power of CPU does not matter but quality has matter only?

[DTL]

For simple script with single MAnalyse and no prefilter the only better is full 4x overlap (search and blend) and possibly pel=4 as better precision of search and blend. So

super = MSuper(mt=false, pel=4)
tr=12
multi_vec=MAnalyse(super, blksize=8, multi=true, search=3, temporal=false, trymany=false, searchparam=2, chroma=true, delta = tr, truemotion=false, pzero=0, pnew=0, mt=false, overlap=4)
MDegrainN(last,super, multi_vec, tr, thSAD=110, thSAD2=100, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9,
thMVLPFCorr=50, adjSADLPFedmv=0.9, MPBthSub=10, MPBthAdd=20, MPBNumIt=2, MPB_SPC=1.5)

Also it have some more 'universal' params settings not optimized for current content. Also remember to adjust thSAD,thSAD2,thSCD1,tr for current content and processing performance.

For possibly better result it require integration in SMDegrain script so it can use different prefiltering and also may need research if different prefiltering for 'current' and 'ref' clips may produce better results. Also may be single or more stages of 'refining blocksize' with MRecalculate may produce better results.

[madey83]

:thanks::script:For simple script with single MAnalyse and no prefilter the only better is full 4x overlap (search and blend) and possibly pel=4 as better precision of search and blend. So

super = MSuper(mt=false, pel=4)
tr=12
multi_vec=MAnalyse(super, blksize=8, multi=true, search=3, temporal=false, trymany=false, searchparam=2, chroma=true, delta = tr, truemotion=false, pzero=0, pnew=0, mt=false, overlap=4)
MDegrainN(last,super, multi_vec, tr, thSAD=110, thSAD2=100, mt=false, wpow=4, thSCD1=500, adjSADzeromv=0.5, adjSADcohmv=0.5, thCohMV=16, MVLPFGauss=0.9,
thMVLPFCorr=50, adjSADLPFedmv=0.9, MPBthSub=10, MPBthAdd=20, MPBNumIt=2, MPB_SPC=1.5)

Also it have some more 'universal' params settings not optimized for current content. Also remember to adjust thSAD,thSAD2,thSCD1,tr for current content and processing performance.

For possibly better result it require integration in SMDegrain script so it can use different prefiltering and also may need research if different prefiltering for 'current' and 'ref' clips may produce better results. Also may be single or more stages of 'refining blocksize' with MRecalculate may produce better results.

:thanks:

[DTL]

As processing of noisy VHS captures shows the settings like MPBNumIt=2, MPB_SPC=1.5 may be too aggressive and leave too much noise too. So if the residual noise level is too high even with big enough tr and thSAD values may be good either to try lower like MPBNumIt=1, MPB_SPC=1.2 settings or even make some linear mix of 2 processed clips - with MPBNumIt > 0 and with 'old' standard blending mode with MPBNumIt=0 (default) to make balance between residual noise and possible keeping of details. The MPB_SPC may be lowered to about 1.02 and still produce visible difference. It should be > 1.0.

[LeXXuz]

Dogway how do I use dfttest as mode with the 3.5.5d version of SMDegrain properly? Any info on the possible parameters?

A lot of parameters cause errors. Right now I just managed to do a simple call like SMDegrain(mode="dfttest", tr=3).
I know tbsize needs to be odd so tr has to be too, I guess?

[kedautinh12]

Dogway how do I use dfttest as mode with the 3.5.5d version of SMDegrain properly? Any info on the possible parameters?

A lot of parameters cause errors. Right now I just managed to do a simple call like SMDegrain(mode="dfttest", tr=3).
I know tbsize needs to be odd so tr has to be too, I guess?

You can use with prefilter=4
https://github.com/Dogway/Avisynth-Scripts/blob/23515d5ca598503c789fca687f8032846a58ac4c/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.avsi#L258

[LeXXuz]

You can use with prefilter=4
https://github.com/Dogway/Avisynth-Scripts/blob/23515d5ca598503c789fca687f8032846a58ac4c/SMDegrain%20v3.5.5d/SMDegrain%20v3.5.5d.avsi#L258

That is NOT what I asked. Version 3.5.5d supports dfttest as a denoising mode not just as a prefilter.

[Dogway]

I didn't notice that requirement. Yes, to make it behave smooth add the next into the dfttest call:
,tmode=(tr-1)%2
What other settings are giving you issues?

By the way, I'm still finishing TransformsPack, just did an overall rehaul to Transfers module. Will focus now on Main module to solve for the chroma location. Chromatic Adaptation fix is reserved for the following (last?) RC version, technically 1.0 final.
ResizersPack will benefit a lot from these so an update is also on due.

[madey83]

F3KDB_3 can better than neo_f3kdb
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/F3KDB_s.avsi

hi kedautinh12,

i did 3 tests:

#1) neo_f3kdb(preset="veryhigh/nograin", dynamic_grain=false, output_depth=10):
preset="veryhigh/nograin" == [y=80/cb=80/cr=80/grainy=0/grainc=0]
neo_f3kdb_8.30 fps_1858.66kbs_QP22.58

#2) GradFun3plus(thr=0.25, radius=10, mask=2, thr_det=2, elast=4, smode=2, UV=1):
GradFun3plus_5.17fps_1962kbs_QP22.42

#3) F3KDB_3(range=10, Y=100, Cb=48, Cr=48, grainY=80, grainC=80, sample_mode=2, dynamic_grain=false, opt=-1, keep_tv_range=true, output_depth=10, chroma=true):
F3KDB_3_8.03 fps_1957.32kbs_QP22.65


and i can see on my eyes that neo_f3kdb produced less details than F3KDB_3 and GradFun3plus.

Between F3KDB_3 and GradFun3plus i do not see any differents but speed is quit big.

i do not really know if this is correct comparision....

[LeXXuz]

I didn't notice that requirement. Yes, to make it behave smooth add the next into the dfttest call:
,tmode=(tr-1)%2

Thanks I'll add that. :)


What other settings are giving you issues?

Never mind that. I got confused with the parameters as I've been using neo_dfttest before which uses different syntax. Btw, you may consider using the neo-fork of dfttest. I've used that one with real.finders dfttestMC for quite a while and it was considerably faster than the original dfttest. Unless there has been a more speed optimized version since 1.9.7 in the meantime.


By the way, I'm still finishing TransformsPack, just did an overall rehaul to Transfers module. Will focus now on Main module to solve for the chroma location. Chromatic Adaptation fix is reserved for the following (last?) RC version, technically 1.0 final.
ResizersPack will benefit a lot from these so an update is also on due.
Very nice! Looking forward to it. :)

As a side note; I've encoded a film twice. Denoised with tr=6 and tr=12. Rest of the parameters identical.
(SMDegrain(mode="MDegrain", tr=6/12, thSAD=300, thSADc=200, contrasharp=true, LFR=true, DCTFlicker=true, refinemotion=true, truemotion=false, blksize=16, search=5, pel=2, subpixel=3, prefilter=3, chroma=true, plane=4))

The file encoded with tr=6 got smaller in total size than the version with tr=12. Not in a huge amount, but noticeably. Like 5.46GB vs 5.53GB for the entire film in 1080p.

I was wondering how to interpret this?
Would this mean the denoising with tr=12 kept more detail or interpreted less content as noise because of higher precision through the higher radius?
Since the tr=12 version took 2 hours longer to encode I was wondering if it's worth the time or merely just a waste? :rolleyes:

[real.finder]

neo_f3kdb produced less details than F3KDB_3

by default F3KDB_3 use neo_f3kdb(smode=4) also it use masking which the same used in GradFun3 (I think it also same in GradFun3plus)

[Dogway]

If you can run dfttest and neo_ benchmarks with different threads, and prefetch combinations I might consider it. When I benchmarked them last year speed difference was negligible (I think dfttest was a bit faster, not sure).

For the tr=12 case, it might be the case of ghosting probably? If not maybe the contrasharp is hitting back harder. You can test with a snippet with contrasharp=false.
For me personally I like keeping a half a second windows, tr=6 for a 24fps source, as it's not either too fast nor too slow for the eye to notice temporal denoising changes. It's a good pacing threshold to capture action.

[DTL]

@anton_foy,

something like this:
https://imgur.com/9VoMBEv

King's face and clothes are clean and fairly sharp (forground), but background behing him is blurred ( i know this is related to camera and recording technics) and it is very clean, without noise.

Is there anything that could separate forground and background and specify different denoise values?

You can check the 'IVS-mask' feature of that release - https://forum.doom9.org/showthread.php?p=1976044#post1976044

It not directly about 3D space fore/back separation but tried to separate sharp detailed areas from flat non-detailed to apply 'anti-blur'/detail saving/restoration' processing only to detailed areas to save from increasing noise on non-detailed.
To generate this mask the property of motion search engines used:
1. At sharp detailed areas the MVs are typically stable enough and random component in MVs length and direction angle is low enough (in completely static areas with ideal search result all MVs = (0,0) so zero length and (zero ? assumed) angle.
2. At flat non scene-detailed but noise added areas the motion search engine typically output a sequence of random enough MVs (in theory the length of each MV may be limited only by search radius) and typical angle is very random too.
So current algo of IVS-mask (it is from 'Is Vectors Stable') calculates acceleration in the MVs from past to the future and angle difference and after multiplication creates some metric of MVs stability for current block to compare with user-provided threshold to make decision if this area are mostly noise without details (and typical degrain settings/processing may be applied) or it is possibly detailed area and details keepig/saving processing may be selected.
If this mask will be good for some other outside MDegrain scripting it may be added special mode of MDegrain to output only 2levels of greyscale mask from this algorithm. Currently it is used only for threshold adjustment and only assumed as detailed blocks are marked with zero-black (not any nice but quick method to mark something - slow but not used in production processing - only for params control/setup).
Currently the generated mask is not very precisely tracks the border between sharp and blurred areas may be because some motion-search engines trying to use area-adaptive methods of blocks tracking with inter-blocks dependencies (like truemotion=true in MAnalyse) so blocks MVs search are not completely independent. So for best results of IVS-mask it is recommended special inter-blocks-independent search mode of MAnalyse (like truemotion = false preset, zero penalties, may be lambda special adjustment too and so on) and seperate input in MDegrain provided if user want to use other search methods for main degraining work (like prefiltered clip for MAnalyse).

[madey83]

You can check the 'IVS-mask' feature of that release - https://forum.doom9.org/showthread.php?p=1976044#post1976044

It not directly about 3D space fore/back separation but tried to separate sharp detailed areas from flat non-detailed to apply 'anti-blur'/detail saving/restoration' processing only to detailed areas to save from increasing noise on non-detailed.
To generate this mask the property of motion search engines used:
1. At sharp detailed areas the MVs are typically stable enough and random component in MVs length and direction angle is low enough (in completely static areas with ideal search result all MVs = (0,0) so zero length and (zero ? assumed) angle.
2. At flat non scene-detailed but noise added areas the motion search engine typically output a sequence of random enough MVs (in theory the length of each MV may be limited only by search radius) and typical angle is very random too.
So current algo of IVS-mask (it is from 'Is Vectors Stable') calculates acceleration in the MVs from past to the future and angle difference and after multiplication creates some metric of MVs stability for current block to compare with user-provided threshold to make decision if this area are mostly noise without details (and typical degrain settings/processing may be applied) or it is possibly detailed area and details keepig/saving processing may be selected.
If this mask will be good for some other outside MDegrain scripting it may be added special mode of MDegrain to output only 2levels of greyscale mask from this algorithm. Currently it is used only for threshold adjustment and only assumed as detailed blocks are marked with zero-black (not any nice but quick method to mark something - slow but not used in production processing - only for params control/setup).
Currently the generated mask is not very precisely tracks the border between sharp and blurred areas may be because some motion-search engines trying to use area-adaptive methods of blocks tracking with inter-blocks dependencies (like truemotion=true in MAnalyse) so blocks MVs search are not completely independent. So for best results of IVS-mask it is recommended special inter-blocks-independent search mode of MAnalyse (like truemotion = false preset, zero penalties, may be lambda special adjustment too and so on) and seperate input in MDegrain provided if user want to use other search methods for main degraining work (like prefiltered clip for MAnalyse).



HI DTL, thank you for that, but to be honest this is to advanced for me. I think i'll stick with SMDegrain or try to learn more about MDegrainN and of course observe the discussion on this topic.... :)

[ENunn]

I'm getting this error whenever I use QTGMC+ with AVISource. I started to get this error after updating it.
https://i.imgur.com/05y3lPt.png

This doesn't happen when I load the video with LSMASH, but loading and scrubbing is just way too slow. Is there a fix?

[Dogway]

I don't recommend you loading interlaced with avisource since it doesn't index the frames.
You can try LSmash, FFMS2, DGIndex or DGDecNV. I prefer the later two because it will detect parity swaps on interlaced content.

[LeXXuz]

I get an error with NNEDI3 (pel=4 & subpixel=4 and set gpuid=-1), I can't pinpoint and need some help:
https://abload.de/thumb/pel4_errormqfvy.png (https://abload.de/image.php?img=pel4_errormqfvy.png)


Furthermore with NNEDI3CL (set gpuid=0), I get the following error:
https://abload.de/thumb/pel4_cl_error3gi5e.png (https://abload.de/image.php?img=pel4_cl_error3gi5e.png)

SMDegrain call:

SMDegrain(mode="MDegrain", tr=6, thSAD=600, thSADc=600, thSCD1=550, contrasharp=200, limitS=true, CClip=ORG, LFR=300, DCTFlicker=true, refinemotion=true, truemotion=false, pel=4, subpixel=4, gpuid=-1/0, prefilter=3, chroma=true, plane=4)



NNEDI3 v0.9.4.60 and NNEDI3CL 1.0.4.0 installed.
Source video is 720x576@25.

pel=4 & subpixel=1/2/3 is working without errors.

[Dogway]

Ok, I think it has to do with the slated updates on ResizersPack because it's working fine on my internal version.
I've been dawdling a bit but hope to release ASAP.

Currently I was testing the new modes in latest RIFE update, specially useful for the faster modes.
Does anyone got the halos_ghosts_judder (https://forum.doom9.org/showthread.php?p=1941970#post1941970)clip? I want to pass it through FrameRateConverterMIX.

[tormento]

@Dogway

Is there a way with the newer Asd-g filter such as RIFE (or anything else) to interpolate frames in reduced real frame material such as anime?

They usually have 23.976 stream fps but the frames are far less, often repeated.

I'd like to match stream fps with "real" frame fps, i.e. replace all duplicated frames with interpolated ones.

[Dogway]

Did you try with "RIFEAnime" preset? I haven't checked though, as I implemented as MysteryX dictated, but I'm currently testing with some live action samples and I don't like the masking logic.

The masking logic treats RIFE as a fallback for MFlowFps ver.1 and MFlowFps ver.2 difference mask. This means, if both match (regardless a match of crap blending or not) the ver.1 is chosen.

I'm now going to reformulate that to pass RIFE always on high motion areas with a motion mask, and then fallback to above logic where RIFE might also be used when both MFlowFps versions don't match.

I want to test this with the halos_ghosts_judder clip. But I think it might also work with Anime, I will give it a look.

[kedautinh12]

Did you try with "RIFEAnime" preset? I haven't checked though, as I implemented as MysteryX dictated, but I'm currently testing with some live action samples and I don't like the masking logic.

The masking logic treats RIFE as a fallback for MFlowFps ver.1 and MFlowFps ver.2 difference mask. This means, if both match (regardless a match of crap blending or not) the ver.1 is chosen.

I'm now going to reformulate that to pass RIFE always on high motion areas with a motion mask, and then fallback to above logic where RIFE might also be used when both MFlowFps versions don't match.

I want to test this with the halos_ghosts_judder clip. But I think it might also work with Anime, I will give it a look.

If it's bug with live action, you can report to Asd-g

[Dogway]

No, it was a problem with FrameRateConverter, but to be honest, I don't see a point on it anymore. RIFE is mostly always better than FRC, and when not the huge performance drop of FRC isn't worth it.

I made a wrapper for RIFE, will add soon to ResizersPack.

#
# OpticalFlowRIFE()
#
# Wrapper around RIFE port by Asd-g
#
# Dependencies:
# RIFE
# ExTools
# TransformsPack
#
function OpticalFlowRIFE(clip a, int "model", int "FrameNum", int "FrameDen", bool "FrameDouble", bool "Anime", int "gpuid") {

isy = isy (a)
rgb = isRGB (a)
rgbp = isPlanarRGB (a)

numD = Defined(FrameNum)
denD = Defined(FrameDen)

num = propNumElements (a,"_DurationDen") > 0 ? \
propGetInt (a,"_DurationDen") : FrameRateNumerator(a)
den = propNumElements (a,"_DurationNum") > 0 ? \
propGetInt (a,"_DurationNum") : FrameRateDenominator(a)
fs = propNumElements (a,"_ColorRange" ) > 0 ? \
propGetInt (a,"_ColorRange" ) == 0 : rgb

pID = color_propGet(a)
bi = pID[7]
pri = pID[2]
mat = "PC"+pri
isHD = (width (a) > 1099 || height(a) > 599)

ver = VersionString()
Assert(Eval(MidStr(ver,FindStr(ver,"(r")+2,4))>=Eval("3689"),"OpticalFlowRIFE: Update AviSynth+ version")

num = Default(FrameNum, num)
den = Default(FrameDen, den)
md = Default(model, 22) # model: 3 - for Anime or lineart, 17,6,22 - for photos or live action (in ascending order of qual/speed)
fd = Default(FrameDouble, !(numD || denD))
an = Default(Anime, false)
gid = Default(gpuid, 0)

ConvertBits(a, 32, fulls=fs, fulld=true)
rgb ? rgbp ? last : ConvertToPlanarRGB() : isy ? MergeRGB(last,last,last) : ConvertToPlanarRGB(matrix=mat,chromaresample="lanczos4")

RIFE(model=fd ? md : max(md,9), sc=true, skip=an, fps_num=fd ? Undefined() : num, fps_den=fd ? Undefined() : den, tta=false, gpu_id=max(0,gid), gpu_thread=isHD ? 4 : 2)

isy ? ExtractR() : \
!rgb ? MatchClip(a,matrix=pri,size=false,props=false) : last
fd ? Interleave(a.propSet("_DurationDen",den*2), SelectOdd()) : last

propCopy(a,true,props=["_DurationNum","_DurationDen","_PictType"], exclude=true) }

[tormento]

But I think it might also work with Anime, I will give it a look.
Please keep us informed and, when ready, show us some script examples.