Dogway's Filters Packs

[Boulder]

Would it be possible to make deep_resize use linear space scaling for downscales by default?

@Dogway,

coud it be an option to add paramiter to SMDegrain to look ahead, let say 24 frames and calculate some avarage value for thSAD to be more "dynamically" insted fix value?
i think this is used for X265 encoder: option: --rc-lookahead - of course i could be wrong.....

[tormento]

@Dogway

I think I have missed some SMDegrain iteration but did you add support for DTL cpu and/or gpu mvtools?

[Dogway]

No I haven't, I think I was waiting for support of blocksize of 16, not sure if that's possible now? Does he have a git?
If so I can make a fork version but I don't think I can benchmark it as I don't have DX12 (Win7 here), will install W10 in a month though, when 22H2 arrives.

@madey83: You don't need to lookahead, you can access the whole scene range stats with ScenesPack, without overflowing into adjacent scenes.
Here's an example for SMDegrain.
And here's another one for GrainFactory3mod so you can get the gist of it.

You can get an idea of the GrainFactory3mod example to derive the thSAD for SMDegrain example.

Originally Posted by Boulder

Would it be possible to make deep_resize use linear space scaling for downscales by default?

Not by default, you can mimic linear space downscaling with SSIM2 kernel, but I can add an option to resize in linear space if it doesn't modify much the structure of the function.

I'm off now to work a little.

[LeXXuz]

What do you use for halo removal on bad DVD sources?

Tried Blinddehalo and Dehaloalpha. But all they did was (over)smoothing the picture no matter what parameters I changed. Although their edgemasks looked fine, the result was poor.

[tormento]

Originally Posted by Dogway

No I haven't, I think I was waiting for support of blocksize of 16, not sure if that's possible now? Does he have a git?

https://github.com/DTL2020/mvtools

Plus a couple of threads in Avisynth Development subforum.

Originally Posted by Dogway

as I don't have DX12

There are a CPU branch and a GPU one, with different (almost) capabilities each.

[DTL]

"CPU branch and a GPU one, with different (almost) capabilities each."

The current sources are single now. The build may be created with DX12_ME define so it will be Win10 (DX12 and later) compatible to load mvtools2.dll and its DX12 dependencies. And this build can use optSearchOption 5 or 6 for MAnalyse for hardware if found. If build without DX12_ME compiler define it will be standard mvtools2 (may be only no 32bit directly because of a few new functions in MAnalyse need to be fixed). optSearchOption 5 or 6 for MAnalyse will throw error message to mark it was no DX12 build. Compatible with Win7 at least.

Block size 16x16 with DX12ME runs somehow buggy and my remote debug do not breaks at crash. May be it is local non-compatibility with my Win 10 + GTX1060 or bug somewhere. So I not use this. Only 8x8. So we need some more developers with complete build and debug system with Win10 and HW accelerator to try to check what cause crash.

Much more sad for Dogway public scripts may be next notes:
1. Currently 2.7.45 mvtools2 version from pinterf was fine tuned for many years to fix possible bugs and support many blocksizes and bitdepths and colour formats. But current my builds have many new features but support only very limited blocksizes/bitdephts/colour formats. Some combinations not implemented, some will use slow C-reference. Some may simply crash. Most new features were only tested at 8x8 8bit YV12.
2. The most sad issue for plugins users - the 2.7.45 and my builds can not be separated in single process because they use equal functions names. And so it looks not possible to run some complex scripts without modifications (like QTGMC). So my build can not be complete replacement of 2.7.45 version - it may cause bugs somewhere (may be crashes). The only typical use case is keep it in current working directory and load as local .dll (not from common plugins folder). Currently it is mostly somehow working tech demo and can be used for production only in very limited scenarios (like simple degrain script). All other use cases of mvtools of 2.7.45 version may or may not work.

Most of new features are about onCPU processing now so not use accelerator. May be I will add SSIM (and/or some more) metric to compute shader someday if tests will show some good use cases for it. Currently I not found them. The SSIM metric only shows some significantly different resulting MVs with onCPU MAnalyse (though direct MDegrain with there MVs also not visibly better) so currently planned use case is higher quality IVS mask of using SAD and SSIM different searches and combined metric from MVs difference. It is to be tested in future.
Most of DX12-dependent simple search features is finished many months ago (planned some fix to compute shader to have more equal SAD result with onCPU but still not finished).

Originally Posted by Dogway

I don't have DX12 (Win7 here), will install W10 in a month though, when 22H2 arrives.

FYI, Windows 10 22H2 has been "out" for a month or more...

[DTL]

Minimum required Windows version is looks like 19H1 - https://devblogs.microsoft.com/direc...on-estimation/ from May 2019.

Originally Posted by Dogway

No I haven't, I think I was waiting for support of blocksize of 16, not sure if that's possible now? Does he have a git?
If so I can make a fork version but I don't think I can benchmark it as I don't have DX12 (Win7 here), will install W10 in a month though, when 22H2 arrives.

@madey83: You don't need to lookahead, you can access the whole scene range stats with ScenesPack, without overflowing into adjacent scenes.
Here's an example for SMDegrain.
And here's another one for GrainFactory3mod so you can get the gist of it.

You can get an idea of the GrainFactory3mod example to derive the thSAD for SMDegrain example.



Not by default, you can mimic linear space downscaling with SSIM2 kernel, but I can add an option to resize in linear space if it doesn't modify much the structure of the function.

I'm off now to work a little.

@Dogway,

thank you answering.

i tried to test the code you pointed me to, but it requires Avisynth 3.7.3 which is not available for download.

[kedautinh12]

Originally Posted by madey83

@Dogway,

thank you answering.

i tried to test the code you pointed me to, but it requires Avisynth 3.7.3 which is not available for download.

Here:
https://gitlab.com/uvz/AviSynthPlus-Builds

[kedautinh12]

Originally Posted by Dogway

neo_f3kdb seems to simply add noise instead of dithering so output is noisier and consumes more bitrate. Yes it's a few times faster but for me quality is first.

The lines you mention are source specific. In this case it's a heavily compressed clip so you want to get rid of blocking. CCD removes chroma macroblocks, while Deblock_QED removes luma blocking. Unfortunately Deblock_QED is not perfect so some high contrast lines (quant1 arg) are left jagged. I run ex_smooth() on top for that, it's a bit destructive but since it's passed through mfilter it will only be applied on motion areas.


####################

By the way, wanted to share some experiment I have been working on these days. Per-shot grain adjustment.

First you output some scene stats (use SceneStats, updated today)

# First Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

SceneStats(mode="Range+Motion",path="Q:\Range+Motion.log")

Prefetch(1)

Now we are going to detect grain level an a per-shot basis. One assumes that the difference between source and a degrained/denoised version estimates the amount of grain level, so that's what we do. First degrain, then compare source with degrained for a diff mask, and subtract edges from it. Export new stats (we want average diff mask values per-shot).

# Second Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

ReadStats("Q:\Range+Motion.log")
ExtractY()
dirt = last
# If this is too slow use a faster spatio-temporal denoiser
SMDegrain(1, 400, ContraSharp=false, RefineMotion=false, LFR=400)

gm  = ex_edge  (cln,mode="frei-chen", scale=1.0, invert=true).ex_inpand(2,mode="disk")
ex_lutxyz(dirt,gm,"x y - abs z range_max / *") # dirt mask
SceneStats(mode="Stats", path="Q:\Stats.log")  # per-shot dirt mask average

Finally we load both stat files and do some parametrization. I intentionally lower grain level slightly when scene has higher motion.

# Third Pass
setmemorymax(2048*3)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

ConvertBits(16)
CCD(15)
mb  = Deblock_QED(quant1=30, quant2=40,UV=1)
mb  = mb.ex_smooth(1, mode="SG", limit=true, sharp=true)
pre = ex_BM3D(sigma=10,preset="normal",radius=2,UV=1,gpuid=0,tv_range=true)

SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
GradFun3plus(thr=0.25, radius=10, mask=2, smode=0,UV=1)

ReadStats("Q:\Range+Motion.log")
ReadStats("Q:\Stats.log")

# per-shot modulated grain
ScriptClip(function [] () {

    avg  = propGetAsArray("_SceneStats")[4]
    mot  = propGetFloat  ("_SceneMotion")
    navg = avg / (mot + 1)
    size = 2*navg + 0.4  # between 0.7 and 1.1 mostly
    str  =   navg + 0.5  # between 0.6 and 0.8 mostly

    GrainFactory3mod(g1str=6*str,g2str=8*str,g3str=7*str,g1size=1.20*size,g2size=1.50*size,g3size=1.40*size,g1cstr=0.5,g2cstr=0.3,g3cstr=0.1,temp_avg=1)
    } )

ConvertBits(8, dither=1)

You can trans it for one pass encode??

[Boulder]

Originally Posted by Dogway

Not by default, you can mimic linear space downscaling with SSIM2 kernel, but I can add an option to resize in linear space if it doesn't modify much the structure of the function.

Hmm, I figured it would only mean calling ConvertFormat with the appropriate scale_space parameter. Of course, if deep_resize could pass that parameter on, it would require only a minor change (I would then just add it to my template).

Originally Posted by kedautinh12

Here:
https://gitlab.com/uvz/AviSynthPlus-Builds

Thank you

[Dogway]

@Boulder: I had a look, it works so unless you use nnedi3. I will study the possibility to call nnedi3wrap through ConvertFormat.

Originally Posted by TDS

FYI, Windows 10 22H2 has been "out" for a month or more...

Those are insider's preview releases, AFAIK the stable release is happening at the end of the month.
Will surely be updated in the wiki when that happens.

@kedautinh12: It's basically the next script, but good luck making it run, it iteratively runs out-of-order calls with the several SceneStats and SMDegrain calls.
You are better off creating a batch script that execute each pass back-to-back, it will run orders of magnitude faster.

# Denoise highly compressed grainy MP4 and regrain back
# First Pass
setmemorymax(2048*4)
DGSource("Q:\source.dgi",cl=0,ct=0,cr=0,cb=0)

SceneStats(mode="Range+Motion")

# Second Pass
ExtractY()
dirt = last
# If this is too slow use a faster spatio-temporal denoiser
SMDegrain(1, 400, ContraSharp=false, RefineMotion=false, LFR=400)

gm  = ex_edge  (cln,mode="frei-chen", scale=1.0, invert=true).ex_inpand(2,mode="disk")
ex_lutxyz(dirt,gm,"x y - abs z range_max / *") # dirt mask
SceneStats(mode="Stats")  # per-shot dirt mask average

# Third Pass
ConvertBits(16)
CCD(15)
mb  = Deblock_QED(quant1=30, quant2=40,UV=1)
mb  = mb.ex_smooth(1, mode="SG", limit=true, sharp=true)
pre = ex_BM3D(sigma=10,preset="normal",radius=2,UV=1,gpuid=0,tv_range=true)

SMDegrain(6, 400, prefilter=pre, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
# Debanding, optional
GradFun3plus(thr=0.25, radius=10, mask=2, smode=0,UV=1)

# per-shot modulated grain
ScriptClip(function [] () {

    avg  = propGetAsArray("_SceneStats")[4]
    mot  = propGetFloat  ("_SceneMotion")
    navg = avg / (mot + 1)
    size = 2*navg + 0.4  # between 0.7 and 1.1 mostly
    str  =   navg + 0.5  # between 0.6 and 0.8 mostly

    GrainFactory3mod(g1str=6*str,g2str=8*str,g3str=7*str,g1size=1.20*size,g2size=1.50*size,g3size=1.40*size,g1cstr=0.5,g2cstr=0.3,g3cstr=0.1,temp_avg=1)
    } )

ConvertBits(8, dither=1)

[Boulder]

Originally Posted by Dogway

@Boulder: I had a look, it works so unless you use nnedi3. I will study the possibility to call nnedi3wrap through ConvertFormat.

My tests did show a difference between the default value "gamma" and "linear" for scale_space in ConvertFormat when using deep_resize to downscale. I've only used the Zopti presets in deep_resize so NNEDI3 is not involved there.

Originally Posted by Dogway

Those are insider's preview releases, AFAIK the stable release is happening at the end of the month.
Will surely be updated in the wiki when that happens.

Oh, OK, well, I was obviously misled there...

But why don't you just go to W11 22H2, that's what I use, and it's great, especially once you "tweak" it up

FrameRateConverterMIX

Would someone be so kind as to give me a basic explanation of what this can do ?

With maybe a simple script

And is it different to this :-

http://avisynth.nl/index.php/SVPflow

[kedautinh12]

Originally Posted by TDS

FrameRateConverterMIX

Would someone be so kind as to give me a basic explanation of what this can do ?

With maybe a simple script

And is it different to this :-

http://avisynth.nl/index.php/SVPflow

Full info for you
https://github.com/mysteryx93/FrameRateConverter

Originally Posted by kedautinh12

Full info for you
https://github.com/mysteryx93/FrameRateConverter

Thanks K, but I have been there several times already.

I'd actually like to know what it is used for...

But as per usual, I can't get it to work for me.

I doubt that I would use it....anyway.