Dogway's Filters Packs

[Guest]

From my experience: KNLmeansCL and BM3D surely will run faster on a RTX 30 card. DGDenoise needs a Nvidia GPU to run at all anyway.

If you think about buying one, do NOT go for a 3050 or 3060 8GB with 128bit memory bus. They are significantly slower than a 3060 with 192bit mem bus. Learned this the hard way.

Well, fortunately I don't use those filters...I'm more curious if even using the "lesser" filters, like MinBlur, and just basic SMDegrain filters use the GPU or CPU...from what I've been able to understand, the gpuid of SMDegrain by default is "0", which is a GPU.

And I just checked my RTX's, and they are only 128bit :(

[LeXXuz]

Well, fortunately I don't use those filters...I'm more curious if even using the "lesser" filters, like MinBlur, and just basic SMDegrain filters use the GPU or CPU...from what I've been able to understand, the gpuid of SMDegrain by default is "0", which is a GPU.


SMDegrain does not use GPU in general, the setting may be a little misleading.

Only for some prefilters like the aforementioned and subpixel processing if you explicitly use subpixel=4 (with gpu=0) which means you use GPU based NNEDI3CL. See its dependencies section at the very beginning of the script.

If you don't use subpixel=4, or prefilter=6,7,8 it will not make any difference if you set gpu=0 or -1. Everything else is CPU processed to my knowledge.

I think Dogway set gpu=0 by default, so unexperienced users will use the faster GPU based processing automatically IF they choose to use any of those prefilters or subpixel processing.


And I just checked my RTX's, and they are only 128bit :(

Don't get me wrong. They will still be faster than processing BM3D or KNLM entirely on CPU. But they are not as fast as a 3060 12GB / 3060TI 8GB and above.
My advice was only for people who think about buying a new graphics card. For those it means: stay away from RTX 3050 and RTX 3060 8GB. Buy a RTX 3060 12GB or RTX 3060TI 8GB instead. ;)

[Guest]

SMDegrain does not use GPU in general, the setting may be a little misleading.

Only for some prefilters like the aforementioned and subpixel processing if you explicitly use subpixel=4 (with gpu=0) which means you use GPU based NNEDI3CL. See its dependencies section at the very beginning of the script.

If you don't use subpixel=4, or prefilter=6,7,8 it will not make any difference if you set gpu=0 or -1. Everything else is CPU processed to my knowledge.

I think Dogway set gpu=0 by default, so unexperienced users will use the faster GPU based processing automatically IF they choose to use any of those prefilters or subpixel processing.



Don't get me wrong. They will still be faster than processing BM3D or KNLM entirely on CPU. But they are not as fast as a 3060 12GB / 3060TI 8GB and above.
My advice was only for people who think about buying a new graphics card. For those it means: stay away from RTX 3050 and RTX 3060 8GB. Buy a RTX 3060 12GB or RTX 3060TI 8GB instead. ;)

Thankyou for all that info...and like I said, I very rarely use those higher filters, and I'm quite happy with the newer Median options recently added, so my RTX's will be fine...anything more hi-end, is so much more $'s :(

[madey83]

Yes, you can adjust the "strength" by adjusting the opacity of the blend somewhere between 0 (no denoise filter effect) and 1.0 (full denoise filter effect)

But if you have just some fine grain better use something different like STTWM. Dogway provided examples for that some posts above and they can also be found in his ExTools package ~line 2194. :)

hi LeXXuz,

to use STTWM as prefilter i should skip below
pre=ex_Median(mode="IQMST",thres=155)
ex_blend(pre,"blend",opacity=0.3).ex_sbr(1,UV=3)

and replace it by one of below?

# STTWM(sw=20,tw=80,aw=100,sthres=255,tthres=7) # Strong temporal
# STTWM(sw=40,tw=60,aw=100,sthres=255,tthres=5) # Strong spatial
pre= STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5) # Balanced
# STTWM(sw=50,tw=50,aw= 10,sthres= 5,tthres=5) # Light

[LeXXuz]

hi LeXXuz,

to use STTWM as prefilter i should skip below
pre=ex_Median(mode="IQMST",thres=155)
ex_blend(pre,"blend",opacity=0.3).ex_sbr(1,UV=3)

and replace it by one of below?

# STTWM(sw=20,tw=80,aw=100,sthres=255,tthres=7) # Strong temporal
# STTWM(sw=40,tw=60,aw=100,sthres=255,tthres=5) # Strong spatial
pre= STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5) # Balanced
# STTWM(sw=50,tw=50,aw= 10,sthres= 5,tthres=5) # Light

Yes. And if you want to use it as prefilter for SMDegrain I would use it without ex_blend for most cases as it is way more subtle than ex_median with IQMST setting.

[madey83]

Yes. And if you want to use it as prefilter for SMDegrain I would use it without ex_blend for most cases as it is way more subtle than ex_median with IQMST setting.


i'm not really sure what you mean. Could you please provide example code for your suggestion?

i see from my test that this pre= STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5) # Balanced

have more details but keep also some grain compare to below

pre=ex_Median(mode="IQMST",thres=15)
ex_blend(pre,"blend",opacity=0.2).ex_sbr(1,UV=3)

i'm very happy with resoults of this STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5) # Balanced, but i would like to know how to adjust it strengh.

STTWM i can adjust it strengh by rising these sthres= 5, tthres=5 ?

[LeXXuz]

If you use it as a prefilter within SMDegrain you may use this:


pre=STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5).ex_sbr(1,UV=3)
SMDegrain("your settings here", prefilter=pre)


STTWM is way less invasive so I don't think there is need for the additional strength adjustment via ex_blend like you may had to use with IQMST before.

If the filtering is too weak try raising "thSAD" and/or "tr" of SMDegrain a little.
If that is not enought the grain of your source may be too strong for STTWM or you need a different prefilter.

In that case try ex_Minblur or ex_Median with "IQM" via the internal prefilters 0-3 if IQMST is too strong and STTWM is too weak for your source.

[madey83]

If you use it as a prefilter within SMDegrain you may use this:


pre=STTWM(sw=50,tw=50,aw=100,sthres= 5,tthres=5).ex_sbr(1,UV=3)
SMDegrain("your settings here", prefilter=pre)


STTWM is way less invasive so I don't think there is need for the additional strength adjustment via ex_blend like you may had to use with IQMST before.

If the filtering is too weak try raising "thSAD" and/or "tr" of SMDegrain a little.
If that is not enought the grain of your source may be too strong for STTWM or you need a different prefilter.

In that case try ex_Minblur or ex_Median with "IQM" via the internal prefilters 0-3 if IQMST is too strong and STTWM is too weak for your source.


:thanks:

edit: Results are very nice with higher TR :)

[madey83]

For sharpening i use this call:
LSFplus(strength=50, preset="LSF", secure=true, Lmode=3, soft=-2, soothe=true, Smode=3, Smethod=3, Szrp=16, Spwr=4, SdmpLo=4, SdmpHi=48, overshoot=1, undershoot=1, Overshoot2=1, Undershoot2=1, keep=20, edgemode=0)

is it any possibility to improve it or replace it by better approach for sharpening?

Could you advice please.... :)

[LeXXuz]

I hardly use Limited Sharpen anymore. I don't like the results, especially on HD sources.

I use contrasharp=true within SMDegrain. And if that is too strong, something more subtle like ex_unsharp() with lower settings after SMDegrain.

try:

SMDegrain(...)
ex_unsharp(0.3) for clean anime

and a double call for films, like:

SMDegrain(...)
ex_unsharp(0.15).ex_unsharp(0.15, Fc=width()/1.5)

The first example sharpens the entire picture the same way, while the 2nd sharpens higher frequencies less to prevent residual grain from being enhanced too much again.

[madey83]

I hardly use Limited Sharpen anymore. I don't like the results, especially on HD sources.

I use contrasharp=true within SMDegrain. And if that is too strong, something more subtle like ex_unsharp() with lower settings after SMDegrain.

try:

SMDegrain(...)
ex_unsharp(0.3) for clean anime

and a double call for films, like:

SMDegrain(...)
ex_unsharp(0.15).ex_unsharp(0.15, Fc=width()/1.5)

The first example sharpens the entire picture the same way, while the 2nd sharpens higher frequencies less to prevent residual grain from being enhanced too much again.


:thanks::thanks::thanks:

[Guest]

I would really like to combine this...

LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
LoadPlugin("%AVISYNTHPLUGINS%\MedianBlur2\MedianBlur2.dll")
Import("%AVISYNTHPLUGINS%\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\GRADE-PACK\Grade Pack.avs")
Import("%AVISYNTHPLUGINS%\LSF-PLUS\LSFplus.avs")
Import("%AVISYNTHPLUGINS%\RESIZERS-PACK\Resizers Pack.avs")
Import("%AVISYNTHPLUGINS%\SHARPENERS-PACK\Sharpeners Pack.avs")
Import("%AVISYNTHPLUGINS%\SMDEGRAIN\SMDegrain.avs")
Import("%AVISYNTHPLUGINS%\Zs_RF_Shared\Zs_RF_Shared.avs")
pre=ex_Median(mode="IQMST",thres=255)
STTWM(sw=50,tw=50,aw=100,sthres=5,tthres=5)
SMDegrain(tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter=pre,str=1.2,refinemotion=true)

with this

LoadPlugin("%AVISYNTHPLUGINS%\masktools\masktools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\mvtools\mvtools2.dll")
LoadPlugin("%AVISYNTHPLUGINS%\RgTools\RgTools.dll")
LoadPlugin("%AVISYNTHPLUGINS%\FrameRateConverter\FrameRateConverter-x64.dll")
Import("%AVISYNTHPLUGINS%\PD_TOOLS\EXTOOLS\ExTools.avs")
Import("%AVISYNTHPLUGINS%\GRADE-PACK\Grade Pack.avs")
Import("%AVISYNTHPLUGINS%\FrameRateConverter\FrameRateConverterMIX.avs")
FrameRateConverterMIX(FrameDouble=true)

They both work on their own, but I'd like to combine them..

I haven't had any luck, it must be the order of the calls that I can't get right :(

Thanks.

[LeXXuz]

May I ask why you manually import and load all those filters and scripts from your plugin folder?

I see no need to put them in separate subfolders and loading them manually. Just put them in your main plugin folder and Avisynth will autoload them by itself.
I have way over 100 files in there without any loading issues when needed. And I don't have to care which plugin I have to load for which script.

But if you have to do so. Load filters first, then import your scripts.

And maybe you should use a mt-modes script, which holds all the tested mt-modes for plugins.
Newer filters auto-register their mt-mode correctly but there are still many old filters which do not.
https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest

[Guest]

May I ask why you manually import and load all those filters and scripts from your plugin folder?

I see no need to put them in separate subfolders and loading them manually. Just put them in your main plugin folder and Avisynth will autoload them by itself.
I have way over 100 files in there without any loading issues when needed. And I don't have to care which plugin I have to load for which script.

But if you have to do so. Load filters first, then import your scripts.

And maybe you should use a mt-modes script, which holds all the tested mt-modes for plugins.
Newer filters auto-register their mt-mode correctly but there are still many old filters which do not.
https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest

Well, I'm one of the few ppl that use RipBot264, and that's the way it's been built...

StaxRip, and I think Hybrid are setup the same way, so not sure what you're using.

MT is controlled elsewhere, too.

Thanks for the tip on the script's, I'll see how I go.

[Lan4]

I have already asked in several places. I'll try to explain. This is off topic, but there are a lot of filter wizards here.

I'm interested in calculating sharpness. For example, I do several video processing options. And I compare the results in values. In order not to strain your eyesight, do not waste a lot of time. And also because the eyes see differently, today it is like this, and tomorrow it will be different, so the opinion changes. Filters perform calculations before starting work. These calculations can be displayed to the user so that this is a styled plugin?

There are examples of this with photos, although they are not popular.

[Heaud]

Regarding ResizersPack.avsi, does it only work with YV12 input? I tested out waifu2xresize from the resizers pack and it does not work properly with any image format that is not YUV420. There is also an error that comes up from the most recent version of this script:

https://i.imgur.com/1XbjsMo.png

Manually going into the script and replacing the values in that line from 4 : 2 to 2 : 1 allows waifu2xresize to run, although I am not sure if my version of w2xncnnvk (1.0.1) is compatible.

[Dogway]

They both work on their own, but I'd like to combine them..

I haven't had any luck, it must be the order of the calls that I can't get right :(
Tested here without any issues calling FrameRateConverterMIX after SMDegrain. I guess it might be something in your setup, as LeXXuz suggested.

Regarding ResizersPack.avsi, does it only work with YV12 input? I tested out waifu2xresize from the resizers pack and it does not work properly with any image format that is not YUV420. There is also an error that comes up from the most recent version of this script
Thanks, it was a bug in TransformsPack for YUV444 to RGB conversion. Just fixed it, will update soon. As for the error, I don't get it, in the readme (https://github.com/Asd-g/AviSynthPlus-w2xncnnvk) it default's to 2, but it says it can be increased if needed. I tested writing 4 in the arg without issues. Maybe a GPU limitation? Or a combination of settings, like HD, gpu_id, etc.

I'm interested in calculating sharpness. For example, I do several video processing options. And I compare the results in values. In order not to strain your eyesight, do not waste a lot of time. And also because the eyes see differently, today it is like this, and tomorrow it will be different, so the opinion changes. Filters perform calculations before starting work. These calculations can be displayed to the user so that this is a styled plugin?

I made a filter for it called SVM in SimilarityMetrics but these days I was playing with it and didn't quite get convinced. It's based on Laplace but with a sqrt flair to it. So the other day I made a new one a bit more elaborated but haven't uploaded yet as I have to review a few things (speed, consistency...)
function SVM2(clip a, bool "texture", int "show") {

rgb = isRGB(a)
bi = BitsPerComponent(a)
fs = propNumElements (a,"_ColorRange") > 0 ? \
propGetInt (a,"_ColorRange") == 0 : rgb
tx = Default(texture, false)
show = Default(show, 0) # 0: output 'diff' 1: output 'clip' 2: output 'diff' with subtitle

ConvertBits(a,32, dither=-1, fulls=fs, fulld=true)
# This is kroon detail mask
ed = ex_edge("kroon",lo=20,hi=255,scale=0.5)
# This is Gaussian diff mask to extract texture detail (with ex_unsharp a discreet lower freq diff is considered as 20%)
bl = ex_unsharp(-0.75,Fc=width()/1.5)
msk = ex_makediff(bl,dif=false,aug=1)
# This is to remove edges from texture detail mask (will add back with kroon edge mask)
ed2 = ex_luts(mode="min/max", pixels=ex_shape(2,mode="disk"),clamp_float=true,exprf=Format(" 60 255 / - range_max 255 150 60 - / * *"))

!tx ? ex_lutxyz(msk,ed,ed2,"x z - 0 max y + 1 255 / - 0 max sqrt ") : \
ex_lutxy (msk, ed2,"x y - 1 255 / - 0 max 0.3 ^")

vde = ex_inflate()

ScriptClip(show == 1 ? a : vde, function [vde] () {
propSet(last,"_PlaneSharpness", AverageLuma(vde), 0) } )

show != 1 ? ConvertBits(last, bi, dither=-1, fulls=true) : last

show == 2 ? ScriptClip(function [] () {
Subtitle("_PlaneSharpness: "+string(propGetFloat("_PlaneSharpness"))) } ) : last
}

[Guest]

Tested here without any issues calling FrameRateConverterMIX after SMDEgrain. I guess it might be something in your setup, as LeXXuz suggested.

Good to know, I will just shuffle the call's around until it works.

I even checked in Hybrid so see how that generates a script using SMDegrain & FrameConverter, and changed my script to suit, as best I could, and it didn't work...or I didn't have the patience to wait for it to start, after several minutes of waiting.

No luck, what script did you use to test ?

[Dogway]

Simply used:
pre=ex_Median(mode="IQMST",thres=255)
STTWM(sw=50,tw=50,aw=100,sthres=5,tthres=5)
SMDegrain(tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter=pre,str=1.2,refinemotion=true)
FrameRateConverterMIX(FrameDouble=true)
in AvsPmod, and I think without Prefetch() even as I only use it when encoding.
You might need to review your workflow, use avs+ test 9, setmemorymax to an appropiate value, and load the mt-modes.avsi. Also test with other source loaders.

[Guest]

Simply used:
pre=ex_Median(mode="IQMST",thres=255)
STTWM(sw=50,tw=50,aw=100,sthres=5,tthres=5)
SMDegrain(tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter=pre,str=1.2,refinemotion=true)
FrameRateConverterMIX(FrameDouble=true)
in AvsPmod, and I think without Prefetch() even as I only use it when encoding.
You might need to review your workflow, use avs+ test 9, setmemorymax to an appropiate value, and load the mt-modes.avsi. Also test with other source loaders.

So I'm guessing that by using AvsPmod, all the required dependencies for SMDegrain & FrameConverter are auto loaded...

I'm also using AVS r3973, I'm also using prefetch 16, and mt, but that is somewhere else in the process, not in the main scripts.

Also using L-Smash.

[Dogway]

So I'm guessing that by using AvsPmod, all the required dependencies for SMDegrain & FrameConverter are auto loaded...

I'm also using AVS r3973, I'm also using prefetch 16, and mt, but that is somewhere else in the process, not in the main scripts.

Also using L-Smash.

Hitting all the marks for a failure. Use pinterf builds, they are the safest. Also load with ffvideosource, and put your plugins and .avsi into the autoload folder. That would be a good start. My recommendation also, if using my scripts separate real.finder scripts into another folder as Zs_RF_Shared that I saw you loading may interfere. I put them all in a folder with a IMPORT_ALL.avsi loading them, which I can in turn then load all of them with a single line.

[Guest]

Hitting all the marks for a failure. Use pinterf builds, they are the safest. Also load with ffvideosource, and put your plugins and .avsi into the autoload folder. That would be a good start. My recommendation also, if using my scripts separate real.finder scripts into another folder as Zs_RF_Shared that I saw you loading may interfere. I put them all in a folder with a IMPORT_ALL.avsi loading them, which I can in turn then load all of them with a single line.

I still have to ask, why then do those scripts work separately, but not when combine them? Or does it just slow down the whole process so much, that it's not worth it?

And I'm pretty sure that an "autoload" folder does not work with the likes of StaxRip, RipBot264 or Hybrid.

And I don't use AvsPmod or MeGui.

I will try your suggestions about AVS & FFMS2.

UPDATE:- Made no difference at all :(

[Lan4]

I made a filter for it called SVM in SimilarityMetrics but these days I was playing with it and didn't quite get convinced. It's based on Laplace but with a sqrt flair to it. So the other day I made a new one a bit more elaborated but haven't uploaded yet as I have to review a few things (speed, consistency...)
OK, thank you. To be honest, I'm surprised that someone is also interested. what will be the implementation? curve, values, tool? will it be one general parameter or several aspects?

[LeXXuz]

Dogway, SMDegrain is calling mfilter, if it's set, for those areas where MVtools found no matching motion vectors (based on the parameters given), and therefore these will not be denoised, is that about right?

I mentioned, I simply use Greyscale() to identify how much of those areas is still present in the output to determine if I should tinker some more with either prefilter, tr or thSAD.

I did some testing with a source where artificial grain was just added to the luma plane, so I saw no reason to process all three planes.
Ergo disabled chroma processing to gain some speed.
To my surprise the output was ALWAYS completely grey as soon as I set plane=0.

For example with this simple call:
ConvertBits(16)
PREm = Greyscale()
SMDegrain(tr=3, thSAD=1000, mfilter=PREm, chroma=false, plane=0)

Is this a bug or supposed to happen?
Because no chroma was processed => therefore no mv data for U and V => so the entire chroma planes are handed over to mfilter (which then flattens them with greyscale())?

I expected if processing is set to luma only, that only what happens in Y plane determines if mfilter is called or not.

[DTL]

" no mv data for U and V "

Single MV data always applied to all planes selected (in MDegrainX). For total processing yon can disable chroma in MAnalyse for better performance (visible enough benefit, may significantly decrease quality of MV data in some cases) and you can disable denoise of chroma planes in MDegrainX (may be very low performance boost, typically disabled only if user not want to denoise chroma for some use case).

[LeXXuz]

Thanks DTL. Yes, I noticed performance gain is very small.
Decided to play it safe and better process all planes again.

[Dogway]

OK, thank you. To be honest, I'm surprised that someone is also interested. what will be the implementation? curve, values, tool? will it be one general parameter or several aspects?
Did you see the posted function? It will be that or a slightly modified version of it. It only has two arguments, 'texture' to isolate texture alone, and 'show', to output source clip with '_PlaneSharpness' frameprop embedded, or a mask with the value subtitled.
My interest sparked from this issue (https://github.com/libjxl/libjxl/issues/1470)in JXL, they use different metrics to assess quality like Butteraugli or SSIMULACRA2, but they are edge centric metrics and don't take into account (texture) sharpness.
UPDATE:- Made no difference at all :(
Sounds like an issue with the software you are using, ripbot, megui or hybrid. Try to ask them about it.
Dogway, SMDegrain is calling mfilter, if it's set, for those areas where MVtools found no matching motion vectors (based on the parameters given), and therefore these will not be denoised, is that about right?

I mentioned, I simply use Greyscale() to identify how much of those areas is still present in the output to determine if I should tinker some more with either prefilter, tr or thSAD.

I did some testing with a source where artificial grain was just added to the luma plane, so I saw no reason to process all three planes.
Ergo disabled chroma processing to gain some speed.
To my surprise the output was ALWAYS completely grey as soon as I set plane=0.

For example with this simple call:
ConvertBits(16)
PREm = Greyscale()
SMDegrain(tr=3, thSAD=1000, mfilter=PREm, chroma=false, plane=0)

Is this a bug or supposed to happen?
Because no chroma was processed => therefore no mv data for U and V => so the entire chroma planes are handed over to mfilter (which then flattens them with greyscale())?

I expected if processing is set to luma only, that only what happens in Y plane determines if mfilter is called or not.
Interesting topic. TBH my workflow recently has been to pass a high blur like ex_blur(6) to mfilter to identify passthrough.
In any case Greyscale() might not be the way to do it as the effect you'd get is simply greyscaled chroma (luma stays the same) which is hard to spot.
A better option is PREm = ex_lut("range_half",UV=1). That would give you a better view of what's going on. Or rendering it in red PREm = Expr("81","90","250",scale_inputs="int"), temporally using plane=4.

Typically from old workflows it is mfilter.MDegrain(input.MSuper(), ..."prefilter MV"...), so as you see it is indeed using luma and chroma from mfilter. Whichever you don't process, then it passes the mfilter plane. This is done for optimization. Do you think I should swizzle unused planes in from input even if it hits performance?

About processing chroma for MV or not, I did some experiments that resulted in increased quality when taking chroma into account, unless chroma is a crap fest like VHS and other sources. I went to the extent to increase chroma weighting for MV as you can see in the scaleCSAD variable.

[Guest]

Sounds like an issue with the software you are using, ripbot, megui or hybrid. Try to ask them about it.

I doubt that it's an issue with the software I'm using...

I am using 2 of your scripts, that work well on their own, but trying to combine them to do both tasks at the same time, just doesn't work for me.

Have I got all the required dependencies or is the order of the calls messed up ??

here (https://forum.doom9.org/showthread.php?p=1985207#post1985207)

[LeXXuz]

This is done for optimization. Do you think I should swizzle unused planes in from input even if it hits performance?

About processing chroma for MV or not, I did some experiments that resulted in increased quality when taking chroma into account, unless chroma is a crap fest like VHS and other sources. I went to the extent to increase chroma weighting for MV as you can see in the scaleCSAD variable.

I think it's fine the way it is. I simply didn't expect this outcome at first. And I will keep chroma processing in my workflow, as I've seen performance benefit is really small and not worth the trade off for losing some accuracy when not processing chroma for mv.

Btw did you have a chance to take a look into ex_blend and those error message overlays from Scenestats in the meantime? :)

[DTL]

" they are edge centric metrics and don't take into account (texture) sharpness."

As I see we still have very few underlying math analysis tools for random fields for image similarity compare. Most of similarity metrics are based on only 2 underlying math statistical tools - variance and covariance. And some higher-level math playing with these 2 computed values to get 'really nice metric ranging in 0..1 or -1..1' .

For texture quality (similarity) part of metric it looks mostly work covariance value. So at some of my MPB-processing for MDegrainN to keep as nice textures as possible (skip most of texture-damaging blocks from blending blocks pool) I found it may be used simple pure covariance. May be the covariance computing may be somehow additionally enhanced to mark texture sharpness even more.

[Dogway]

LeXXuz, try now, I updated SMDegrain with some improvements. First unprocessed planes are copied from input to mfilter clip when using mfilter obviously, this makes more sense.
Also multi (tr > 6) happened to don't be correct for some modes like LFR, mfilter or mode!="MDegrain", that is fixed now, and also with nice for loops, so less code lines.

About SceneStats that's a delicate issue due to the complexity of the filter. First I'm taking things slower now, and second I decided to wait until the issue happens to me again instead of searching for it, but it's acknowledged and will tackle with it when that happens.

DTL, do you have literature about this approach (covariance for texture/sharpness metrics)? What variables would covariance employ in this case?

[LeXXuz]

Thanks Dogway, updating now. :)

I don't know if that would be any simpler, but if you could just disable that error overlay that would be a big help already.

Otherwise I'll wait with these sources, no problem at all. Enough other stuff to work on from my collection. :D

[Dogway]

I don't think it's that easy, as I said the filter is quite complex, and even if I fix the eventual errors (at least the ones I was having) fades in/out will still need to be trimmed out. If someone comes up with a fade detector that would be welcomed.

[LeXXuz]

I don't think it's that easy, as I said the filter is quite complex, and even if I fix the eventual errors (at least the ones I was having) fades in/out will still need to be trimmed out. If someone comes up with a fade detector that would be welcomed.

Okay, no problem. And thanks for the suggestion with Expr() and ex_lut() for mfilter preview. They show way better which areas are processed. :)

[DTL]

"do you have literature about this approach (covariance for texture/sharpness metrics)? What variables would covariance employ in this case?"

Covariance is major part of most 'complex' metrics (more complex in compare with 'simple' SAD). SSIM and VIF widely uses covariance component. SSIM have 'structure' part based on covariance and VIF also uses covariance in output computing formula. You can look into SSIM and VIF computing functions in mvtools to see how it calculated from blocks arrays samples:

SSIM https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Sources/SSIMFunctions.cpp

Full SSIM is SSIM_Light * SSIM_Contrast * SSIM_Structure . SSIM_Light is sort of simple average of samples. SSIM_Contrast is some function of (variance_a, variance_b) where variance_a is fsX and variance_b is fsY at https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/SSIMFunctions.cpp#L57

And SSIM_Structure is function of covariance(a,b) (fsXY) - https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/SSIMFunctions.cpp#L66 . And after getting all 3 components of SSIM output is simple multiplication.

But for some specific analisys each part of SSIM may be used separately. Most interesting for textures may be SSIM_Structure calculated from covariance.

VIF is https://github.com/DTL2020/mvtools/blob/mvtools-pfmod/Sources/VIFFunctions.cpp

It uses covariance of DWT - https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/VIFFunctions.cpp#L66 (DWT_a component) for VIF_A part and also some covariance for VIF_E part - https://github.com/DTL2020/mvtools/blob/0a6b093507bc757457783d537bc6cfaaa273989d/Sources/VIFFunctions.cpp#L98

It may be possible to convert C-computing to AVS scripting computing and you can make full or structure-only SSIM in AVS function.

The VIF DWT-based require DWT (it is not complex but more slow). The most diffrerence between SSIM and VIF for structure part is SSIM compute from direct input blocks samples covariance and VIF compute covariance from DWT-transformed blocks (may be sort of low-pass filtering or twice reducing ?).

[Dogway]

Thanks for the links, I will have a look later. If none of covariance components come from a texture analysis it would be a moot point. I have SSIM ported to a function in SimilarityMetrics, but I don't recall any texture extraction.
VIF doesn't sound familiar to me so will have a look (EDIT: ok, VIF -Visual Information Fidelity- from my metrics issue (https://github.com/Dogway/Avisynth-Scripts/issues/36)). DWT is frequency based so that might be useful.

The problem probably with blocks is that small details vanishes from the block average, unless augmentation is performed, that or dynamic sized blocks come into play like x265.

[Boulder]

For visualizing the effect of filters outside MDegrain, I've used the Subtract method like Subtract(finalclip_nonblurred, finalclip.mergechroma(finalclip_nonblurred)).levels(127*256, 1, 132*256, 0*256, 255*256).subtitle("Effect of blur in result", y=40). Here 'finalclip' is the normal output, finalclip_nonblurred the one where there is no denoising other than MDegrain and then I just want to see how luma looks like so the chroma planes are merged from the normal output.

Funny thing is that the filter gets quite a big weight even if you use something like thsad=10000 in MDegrain, it won't change the output much if you limit the effect of MDegrain itself. I was always under the impression that thsad can be used to control which parts of the frame get processed by MDegrain and which parts are skipped because the calculated thsad is too high.

[DTL]

" If none of covariance components come from a texture analysis it would be a moot point. "

Covariance in SSIM is direct math covariance between samples of compared blocks. It shows how the samples of the compared blocks changes in similar way at each sample. If one block is blurry and second is sharp (non-damaged by blur) - the covariance metric will be lower.

Covariance is
fsXY += (float)((pWorkSrc[x] - isuX) * (pWorkRef[x] - isuY));
- sum of multiplication of (blocks values - mean block value (mean DC component)). So if samples changes in same direction from mean value - the mul operation gives higher result. Also it is positive for both positive and negative difference from mean value (DC) if both differences have same sign.

So if some processing start to damage texture in compare with 'initial source' - the covariance metric become lower and it can be detected and the processing may be adjusted to keep texture more non-damaged. Also the processing may be optimized (zopti ?) to maximize covariance metric between input and output.

[LeXXuz]

Funny thing is that the filter gets quite a big weight even if you use something like thsad=10000 in MDegrain, it won't change the output much if you limit the effect of MDegrain itself. I was always under the impression that thsad can be used to control which parts of the frame get processed by MDegrain and which parts are skipped because the calculated thsad is too high.

I think without raising thSCD as well you will not do much, even with thSAD that exorbitant high, since every frame is probably considered a scene change now?

[Boulder]

I think without raising thSCD as well you will not do much, even with thSAD that exorbitant high, since every frame is probably considered a scene change now?

thSAD is not directly connected to scene change detection, you can use a higher thSAD value than thSCD1 and the frame will still show motion vectors with MShow (which is very useful with showsad=true). As far as I know, thSCD1 sets the threshold (per block) and thSCD2 then sets the max amount of blocks triggered by thSCD1.

[DTL]

If you set thSAD to very high value and scene detection still not skip frame - you simply lost protection from bad blends and MDegrainX will work as linear motion compensated blending engine (may be equal to MCompensate + Average()). If MVs are not perfect or there is other non-supported and not-compensated transforms left it will cause either blurring or visible several different blocks blended. thSAD adjustment in MDegrainX is only to protect user for possible artifacts of the very very simple temporal denoise method of different frames blocks weighted averaging.

In a perfect world with ideal transforms compensation no any protection required (see RIFE + Average() denoise example).

thSCD protects from more general error of blending different scenes in single frame.

[Guest]

Simply used:
pre=ex_Median(mode="IQMST",thres=255)
STTWM(sw=50,tw=50,aw=100,sthres=5,tthres=5)
SMDegrain(tr=2,thSAD=200,thSADC=100,contrasharp=true,prefilter=pre,str=1.2,refinemotion=true)
FrameRateConverterMIX(FrameDouble=true)
in AvsPmod, and I think without Prefetch() even as I only use it when encoding.
You might need to review your workflow, use avs+ test 9, setmemorymax to an appropiate value, and load the mt-modes.avsi. Also test with other source loaders.

I think I might have figured out what's going on...

What type (resolution) video did you test this script on ?

I am trying to use FrameConverter on 4K HDR10 content, and I think the combination is just too much for it, 'cause it seems to be ok, but slow, on lesser resolution's.

[LeXXuz]

I think I might have figured out what's going on...

What type (resolution) video did you test this script on ?

I am trying to use FrameConverter on 4K HDR10 content, and I think the combination is just too much for it, 'cause it seems to be ok, but slow, on lesser resolution's.

Could be a memory problem. How much RAM does your system have? If you have 32GB for example, try adding


SetMemoryMax(20*1024) # max RAM usage in kilobytes
SetCacheMode(0)

at the very beginning of your script. If your machine has more RAM you can raise that even more.

I have some script combinations which have very high RAM demands and either will not start or be very very slow on higher resolutions, if I don't set MemoryMax correctly.

Also, some filters don't really like caching, especially in multithreading with a high prefetch count, so I have to set cache mode explicitely to zero.

[Guest]

Could be a memory problem. How much RAM does your system have? If you have 32GB for example, try adding


SetMemoryMax(20*1024) # max RAM usage in kilobytes
SetCacheMode(0)

at the very beginning of your script. If your machine has more RAM you can raise that even more.

I have some script combinations which have very high RAM demands and either will not start or be very very slow on higher resolutions, if I don't set MemoryMax correctly.

Also, some filters don't really like caching, especially in multithreading with a high prefetch count, so I have to set cache mode explicitely to zero.

I do only have 32Gb....

And I have this in every script:-

SetCacheMode(1)
SetMemoryMax(16384)

And the Prefetch is 16, but it's not included in the scripts.

[LeXXuz]

I do only have 32Gb....

And I have this in every script:-

SetCacheMode(1)
SetMemoryMax(16384)

And the Prefetch is 16, but it's not included in the scripts.

Try with cache mode 0.

[Boulder]

If you set thSAD to very high value and scene detection still not skip frame - you simply lost protection from bad blends and MDegrainX will work as linear motion compensated blending engine (may be equal to MCompensate + Average()). If MVs are not perfect or there is other non-supported and not-compensated transforms left it will cause either blurring or visible several different blocks blended. thSAD adjustment in MDegrainX is only to protect user for possible artifacts of the very very simple temporal denoise method of different frames blocks weighted averaging.

In a perfect world with ideal transforms compensation no any protection required (see RIFE + Average() denoise example).

thSCD protects from more general error of blending different scenes in single frame.
My case was just an example of trying to emphasize the effect. My default values for thSAD and thSCD1 are 400 and 600 respectively. thSCD1 sometimes needs to be bigger if the source is very grainy (despite heavy prefiltering).

[Guest]

Try with cache mode 0.

Isn't 0 default, so I could simply remove that call...

[LeXXuz]

I don't trust defaults. :D

[Guest]

I don't trust defaults. :D

I might as well raise the "memorymax", too...

[LeXXuz]

You probably have to. 16GB is not very much for more complex filtering of 4k content.