Dogway's Filters Packs

[tormento]

It's belong to TransformsPack - Main
My fault. I updated from 2.0.0 to 2.2.0 and I had 2 copies of the same file for main and models.

Thank you!

[DTL]

Users frequently ask for the use of some new features of mvtools2 in the SMDegrain script. As full-featured latest releases (marked with -a.XX tag) are not stable in many use cases I make some small extract of the most simple and useful features expected to be safe from bugs.

It is release https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-e.01

Though it is based on latest commits by pinterf in 2021 after 2.7.45 release and not much tested.

It has only 2 new features added:
1. "SuperCurrent" second super clip input in the MAnalyse - so in the use cases with prefiltering or multi-generations MVs refining (when MDegrain used as 'prefilter' from previous generation MVs) 2 super clips may be feed to MAnalyse. The 1st may be original non-distorted by any processing input 'super' clip and second after some pre-processing. It may decrease MVs search errors.

If SuperCurrent input clip is not provided - MAnalyse uses a single super clip for both inputs of MV search algorithm for 'current' and 'reference' frames. As in the versions of 2.7.45 and before.

2. Auto-thSAD feature for MDegrainN - described in the https://forum.doom9.org/showthread.php?p=1990183#post1990183

These new features are expected to be independent from block size / bitdepth and work with any combination.

Extracting of more useful features for MDegrainN like interpolated overlap and MVLPF is significantly more complex (also may be more buggy) so expected in some time later if this e.01 extract release will be tested as stable enough. They are also independent of blocksize and bitdepth so may be used in complex universal scripts like SMDegrain.

[LeXXuz]

2. Auto-thSAD feature for MDegrainN - described in the https://forum.doom9.org/showthread.php?p=1990183#post1990183


Oh I want that. :eek: Pretty please with a cherry on top @ Dogway :rolleyes::D:o

[kedautinh12]

No, hard to use cause DTL's MVTools change many things and don't stable with many scripts use Pinterf's MVTools. If copy and change name, it will have 2 function duplicate

[DTL]

Build with -e.01 tag is special edition expected to be compatible with scripts running with 2.7.45 by pinterf. It is based on latest commits to pinterf repository in 2021 (only a few after 2.7.45 release). If it still not stable - I can try to use exact 2.7.45 sources (but they cause some debug assert fail in MDegrainN with default block size 8x8 and it was fixed by pinterf later - looks like this commit https://github.com/pinterf/mvtools/commit/2c30382aaea57f4224553359debe475b1b2e045c ).

[anton_foy]

Build with -e.01 tag is special edition expected to be compatible with scripts running with 2.7.45 by pinterf. It is based on latest commits to pinterf repository in 2021 (only a few after 2.7.45 release). If it still not stable - I can try to use exact 2.7.45 sources (but they cause some debug assert fail in MDegrainN with default block size 8x8 and it was fixed by pinterf later - looks like this commit https://github.com/pinterf/mvtools/commit/2c30382aaea57f4224553359debe475b1b2e045c ).

Lovely! Now I think it is time for me at least to make it into Clay. So the base is Pinterf's build with added features from you DTL? Does mrecalculate not work with this build and mdegrain2 aswell?

[DTL]

The Auto-thSAD added only to MDegrainN. All other filters are expected to work the same as in the 2.7.45 release. Though latest commits by pinterf in 2021 really have some additions to MAnalyse too (like optSearchOption and optPredictorsType) but I hope if users will not enable them - it should run stable. If some issues with old scripts will be found - I can try to make additions of dual-input to MAnalyse and Auto-thSAD to exact source version from 2.7.45 release by pinterf. I hope I can temporarily disable that debug assertion from MDegrainN with debug build. But as a first attempt to make more new builds with only a few and simple new features added, I get the latest sources from the pinterf's repository also to test if it is still not loaded with newer bugs or non-compatibility with old scripts.

"So the base is Pinterf's build with added features from you DTL?"

It is latest sources from pinterf's repository with some post-2.7.45 commits in 2021 year and added only 2 new simple features of second input to MAnalyse and Auto-thSAD to MDegrainN. Uploaded as separate branch to github - https://github.com/DTL2020/mvtools/tree/pinterf_last2021 . This branch is 2 commits ahead of pinterf:mvtools-pfmod.

"Does mrecalculate not work with this build and mdegrain2 aswell? "

Everything old expected to work. MDegrainX(1..6) will not have thSADA_a and thSAD_b new params. I do not have lots of required plugins and script libraries installed to test it with 'complex' scripts like QTGMC() or SMDegrain(). But I hope it will work.

Update: first small bugfix - https://github.com/DTL2020/mvtools/releases/tag/r.2.7.46-e.02 . Found while trying to use Auto-thSAD with > 8bit clips.

[LeXXuz]

Dogway I have a problem with sharpening and noise I don't quite understand atm. Does contrasharp work on the entire picture or only the areas SMDegrain denoised. So far I thought the latter to not increase strength of noise in areas mvtools didn't find a match and SMDegrain did NOT denoise.
But I think that's not what I'm actually seeing in my testclip. How does it behave with LSFplus mode and limitS=true?

[Dogway]

@LeXXuz: ex_ContraSharpening() works on denoised (aka blurred) areas, but it incorporates a limiting so to not sharpen noise. The limiting is basically a DoG (edge mask) of the prefiltered (with ex_minblur) denoised clip, so safe to use, this when using spatial limiting of course.

LSFplus is similar in contrasharp mode but it uses different prefilters+edgemasks and additionally adds a non-linear sharpening, what this means is that it does some local contrast to such edgemask to tighten the area (edges) to be sharpened.

[LeXXuz]

Ah, thanks! I have some content here where noise was added in some areas and that noise is so strong that I would have to use settings which would oversmooth the cleaner areas and kill lots of detail which would be worse. So I decided to keep it in there, although I think partially denoised material looks even more ugly than a noisy source...

So, for content that is not completely clean after denoising it may be safer to use contrasharp option in SMDegrain instead of running an external sharpener afterwards? I know I could lower frequency f.e. in ex_unsharp to avoid noise gain, but then I would lose the ability to sharpen very fine detail in other areas.

[Dogway]

If you want to sharpen noisy/grainy content you have to denoise first as you would normally do. Then feed that to ex_ContraSharpening() as denoised and source as source. Then you make a diff from the sharpened and denoised and add that to source, this way you don't sharpen grain/noise but only details.

source
src=last

SMDegrain()
den=last

ex_ContraSharpening(src) # or LSFplus in contra mode

ex_makeadddiff(last,den,src)

[tormento]

@Dogway

Do you plan to use the new features of DTL MVTools (both the standard ones and the DirectX porting) soon? I am really curious to see improvements in speed and quality, if any.

[Dogway]

No, it looks like big work as I'd need to refactor SMDegrain in its entirety, also it looks GPU motion vectors could be a reality soon with either DLSS 3 or FSR 3 if they provide an API.

I'm semi-retired from AVS, also starting classes tomorrow on Flutter+Dart.

[Boulder]

If you want to sharpen noisy/grainy content you have to denoise first as you would normally do. Then feed that to ex_ContraSharpening() as denoised and source as source. Then you make a diff from the sharpened and denoised and add that to source, this way you don't sharpen grain/noise but only details.

source
src=last

SMDegrain()
den=last

ex_ContraSharpening(src) # or LSFplus in contra mode

ex_makeadddiff(den,src)

Isn't the contrasharpening call an NOP here since ex_makeadddiff uses variables which are not affected by it?

[Dogway]

Yes, sorry I forgot that ex_makeadddiff() also accepts two inputs so I was thinking on implicit "last".

it should be:
ex_makeadddiff(last,den,src)
Also fixed above

[DTL]

it looks GPU motion vectors could be a reality soon with either DLSS 3 or FSR 3 if they provide an API.


I do not expect big quality gains of MVs from DLSS/FSR motion estimation engines. Because it looks like they are designed to work with initially clean sources and with low latency (may not allow complex algorithms to run).

It may be good for smooth-video projects acceleration like SVP to free CPU when running with clean noise-free content and to get more intermediate frames generated.

For clean sources even old simple mvtools single pass based on SAD works good enough. You have clean src block and clean ref block and if motion is only translation the SAD-based search (with exhaustive method and enough radius) will output absolutely the best MV with single search. Nothing more to do.

With natural noised content things are going much worse - we do not have clean src block and clean ref blocks. So a simple search algorithm with single pass (based on 2 frames only) will output lots of false MVs and they are correct for the algorithm used - SAD also lowest. So no increase of pel/radius/search method can help in getting more correct MVs.

Motion search for natural noised content needs to use more complex algorithms with many frames analysis and using properties of natural motion to be not very random. Some algorithm targets may be to restore lowest temporal changing blocks over a long sequence of frames (sort of original objects texture restoration).

Sad idea about modern motion search engines - they are not designed to work with noised sources because new video footage computer generated (or from low noise video cams) are of low noise. So denoise is no common public task today and the digital industry does not invest into temporal denoise accelerators development (as software algorithms or hardware ASICs or GPU general compute blocks implementation like ML). Denoise of old film footage and some not common new shoot applications (like low light shots) is a limited task today to amateurs or old content restoration or rare commercial firms. So the general computer hardware industry can not make good money on developing good denoise engines for such a small market.

Possibly best MVs quality from general public hardware engines can be expected from modern MPEG encoders because it more frequently works with natural shot noised content and better motion compensation may make better MPEG compression performance (in theory). But the quality of ME of MPEG encoder also not needed to be best because all not compensated differences between src and ref block may be encoded as additional bits and also make output bitrate higher (if allowed by current rate control). Also MPEG encoders need to run fast so motion search radius and complexity may be limited for better performance.

[Dogway]

I do not expect big quality gains of MVs from DLSS/FSR motion estimation engines. Because it looks like they are designed to work with initially clean sources and with low latency (may not allow complex algorithms to run).

You can use prefilters anyway but I think denosing is also an area of interest sooner or later as shown with OptiX denoiser and RTX VSR.

Since it is accelerated with specific GPU chips I don't think it would be hard to surpass MVTools quality where the core algos haven't been changed in years.

It would be be interesting to know if it introduces also rotation, zoom and other transformation predictors. But all this depends onthe API options.

[DTL]

I still not see progress in new temporal denoisers for natural motion pictures data (from both film and silicon based sensors). The current industry denoisers are about decreasing artificial noise produced by too low quality raytracing engines.

I found demo of NVIDIA OptiX https://developer.nvidia.com/optix-denoiser and also some open denoiser from intel - https://www.openimagedenoise.org/ . But as I see in short description it is spatial single frame denoisers mostly ?

Intel open image denoise may be also interesting to make into AVS plugin to test and maybe test as prefilter. It promised to run at most of todays platforms from SSE4 CPU to many new hardware accelerators.

But I can not found anything about RTX VSR.

So it looks image processing industry still not (or already not) make anything good for natural noised imaging. Looks like close to no one (in general puclic) need to make clean natural images. We lost most of new end-users video cameras market. And smartphones video uses internal image processing and MPEG encoding. So new denoise engines oriented mostly on fixing bad computer renders with raytacing.

[poisondeathray]

I found demo of NVIDIA OptiX https://developer.nvidia.com/optix-denoiser and also some open denoiser from intel - https://www.openimagedenoise.org/ . But as I see in short description it is spatial single frame denoisers mostly ?

Yes, single image, using multiple layer inputs. OptiX and Intel OIDN are designed for 3D render engines, not for "flat 2D video". You need multiple passes and layers (albedo, normals) for best results. Otherwise the results are not good. Of course you don't have that type of data for normal "video"

[Dogway]

Yes that's true it's mainly designed for raytracing noise and makes use of normals and other passes, but that's like a bonus for renders, for natural images the raytracing denoising should be good enough on its own as it resembles typical film grain, for example on unbiased (or simply path traced) render engines it's typical to not let finish the render at all to have a natural looking film like grain.
In DLSS 3.5 presented last week the algorithm was improved so the denoiser takes into account the motion vectors and thus avoid the common ghosting and blurring issues it was suffering on previous versions. see the video here (https://youtu.be/sGKCrcNsVzo)

RTX VSR is simply DLSS applied to video (super resolution)

[LeXXuz]

If you want to sharpen noisy/grainy content you have to denoise first as you would normally do. Then feed that to ex_ContraSharpening() as denoised and source as source. Then you make a diff from the sharpened and denoised and add that to source, this way you don't sharpen grain/noise but only details.

source
src=last

SMDegrain()
den=last

ex_ContraSharpening(src) # or LSFplus in contra mode

ex_makeadddiff(last,den,src)

Well this way I get a picture that is almost as noisy as the original but with the grain being static now. I already noticed the missing last argument in the first version of your post and corrected that.

I could post screenshots, but to spare me the hassle with an image provider, I'll just post SAD values from mvtools. Hope you'll take my word for it.

untouched: 26455
denoised: 7720
denoised with internal contrasharp: 8477
denoised with external cs and makediff: 25965

mvtools script:
super = MSuper(mt=false, pel=2)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, search=3, chroma=true, mt=false)
MShow(super,forward_vec1, showsad=true)

(In case someone wonders about those numbers, they are in 16bit´. You have to divide those by 256 if you want a relation to those numbers you use with SMDegrain.)

Idk, maybe it is intended that way. But the denoised areas should still be clean which they are not and those numbers reflect that obviously.

[Dogway]

Maybe the principle was ill-founded, try to do an ex_makediff between src and denoised -with LFR- (grain extract) then add that to the sharpened-denoised clip

ex_makeadddiff(den,src,last)

[kedautinh12]

I have that error when you SMDegrain
SMDegrain(6, 400, prefilter=5, ContraSharp=true, RefineMotion=true, plane=4, limits=false, DCTFlicker=false)
https://i.imgur.com/yuxGTcE.png

[Dogway]

I have that error when you SMDegrain
SMDegrain(6, 400, prefilter=5, ContraSharp=true, RefineMotion=true, plane=4, limits=false, DCTFlicker=false)

Thanks! Just fixed. I forgot to update the new Super global variable that I changed in SMDegrain to align to QTGMC+

[kedautinh12]

Hi Dogway, I tried QTGMCp but can't fix the interlaced video of some scenes, can you help me how to fix that??

Frame 920
https://i.imgur.com/Q4PppXn.png

AssumeTFF()
QTGMCp(Preset="Medium", Sharpness=0).SelectEven()

Sample:
https://drive.google.com/file/d/1oatclOhzlOtYIs527liyP72h7x5hY_mt/view?usp=sharing

[Selur]

resize to 1920x540, then upscale back to 1920x1080

[Dogway]

The moiree effect is present even in the fields so you might need to target the issue locally as postprocessing I think.

[kedautinh12]

The moiree effect is present even in the fields so you might need to target the issue locally as postprocessing I think.

Can you have any suggestions script for that??

[kedautinh12]

resize to 1920x540, then upscale back to 1920x1080

resize after or before Deinterlaced??

I tried and got better results but I still have errors
propSet("_FieldBased",0)
deep_resize(1920,540,grain=0)
deep_resize(1920,1080,grain=0)
AssumeTFF()
QTGMCp(Preset="Medium", Sharpness=0).SelectEven()

[kedautinh12]

After updated QTGMC+ to latest ver with moiree option, I tried but still doesn't work
AssumeTFF()
QTGMCp(Preset="Medium", moiree=true, Sharpness=0).SelectEven()

[Selur]

I did have an additional look the sources:
a. seems like DVD resolution at best, so you could probably downscaling to dvd resolution without loosing detail.
b. Dogway is right, this isn't caused by deinterlacing and won't be fixed by it, this would require some specific local processing like copying parts of a previous frame/field over to the broken ones
=> probably not worth the effort this would require

[kedautinh12]

I did have an additional look the sources:
a. seems like DVD resolution at best, so you could probably downscaling to dvd resolution without loosing detail.
b. Dogway is right, this isn't caused by deinterlacing and won't be fixed by it, this would require some specific local processing like copying parts of a previous frame/field over to the broken ones
=> probably not worth the effort this would require
a, i try downscaling to DVD but errors won't gone

[kedautinh12]

Ok, I tried other Deinterlaced scripts and work very well. Some scenes have ALIASING after Deinterlaced. Therefore, I fixed it with ANTI-ALIASING
AssumeTFF()
AnimeIVTC(mode=1)
daa3mod()

[Boulder]

@Dogway: do you have any recommended method for stabilizing luma changes for a more accurate scene change detection with ffmpeg? I tend to get quite a lot of false detections just because the brightness between frames seems to change a bit (Withnail and I from the early 80s, rather LQ Blu-ray). If I raise the detection threshold, it starts missing too many scene changes to be useful. I'm not looking to catch all of them so I have some wiggling room. I've tried downscaling, severe blurring and using ex_luma_rebuild but no real help there.

[Dogway]

If it's some kind of luma flicker try ex_autolevels() (example (https://forum.doom9.org/showthread.php?p=1982586#post1982586))

Otherwise yes, you have to temporal blend the frames, except those for scene changes, so a catch-22 situation.

It depends if you want surgeon denoising, or a simple temporal blend, in such case replace the denoise with a TemporalSoften().
With SceneStats you can work on a scene basis, also it inherits SC weight from the source frame type.

Here are some examples:
https://forum.doom9.org/showthread.php?p=1982586#post1982586
https://forum.doom9.org/showthread.php?p=1985953#post1985953
https://forum.doom9.org/showthread.php?p=1976405#post1976405
https://forum.doom9.org/showthread.php?p=1969107#post1969107 (I recommend putting SMDegrain/TemporalSoften outside ScriptClip)

[alexx7777]

18482Thank you very much for the work done. I'm getting an nmod error. For what reason and how to solve?

[kedautinh12]

18482Thank you very much for the work done. I'm getting an nmod error. For what reason and how to solve?

Up image to another sever or you will waiting for mod approval forever :D

[alexx7777]

gives an error message (https://disk.yandex.ru/i/HKKUTOkRI7eprQ)

[kedautinh12]

gives an error message (https://disk.yandex.ru/i/HKKUTOkRI7eprQ)

Did you update all scripts of Dogway to latest ver??

[alexx7777]

Did you update all scripts of Dogway to latest ver??
Downloaded from here (https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack.avsi)

[Dogway]

@akexx777 That's an error on line 215, basically a nmod() call in the Defaults definitions. It works for me so ensure everything is updated and no other older ResizersPack is present.
You can also try with this call nmod(8/2,2,4) which is what is failing on line 215

[alexx7777]

@akexx777 That's an error on line 215, basically a nmod() call in the Defaults definitions. It works for me so ensure everything is updated and no other older ResizersPack is present.
You can also try with this call nmod(8/2,2,4) which is what is failing on line 215
Thank you very much, everything worked!

[tormento]

What about using avs_libplacebo in some of your scripts? It's some weeks that I am playing with it and it's really fast and powerful.

The mitchell downsizing filter is something astonishing to get native anime resolution.

[DTL]

I'm still struggling with a bluring effect with SMDegrain in moving areas which really disturbs the overall look of the picture.
To elaborate: I've uploaded four samples of a short scene. Original, 2x denoised with SMDegrain (thSAD 400 and 200)


New tests with 2 stages (non-linear median-like and linear low pass with Gauss kernel) MVs filtering in new MDegrainN release of 26.09.2023 - https://forum.doom9.org/showthread.php?p=1992038#post1992038

Still first generation of MVs (no refining) and no prefiltering. mvtools2 script only.

[LeXXuz]

Looks clearly better, especially the skin tones and areas. :)

[tormento]

I am swearing with the ending of an anime.

The first part (https://pixeldrain.com/u/6o3sShft) of the ending is a resized 1080i 59.94, the second part (https://pixeldrain.com/u/3tedTfUr) is a true (I think) 1080i 59.94.

I want to convert them to 23.976 as the rest of the anime. For the second part, I tried to deinterlace and apply IVTC via DGTools but it becomes a mess, as its not a true telecined video.

For the first part I have no ideas too.

:thanks:

[Selur]

you could try TFM+sRestore

[tormento]

you could try TFM+sRestore
Is proper to deinterlace without downscaling?

Talking about the first video.

[kedautinh12]

I tried this script with the second part and it's a good result but you need to add some dehalo/dering and derainbow to get a better result

QTGMCp(Preset="Very Slow", Sharpness=0).SelectEven()
prefetch(2)

https://drive.google.com/file/d/1f2dMzKeZbTqxOwWrN2KoyPwg-WmZ3L4y/view?usp=drive_link

[tormento]

I tried this script with the second part
Thanks but the problem isn't deinterlacing but get a decent 23.976 ;)