mClean spatio/temporal denoiser v3.2 (01 March 2018)

[burfadel]

I'll see what I can do regarding memory when I can, most likely tomorrow. I think I know the cause and possible useful alternative options.

[MysteryX]

Originally Posted by Groucho2004

If the application that uses 32 Bit Avisynth is built with the LARGEADDRESSAWARE linker flag it can address up to 4 GB on a 64 Bit OS.

I'll have to test this some more

Without SetMemoryMax

FPS (min | max | average):      0.762 | 28.59 | 6.270
Memory usage (phys | virt):     1710 | 2022 MiB
Thread count:                   35
CPU usage (average):            77%

With SetMemoryMax

FPS (min | max | average):      0.264 | 61.48 | 6.238
Memory usage (phys | virt):     2201 | 2510 MiB
Thread count:                   35
CPU usage (average):            79%

I'm not seeing much of a difference here (8-bit)

Yes, I noticed that 16-bit is a LOT slower; it's either FFT3DFilter or MvTools2 that isn't well optimized. To know for sure, we'd have to test on a Y8 clip. If it's 2x slower in 16-bit, it's FF3DFilter. If it's 5x slower, it's MvTools2.

mClean crashes on a Y8 clip!

[Groucho2004]

Originally Posted by MysteryX

I'm not seeing much of a difference here (8-bit)

Try it with a 1080p 16 bit source.

[MysteryX]

Originally Posted by Groucho2004

Try it with a 1080p 16 bit source.

Generally not much point in running MvTools2 in 16-bit anyway.

Unless your source is a 10-bit clip, this goes at the beginning and your source is 8-bit.

Unless you have a Deblocker (it doesn't support 16-bit yet) and you get that output in 16-bit; then perhaps it would be best to have that deblocking integrated within mClean.

[Groucho2004]

Originally Posted by burfadel

I'll see what I can do regarding memory when I can, most likely tomorrow. I think I know the cause and possible useful alternative options.

Considering the results I got, I don't think the memory consumption is excessive. MCTemporalDenoise for example uses a lot more memory from what I recall.

Of course, if you run 8 threads with a temporal filter you need plenty of memory.

[burfadel]

Well, no luck trimming the memory down, but on x64 systems it shouldn't be an issue at all! It's a limitation of MVTools2, I think there may be a way to trim it a bit (script wise) but I'll have to test it.

In terms of testing, I've posted an updated script for people to play with. Only 'minor' changes (this term is relative depending on what you are referring to!).

[MysteryX]

Have you made corrections to color distortions?

It would be useful to know what the changes are so we can know what to look for.

[burfadel]

Difference is just heavy tweaking to the noise analysis for speed and quality. It's based on the latest MVTools2 and Avisynth by pinterf, not sure how it will perform using the old versions. The analysis is used for two different things, so the settings chosen are based on leveraging the most ideal settings for both. Fvec and bvec 2 through 4 is noise analysis purely, to a large extent at least the motion analysis is meaningless apart from determining what are moving objects and what is temporal noise. The quality of the motion therefore isn't overly important considering the temporal radius. The important motion analysis is either side of the current frame, which is why the analysis is more precise there. I believe slight colour issue is resolved, it was just an issue with tweaking in preparation for the next feature.

[MysteryX]

Differences with v1.7 are very minor in most cases. This frame shows more differences.

no denoise / mClean 1.3c / mClean 1.7(enh=7) / mClean 1.7b



First, enh is now almost good with its default value. Just slightly too strong. However, because it's a value relative to the resolution, it's hard to tweak it manually. Let's say I want to always set it 10% weaker, I need to look at the script and calculate it based on the resolution every time.

Then, there are slight improvements over 1.7

However, I still get better results with v1.3c!

Finally, look at "Home Karaoke", it looks fine in v1.3 but appears ugly and distorted in both v1.7 and v1.7b

Does v1.7 work better in other clips such as the snowflakes clip?

There are still color distortions but it's slightly better than 1.7. It might be enough to cross the line where it's 'acceptable', although it is still an undesirable effect for a denoiser.

v1.3c is still the one that works best according to my tests. Perhaps there could be an option for an alternative mode (MDegrain2 or MDegrain4) when a clip doesn't react well.

[lansing]

Ok here's a good all around sample clip with temporal and spatial grain for the OP to test. It's a pain to watch MysteryX testing a denoiser with videos that have no grain and vcd that has nothing but compression artifacts.
sample

[burfadel]

Thanks, I'll check that out as soon as I am able .

[manolito]

Out of curiosity I applied my usual weapon of choice (DegrainMedian plus FineSharp) to lansing's sample. I had to convert the source to DVD compliant MPEG2, my computer does not do HD.

I liked the result quite a bit, and just for fun I made another conversion using Kassandro's old DenoiseSharpen plugin in mode 22. And even this conversion doesn't look too bad.

Needless to say that I cannot use mClean on my old machine, and even on a more capable Core i5 CPU the low speed is not worth the small improvement (which almost nobody will be able to detect when watching the clip - not stills).

Get the results here:
http://www101.zippyshare.com/v/v89ieI57/file.html
https://www.sendspace.com/file/kvtmts


Cheers
manolito

[burfadel]

I'm not doing mClean to support older systems, I'm trying to leverage the latest versions of the plugins used without compromise if possible. Temporal denoisers work differently to spatial denoisers. Spatial denoisers work on luma/choma differences over a given area for a given frame. They can remove noise but can also wipe out finer detail. They also don't work well on temporal changes as you can still see temporal noise in the video. As a result of the way spatial denoisers work, areas of difference such as lines etc also become less clear, so the result is effectively a lower spatial resolution image in terms of the detail. This is bound to happen due to the averaging over a given area of pixels that the spatial denoiser does. The stronger the spatial denoiser (like removegrain(mode=20), the lower the effective spatial resolution.

I just tried mClean against knlmeanscl(s=5,d=1,h=2.0), removegrain etc, I have to say I like the results of mClean best using just mClean() default settings... obviously you would expect me to say that, but it's true! I'll tweak some things a little more on the luma side of things, then go to work on the chroma adjustments. I've already made some minor adjustments that result in a tad better speed.

[MysteryX]

Originally Posted by lansing

It's a pain to watch MysteryX testing a denoiser with videos that have no grain and vcd that has nothing but compression artifacts.

It's good to test on a variety of formats
- SD with encoding artifacts
- HD with grain only in limited areas
- grainy video

You're testing as a degrainer, but it's doing a pretty good job as a denoiser.

Burdafel... Burfadel, anything you can do to fix the image distortions pointed above? I'd want to move on from v1.3c

[burfadel]

I wasn't going to post it as there are only minor changes, but try 1.7c. I'm not claiming it will fix what you described, but the output may be slightly different.

[Taurus]

@ lansing:
Thanks for the sample!
After all this illnoised stuff MysteryX posted, what a relief
I think the dancing grain is artificial generated?
Mad producers .
So far good ol TemporalDegrain seems to be the only script that can handle this stuff eyepleasing.
(Thanks to Sagekilla and Didee for this wonderful piece of work!)
I shranked your 55Mb file down to 5.05Mb downscaled with BicubicResizeMT(1280,720,-0.4,0.20) + Temporaldegrain
With mClean nearly the same (5.44Mb).
Warning! Temporaldegrain is deadslow, but worth the effort!
Sorry for being a little offtopic.

[burfadel]

It's not really off topic, it's an aim of mClean to not be too slow so that's good. The are still other minor improvements that can be done. There is something I can do that could be considered overkill, but if it doesn't affect speed too much and it is useful then why not .

[lansing]

Originally Posted by MysteryX

It's good to test on a variety of formats
- SD with encoding artifacts
- HD with grain only in limited areas
- grainy video

You're testing as a degrainer, but it's doing a pretty good job as a denoiser.

This just goes down to common sense, you don't test a vacuum cleaner on a clean floor.

Originally Posted by burfadel

then go to work on the chroma adjustments. I've already made some minor adjustments that result in a tad better speed.

Just take off the fft3dfilter filter on chroma, desaturation problem fixed.

Originally Posted by Taurus

@ lansing:
Thanks for the sample!
After all this illnoised stuff MysteryX posted, what a relief
I think the dancing grain is artificial generated?
Mad producers .
So far good ol TemporalDegrain seems to be the only script that can handle this stuff eyepleasing.
(Thanks to Sagekilla and Didee for this wonderful piece of work!)

MCTD surpassed it. And KNLMeansCL(s=4,h=3.0) did pretty good as well

[burfadel]

I found knlmeamscl to cause detail or edge sharpness loss. Denoising is ineffective if it also limits visual fidelity. The idea for mClean is to have something that's suitable for most situations, and be simple to use, and also not to be too slow. There's still a lot to do on it. As for chroma, it's just the sigma settings. There's a couple of things I haven't added for that yet also.

The texture like noise isn't removed so much in that anime image, also remember you can set renoise to 0. Noise removal doesn't have to be absolute, some noise can be visually pleasing watching at a normal distance, without those overly flat surfaces. This of course depends on the type and quality of noise. mClean is temporally stabilised which is why it is relatively bandwidth friendly with renoise.

[MysteryX]

Originally Posted by lansing

This just goes down to common sense, you don't test a vacuum cleaner on a clean floor.

You have the pre-conceived idea that this is a degrainer that should be compared to TemporalDegrain and other degrainers.

Originally Posted by burfadel

The idea for mClean is to have something that's suitable for most situations, and be simple to use, and also not to be too slow.

This filter was never meant to be limited to grains. It is more meant as a replacement to KnlMeansCL denoising while preserving more details.

In fact, the tests so far show that it's giving better or more consistent results as a denoiser than as a degrainer.

Since it's meant to be suitable for most situations, it's good to test various scenarios. In my case, I'll test the videos I'm actually using it on (and the most difficult cases). I have very little videos with grain, except camera footage.