Degrain/noise filter

[Chengbin]

I'm using MVDegrain 3 as my filter right now. Is there any other filter that's better? I don't care about time. I've heard some GPU based degrain/noise filters, how are those compared to MVDegrain 3 in terms of quality?

[Bablu]

speaking from less experience, but give a try at FFT3DFilter, it's damn slow, but it gives the fruit is really sweet afterwards. As for MVDegrain, I have never tried so I don't know how that compares to FFT3DFilter.

[Terka]

if you want something good and slow, try nlmeans. you will care about time then.

[Sagekilla]

tnlmeans, FFT3DFilter, MVDegrain3, and dfttest are all very heavy weight denoisers. Some of the slowest I know of (that aren't a complex filter chain) but very good quality.

[Didée]

Of course, it also depends on the filter's parametrization if it does heavy weight denoising, or only a gentle one. Then, it's difficult to give a general advice, since sources can be so very different.

However: if a source has only rather faint noise or grain (me says: "MickeyMouse grain"), then almost any noise filter will do something reasonable, and the arguing will be about rather small differences - hey look at this pixel, but no, look at that tiny spot, this one is a bit sharper, but no, that one is more homogeneous ... splitting hairs is big fun.

I like to look at what a noise filter is able to achieve on more demanding sources. A filter that can cope with strong noise can always be adapted to act weaker, if needed. On contrary, if a filter can not deal reasonably with strong noise, then it can not. That's a difference. It's interesting to see where a filter reaches its limits.

Let's see ... what happens with one of my usual grain test sequences:



=> Sample <= (13MB, MediaFire)

(Note: intentionally decreased fps to 25% PAL, for easier evaluation of the differences.)


I won't comment too much on that. Shortly:

- FFT3DFilter gets quite soft in order to get a reasonably stable result. (DFTtest most probably would be slightly better and slower, but its general characteristics are quite similar to FFT3D.)

4 fps are 4 fps.


- MVDegrain3 with ME-prefilter is detailed and stable.

1.5 fps are 2.66 times slower than FFT3D, but the time seems well spent.


- TNLMeans is almost 27 times slower than FFT3D. Could've been set up to be more stable but kill even more detail, or to keep more detail, but keep the nasty LF flicker.
Might be a good choice for masochists when dealing with low-noise sources. For strong grain, the effort/result ratio seems rather poor to me, somehow.
(Remember those highly stunning before/after examples in the PDF paper?)

[Archimedes]

TNLMeans break out into blossom with the right parameters (e. g. Ax=Ay=10, Sx=Sy=3, Bx=By=1 or 0). But now we talk about fph.

[Didée]

Perhaps I've not the tune for TNLMeans, I'm not used to it. Suggest some more exact settings to try for such a grainy source? No matter which knobs I turn in which direction by how much, it seems to keep flickering and floating.

[*.mp4 guy]

Well its important to keep in mind that the ideal settings for tnlmeans would be.

ax=largest image dimension/2
ay=largest image dimension/2
az=framecount

bx=0
by=0

sx=largest image dimension/4
sy=largest image dimension/4

Under those conditions it would outperform everything by a large margin.


by using "sane" settings you are severely handicapping it, if you want an apples to apples comparison, quality (not speed) wise, you need to set:

ax=mvrange/bw*1.5 (assuming ow=bw/2)
ay=mvrange/by*1.5 (assuming oy=by/2)
az=mvdegrain number/bt number -1 divided by 2 rounded up

sx=blksize/ax
sy=blksize/ay

bx=0
by=0

[Didée]

Under those conditions it would outperform everything by a large margin.

Sure. And if Elementary Charge would've happened to be double as big as it is, then all matter would be just one quarter in size. Or not?

Short of running some several-weeks-running tests, I'll try those:

Common sense argument:
With not unreasonable settings, TNLMeans was already 10 times slower than MVDegrain, and still it looks definetly worse. To me, at least.
Okay, what now ... let's try settings that are 100 times slower? If not sufficient, lets try settings that are 500 times slower? If still not sufficient, suggest settings impossible to be carried out? Hooray!

Technical argument:
TNLMeans is based on a gaussian-weight, zero-mean assumption. Now, strong Film Grain compressed by Mpeg-2 at DVD-ish bitrates does not really fit that model ... and the difference between ideal model and rude reality will propagate into the result. When making TNL's similarity comparisons less blurry, it keeps the fingerprint of the base noise. When making the comparisons blurry enough to obfuscate the theory/practice difference, the fingerprint fades away. Along with the detail.

[*.mp4 guy]

TNLMeans is mostly usefull for still images, where a very long processing time can be accpetable under some conditions.

also, tnlmeans always outperforms denoisers with the same processing area it is using, which was primarily my point in the previous post. You are comparing a spatial radius 2, by temporal radius 2 filter versus much larger filters.

The common sense argument is valid, but the technical one is not, because all denoisers are based on similarly stupid assumptions. such as "representing non critically sampled images as a sum of periodic waves is a really good model for the content of the image". That assumption is still usefull though, because it allows fast denoising of low frequency noise, which cannot be achieved without making such sacrifices.

[Didée]

Well, TNLMeans definetly is a spatial denoiser, yes. Still images is what it originally was designed for. And funnily, even if the temporal extension with Az>0 is used, then in a sense it still is a spatial denoiser. It just gathers more spatial evaluation from the temporal neighbors. (As long as the search- and the support window are no collapsed to size 0 -- in that special case, TNLMeans becomes similar to ttempsmooth with a blurred 'pclip').

I really don't want to battle against TNLMeans. It's a good filter for the cases where it's appropriate. But after all, the topic started about video processing, not still images.


It's just when TNLMeans is touted as "THE" denoiser of all, if, oh if only you crank up the radii and are willing to invest hours ... days ... months ...

... exactly then it's time to point out that TNLMeans is in no way the holy grail, either. Often other filters can do a better job in less time.


BTW,

You are comparing a spatial radius 2, by temporal radius 2 filter versus much larger filters.

that's 5x5 x (2+1+2) = 125 pixels that TNLMeans had available to compute each output pixel. MVDegrain3 had 3+1+3 = 7 pixels available to compute each output pixel. That's not a "much larger" filter, no?

[Archimedes]

I only have experience with TNLMeans in spatial mode (image denoising). But I'm not working with the default values (Ax=Ay=4 and Sx=Sy=2). I don't like the results. Normally i'm using a search window of 21x21 (Ax=Ay=10) and a neighborhood window of 7x7 (Sx=Sy=3), as described in the paper. And, if the image has many details, I go with Bx=By=0. The results are much better than with the default values.

Here is an example. First image is the original, second is the denoised version with TNLMeans (spatial mode). Zoom in, 200 %, 400 % or more and look at the letters. TNLMeans only touch the noise. I never beat this results with other spatial denoisers. May be the results can be improved with higher values of the search and neighbourhood windows. I never tested it.

Original


TNLMeans


I never tested TNLMeans for video processing. But denoising still images (may be not all kind of source), the results are remarkable. And when i see, how long TNLMeans works for only one image (for the above image my notebook needs 4 min. and 18 sec. for denoising all 3 channels extra), i haven't any interessests using it for video processing.

[*.mp4 guy]

Originally Posted by Didée

BTW,
that's 5x5 x (2+1+2) = 125 pixels that TNLMeans had available to compute each output pixel. MVDegrain3 had 3+1+3 = 7 pixels available to compute each output pixel. That's not a "much larger" filter, no?

no, mvdegrain had (searchradius^2)*pi*7 pixels to use, so for searchradius 32 (on the low end, iirc) thats 8182.91712 pixels, not counting sub-pixels.

[Archimedes]

In general, TNLMeans takes advantage on the high degree of redundancy of an image (even on natural images).

So, another excellent field for TNLMeans is removing jpg artefacts and deblocking.

Original (saved with only 20 % jpg quality!)


Look, what TNLMeans are doing with this image.

TNLMeans (Ax=Ay=10, Sx=Sy=3, Bx=By=0, a=1.0, h_y=4.8, h_u=4.8, h_v=4.8)


You would think, the last image is the original.

But now, let's go back to the original discussion.

[Didée]

Originally Posted by *.mp4 guy

no, mvdegrain had (searchradius^2)*pi*7 pixels to use, so for searchradius 32 (on the low end, iirc) thats 8182.91712 pixels, not counting sub-pixels.

Impressing numbers of little interest. Point is, you're stretching the term "making use of" to fit your arguing. And imo you're overstretching quite a bit.
(BTW, I can't follow your actual calculation. "(searchradius^2)*pi*7" with "searchradius 32" is (32^2)*Pi*7 = 22518.93614 , not 8182.91712. ? )

Sure, MVTools will evaluate a gross number of pixels in the motion estimation stage. But most (almost all) of them will be *discarded*, hence not being used in the end.
Further, the "looking at thousands of pixels" is done by MVAnalyse, not by MVDegrain. What MVDegrain gets from MVAnalyse is the one single result that was sorted out as being the best-fitting. Hence, my statement was that MVDegrain3 uses 3+1+3 = 7 pixels for each input pixel to form the output pixel. This is correct. What's actually used in the end are only 7 pixels, not some thousands.

In difference, TNLMeans does indeed actually use all pixels within the Ax*Ay*Az search window. Each single pixel does get a weight and therefore is used to form the output pixel. So, with the settings I had used, for each input pixel TNLMeans did use (4+1+4)^2 * (2+1+2) = 405 pixels to form the output pixel. (Corrected from above ... the default radius of the search window is Ax,Ay=4 ... you had mentioned a spatial radius of 2, which is the radius of the support window, and I took it. Averaging is done over all pixels of the search window, not of the support window.)
_____

I don't see much point in beating this horse any more. Simplificated, it boils down to the fact that TNLMeans is a spatial denoiser (even with the temporal extension switched on). The difference in characteristics between spatial denoisers and temporal denoisers is well known.

[rkalwaitis]

Didee what script did you use for MVDegrain3? If you so not mind sharing.

[Sagekilla]

@Didee, can you show me an example of settings on (MVDegrain|FFT3DFilter)that would produce low frequency flicker? I don't know what it looks like, or at least my settings seem to have never produced it.

[*.mp4 guy]

Didée, by your logic, a median filter is only utilizing one pixel. also, motion compensated denoisers are spatial-temporal, not pure temporal.

(also, I have no idea what I did wrong when I solved the equation, it is indeed 22518.91712)

[Sagekilla]

I don't see how that works. If you look at a simple 1 px median filter, then you'd have spatial averaging of 1 center + 8 neighbor pixel. MVDegrain3 would do 1 center + 3 from backward frames + 3 from forward frames.

[*.mp4 guy]

Median filters do not average anything, they are nonlinear and only 1 pixel value is used in the output, much the same way only 1 pixel value is the output of mvanalyze->mvcompensate, even though both filters use many pixel values as inputs. My point is that the act of choosing between the inputs to find a good output is filtering in and of itself, so you must count the total number of inputs, not the total number of inputs that are actually used.