VagueDenoiser 0.35.1: a 2D Wavelet denoiser/smoother

[Lefungus]

A 2D wavelet denoiser/smoother.

It transforms each plane into the wavelet domain, then apply one kind of thresholding on the obtained coefficients, to reduce data that should be noise. With a low threshold you'll erase noise, whereas with higher threshold, you'll smooth the picture. It's somewhat effective against blocking too. Judge it by yourself :)

Grab it on my site (http://perso.wanadoo.fr/reservoir/) or on Kurosu site (http://kurosu.inforezo.org/avs/VagueDenoiser/index.html)

See the proper documentation into the package on how to exactly use it.

Required:
Avisynth 2.52
YV12/YUY2 colorspace

[Edited]
Removed obsolete things

[Kurosu]

Originally posted by Lefungus
To be updated soon, Fastsymmlet8 seems to give good results for deblocking ( don't ask me why :) )

Because Symlets are approximately symetric (d'oh) and Daubechies wavelets aren't?

*acts as if he really knew what he was talking of*

Anyway, that's a very good start (awaited in its results by many user), and indeed there can be many things to expect from the update. Is the method used now somehow linked to "waveshrink" (ie zeroing of too small coefficients) ?

[Lefungus]

OK, i've removed the bug
Now symmlet works again

it's quite slow though

For low filtering, begin at 0.5
strong filtering at 1-2

The method i use is soft thresholding, ie same as waveshrink it seems
All coefficients under the threshold are zeroed and those above are reduced to limit artefacts.

[d'Oursse]

hello,

i wanted to try your plugin but it requires libmmd.dll. I think it is the library that comes from the intel compiler. Am I right ?

Another possibillity is that you compile statically your plugin.

thanks !

[Dreassica]

U replaced the old dll with the new one yet? Cuz i am still getting a garbled picture when using the FastSymmlet8 routine (1).

[Lefungus]

Updated to v 0.12

-Precisions problems corrected
-The dll shouldn't need any other dll
-Symmlet8 supported

The source is also released, free.
I have indeed used Daniel Lemire code, the original file is included with the package.

Next step would be to implement other wavelets like Haar and also include other colorspaces.

[SILICON]

What is the Difference between VagueDenoiser and NIc's DCT filter?

Both make DCT and zeroed coefficients under one threshold.

[Kurosu]

The biggest difference (and I assume that's the reason why this filter is good at smoothing blocking artifacts) I see is rather the DCT is done locally, on a 8x8 block with no regard to neighbours in the image, while DWT operates on a whole image at once. Therefore, this introduces in the case of DCT discontinity problems on the 8x8 block' borders.

But maybe Lefungus has chosen an intermediate solution , though processing times are a hint that he's transforming a whole planar (plane?) at once.


So, what about to code Haar wavelet and verify if it is even better?(just kidding)

Actually, it is because symlet8 have 8 vanishing moments. DB4 have only 4. The smoothness of wavelet reconstruction increases with the number of vanishing moments.

Indeed, Haar has only one moment and doesn't have such a good result. Though I would have though wavelets with more than 3 vanishing moments (ie whose recontructed data is at least C² and therefore should smooth out all blocking artifacts) would have been sufficient, hence it wouldn't explain such a difference between the 2 used in this case.

Anyway, this gives me ideas for detecting other kinds of noise...

[edit]
Another (http://infolab.snu.ac.kr/thesis/IJ2.pdf) approach, that tries to identify the part of the decomposition that is due to blocking.

[Lefungus]

Properties of wavelets transformations allow them to be used on the whole picture, so that's what i've done. That's one of the strong point of this filter, i think.

I've played with it a bit, some kind of distorsion appear when the threshold is too high, like vertical or horizontal bars. I'll experiment with hard-thresholding to see if it change anything. But for denoising/smoothing purposes, a better way than thresholds should be found.

In the end i'd like to have a multi-purpose wavelet filter, any help would be appreciated :)

[Acaila]

I've played with it a bit, some kind of distorsion appear when the threshold is too high, like vertical or horizontal bars. Zeroing high frequencies will create ringing artifacts near edges (can easily be seen when you encode a video/image with a low bitrate), is this the kind of artifacts you are refering to? If so then maybe weighting the threshold based on the frequency could reduce this, i.e. a higher threshold for low frequencies and a lower threshold for high frequencies.

Just a thought.

[Lefungus]

Unfortunately, if i have understood correctly wavelets, left coefficients are not low frequencies and high coefficients, high frequencies like in DCT domain. So it's more complex.

Anyway it may be interesting to scale/reduce coefficients instead of zeroing them.

[Kurosu]

It seems to me it's more a result like http://members.optushome.com.au/emikulic/gfx/haar/cat-output.png

[MfA]

As far as ringing is concerned the cause is the Gibb's phenomenon same as with DCT, no changes there.

The horizontal/vertical lines however are not ringing, just typical artifacts of these kinds of denoisers ... I dont really know the cause, but my guess it has to do with the isotropic nature of the basis functions and the aliasing inherent in critically sampled transforms.

Using (pseudo) translation invariant transforms usually helps a lot (in fact with the undecimated wavelet transform for instance you can just use hard thresholding without having to suffer the usual artifacts). By their very nature these transforms have to be expansive though, so the coefficients will outnumber the pixels ... bringing with it higher memory and computation requirements.

[Lefungus]

Version updated to 0.2: lots of changes included, lots of wavelets implemented (15!)
a little faster
See the first post for more informations

There are no artifacts anymore when you select an high threshold, the picture is just highly smoothed
And you can finally select the number of steps, so Haar wavelet with 3 steps should be faster than Antonini with 8 steps.

Sometimes you can have a wrong picture, it means you've used too much steps, so reduce it until it works again.

now i just need to test all this :)

[Lefungus]

In my first tests Brislawn2 wavelet seems to give really good results

[Edited]

On Ghost In The Shell movie, i'm trying this :

VagueDenoiser(threshold=2.5,wavelet=7,method=1,nsteps=4)

It smooths blocks nicely, reduce interlaced artefacts, but still keep details and edges. And noise is killed :)

[Lefungus]

I think i will make some presets like convolution3d.
It definitely seems that the Brislawn 2 function gives the best results. At least it was on my own tests, both on an anime (GITS) and and a movie (Brotherhood of the wolves)

Speed isn't so bad after all, with 6 nsteps on a 640x256 movie, it's ~20 fps on my p4 2.4GHz

So here are my findings:

For a high quality movie:

VagueDenoiser(threshold=0.75,wavelet=7,method=1,nsteps=6)

Threshold can be increased up to 1/1.25, even 2 but you may lose fine details, it's visible a little on skins for example.

For a high quality anime:

VagueDenoiser(threshold=2,wavelet=7,method=1,nsteps=6)

Some may find 2 a little too strong.

On these movies, i've put VagueDenoiser before resizing, if you put it after, you may need to decrease the threshold.

Now i'll seek bad quality movies/anime and try to enhance them :)

[lamer_de]

k, here's a short test result:


avisource("M:\UM Raws\um7.avi",false)
Levels(0,1.000,236,0,255)
VagueDenoiser(threshold=4,wavelet=7,method=1,nsteps=4)


Using Athlon XP 1700+, Avisynth 2.5.2, VDubMod 1.4.13 build14328. I'm encoding to lossless VBLE in YV12 at 2fps with the script above (The levels is just in there cause the source is to dark for my eyes). The same script without VagueDenoiser gives me around 30 fps. The source is an Anime, 640x480, which has some slightly visible blocks in ultra-high motion scenes, only ringing in the rest of the episode. Lower thresholds gave the same speed, but almost no visible difference. With this high threshold, the blocks got slightly smoothed out (but were still visble). Ringing wasn't touched at all in my eyes.

The speed/visible quality gain is too low for me to put this filter in use.

The smoothing effect was better on a lq real life footage video full of blocks, but BlindPP delivers better results with more speed imho (If you have to work with such horrible sources full of blocks)


CU,
lamer_de

[Lefungus]

Well, it's not really designed to remove blocks but noise instead. Maybe it'll be possible later with a fine analysis of the wavelet domain. Wavelets open many possibilities, i've just tried the most simple one.

Anyway, any tests are welcome.

About speed, you're right, it's not usable yet to rip, too much slow. I only use it under vdub now, doing comparisons. i guess what i really need is a fast asm-optimized dwt transform like xvid has for DCT. Now it's pure C code full of memory leaks so that doesn't help :) !

i'll try to improve it, but don't expect much from me on the speed side.

[sh0dan]

There are (fortunately) several ways of optimizing this routine - some that springs to mind.

Edit: Silly me hadn't seen you had already done 1) in 0.2 - I'm leaving it there for understanding. ;)

1) Use floats instead of double. There seem to be (unused?) routines that already does this. It would make assembler optimization possible, and enable better cache efficiency. I don't know if the precision is good enough - Lefungus - what do you say?

2) Assembler int -> float -> int conversion. Since we have a large array this is fairly trivial. I havne't looked at the access patterns in the transform, but it might be a good idea to actually do the conversion there.

3) SSE / SSE2 transforms. This should be able to at least double the speed of the transform/inv transform. Not that trivial, but probably this is where the biggest gain can be found with the current algorithms. Unfortunately none of my current CPU's support SSE, so I might have to get a new MB first. ;)

4) The Intel C++ compiler should be able to squezze out some percent more performance.

If you would try out implementing 1), I wouldn't mind doing 2) and 3).
4) Can be tried now, if anyone has the compiler, and knows how to use it.

We should probably move further dev. talk to the dev. forum.

Edit 2: Did some profiling - the results are here (http://forum.doom9.org/showthread.php?s=&threadid=57084).

[Lefungus]

Version upgraded to 0.21
-The filter doesn't require SSE anymore
-Some improvements from Shodan and Bidoche increase performances up to 30%

[MfA]

Why wouldnt it be true?

[Lefungus]

I've made some comparisons on GITS

the original avisynth script is:

mpeg2source("C:\Vobs\Ghost In The Shell\Movie.d2v",CPU=0,IDCT=5)
telecide(order=1,guide=2)
crop(6,14,708,546)


It's for the first picture

For the second picture, same script but with CPU=6

And third picture, same script, CPU=0, but with VagueDenoiser

You'll see it clean quite well the last interlaced artifacts that have passed through Decomb.

Anyway, results are good too even without interlaced artifacts, i think.

Small comparison (http://perso.wanadoo.fr/reservoir/)

[Lefungus]

Version updated to 0.22:
- YUY2 colorspace support
- Optionnal chroma filtering, (chroma=true/false)
- Cosmetic changes, code cleaned
- Html documentation
- Villa 10/18 filter added

"wavelet" parameter has been renamed to "filter".
The wavelet filters has been sorted.

[unplugged]

Sorry, latest version crashes VDub/Avisynth :(, even with suggested values
I have tried 768x576 and 720x576 res with YUY2 and YV12.

[Lefungus]

I've encountered the "stupid typo error" bug, sorry, i've uploaded version 0.221

[Lefungus]

In order to improve speed, the filter may have to lose flexibility and choose one filter over the 15 today.

My opinion is that Brislawn 10/10 (filter=7) gives the best results.
Haar and any Daubechies filter are always ugly. The others gives fairly similar outputs.

With high tresholds (15-20), Brislawn keep more features than any other filters. That's why i think it could be better. But i use the denoiser with thresholds around 2 so it may be different.

On thresholding, i've found that soft thresholding (method=1) gives a more smoothed result so a lower threshold is required. But the output is more natural, and i've seen some artifacts with hard-tresholding (method=0). So i think to stick with method=0 is a good idea.

More results on various sources will be helpfull, as i can't do as many tests as i'd like to do.

[sh0dan]

Otherwise you could always just call the optimized functions for filter 7. But it seems like a good idea.

Minor stuff:
__inline unsigned char VagueDenoiser::realToChar (float x) {
return (x>255?255:(x<=0?0:(unsigned char)(x+0.5))); };

If you did an unrolled transform/invert of filter 7, you might already get a speedup - and it would be easier for us to assembler optimize it. :)

Edit: Why is there smilies in my [code] section? :(

[Leuf]

I can't seem to get the plugin to load, I just get LoadPlugin: Unable to load vaguedenoiser.dll. Do I need something in addition to 2.5.2?

[Lefungus]

I really don't know :/
i don't even use loadplugin on my computer, i just call it in my script.
And i only use avisynth 2.5.2, nothing exotic installed.

[Lefungus]

Version updated to 0.23:
-Code cleaned, filters class removed.
-Better parameters for compilation. Should really works on every cpu now.
-Removed all filters except Brislawn 10/10, so filter parameter has been removed.(no more filter=7).
-A little speed increase (3-5fps on a 640x256 frame with nsteps=6).

[Lefungus]

Version updated to 0.24:
-Another speed increase, some critical loops unrolled (To infinity and beyond!) :)

I reach 18 fps for a 640x256 YV12 movie. It's still slow but better than 12 fps ! Quality should be unchanged.

[Leuf]

Well, I don't know how to explain it but I was able to compile 221 and it works, my compiled version is 68 kb vs the 96 kb in the download - so we definitely have some different compiler settings. I can't see how that would make it not load though. I have the same problem with 24, and I can't get the source to compile. At least I have something to play with now though.

[Kurosu]

@Leuf
If you have duplicated symbols (namely the float arrays), move them from wavelet.h (Vaguedenoiser doesn't need them) to wavelet.cpp. Another needed modification is replacing
#include "wavelet.cpp"
by
#include "wavelet.h"

If that still doesn't work (it worked for me), use the usual trick

#ifndef __SOMETAG__
#define __SOMETAG__
...
#endif


This way, the .h file doesn't get parsed several times.

To reply to your other concern, the filesize difference is probably due to the compiler trying to do costly optimizations such as loop-unrolling, and maybe padding.

[Lefungus]

There's a bug in latest version, so until it's fixed keep sure res is at least mod16 or even mod32.

[Lefungus]

Version updated to 0.241:
Visual artifacts were seen when clip width or clip height weren't mod4.
So i've added checks at initialisation.

It should be bug-free and crash-free again. And visual output should be exactly the same than with older versions with filter=7

Thanks Kurosu for your trick with "ifdef", it's corrected too.

@Leuf:
Can you try the latest version and tell me if you have problems using it ? Which compiler are you using ?
I use icl 7.1 and checked Blended option. That mean it should works on every cpu on the planet. i don't see any other option that could prevent you to use it. Maybe someone experienced with icl could help there.

[Lefungus]

another version: 0.242 ("The bug wasn't dead")
-I've really removed the nasty bug now. it should be pixel-perfect equal with output given by 0.221, and this with no restrictions on width and height. So no more mod4.
-Some improvements from Kurosu.

Note that YV12 is the favorite mode of this filter, YUY2 will work but slower.

[Leuf]

Your latest compiled dll still doesn't work for me. I'm using MSVC 6 sp5/pp5.

[Lefungus]

Originally posted by Leuf
Your latest compiled dll still doesn't work for me. I'm using MSVC 6 sp5/pp5.

I'm confused, what doesn't work on your computer, my own compiled dll or are you trying to compile it yourself ?

[Leuf]

I can compile the source fine with msvc and it works, but your dll doesn't work for me. I get the error right at LoadPlugin, it just says unable to load the file as if it couldn't be found. The only thing I can think of why it wouldn't work for me but does for others is I'm on win98. I would think the problem would center on avisynthplugininit2() and how it is getting exported. But if it works for everyone else *shrug* I wouldn't worry about it too much.

[Kurosu]

@leuf
What's exactly your CPU ?

[Lefungus]

version updated to 0.243:
-Code cleaned

No more versions from me for a little while. I've squeezed speed as much as possible with pure C.
Compilation parameters should be almost optimal too. It could be made a little faster for p4, but it'll require two dlls.

Time to get some holidays now :) !

[Kurosu]

I finally managed to get some optimization done into this filter... Using 3DNow! (ie only AMD processors will be able to use it, sorry, SSE2/P4 enthusiasts), I got an mind-breaking speed improvement of... 17% for 0.241, on which it's based.

For people really using that filter, and after a speed improvement, even the tiniest, get the filter here (http://kurosu.inforezo.org/avs/VagueDenoiser.zip). And for those that can't run the filter for whatever reason, a VC6 compile is included...

[Lefungus]

Version updated to 0.25:

-some speed increase (profile & vectorization)
-defaults added (threshold=2,method=1,nsteps=3,chroma=true)
-a visual.net dll is also included for compatibility purposes. It's a lot slower though


3Dnow optimisations from Kurosu aren't included yet, i'll try to include it later.

Output could be slightly different from previous versions with height and width not mod32. But it's not noticeable and have no impact on compressibility. (I have 97db instead of 103 db on my compared clips when i use threshold=0)

I have now 22fps on a 640x256 YV12 movie with these settings
Vaguedenoiser(threshold=2.5,nsteps=6)
It's above realtime when i don't process chroma.

[morsa]

As Leuf said: "I get the error right at LoadPlugin, it just says unable to load the file as if it couldn't be found. The only thing I can think of why it wouldn't work for me but does for others is I'm on win98."

Well the same applies to my situation, WIN98SE.
No way to load it, and I'm the most stupid guy in the world for compiling soft.

[Lefungus]

Originally posted by morsa

No way to load it

Could you try this version (http://perso.wanadoo.fr/reservoir/dl/VagueDenoiser-0.251.rar), and tell me if it works better ?

[morsa]

Well managed to compile it myself and....It works now!!!
Don't know why but it is functioning perfectly and is very fast for me.
I'll try your new version tonight.
Thank you

[Boulder]

What's the status of this fine filter, any new versions coming out?

[Lefungus]

0.26 was released a while ago in avisynth development forum. And 0.27 is coming too with nice improvements, thanks to Kurosu.

[Kurosu]

And here is 0.27 (http://kurosu.inforezo.org/avs/VagueDenoiser/index.html)

This time, I'd like some report. I suspect 0.26 could crash easily, yet nobody reported it...

@Lefungus
Feel free to mirror. FU2 Development thread.

[edit]
Even better maybe, could you edit your first topic so as to link to it (mine or the one you would host)?

[Mr_Khyron]

can VagueDenoiser be used on interlaced videos or do i have to deinterlace?
:confused: