my experimental (useless?) denoising filter

[E-Male]

ok, i had an idea and some time to program it:
http://e-rels.dyndns.org/downloads/nearavg.dll
http://e-rels.dyndns.org/downloads/nearavg.cpp

it's an early version without parameters and slow as hell, oh and yv12 only

what it does is this:
it takes the pixel and it's 8 spatial neighbors and then replaces the pixel by the one of the 9 that is closest to the average of them
luma and chroma are processed separately

maybe someone with much too much time will try it
maybe it can be developed further into something usefull
maybe not...

cu
e-male

[LeoH]

I'm getting 404 on that... :confused:

[E-Male]

whoops
fixed

[Dreassica]

It won't even load the dll for me.

[LeoH]

Hmm, well my first impressions are that it is very subtle at noise removal - slightly reducing some grain in my test of a VHS cap. But it softens the image more than I'd like for such a small noise reduction. I think some tweakable parameters would help this obviously. Still a good idea and I hope you keep working on it!

LeoH:)

[Didée]

it takes the pixel and it's 8 spatial neighbors and then replaces the pixel by the one of the 9 that is closest to the average of them.Ah, seems you're thinking in a similar direction I was thinking for several weeks now. (Ever heard of "Morphogenetic Fields"? A hypothesis by Rupert Sheldrake...)


maybe it can be developed further into something usefullYes, it can. OH YES, IT CAN!


/*Falls down on knees, makes praying hands*/ E-male - please oh pretty please, with sugar'n'honey on top, try to do the following:


1. Basic idea:

Take the 4 (or 3) out of the 8 neighbors that are closest to the center pixel. Replace center pixel with the average of those 4 (or 3) pixels.


Well, doing this directly probably won't give very impressing results. This must be modified:

2. Make a smart blur of the input frame. This is the reference frame. For development, a simple 3*3 blur is fully sufficient.

2a. Now, within the reference frame, search for those pixels most similar to the center.

2b. Then, in the input frame, replace the center pixel with the average of those pixels which positions were found in the reference frame.

3. Bingo! (hopefully)


The idea behind that is the following:

If we look on a frame at the level of 3*3 or 5*5 cells, we'll find that we are on the "detail level". In these small cells, the finest detail level is present: each pixel belongs to either one or the other side of a detail's edge. (Basically, at least.) Therefore, each of these small cells can be divided in two groups of pixels: the brighter ones, and the darker ones. What should be done is simply to average all pixels together that belong to the same group.

But: The input is noisy. The noise greatly disturbs the information what group each pixel belongs to: a "bright pixel" may appear to be a "dark pixel", and vice versa, due to the noise fluctuations.

Therefore, the evaluation of the pixels must be done on a blurred frame. Blurring will soften detail, but that doesn't matter. In contrary - after blurring, there is *much* cleaner information available if each pixel is lying on the bright or dark side of the detail snipplet presented in the cell. Then the center pixel must be replaced by the average of the other pixels that belong to the same group as the center pixel. And that's just it :)

This should work pretty good on 3*3 cells already. Even better would be to work on 5*5 cells, and to divide each cell in 3 groups of pixels: the "brights", the "darks", and those that are "on edge" - the "gradients".


I tried to script it, but practically it's almost impossible. By scripting, there is no possibility to "calculate around" with pixels, and then make per-pixel decisions based on the calculation's result. And what must be done here basically is: sorting pixel values, and finding their median value ... no chance to script that.
Then I tried to make it through logics, LOTS of comparisons, and creating crazy masks, in order to achieve the desired effect. It is possible, but the script runs at << 0.01 fps, eats GigaBytes of memory for breakfast, and is not even working fully correct. This simply is not within the scope of an avisynth script. /*scratches head*/


Try it, E-male. Or someone else with a little time at hand. I am absolutely convinced this will give a wonderful spatial denoiser!


- Didée



P.S.

In principle, this idea can be transported to the temporal domain, too ... ;)

[E-Male]

sounds interesting, and shouldn't be too hard to add to the existing filter
when i'm back home at sunday i'll look into implementing it

i'd need some details:
to 2.
what you mean with "smart blur"?
to 2a.
most similar to the center-pixel in the referene-frame, not the input-frame, right?
to 2b.
so i use the positions found in the reference-frame but the values from the input frame, right?

[Mug Funky]

didee, would that work? i'm thinking edges will be preserved after a blur, but "corners" will be destroyed in the process.

there's a catch-22 here - to distinguish detail from noise we must first eliminate the noise, but to eliminate the noise we must distinguish it from detail.... and so on.

however, a smart blur could help this... it depends how smart it is :)

(did i not get the point? i'm not sure...)

[morsa]

Smart blur....Maybe 2D cleaner, or MipSmooth???

[cypher_soundz]

Could some one up the .cpp file plz?
The link to it is not worknig.
regards
cyph

[Manao]

Didee's idea in filter (http://jourdan.madism.org/~manao/DideeBlur-v0.1.zip)

[rfmmars]

Link is still dead for me.

http://e-rels.dyndns.org/downloads/nearavg.dll
http://e-rels.dyndns.org/downloads/nearavg.cpp

Still dead as 6-17-2004 11:00am MST




richardLink OK 6/19/04

[Didée]

Manao, big thanks you for putting this together! Runs indeed a little faster than my script ;), and with more reasonable results.

Hmh. Seems not too impressing. Obviously, taking 4 pixels for averaging is too much for 3*3 cells. Could it be parametrized wether to use 4, 3 or 2 pixel positions from the reference frame? I assume 2 would be optimal, 3 could also be usable.
Or perhaps: take 3 most-similar pixel positions, and the closest 1 of the non-similar? - Probably not, but simply trying it out could help.

But the current one is not bad as it is, if we:

- fall back to "subpixel operation" (supersampling)
- use the filer itself as smartblur

Like this:
factor = 1.5
orig = last
ox = last.width
oy = last.height
xx = round(ox*factor/8)*8
yy = round(oy*factor/8)*8

orig.lanczosresize(xx,yy)

blurr = blur(1.0)
test1 = dideeblur(last,blurr,15)
test = dideeblur(last,test1, 7).lanczosresize(ox,oy)

# stackvertical(test,orig)
# interleave(orig.selecteven, test.selecteven)
test

return last
This looks much better, but of course is much slower.

I'd be curious how a filter variant working with 2 pixels would turn out. Should be somewhat similar to kassandro's RemoveGrain, but (perhaps) even better. I'm still convinced it has advantages to reference an already de-noised frame, instead of the original. It *IS* a catch22-problem, Mug Funky ;) - but OTOH, when doing anything thresholded on a noisy input, you'll always either miss targets because they were brought beyond threshold by the noise, or do much unneeded damage by raising the thresholds. Working with a de-noised reference should help a lot. It only needs some fiddling to find out a well suited method.


- Didée

[E-Male]

sorry for the dead links, my pc was off when i was away over the weekend, back online now

about the filter didée and manao pu together:
i'll work on another version on it soon, got some tweaks in mind