New filter - DeGrainMedian for grain removal

[Fizick]

Almost all filter-writers had wrote at least one denoiser.
Now i also did it.
DeGrainMedian - Spatio-Temporal Limited Median filter for grain removal
Plugin for Avisynth 2.5
Version 0.1 - August 13, 2004
DeGrainMedian plugin is mainly for film grain removal, but may be used for general denoising.
It uses some spatio-temporal limited median method.

The plugin is mostly based on two ideas, used at two stages of processing.
The first idea is from STMedianFilter plugin by Tom Barry -
I also use part of its great optimized code.
Shortly it is 3D cube, and 13 opposite pairs.(see doc)

STMedianFilter at second stage don't filter pixel if it would change it more than some threshold.
(More correctly, it firstly makes temporal detecting and thresholding,
with following spatial detecting and thresholding)
The most denoisers also do not filter pixels with big noise (big luma differences).
But both such single pixels are very visible on flat smoothed regions,
and group of them forms very annoying edge artifacts.
Moreover, pure temporal denoisers often produce ghosting even for not very big thresholds.

DeGrainMedian at this second stage uses other (different) method of pixel processing.
Here I use second idea, borrowed from Dust plugin by Steady. See part of Dust doc here:
"Limit=5
Sets the strength of the temporal filtering. (How much it can change the
original pixel)." (The end of Steady citation).
So, if filtered pixel would be changed more than some threshold (limit), it will NOT be restored to original in DeGrainMedian,
but will be changed by LIMITED value.
(separately for Luma and Chroma)

DeGrainMedian plugin filters ALL pixels, but with limited strength.
Therefore, all grain and strikes are (fully or partially) smoothed.
Filter almost don't produce edge artifacts.
Ghosting is also minimal, since it is automatically switched to spatial smoothing as more nearest pair by values.

So, DeGrainMedian have a speed of STMedianFilter (true) and strength of Dust (true by limit value :)!

It is almost not joke :devil:

What is bad, really?

1. Some blurring (you want denoising without it ? :) .
2. It can not produce flat fully smooth picture, some noise always remains (you like oversmoothed and blocked clips ?) .
3. Motion compensation is limited to 1 pixel value (of course, it is NOT Dust, but what about external motion compensation ?).

Seriously, it is not super filter and still experimental, but for my noisy grainy films it makes quite good results.
Firstly I developed it as prefilter (to Vaguedenoiser etc), but it seems now, that sometimes it can be used alone...

Probably, it can help somebody else. That is why I release it.

http://bag.hotmail.ru

[Didée]

Well, I haven't tested it yet, but after reading the docs I can say that I like the concept. :)
Some additional scenechange protection would be nice. Although there won't occur major blending artefacts on changes, thanks to the limited temporal processing, some noticeable artefacting may come up when relatively big thresholds are used.

Thanks for sharing this, Fizick.

- Didée

[Fizick]

Some additional scenechange protection would be nice
and may be added later if "concept" will well not only in docs, but in real tests. :)
I wait its, welcome.

[krieger2005]

Hi,

i've done some tests on a VHS-Capture with dot-crawls. While your filter makes with high thresholds the picture little smooth i used a mask only to use it for edges. The result was the removing of dot-crawls (wibbling edges). I don't look enoght at different results at the moment, i mean what does the filter done when there was motion or in still images... As long as i remember the picture get smooth in motion...

A second try was on a use on a DVD-source, which was compressed littlebit to hard, so the deinterlacing produce some artifacts (with KerneDeInt, still with high thresholds). Here i used low thresholds of your filter and deinterlace then. The Result was a clean image.

So for now i think, this is right good and should be developed... for example to get a "interlaced"-flag ;). I will try it even more..

Thanks for your work

[Fizick]

Krieger2005,
Thanks for test report.
I think about "interlaced" flag.
But seems it works without it somehow :confused:

But firstly I must fix one bug.
For spatial part of YV12 mode smoothing, the neibours distance form central pixel was 2 pixels instead of 1.

New version 0.1.1 is released.

[numlock]

Will this filter work with interlaced sources also ? Or is this only for progressive ?

[krieger2005]

Will this filter work with interlaced sources also ? Or is this only for progressive

So for now i think, this is right good and should be developed... for example to get a "interlaced"-flag .

So for now it has no interlaced flag and should used with
mpeg2source(...)
SeparateFields
Interleave(SelectEven.DeGrainMedian, SelectOdd.DeGrainMedian)
Weave

[Serg Belyansky]

I should say that judging on fine-but-contrasty grain removal this filter is the best of all I tried. Its effect reminds me he effect of high-end DVNR devices I've seen at TV-stations recently.

Also I can't see any need for scene change detection: with parameters set at (7, 10) there are positively _no_ scene change artifacts.

(BTW, thank you for DeSpot filter. Too bad E-Gold still refuses my payments :angry: )

[Yuri]

Fully agree with Serg Belyansky.
With my DVD content it worked a miracle. Blurring is not as strong as Dust's. With film content it was teamed with removedirt and that time both of them worked a miracle! Thanks for great filters.

[Fizick]

Serg Belyansky,
Yes, scenechange detection is not very needed, since the filter is adaptively switched to spatial part.

Yes, most other filters do no remove contrast grain.

Yuri,
Thanks for test report and warm words!

But somebody (i4004) notes some oversmoothing.
I too. Anybody else? :)
Probably I can add more safe (but not so effective) mode in new version soon.

[Fizick]

New version 0.2 released.
Since version 0.2, I add to DeGrainMedian also some other more safe operation modes 1-4 (code borrowed from RemoveGrain plugin by Kassandro).
We consider the same 3x3x3 cube.
New pixel candidate value (newp) is clipped by values of neighbor pixels from pair (bound1 and bound2).
But mode1-mode4 use more safe criterion (weight) of optimal pixel pair,
We take into account also the difference of new value from central pixel old value (oldp) .
Mode=0 of DeGrainMedian (similar to mode=9 of RemoveGrain) uses weight=|bound1 -bound2|, it is strongest (used in v.0.1).
Mode=1 of DeGrainMedian (similar to mode=8 of RemoveGrain) uses weight=|oldp - newp|+2* |bound1 - bound2|
Mode=2 of DeGrainMedian (similar to mode=7 of RemoveGrain) uses weight=|oldp - newp|+ |bound1 - bound2|
Mode=3 of DeGrainMedian (similar to mode=6 of RemoveGrain) uses weight=2*|oldp - newp|+ |bound1 - bound2|
Mode=4 of DeGrainMedian (similar to mode=5 of RemoveGrain) uses weight=|oldp - newp|, it is weakest.

Set mode parameter to bigger value if you found picture as oversmoothed (try!).
The more mode number, the bigger limit values is safe.
It seems, that mode=1 (now default) is good compromise, but it is dependent from source.
With mode=4, the result changes are invisible, but some of "hot" pixels are removed.


Try, and please post test reports.

[Serg Belyansky]

Mode=1 is the best default indeed. Though I think that i will use Mode=0 from time to time, since it removes very-high-contrast grain somewhat better.

[Mug Funky]

fizick: bravo, man. this denoiser is very cool indeed.

i always liked removedirt, but am prevented from using it for real-world situations, because of a nasty problem it has with "earthquake" style shaking, like you find in almost all anime at some point.

degrainmedian gives similar results but doesn't "remove" large sections of the frame in juddering motion. :)

i've only used it at default settings, so i'll see what i think of it after some experimentation (i'm working on converting Eva platinum edition to PAL. i never got a good look at the old eva, but considering that the "renewal" edition was seen as a dramatic improvement, the originals must have been complete crap, because the new ones are pretty awful too.)

[Fizick]

Yes, it is "cool" - it removes "hot" pixels :)
BTW, for numlock et al, I release "interlaced" version 0.3 today.

[Soulhunter]

Finally I found some time to do a quick test with this filter !!!


First conclusion...

Its very fast (14fps -> 12fps) !!!

It drops the filesize 5-10 % (regular movie) without blurring too much... :)


Bye

[Yuri]

Fizick,
great filter!
Are you planning to incorporate despot/descratch features into degrain? That would be splendid if you did!
Thanks and regards.

[Fizick]

incorporate despot/descratch features into degrain? :confused:

[Yuri]

To simplify the script chain: despot-descratch-degrain ...

[Fizick]

No.

[Fizick]

I release version 0.4, September 10, 2004 - Fixed possible bug with different frame pitches (prev, current, next).
Quite exotic, but possible case.
I forget to note this limitation of previous versions.

[kassandro]

Fizick,
I just made a look at your solution of the non-constant pitch problem. It is fairly optimal, i.e. bitblts are minimised, but instead of malloc you should use alloca. Executing malloc 100000 times you risk that heap fragmentation may exhaust memory. Use alloca instead. alloca simply allocates the memory on the stack and automaticly releases it, when you leave the stack frame of the function, where it was allocated.

[Fizick]

Kassandro,
I think my solution is better.
1. The stack size is limited.
2. Why do you think that all 100000 frames have different pitches?

[kassandro]

Originally posted by Fizick
Kassandro,
I think my solution is better.
1. The stack size is limited.
2. Why do you think that all 100000 frames have different pitches?
Fizick,
take a 720x576 with 100000 frames and execute the following somewhat bizarre script:

MPEG2Source("input.d2v")
cropped=crop(128,0,0,0) # the luma pitch is 720
even=selecteven(cropped).undot() # the resulting pitch is 720 - 128
odd=selectodd(cropped)
interleave(even, odd) # the pitch alternates between 592 and 720

if you do so, the pitch will alternate all the time and all in all malloc/free will be called 200000 times by your most recent DeGrainMedian. That should not happen in a good program. Now the malloc chunks alternate between two sizes. When I tried the above script it did execute correctly without heap fragmentation. But if the situation gets more complicated it may happen.
Now alloca - unlike malloc - is fast and simple. Essentially only the size of the memory chunk is subtracted from the stack register, which is restored to its value before the function call, when you leave the function. The problem is that the stack is very limited, though 800 kb should not be a problem at all. I really don't understand why the stack is limited. If the stack is exceeded and a page fault happens the system system should simply supply new real memory to enlarge the stack.

[Fizick]

But it is limited.
And why you do not consider resizing to 4096x4096 ? ;)

[kassandro]

Originally posted by Fizick
But it is limited.
And why you do not consider resizing to 4096x4096 ? ;)
One should try it, to see what happens.

The heap after all is also limited by virtual memory.

[Leak]

Originally posted by kassandro
if you do so, the pitch will alternate all the time and all in all malloc/free will be called 200000 times by your most recent DeGrainMedian.

Why not just keep the last used buffer around instead of freeing it and only replace it if it's too small? Or free it only after it hasn't been used for a number of frames...

np: Funki Porcini - King Ashabanapal (Parts In The Post (Disc 2))

[kassandro]

Originally posted by Leak
Why not just keep the last used buffer around instead of freeing it and only replace it if it's too small? Or free it only after it hasn't been used for a number of frames...


The problem with this solution is, that the size of the buffer depends on the pitch of the frame in the middle and in theory this pitch may change from frame to frame. If it gets bigger, the old buffer is simply too small and you have to reallocate. On the other hand, you don't want to spend too much time with this kind of problem, because in practice it virtually never happens.

[Leak]

Originally posted by kassandro
The problem with this solution is, that the size of the buffer depends on the pitch of the frame in the middle and in theory this pitch may change from frame to frame. If it gets bigger, the old buffer is simply too small and you have to reallocate. On the other hand, you don't want to spend too much time with this kind of problem, because in practice it virtually never happens.

That was what I meant, just keep the buffer around and only re-allocate it when it's too small - unless you have a video where the pitch grows with every frame (which seems rather unlikely to me) you'll just re-allocate a few times throughout the video and not on every frame as in the worst-case scenario you mentioned before...

np: Lali Puna - Middle Curse (Scary World Theory)

[SpikeSpiegel]

Great work, Fizick!

Anyway I wanna make a few considerations:

-Usually, the only way to clean correctly (with no artifacts risk) low-motion areas is to use a spatial smoother (or a spatio-temporal denoiser in this case), but, this way, blurring is inevitable.
-Temporal filters can easily erase (almost) all noise in static areas with no definition lost, but they behave randomly with motion ones (especially when the chroma difference between details is really low).

I noticed that, using DeGrainMedian, "mode2" is the best compromise between blurring and definition (IMHO, mode1 is too aggressive), but, this way, the filter can't remove enough noise in static areas, omnipresent in DVD stuff.
That's why I'd like to suggest this combination:

DeGrainMedian with mode2/3
+
RemoveDirt with low mthreshold (<40 should be fine), to let it work only on static areas

I did a few tests and results seem very good!

[Fizick]

SpikeSpiegel,
You of course must select the most appropriate filters and parameters combination for your source.

BTW, you also may try to use several DeGrainMedians with sililar of different parameters:

DeGrainMedian(4).DeGrainMedian(3)

:)

or

DeGrainMedian().DeGrainMedian().DeGrainMedian().DeGrainMedian()

:D

[SpikeSpiegel]

Originally posted by Fizick
SpikeSpiegel,
You of course must select the most appropriate filters and parameters combination for your source.

BTW, you also may try to use several DeGrainMedians with sililar of different parameters:


Sure, anyway I'm not saying that this is an universal solution, but, in this case (I forgot to write that it should be used with hi/medium-quality-DVD-source, xcuse me), considering how the two filters work, it could be the best way to get a good cleaning (and a better compressibility) without compromising too much definition.

This is because I think that it's (almost) impossible to get total adaptiveness (for each "variable", like motion and color characteristics, in the same frame) with only one filter&setting (even if DeGrainMedian's pretty close :) ), so the better way should be to use different filters only in the "situation" (frames and part of frame) where they work better.

Anyway, it seems that for every other purpose (4example cartoons), there's no need of anything else! ;)

[Fizick]

SpikeSpiegel,
You also may try TTempSmooth temporal filter by Tritical.
It is qute good (for classical pure temporal), since it can uses 5-7 frames.

[SpikeSpiegel]

Originally posted by Fizick
SpikeSpiegel,
You also may try TTempSmooth temporal filter by Tritical.
It is qute good (for classical pure temporal), since it can uses 5-7 frames.

Thanks for the tip!
I was searching for a precise temp. filter with no mod8-resolution limitation (unlike RemoveDirt), so it may be really useful!

[Fizick]

BTW, I realy have got good result for VHS with this double without other denoisers:

degrainmedian(mode=1,limity=8,limituv=8,interlaced=true)
degrainmedian(mode=0,limity=4,limituv=4,interlaced=true)

[HighInBC]

This is a great filter, but would it be possible to have a variable radius on the median blur? so that I can have a far more pronounced effect. This of course would make a video look very blurred, but it would be perfect for alpha masks

[Fizick]

Do you understand, that DegrainMedian is not spatial, but spatial-temporal denoiser? ;)

[HighInBC]

Oh, well I would give my left eyebrow for a purely spacial, variable radius median filter

[Fizick]

New Version 0.5, March 23, 2005 - Use more correct cache settings. Thank to Turyst04 for stablity problem report.

There was some problem of v.0.4 with my GetDups plugin.
Setmemorymax(128) help a little, but...
I did not use cache settings.

Now I use CACHE_RANGE=2.

I really wonder, in what case we must use CACHE_RANGE=1 ?

[kassandro]

Originally posted by HighInBC
Oh, well I would give my left eyebrow for a purely spacial, variable radius median filter
My RemoveGrain filter is purely spatial, has many more modes and is a lot faster, especially if you have a SSE3 capable CPU. However, as DeGrainMedian it is radius 1 only anf there are very good reason for it. The execution would explode quadraticly with the radius and there are other implementation specific issues. It is faster (execution time increases only linearlily) to iterate these filters and the effect is very similar. For instance radius=4 is similar to a fourfold iteration of these filters.

Originally posted by Fizick

Now I use CACHE_RANGE=2.

I really wonder, in what case we must use CACHE_RANGE=1 ?

CACHE_RANGE=1 is always bad, because you use the previous and the subsequent frame and there difference is simply 2 and not 1. If you would use CACHE_RANGE=1, then each input frame is processed twice. If you would use CACHE_RANGE=0, then each input frame would be processed processed even three times. CACHE_RANGE=1 would be sufficient if you use only the previous or the subsequent frame but not both. Of course, the speed gain, which you will achieve with CACHE_RANGE=2 heavily depends on the speed of the filter immediately before DeGrainMedian.

[tsp]

My RemoveGrain filter is purely spatial, has many more modes and is a lot faster, especially if you have a SSE3 capable CPU. However, as DeGrainMedian it is radius 1 only anf there are very good reason for it. The execution would explode quadraticly with the radius and there are other implementation specific issues. It is faster (execution time increases only linearlily) to iterate these filters and the effect is very similar. For instance radius=4 is similar to a fourfold iteration of these filters.

http://forum.doom9.org/showthread.php?s=&threadid=84636
Although this filter was first created a couple of weeks after HighInBC requested it and it's not that slow because the speed only decreases linearly with radius (using the same principle as bucket sort). This algorithm doesn't work with MMX so for the small radius a batcher sort is used (with increases n*(log2(n))^2).

[Fizick]

Is CACHE_RANGE directional (forward, backward)?
or it is a radius from current frame?
or it is a simply counter (limit) of cached frames?

[kassandro]

Originally posted by Fizick
Is CACHE_RANGE directional (forward, backward)?
or it is a radius from current frame?
or it is a simply counter (limit) of cached frames?
Last time, when I studied the Avisynth source code, it was a radius from the last processed (not necessarily the current, i.e. last requested) frame. Meanwhile the Avisynth cache seems to have undergone substantial changes. As this subject is quite important for filter writers to achieve maximal performance without wasting memory, it should be better documented.

[Fizick]

kassandro,
thanks for respond.
Yes, it would be nice. Now i try change radius and look to memory usage, speed, faults, etc.

[AVIL]

Hi:

By suggestion of Ficik i continue here the trhead :

http://forum.doom9.org/showthread.php?s=&threadid=92327

[AVIL]

Hi:

Sorry for my inexperience, i can't post the attachement. I pick the file ( extension jpg, size 103 kb ) whith the button "Examinar" (in spanish). The name of the file appears in the window but, alas, no attachement is made. Surely i'm doing anything erroneus.

Help please.

[Soulhunter]

Originally posted by AVIL

Sorry for my inexperience, i can't post the attachement. I pick the file ( extension jpg, size 103 kb ) whith the button "Examinar" (in spanish). The name of the file appears in the window but, alas, no attachement is made. Surely i'm doing anything erroneus.

Help please.

Just upload ya pic here... (http://imageshack.us/) ;)


Bye

[AVIL]

Tanks Soulhunter, you have saved one soul (the mine).

This is the image (click to enlarge):

http://img93.exs.cx/img93/4748/degrainmedian9xs.th.jpg (http://img93.exs.cx/my.php?loc=img93&image=degrainmedian9xs.jpg)

The first part is the image as is, the second part is the difference ( luma only ) after apply degrainmedian(mode=1,limituv=0) in YUY2 and done posterior conversion to YV12. The third part is the difference after degrainmedian(mode=1,limituv=0) directly in YV12. I have expected the same luma difference ( or very approximate ).

Ouch, post a problem is near as difficult as doing complex scripts.

[Fizick]

New Version 0.6, April 2, 2005 - fixed some bugs with YUY2. Thanks to AVIL for report.

But I am stil can not get STRICTLY same output foe YUY2 as YV12.:confused:

Ouch, fix bugs problem is near as difficult as doing complex scripts.

[Guest]

Re: Attachments.

There is a delay because the mod has to approve them and we don't check every 10 minutes for pending ones. :)

So my personal policy is that I welcome a PM from you when making an attachment to notify me that it is pending.

Thank you for your interesting contributions to the forum.

[JuanC]

@Fizick: I've been using this filter lately, I like it a lot, but I have just noticed that my interlaced clips are getting encoded with the field dominance reversed: They are originaly TFF, but just after using DeGrainMedian (interlaced=true) they become BFF :confused: Is there anything I am missing ?

My clips are originally mpeg2 video clips (using dgdecode) either 352x480 or 720x480 (NTSC, 29.97 fps). My scripts are very simple, something like the following will be enough to reproduce the situation I am describing:DGdecode_mpeg2source("Paulita-mpeg2-corta.d2v")
#Return AssumeTFF().SeparateFields()
Return DeGrainMedian(interlaced=true,mode=2).AssumeTFF().SeparateFields() Then I encode them to mpeg2 interlaced.

Thanks for your comments (and the filters ;) )