Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th August 2004, 16:45   #1  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
New filter - DeGrainMedian for grain removal

Almost all filter-writers had wrote at least one denoiser.
Now i also did it.
DeGrainMedian - Spatio-Temporal Limited Median filter for grain removal
Plugin for Avisynth 2.5
Version 0.1 - August 13, 2004
DeGrainMedian plugin is mainly for film grain removal, but may be used for general denoising.
It uses some spatio-temporal limited median method.

The plugin is mostly based on two ideas, used at two stages of processing.
The first idea is from STMedianFilter plugin by Tom Barry -
I also use part of its great optimized code.
Shortly it is 3D cube, and 13 opposite pairs.(see doc)

STMedianFilter at second stage don't filter pixel if it would change it more than some threshold.
(More correctly, it firstly makes temporal detecting and thresholding,
with following spatial detecting and thresholding)
The most denoisers also do not filter pixels with big noise (big luma differences).
But both such single pixels are very visible on flat smoothed regions,
and group of them forms very annoying edge artifacts.
Moreover, pure temporal denoisers often produce ghosting even for not very big thresholds.

DeGrainMedian at this second stage uses other (different) method of pixel processing.
Here I use second idea, borrowed from Dust plugin by Steady. See part of Dust doc here:
"Limit=5
Sets the strength of the temporal filtering. (How much it can change the
original pixel)." (The end of Steady citation).
So, if filtered pixel would be changed more than some threshold (limit), it will NOT be restored to original in DeGrainMedian,
but will be changed by LIMITED value.
(separately for Luma and Chroma)

DeGrainMedian plugin filters ALL pixels, but with limited strength.
Therefore, all grain and strikes are (fully or partially) smoothed.
Filter almost don't produce edge artifacts.
Ghosting is also minimal, since it is automatically switched to spatial smoothing as more nearest pair by values.

So, DeGrainMedian have a speed of STMedianFilter (true) and strength of Dust (true by limit value !

It is almost not joke

What is bad, really?

1. Some blurring (you want denoising without it ? .
2. It can not produce flat fully smooth picture, some noise always remains (you like oversmoothed and blocked clips ?) .
3. Motion compensation is limited to 1 pixel value (of course, it is NOT Dust, but what about external motion compensation ?).

Seriously, it is not super filter and still experimental, but for my noisy grainy films it makes quite good results.
Firstly I developed it as prefilter (to Vaguedenoiser etc), but it seems now, that sometimes it can be used alone...

Probably, it can help somebody else. That is why I release it.

http://bag.hotmail.ru
Fizick is offline   Reply With Quote
Old 14th August 2004, 15:45   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
Well, I haven't tested it yet, but after reading the docs I can say that I like the concept.
Some additional scenechange protection would be nice. Although there won't occur major blending artefacts on changes, thanks to the limited temporal processing, some noticeable artefacting may come up when relatively big thresholds are used.

Thanks for sharing this, Fizick.

- Didée
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 14th August 2004, 18:15   #3  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Quote:
Some additional scenechange protection would be nice
and may be added later if "concept" will well not only in docs, but in real tests.
I wait its, welcome.
Fizick is offline   Reply With Quote
Old 18th August 2004, 10:37   #4  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
Hi,

i've done some tests on a VHS-Capture with dot-crawls. While your filter makes with high thresholds the picture little smooth i used a mask only to use it for edges. The result was the removing of dot-crawls (wibbling edges). I don't look enoght at different results at the moment, i mean what does the filter done when there was motion or in still images... As long as i remember the picture get smooth in motion...

A second try was on a use on a DVD-source, which was compressed littlebit to hard, so the deinterlacing produce some artifacts (with KerneDeInt, still with high thresholds). Here i used low thresholds of your filter and deinterlace then. The Result was a clean image.

So for now i think, this is right good and should be developed... for example to get a "interlaced"-flag . I will try it even more..

Thanks for your work
krieger2005 is offline   Reply With Quote
Old 18th August 2004, 18:13   #5  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Krieger2005,
Thanks for test report.
I think about "interlaced" flag.
But seems it works without it somehow

But firstly I must fix one bug.
For spatial part of YV12 mode smoothing, the neibours distance form central pixel was 2 pixels instead of 1.

New version 0.1.1 is released.
Fizick is offline   Reply With Quote
Old 19th August 2004, 00:20   #6  |  Link
numlock
Registered User
 
Join Date: Dec 2002
Posts: 171
Will this filter work with interlaced sources also ? Or is this only for progressive ?
numlock is offline   Reply With Quote
Old 19th August 2004, 07:47   #7  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
Quote:
Will this filter work with interlaced sources also ? Or is this only for progressive
Quote:
So for now i think, this is right good and should be developed... for example to get a "interlaced"-flag .
So for now it has no interlaced flag and should used with
Code:
mpeg2source(...)
SeparateFields
Interleave(SelectEven.DeGrainMedian, SelectOdd.DeGrainMedian)
Weave
krieger2005 is offline   Reply With Quote
Old 20th August 2004, 08:58   #8  |  Link
Serg Belyansky
Registered User
 
Join Date: May 2002
Posts: 18
I should say that judging on fine-but-contrasty grain removal this filter is the best of all I tried. Its effect reminds me he effect of high-end DVNR devices I've seen at TV-stations recently.

Also I can't see any need for scene change detection: with parameters set at (7, 10) there are positively _no_ scene change artifacts.

(BTW, thank you for DeSpot filter. Too bad E-Gold still refuses my payments )
Serg Belyansky is offline   Reply With Quote
Old 20th August 2004, 21:32   #9  |  Link
Yuri
Registered User
 
Join Date: Aug 2003
Posts: 55
Fully agree with Serg Belyansky.
With my DVD content it worked a miracle. Blurring is not as strong as Dust's. With film content it was teamed with removedirt and that time both of them worked a miracle! Thanks for great filters.
Yuri is offline   Reply With Quote
Old 20th August 2004, 22:12   #10  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Serg Belyansky,
Yes, scenechange detection is not very needed, since the filter is adaptively switched to spatial part.

Yes, most other filters do no remove contrast grain.

Yuri,
Thanks for test report and warm words!

But somebody (i4004) notes some oversmoothing.
I too. Anybody else?
Probably I can add more safe (but not so effective) mode in new version soon.
Fizick is offline   Reply With Quote
Old 21st August 2004, 20:52   #11  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
New version 0.2 released.
Since version 0.2, I add to DeGrainMedian also some other more safe operation modes 1-4 (code borrowed from RemoveGrain plugin by Kassandro).
We consider the same 3x3x3 cube.
New pixel candidate value (newp) is clipped by values of neighbor pixels from pair (bound1 and bound2).
But mode1-mode4 use more safe criterion (weight) of optimal pixel pair,
We take into account also the difference of new value from central pixel old value (oldp) .
Mode=0 of DeGrainMedian (similar to mode=9 of RemoveGrain) uses weight=|bound1 -bound2|, it is strongest (used in v.0.1).
Mode=1 of DeGrainMedian (similar to mode=8 of RemoveGrain) uses weight=|oldp - newp|+2* |bound1 - bound2|
Mode=2 of DeGrainMedian (similar to mode=7 of RemoveGrain) uses weight=|oldp - newp|+ |bound1 - bound2|
Mode=3 of DeGrainMedian (similar to mode=6 of RemoveGrain) uses weight=2*|oldp - newp|+ |bound1 - bound2|
Mode=4 of DeGrainMedian (similar to mode=5 of RemoveGrain) uses weight=|oldp - newp|, it is weakest.

Set mode parameter to bigger value if you found picture as oversmoothed (try!).
The more mode number, the bigger limit values is safe.
It seems, that mode=1 (now default) is good compromise, but it is dependent from source.
With mode=4, the result changes are invisible, but some of "hot" pixels are removed.


Try, and please post test reports.

Last edited by Fizick; 22nd August 2004 at 22:07.
Fizick is offline   Reply With Quote
Old 26th August 2004, 12:14   #12  |  Link
Serg Belyansky
Registered User
 
Join Date: May 2002
Posts: 18
Mode=1 is the best default indeed. Though I think that i will use Mode=0 from time to time, since it removes very-high-contrast grain somewhat better.
Serg Belyansky is offline   Reply With Quote
Old 27th August 2004, 16:44   #13  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
fizick: bravo, man. this denoiser is very cool indeed.

i always liked removedirt, but am prevented from using it for real-world situations, because of a nasty problem it has with "earthquake" style shaking, like you find in almost all anime at some point.

degrainmedian gives similar results but doesn't "remove" large sections of the frame in juddering motion.

i've only used it at default settings, so i'll see what i think of it after some experimentation (i'm working on converting Eva platinum edition to PAL. i never got a good look at the old eva, but considering that the "renewal" edition was seen as a dramatic improvement, the originals must have been complete crap, because the new ones are pretty awful too.)
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 27th August 2004, 22:06   #14  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Yes, it is "cool" - it removes "hot" pixels
BTW, for numlock et al, I release "interlaced" version 0.3 today.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 28th August 2004, 00:46   #15  |  Link
Soulhunter
Bored...
 
Soulhunter's Avatar
 
Join Date: Apr 2003
Location: Unknown
Posts: 2,812
Finally I found some time to do a quick test with this filter !!!


First conclusion...

Its very fast (14fps -> 12fps) !!!

It drops the filesize 5-10 % (regular movie) without blurring too much...


Bye
__________________

Visit my IRC channel

Last edited by Soulhunter; 28th August 2004 at 00:50.
Soulhunter is offline   Reply With Quote
Old 28th August 2004, 10:58   #16  |  Link
Yuri
Registered User
 
Join Date: Aug 2003
Posts: 55
Fizick,
great filter!
Are you planning to incorporate despot/descratch features into degrain? That would be splendid if you did!
Thanks and regards.
Yuri is offline   Reply With Quote
Old 28th August 2004, 19:48   #17  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
incorporate despot/descratch features into degrain?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 28th August 2004, 20:26   #18  |  Link
Yuri
Registered User
 
Join Date: Aug 2003
Posts: 55
To simplify the script chain: despot-descratch-degrain ...
Yuri is offline   Reply With Quote
Old 29th August 2004, 20:43   #19  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
No.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 12th September 2004, 20:29   #20  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I release version 0.4, September 10, 2004 - Fixed possible bug with different frame pitches (prev, current, next).
Quite exotic, but possible case.
I forget to note this limitation of previous versions.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.