preparing a filter comparison: removing grain

[E-Male]

i'm planing a filter comparison and would like your input

the target is: removing grain
to be more precise make grainy clips (mostly old movies) as compressable as possible without killing too much detail (avoiding a "flat" look)
i do not want to make the video look better, just compress better (so the compressed result will look better at same b/p)

besides my own filter (if i get it working in time) i by now plan to try these:
-Convolution3D
-FluxSmooth
-VagueDenoiser
-Dust
-MVDenoise

any recommendation for other filters, recommended parameters to start with, tips or warnings for doing the comparison, ... please post them here

thx
E-Male

[lamer_de]

Please include MipSmooth (http://forum.doom9.org/showthread.php?s=&threadid=64940&highlight=mipsmooth)in your comparison.

Can't help you with the rest, I only encode anime which has no film grain :P

CU,
lamer_de

[MasterYoshidino]

heres a couple of filters which I like to use when I don't like to spatially filter a source.

ttempsmooth (http://forum.doom9.org/showthread.php?s=&threadid=77856) and removedirt (http://forum.doom9.org/showthread.php?s=&threadid=70856).

only spatial filters that tend to avoid that 'cartoonized' look are mipsmooth (http://forum.doom9.org/showthread.php?s=&threadid=64940) and vaguedenoiser (http://forum.doom9.org/showthread.php?s=&threadid=56871)

settings I tend to use are...
mipsmooth(method="SuperStrong",spatial=4,temporal=0
,spatial_chroma=4,temporal_chroma=0,scalefactor=0.5
,downsizer="bilinear",upsizer="bicubic",weigh=true)
vaguedenoiser(method=1,threshold=2,nsteps=6,chroma=true)
ttempsmooth(9,4,4,2,2)
RemoveDirt(athreshold=50,mthreshold=50,pthreshold=10,mode=2)
#note removedirt I personally only call
#in a conditional filter scheme, as it destroys motion
#mipsmooth line needs it's commas shifted to the first line

[Didée]

Are only plugins allowed, or are avisynth scripts valid, too?

I'm currently scripting some things around some ideas ... (and some seem quite promising ;) ) - although these, if proved useful, should be coded as plugins anyways: toooo much overhead through scripting.

BTW, when comparing existing plugins, my bets go to Dust. All of my bets.


#mipsmooth line needs it's commas shifted to the first line
For this there is the "\" sign (line wrapping) ...

- Didée

[E-Male]

@lamer_de& MasterYoshidino:
thx, i'll have a look at those

@Didée:
i didn't think of scripts/functions
name them, i'll look at them, then i'll see

[yaz]

@e-mail
yes, removedirt, vague & mip must be considered too but ... if grain is what i think of it'd be quite frustrating. i've wasted tremendous amount of time with this 'hopeless' task. 'complete' removal may 'cartoonize' the film, 'incomplete' operations may even worsen. instead, you (& i & we :-) should try to decrease it to a reasonable level & give enough bit to encode the remaining. yeah, quite cheap solution but the result seems much better for me than any other (ok, just my personal taste)

@masteryoshidino
khmmm ... a lot of dirty incompressible cartoon ... isn't it ? :-)

the bests
y

[E-Male]

i know there is no perfect solution
that's why i wanna compare the filters
to see which comes closest

[Manao]

E-Male : I made an new release of the MVTools, because I rewrote MVDenoise, so you should get the new version before testing. Get it here (http://jourdan.madism.org/~manao/).

To use it, follow the example :backward_vectors = source.MVAnalyse(isb = true, lambda = 2000)
forward_vectors = source.MVAnalyse(isb = false, lambda = 2000)

return source.MVDenoise(backward_vectors, forward_vectors, tht = 10, sadt = 300)

There are new undocumented parameters :
* for MVAnalyse : lambda is used to give some coherence to the field of vectors. The higher the lambda, the more coehrent the field. Try values between 1000 and 10000. More will give you bad vectors, sad-wise. Less will make the field to sensitive to noise.

* for MVDenoise : 'tht' is a threshold which works as luma_threshold for TemporalSoften, but motion compensation allows you to use ( theorically ) much higher thresholds without having ghosting. 'sadt' is the sad threshold. If the block moved has a threshold higher than sadt, it isn't used to denoise. 'sadt' is a measure of the validity of the motion vectors, and 300 is a good start value. Try between 100 and 500.

I would say ( without much testing ) that :
* tht = 5-10, sadt = 100-200 : weak denoising
* tht = 10-15, sadt = 200-300 : medium denoising
* tht = 15-25, sadt = 300-500 : strong denoising

Finally, you'll see that the second frames before and after a scenechange have artifacts : I know it, and it will be corrected soon ( I hope ).

For the rest of the set of tools, the core engine is much faster ( 120 fps on a 704x416 frames, it's faster than MPEG2 decoding in avisynth ), but subpel refinement is disable yet, and the filters who use the vectors are slow. However, you should still see some speed up for MVMask and MVCompensate.

[Fizick]

E-Male, is your test sources really grainy ? ;)
I have very grainy films. See my post in
http://forum.doom9.org/showthread.php?s=&threadid=63270
Also in that thread you may foun a link to Kassandro RemoveGrain filter (pre-release), and one my script for degrain.

If speed is not important, try new Vaguedenoser with WienerChop method (fps/2), and averege of 2 denoised shifted clips method (fps/2 plus) :)

[E-Male]

looking forward to trying that
i couldn't try mvdeboise yet
the concept seems promising to me, but i wonder how well it'll work with grainy material

[Manao]

It averages 5 consecutives frames, compensating the previous two and the next two to bring them on the current frame, and taking into account only pixels under the threshold. There are no spatial filtering at all ( thought since it only makes pel compensation, it's a kind of spatial filtering, but not intended ).

I don't know yet how I could take advantage of the motion vectors to make a spatial filtering in addition to the temporal one. Perhaps a slight blur whose strength is adapted to the movement, dunno.

If you want to try further on that path, you can try the following script :backward_vectors = source.MVAnalyse(isb = true, lambda = 2000)
forward_vectors = source.MVAnalyse(isb = false, lambda = 2000)

spatially_filtered = source.deen(a2d, 10, 0, 2)
motion_mask = source.MVMask(fvectors)
temporally_filtered = source.MVDenoise(backward_vectors, forward_vectors, tht = 10, sadt = 300)

result = temporally_filtered.MaskedMerge(spatially_filtered, motion_mask, Y=3, u=1, v=1)

return resultAnother detail, the previous filtering only filters the luma channel. I still need some interpolating & fast functions to do the chroma in MVDenoise, but I prefer first to add half pel precision to the whole set of filter.

[kassandro]

Thank you, Fizick for mentioning the prerelease of my RemoveGrain filter. I have just released the first official release of this filter. It is a lot faster and 5 new modes have been added. For the competition I recommend RemoveGrain(mode=4) (identical output as the prerelease with RemoveGrain(threshold=4). For more information goto www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf).

[E-Male]

in a very rough test ttempsmooth clearly won, closely followed by mvdenoise

i start wondering if instead of one comparison, i should just make it a continouse project, that might result in some tweaked filters

[Wilbert]

Did you try PeachSmoother (with dot=true and readout=true to get the noise estimate)?

[E-Male]

added to mental list

[Fizick]

in a very rough test ttempsmooth clearly won, closely followed
by mvdenoise
I think, that we need in some objective comparison method (and not only for this grain test).
May be, you (we, they) must:
1) start with clear original clip,
2) add some controlled value of noise (with some plugin)
3) clean clip with various filters,
4) make comparison of cleaned clips with original :
4a) by subjective visual criterion,
4b) by some objective criterion (for example, with SSIM plugin by Lefungus)

Similar technics were sometimes used in film restoration reports.

I have once more question. ;)
Is the grain noise additive or multiplicative? For grained films it is certainly multiplicative (the noise has bigger value in lighter regions). But it seems, that no one filter has such cleaning method (i now think about adding such option to Vaguedenoiser). Or it is unnecessary ? :confused:

[E-Male]

i personalyl will keep using real grainy sources and compare the result with my own 2 eyes
because i don't want to reverse an algorithm, i want better compression for video taht comes from grainy sources

about your question:
you suggest cleaning darker areas less than lighter ones?
i think many people here wil ltell you the oposite

[dragongodz]

you could probably add Undot to the list(and no i do not mean using it with deen for those that like that combination).

[E-Male]

undot alone is extremely mild
i'll only try it in combinations

[Fizick]

real grainy sources
May be clear original source of these exist somewhere?

[SansGrip]

New FluxSmooth?

:D

[Bulletproof]

Hi, try adding the TemporalSoften (mode 2) that comes with AVISynth to the comparison, it is very good on its own.

[E-Male]

thx, adding to list

Originally posted by Fizick
May be clear original source of these exist somewhere?
AFAIK grain is caused by the film-material the movie is shot with, so i don't see where a (non-filtered) clean source should come from
and if there is one, why use teh grainy version? :)

[dragongodz]

undot alone is extremely mild
i'll only try it in combinations
yes i know but sometimes just light filtering can look better. was just a thought. :D

[scharfis_brain]

some days ago, I had to deal with grainy Video cam footage.
It was filmed only from one point of view (a balcony) but with lots of shakes and panning.

so I did a preprocessing with deshaker to smooth out the shakes.

It left me with a *alloverthetime* panning video.

so, I firstly decided to use piexiedust, cause of its motion compensation. but pixiedust didn't remove all noise.
while (for example) temporalsoften(2,4,10) did, but left much artifacts due to panning, like all other denoisers.

so I decided using depan from fizick. My video is now clean as hell :) no visible artifacts, just a noise free & clean video :)

maybe depan should be added to this comparision as denoising-subset for all denoisers...

[E-Male]

if tried such a stabilizing filter (don't remember wihch) on a dvd rip and it didn't really work well
but if you think it'll help i'll look into depan

EDIT: i think i might have misudnerstand depan
could you go into detail a bit more
maybe post a script sample

[Fizick]

Yes, grain is caused by the film-material the movie is shot with
where a (non-filtered) clean source should come from It is really not simple problem, but it must be solved even if
you want deal with real films only (and not with artifical noise of AddGrain plugin by Tom Barry or similar).
For example, a long time ago I was a child and have a few 8mm films (cartoons copy), which are available on DVD now.
I don't know where are my films now, but probably similar source may be found (and captured).

why use the grainy version? - of course, for test only, not for real encoding.

Yes, Depan is a great tool :) for pure panning video.
But MVTools also may be used as motion compensation tools for different denoisers.

It seems, this thread never finished (too big "to do" list). :)

[E-Male]

yeah, list keeps growing

that's why i said i won't do one comparison, but instead make this a long-time project

[Arcon]

i like TemporalCleaner (http://forum.doom9.org/showthread.php?s=&threadid=37620), if it's not yet on your list.

[Fizick]

E-Male,
now you may add my special DeGrainMedian filter to list. :cool:

[E-Male]

just got home from "holydays", driven 4 hours and find work here, nice :)

[snek_one]

i am very interrrested in the overal outcome of this thread..
currently i have a bunch of old Spawn dvd's here which are, imho, pretty bad quality for being dvd's...

they are very grainy and there is interlacing during high action scenes..

now, it being a cartoon, which filter would get rid of the most mosquito noise? because there's lots and lots of it! there's also some slight blocking on some one-color-surfaces..

i'd be happy to test some filters myself, but if someone already knows the answer it would save me a couple of hours which i can use to study ;)