a smarter adaptive noise removal?

[McQuaid]

I think this idea has been thrown around before but here goes:

I capture alot of stuff off tv and various shows/commericals fade to/from black. When a fade reaches black, noise is at its most apparent. If spliced all together this black area might add up to 30secs of video.

Could some type of adaptive noise filter analyse this and become some type of smart filter better than what we already have? Or because noise is normally random that to the filter's it doesn't make a difference if it can have an 'only noise' sample?

[tenebrenz]

I too think this warrants a little more interest. One idea would be for it to work in the frequency domain of the video in a similar way to audio noise reduction. I think some preliminary work was done some time ago in this area and some filters for AviUtl were released, unfortunately it was all in Japanese and didn't get very far around here.
I see getting the noise sample to be a little more difficult than using black frames because often video, especiallly tv, can have coring(?) where everything below a certain threshold is set to black, this means that the amound of noise is black areas may be lower than the actual average of noise in the video.
Also I previously had an idea of an adaptive filter that used a combination of spatial and temporal filtering depending on whether an area of the image was considered moving or not, a non-moving area would use temporal smoothing and retain full detail whereas a moving area would be treated to spatial smoothing. I wrote an avisynth script that showed this worked but it was very slow (and a little unstable), if an optimised filter were produced I'm sure many would find it useful for dealing with very noisy video.

[dividee]

About filtering with a noise sample, I'm not expert on the subject, but I think like tenebrenz that it should be done in frequency domain (and this is quite a bit over my head).

@tenebrenz: Did you use MotionMask to achieve that?

[Edit:] Oops I first believed you wrote a filter.

[tenebrenz]

@tenebrenz: Did you use MotionMask to achieve that?
Yes, from your mpeg2dec dll :) One problem I encountered was that if I used noise filters other than avisynth's built-in ones, MaskedMix didn't always put the same 2 frames together (probably a bug in avisynth itself.)

[Edit:] Oops I first believed you wrote a filter.
Unfortunately I have no programming skills, except a vague rememberance of writing machine code for a Z80 processor a very long time ago at college.

[High Speed Dubb]

@McQuaid

It turns out that noise on all black screens differs substantially from noise anywhere else, so using only black screens as a baseline isn’t a good choice. It is possible, but you can already get a good noise estimate without the use of fadeouts. (Using repeated fields from 3:2 pulldown material, on the other hand, is probably worthwhile, though you wouldn’t be able to get the noise in the source material that way.)

@tenebrenz

Working in the frequency domain would probably be necessary to deal well with many kinds of interference. But for white noise, (adaptive) filtering using the pixel values directly can already work very well.

PS: Whoops — I should have read your post more carefully. Your explanation of noise in a black screen is right on the mark. And yes, a combination temporal/spatial filter does make a lot of sense. One of these days I’ll probably even write one up.

[trbarry]

I have been pestering Lindsey for some time to port his DScaler adaptive noise filters over to Avisynth, but with no success. ;)

- Tom

[Wilbert]

Where can I find these filters?

[High Speed Dubb]

They’re at
http://students.washington.edu/ldubb/computer/Noise_filters.zip

Remember that these are DScaler filters, not AVISynth tools. So don’t try to use them with AVISynth.