Thoughts about quantifying noise

[vlad59]

Hi all

I got an interesting point for you. As soon as someone talks about noise it's always difficult to have one answer for all cases as it's difficult (impossible) to really say how noisy is a source.

So I thought (Yes it happens sometimes ;) ) of way to quantifiy noise. So my point is to explain my method and get as many feedbacks as possible to have a good filter.

So my method is based on using a non moving part of a movie and to analyze it temporally.
If this part is not moving : all the variations in Y, U, V could be due to noise (I'm not totally sure of it).

So a typical use of the attached filter is :
# to start at the good time
Trim (XX, 0)
#Analyze a rectangle starting at pixel (100, 50) and
#ending at pixel (100+10, 50+20)
NoiseAnalyzer (100, 50, 10, 20)

Then open Vdub, go frame by frame and some stats will appear :

For each pixel, I store min & max values
MeanDelta = Mean (max - min) for all values of the rectangle
Min = Min (max - min)
Max = Max (max - min)
Mean = to be explained better

Example :

Frame 1 Frame 2 Frame 3
10 20 11 15 09 40

For Pixel 1
max - min = 11 - 9 = 2
Mean = (10 + 11 + 9) / 3 = 10

For Pixel 2
max - min = 40 - 15 = 25
Mean = (20 + 15 + 40) / 3 = 24,67


So on frame 3 I'll display :
MeanDelta = 2 + 25 / 2
Min = 2
Max = 25


You can download the dll
here (http://membres.lycos.fr/tempask/NoiseAnalyzer.dll)

[High Speed Dubb]

Yes, just about all differences in stationary areas will be due to noise, though a little will come from quantization.

Mean(max - min) is going to be a difficult measure to interpret — Its meaning will depend on the number of frames which have been anayzed. A more easily interpretable statistic would be:
(1/n)Sum_over_n(Sum_over_pixels(|color_difference|))
where n = the number of temporally adjacent frame pairs = frames - 1

In other words, just average the absolute temporal differences.

There are a couple catches to this method. Clearly, it needs to be used in a stationary area of the picture. More subtly, it will not work correctly in very dark areas of the picture — pixel differences in dark areas are suppressed. Otherwise it’s a very reliable estimator.

[High Speed Dubb]

I forgot to mention a third catch. Color crosstalk will inflate the noise figure. In some sense, color crosstalk is noise, so that’s not a problem. But crosstalk is noise which depends on the amount of detail in the picture, so you might not want to include it. There are a couple easy ways to get rid of crosstalk in stationary areas.

NTSC:
1) Sum pixels at y=0 and y=1 before determining temporal differences
or
2) Take differences at t=0 and t=-4

PAL:
1) Sum pixels at y=0 and y=3 before determining temporal differences
or
2) Take differences at t=0 and t=-8

Another problem to avoid is hand-held camera material. There, supposedly stationary areas can have a lot of subtle motion.

[bb]

I'd rather calculate the correlation coefficient of neighbouring pixels in time and space (remember Convolution3D :D ), and use the average of all coefficients as the measure.

It's probably better to do this separately for luma and chroma, as some sources tend to have luma noise only, others have much chroma noise (e.g. low light DV shots).

bb

[vlad59]

@High Speed Dubb
I'll follow your advice to have simple Absolute difference between temporally adjacent frame. I'll add that

I'll have to re-read your point about color crosstalk because I haven't understood it at all ;) ;) .

@bb

My point here isn't the same as in Convolution3D : it's more difficult to find a non moving with an uniform color (so you can anlyze it with your algo) than only analyzing it temporally and have a larger choice in the rectangle to analyze.

[bb]

@vlad59:
How are you going to find a static area automatically, I mean without the user specifying it manually? I believe you need a statistical approach, which takes into account spatial and temporal noise (you want to calculate video noise, not just 2D picture noise, right?). Evaluating larger numbers of pixels would average out movements. Comparing highly correlated neighbour pixels (in 3D) will make it possible to distinguish between correct values and noise pixels. It might be enough to just count the noise pixels, but I'm not sure.

I may work out a pseudo algorithm this evening, if you like.

bb

[High Speed Dubb]

I was figuring on a user-specified stationary region. Things definitely get hairier if you need to infer a lack of motion.

There is a good justification for using the sum of squared differences as a statistic. If noise is distributed according to a normal density, then the sum of squares contains all the information present in the full data for estimation of the noise variance. (It’s a “sufficient statistic.” )

My suggestion of sum of absolute differences instead of sum of squares comes from my experience with trying to simultaneously sort out noise and motion. An absolute difference is less sensitive to messing up the motion estimate. (And it’s also less sensitive to non-normal interference.) But now that I think about it, if the user is manually specifying the stationary region, then it probably makes more sense to use sum of squares — You don’t have to worry nearly as much about the accuracy of the motion estimation.

The normal density assumption for noise is a very good one with uncompressed images. Compressed formats could be another story, though.

The reason I mentioned excluding color crosstalk is that it can’t be accurately estimated as a single number — Crosstalk depends on the frequencies in the image. So any estimate of it is only meaningful for that particular frame. But since it’s an artifact of the color encoding format, it follows a very sterotyped pattern — and there are some tricks which can remove it.

[SansGrip]

HSD: There is a good justification for using the sum of squared differences as a statistic. If noise is distributed according to a normal density, then the sum of squares contains all the information present in the full data for estimation of the noise variance. (It’s a “sufficient statistic.” )

Good God. Any chance of PM'ing me with this in English? ;)

Seriously though, I'd love to know what you're talking about. By sum of squared, do you mean simply take the square of each pixel value difference and sum them all together? Or do you mean take the difference of the square of each pixel value? What would you use as a "reference"? What's noise variance? What's a normal density?? Should I just take a math class??? heheheh :D

[High Speed Dubb]

SansGrip wrote
By sum of squared, do you mean simply take the square of each pixel value difference and sum them all together?
Yes — Though I’m really glad you asked, since I didn’t get it quite right. The statistic should really be the sum of the square of differences between each pixel value and the mean value at that pixel.

What's noise variance? What's a normal density??
Those are pretty much the same question. A probability density function is a representation of the probability of each outcome from a random process. Here, the “process” is noise — the probability density function shows the chance that noise = -4 from the mean, -3 from the mean, etc. for any difference from the mean.

A normal density (aka a Gaussian density, aka the bell curve) is a family of probability density functions. In other words, there are a bunch of different probability densities functions which are called “normal.” These densities all have the same “bell curve” look, but vary in their center (defined by the mean mu) and how spread out they are (defined by the variance sigma^2).

Normal distributions turn out be important because they show up all over the place in data. Generally, if there are a bunch of factors which add up, the summed effect will turn out to follow a normal distribution.

So summarizing that into English — A normal density is a bell curve. And the noise variance describes how much variation there is in that bell curve. In terms of noise, the variance describes how much pixel values tend to vary.

[SansGrip]

HSD: The statistic should really be the sum of the square of differences between each pixel value and the mean value at that pixel.

I'm kinda following up to here, but the rest is still over my head. I was going to respond with more questions, but I fear that would pollute the thread too much.

I think I need to pick up a book on statistics or something, because I'm not even sure if a bell curve really is a curve shaped like a bell ;).

Thanks for trying though. I'll ask Dr Google and see if I can find some introductory material addressing this kind of thing.

[High Speed Dubb]

Okay — Yeah, this is mostly introductory statistics stuff, and a good website or book would probably be more helpful than my ten sentence explanation.

Here’s a link to a normal distribution, though, just to make things easier to visualize
http://www.atmos.washington.edu/~prowe/atm211/atm211pics/gaussian.jpg
(s here is sigma, the square root of the variance.)

[vidiot]

@vlad59 and High Speed Dubb:

Is there a chance not only to quantify the noise, but to profile it?
I was thinking about that since I use audacity to clean the noise in sound-files out of (tv-)captured videos (but it´s also good for home made stuff...).

In this small, but fast Software I define a small part with nearly no sound (if possible) and it was analysed (profiled) and then applied on the whole *.wav file as an (inverted?) noise filter.


Imagine a captured video with some black or white transitions where you can tell pretty sure whats noise and whats not.

I know that this will be somewhat very special and you have to handle FFT (I don´t really know how that works), but maybe it is a chance...?

Vidiot

[vlad59]

@High Speed Dubb

I'll make some tests with your stats tonight, And I'll post tomorrow morning.

@All

Thanks for your interest and your help

[High Speed Dubb]

@vidiot,

I’m not sure what distinction you’re making between quantifying and profiling. Are you suggesting that interference patterns be inferred? That’s an interesting possibility for over the air material.

Using black areas to estimate noise would work, but would be a little tricky. Problem is, black areas have less noise than anywhere else. But you could still use black areas for noise estimation if you take that into account.

[vidiot]

@High Speed Dubb,

the distinction between both is (in the way I´m thinking):
- quantifying: To tell how much noise is in the movie and where is it (in the picture) and in which pictures is how much noise...
- profiling is heavily build on that, but I hope that one can determine "typical" noise, that is introduced by the power supply or by the vent of your graphic board.
And this by analyse just a small amount of video.
If you take the time to analyse the whole video you can easily go with a combination of spatial and temporal filtering like MAM for example.
And yes, - over the air interference like rainbow artifacts could maybe handled the same way by profiling (hopefully).

...as I said: The idea came from sound cleaning...

[High Speed Dubb]

Yes, inferring interference patterns using black areas is a possibility. Dealing with edge effects would be very difficult, though.

Note that cancelling interference patterns isn’t the same as getting rid of noise. Everyday white noise doesn’t have a pattern to detect.

And this by analyse just a small amount of video.

Then you’ll only be able to detect high frequency interference.

If you take the time to analyse the whole video you can easily go with a combination of spatial and temporal filtering like MAM for example.

Easily? You’re welcome to code it.

And yes, - over the air interference like rainbow artifacts could maybe handled the same way by profiling (hopefully).

No, that wouldn’t work at all.

[vidiot]

@ High Speed Dubb:

Thanks for clearing up my questions and suggestions...

With easily I meant: It´s already there!
The only things I could code are some simple and stupid LISP programms for ACAD... :D

[MfA]

Originally posted by High Speed Dubb
Everyday white noise doesn’t have a pattern to detect.

Not all noise is white though, especially after gamma correction.

BTW what is rainbowing? Is it simply ringing in the chroma planes?