Edge Sensitive High Freq Noise Filter?

[MrBunny]

Hey everyone,

I had an idea that I wanted to toss around and see if it was of any interest. A major problem with high frequency noise filters (or even the MPEG-type quantization process) is that it affects both types of high freq elements: "noise" and edges. Even the advice given in the documentation of the Wavelet noise reduction filter indicates that the threshold (of high frequency components to be removed) should be increased such that the noise is removed with minimal effect on the edges.

My idea (and probably not a new one), is to use one of the edge detection algorithms that exist and use it to prevent a high frequency noise filter from removing from that area. The noise filter itself could either remove or soften (like how the DCT filter could be used), and be guided to avoid edges by the edge detection algo.

If I weren't such an absolutely horrid coder (and had more time on my hands), I would take a look at coding this, but if people think this might be useful in some way and have the time, please think about it :)

Oh, and I know I probably made a few technical or theoretic errors in what I said, please correct them. Sometimes I feel completely out of my league in here, but I really don't mind learning ;)

Mr. B

[Guest]

My filter MSmooth already implements this idea. But by all means, have a go at it. There are still some serious issues to be solved.

[MrBunny]

After having looked over the documentation again (and what I remember about the threads discussing msmooth), it seems that msmooth is an edge guided spatial smoother, whereas what I was thinking about was a frequency based noise reducer. The edge guidence on a filter like that would be more useful because edges will contain high frequency components and thus will need the guidence to prevent it from lightening or distorting the edges excessively. As a result, it should only remove the non edge high freq components (mostly grain and real noise). I must say, I'm definately not suprised that something along those lines already existed ;)

Having played around a lot with XviD, one of the problems is the resulting edge mosquito noise when the bitrate is too low and the codec is forced to clip some high frequencies. It was along this train of thought that I was proceeding.

Mr. B

[Guest]

A smoother by definition attenuates higher spatial frequencies. What is the distinction that you are drawing?

[Steady]

Perhaps a slightly different but related approach would be useful.

If you look at what a filter actually changes with Subtract(orig,filtered) followed by a levels, you will see a lot of noise with some edges mixed in. Possibly you could process this difference data to separate out the edge info and then add it back in to the original.

Sometimes I find that a very light sharpening filter will restore the edges while the noise will fall below the quantization threshold.
Such as this GeneralConvolution filter:
-2 -4 -2
-4 280 -4
-2 -4 -2

[MrBunny]

I'm sorry, as I said, I might be just completely confused and wrong. From what I understand, a spatial smoother will manipulate a pixel based on its relationship with other pixels in it's spatial vincinty. Thus, each pixel, whether or not it's "noise" will be processed and possibly changed.

From what I understand of a frequency domain based filter is that within an area (be it 8x8 for DCT or the whole image), "noise" and edges and similar effects will be of much higher frequency than the rest. Therefore, if you convert the source into the frequency domain, the filter will more easily be able to identify the noise since it will be of a higher frequency. I about to say that using the frequency information and the location of the edges the filter could tell what high freq was noise and what was edge...but in while writing this I figured it out :(
It can't work because the frequency domain information can't be directly correlated to any spatial location (not easily at least). Thus if you clip the frequencies, you can't specify that you only want to clip frequencies if they aren't at certain spatial locations because it has no way to reference their locations. I guess if you used a transform that used small enough blocks, you could simply choose to ignore any block that contained an edge, but at the same time, the frequency information would be less indicative of the overall frame.

However, I think Steady's idea has great merit. Manipulating it a bit, this is what I thought of: If the filter could do something like this, it might work out:
1. Store edge information from edge detection algo
2. Transform the image into some type of frequency representation (the exact transform used is up in the air)
3. Low pass filter relative to some threshold
4. InverseTransform
5. Superimpose the orignal edge information onto the resulting image

Well, it's not quite what Steady was saying, but I drew some inspiration from it. Steady's idea also does sound quite interesting, as that would be another easy way to preserve the edge information.

Mr. B

P.S. I feel WAY over my head and hope I'm making some sense and not offending anyone ;)

[Guest]

Originally posted by MrBunny
5. Superimpose the orignal edge information onto the resulting image
That's not a good idea, because smoothing would occur across edges. E.g., a green area and a red area separated by a black line would get blended together in the area of the line, and that's exactly what you don't want to happen. You need to read the MSmooth thread. :)

High spatial frequencies are easily filtered with a small smoothing kernel. There is nothing special that is gained by going to the frequency domain and, as you say, spatial localization is lost in the frequency domain.

[Bulletproof]

What if you were given a reference image (no compression artifacts at all) and a resulting compressed image, would you be able to use the information you recieve from the difference between the two images to smooth away the artifacts?

[Guest]

Originally posted by Bulletproof
What if you were given a reference image (no compression artifacts at all) and a resulting compressed image, would you be able to use the information you recieve from the difference between the two images to smooth away the artifacts? LOL.

Who needs the difference? Just toss out the compressed image and use the reference image!

[JuanC]

Mr. B. you made me remember the enhancements I have been doing to my old vinyl records by digitizing them and then, by using a wave editor (as the one included in Nero Burning Rom, or as the more powerful Cool Edit Pro). The principle of the process is to select a small “silent” portion of the wave (like the ones between the tracks on a vinyl LP record) and create a spectral profile of the selected part: the noise you want to eliminate, and then “subtract” that profile from the entire wave using an adjustable reduction level.

This has worked miracles with my old recordings. Could something similar be done to the signal of the video captures? What about this:

A tool that would enable you to select the “silent” portion of the video: a whole frame or a portion of it (a 32x32 block?) without information (no edges!?) containing just the “noise” you want to reduce in your whole capture, and then in the frequency domain “subtract” the spectral profile of the “noise” portion from the whole video?

May be by automatically selecting new “silent” portions (no edges) it could be adjusted as to dynamically get new noise profiles during the processing of the video and adapting to different kinds of contents in the same clip.

I hope this sounds reasonable, :J

[vidiot]

...it does (at least to me)

http://forum.doom9.org/showthread.php?threadid=36173&perpage=10&highlight=Profiling%20noise&pagenumber=2

Look for Highspeeddub answers...

KR
Harald

[Bulletproof]

Originally posted by neuron2
LOL.

Who needs the difference? Just toss out the compressed image and use the reference image!

I should've been more specific. I actually meant this in a post-processing sense. Once you have the information about how the image is changed you could use the information during post-processing to smooth out the image. Obviously using it over the reference image before encoding would just make the process redundant. From my current understanding I believe the current deblocking routines are not using any reference information and is basically applying it to the whole image, possibly blurring edges in the process, or working like a less complex version of a smart smoother, I think having some reference information about how the original image was would help in aiding areas to deblock and could possibly be faster.

[WarpEnterprises]

IMHO going to frequency domain works only well when there are "peaks" at some frequencies, which is sometimes but not so often.
Try the wavelet filter and you see - it's NOT general purpose but when it bites you can't beat it.