Conditional Temporal Median Filter

[kevina]

I have created a Conditional Temporal Median Filter. The filter is designed to remove temporal noise in the form of small dots and streaks found in some videos. A common cause of this is dirty VHS heads but I have also seen small black or white streaks in broadcast material.

Unfortunally I do not have the ability to create AVISynth filters since I don't have VC++. What I have done was create a command line utility that reads in a yuv4mpeg stream from stdin and write a yuv4mpeg stream to stdout. Yuv4mpeg is a raw planer format used my MJPEG Tools.

Anyway I am posting it here in the hope that someone will take an interest in it and port it. While non-trivial it should be fairly easy to do.

Either that or create an Compiler independnt interface for writing AVISynth filters. :)

You can find it at http://kevin.atkinson.dhs.org/temporal_median.

It works as follows:

1) Find pixes that are different from its neighbors by at least P1.

1b) Enlarge outliers based on P2.

2) Determine the size of the specks and reject all thoselarger than PWIDTHxPHEIGHT

3) Find moving areas of an image by simply comparing each pixel to the previous frame and considering all those which are greater than MTHRES.

3b) Remove pixels determined as noise from the motion map.

4) Denoise the motion map by diffusing and then diluting. This is probably the most important step.

5) Only remove the specks in which there was no motion in the current or the next frame.

[Kurosu]

The way the ASM is used in your source is a major problem. It would take more time to decipher it than port the architecture to VC++ 6.. Besides, you don't provide mmintrin.h and xmmintrin.h so that's even more bothersome to understand what's going on (for instance, I would figure d = _mm_setzero_si64() is equivalent to pxor d,d with d a MMX register - but I'm not sure, and there's so many other stuff like this)

One suggestion to make the port possible (rather, so that someone wants to port that ugly stuff) is to port your ASM to NASM syntax, unless someone is really up for that boring task. It would then only be a matter of writing the avisynth interface to your code.

[Dark-Cracker]

could u explain what is the difference beetween your filter and the filter made by trbarry ?

http://forum.doom9.org/showthread.php?s=&threadid=56107

Bye.

[kevina]

The headers mmintrin.h and xmmintrin.h are from the intel compiler.
Either sse-gcc or sse-intel is used, not both. Sse-gcc might be easier to understand. See http://gcc.gnu.org/onlinedocs/gcc-3.3/gcc/Vector-Extensions.html.

I don't ever directly write assembler instead I use builtin functions which call the equalevent to the assembler instuction, however the compiler handles the register allocation and the like.

You don't nessary need to port med-sse.cpp as med-f provides non assembler versions of all that code. If it will help I can provide the generated assembler code from med-sse.cpp.

[kevina]

The STMedianFilter does not remove specks like my filter does.

[unplugged]

I don't think to have fully understood all the things that your filter does, but sure it interests me a lot!
I think we lack this kind of filter, what is important is the concept of weightening, grouping areas/pixels for more effective threshold.

Indeed, the right test bed for smoothers is TV capture, where we have often luma noise at impossible thresholds, or, in better conditions, with low noise we can see anyway lots of orphan dots (dots with high luma variation).

All other filters that I have tried until now seem based by "simple" pixel-to-pixel threshold comparison, so you want smooth and cope the noise you must use very high thresholds as parameters, resulting in ugly output with posterization (very poor dynamics) and visible orphan dots over all the image.
The only way that I have found to separate and wash image from the high-contrast luma noise and from spikes is this:
eDeen(7,0,16,0,2,false) # cleaning chroma banding
Blur(1.0) # averaging lowers sharpness but greatly washes high-contrast noise and preserve image dynamics (spatial smoothers really don't)
ConvertToYUY2(interlaced=false)
PeachSmoother(NoiseReduction=35,Stability=0) # thanks to that stupid blur it does the job!
LanczosResize(...) # final downsizer
That's incredible, but with this kind of noise the blur effect is most effective spatial smoother. Its averaging "prepares" the image for a much better job by temporal smoother (like PeachSmoother), this because we can further use much lower and non-destructive thresholds on the temporal side (to avoid ghost effect).

[kevina]

Although my filter is relatively harmless it should not be applied to any video. For one is it is a fairly involved filter so it takes a decent amount of time to run. Also, my filter has the tendency of mistaking small moving dots (such as a button on someone shirt) as noise and will thus remove some detail from the image. Finally, the default paramaters are tuned to one particuler video. Others will proably require some carefull adjustment to the paramaters for best results.

However, it will do wonders for images which trully have lots of random black (or white) specks in the image. It will remove this type of noise with very little image degradation. Something that no other filter for AVISynth will do.

Also, please note that my filter only works with luma information. It leaves the chroma untouched. Thus, it will not remove sharp spikes in the chroma channels.

If you want to try it out as is you will need mjpegtools and cygwin.

As far as porting goes, I can post generated assembly code if it will help. However, does VC++ have builtin functions for using MMX/SSE instructions, if so than all that would be required would be to copy sse-gcc (or sse-intel) to sse-vc and change the gcc (or intel) builtins to use VC++ builtins.

[kevina]

An AviSynth version of my filter is now available using my new C API Plugin. Find it at http://kevin.atkinson.dhs.org/temporal_median/.

Developers: Please test it and try to make it crash. AviSynth seams to crash when using it, but not when actually using the filter, rather when retriving a cached copy. It will crash very fast when using Version 0.10 of my API by moving forward in the video and then backwards. With version 0.11 it won't crash so easily but it still will. Before it crashed the cached copies sometimes gets messed up.