Log in

View Full Version : idea for temporal filters


UGAthecat
28th June 2002, 02:39
ok, First post, and I searched repeatedly, and couldn't find any previous posts that relate to this, even though to me it seems like an obvious idea, and since i'm an avisynth addict and love the fact that filtering can be done with it, i posted here instead of VirtualDub forum.
Also, I'm gonna get kind of long-winded here, so if you don't want to read the whole thing, the gist is: if GMC was added into a temporal filter you could use stronger filtering and get the same (high) strength filtering in scenes where a basic pan is being done as in a scene with no motion at all, which would be great for noisy sources.
The biggest minus I see is that it will make encoding take quite a bit longer I think, but I think it would be worth it.

I hate to sound like a beggar, and I would gladly work on it myself, but the only programming I can do is in VB. If anyone has suggestions on sites/guides to get started with C and video filter programming, I'm all ears.

And now for the longwinded verison... (if everyone thinks the description above is more than enough i'll take this out)
I've been working with an anime DVD where the source is very noisy. With no filtering, or light spatial and temporal filtering, I get a video that doesn't look too noisy, but has a lot of artifacts (can't describe them other than it looks like macroblocks are floating around like swarms of flies, but it only happens in 'smooth colored' areas).

For parts of the video where there is relatively low motion, I have been able to get rid of the swarming artifacts (while also improving the picture and reducing file size), but i have to turn up temporal smoothing to have thresholds as high as 80(0-255 scale) and sometimes higher. This obviously introduces tons of ghosting when the motion starts.

So the thought sprang into my head "if these temoral filters had basic global motion compensation/estimation algorithms in them, I could use stronger temporal filter settings, and not get as many ghosts during camera pans at least"
then i could make an avisynth script that applied

Adding normal motion compensation (in addition to or instead of GMC) would probably help in regular films as well, but using normal motion compensation would probably not fix the problem of the swarming macroblocks because the motion estimation in the filter would filter in blocks that move in the same way that the macroblocks swarm(which I also have with another DVD source that was made from old film).

oh yeah, another filter idea (I had lots more but I'm tired now and can't remember them):
add padding option to temporal filters so that it won't filter pixels in X radius around pixels that dont meet threshold to get filtered (to fix ghosting at the edges of objects that are moving, negative effect will reduce effectiveness of getting rid of macroblock artifacts)


The Cat

WarpEnterprises
28th June 2002, 11:38
I had a similar idea:
Use two paths of filtering, one for low-motion (strong temporal filter but weak blurring) and one for high-motion-parts (weak temporal and strong blurring).
There are functions in dividees mpeg2dec.dll (MaskedMix) which can do such a job. But I haven't completed the evaluation.
Difficult was the generation of a good motion mask.
Is this a similar idea to yours?

UGAthecat
28th June 2002, 21:06
I was thinking more along the lines of an motion estimation algorithm so that if the algorithm detects that the full frame motion. like if the camera pans to the right, the algorithm would detect from frame 1 to frame 2 the picture is the same, just moved over by X pixles, and then instead of averaging or locking the same pixel from one frame to the next, it would lock the pixel in frame 2 that is offset by X from the pixel in frame 1, as long as it meets the threshold setting of course.

Richard Berg
28th June 2002, 23:54
While I'm usually not a fan of temporal filters, if we made this one "smart" enough it could be really interesting. Removing ghosts, for example, could be done much better than any of our current tools.

What we need is some kind of warping tech -- VDub has "warpsharp" which does good edge detection, but on top of that we'd need some real morphing technology in order to make the changes intelligently. It's over my head at this point, unfortunately.