YATS temporal smoother, porting to avisynth [Archive]

blivit

11th January 2003, 23:53

I've written a temporal smoother for VirtualDub called YATS (Yet Another Temporal Smoother). I'm posting in the avisynth forum, rather than the virtualdub forum, for several reasons. 1) I am thinking about porting it to avisynth and want to know if it is worth the effort. 2) It looks like most of the temporal smoother developers are writing avisynth plugins, so this is the best place to post if I want them to notice it, try it out, and give me any feedback they might have. 3) I want general feedback from everyone, and since many people seem to ignore virtualdub plugins these days, even though avisyth can use them just fine, I figured this was a good place to post for that too.

The filter is available for download from www.doom9.org, under the Filters download section. To use it from avisynth, use a script similar to the following (default settings):

# When loaded in an avisynth script, YATS doesn't work right if you try
# to step backwards through the video. It does not have this problem in
# VirtualDub.

LoadVirtualdubPlugin("c:\virtualdub\plugins\yats.vdf","_VD_YATS")
function VD_YATS(clip clip)
{
return clip._VD_YATS(6, 2, 6, 2, 0, 1, 1, 0, 25, 2, 6, 4, 155, 1, 0)
}

video = AVISource("shimmer_huf.avi")
video = video.ConvertToRGB()
video = VD_YATS(video)

return video

The order of the function arguments is the same as that displayed in the virtualdub GUI: 1st vertical column, then 2nd vertical column. Try using the default settings first. They are a bit aggressive, but they work very well for everything I have tested them on, giving little to no noticable artifacts.

What does YATS do differently than other temporal smoothers? It does not process any pixels that have moved since the previous frame(s). Motion detection (or, more accurately, stationary pixel detection) is done a bit differently than any of the other filters I have used or read about. It appears to do a good job of it, since it avoids most motion related temporal smoothing artifacts, even at fairly aggressive settings. This alone makes it quite useful. It also has fade and edge detection. Frames involved in a fade are not processed, since fades can cause many ugly temporal smoothing artifacts. Edge detection is useful, since edges can tolerate much higher levels of temporal smoothing than non-edges. This all adds up to a filter that can remove a good amount of temporal noise without introducing noticable artifacts. Increased compressability is a bonus :)

Why would I not want to port it to avisynth when working in YV12 space can gain me so much speed? The thresholding / motion detection is done in RGB space. I originally wrote the filter to work in YUV space, which gave decent results, but not as good as I had hoped for. I switched over to RGB space, and everything looked much better. A single distance in RGB space seems to work better than two distances in luma and chroma space. So, since I would be converting to RGB internally for the thresholding, there goes the speed advantage of working in YV12 space. Color edge detection only makes
sense in RGB space. Edge detection in luma space is fine, since all edge detection mehtods are originally based on greyscale image processing techniques. But edge detection in the chroma planes doesn't make any sense to me at all. There is no "light" and "dark", only different shades of color, so the results of the edge detection in chroma space might be a bit odd.... For instance, dark red next to dark blue would not be detected as an edge in RGB space, but they would in chroma space, since red is far away from blue. So, since the entire filter works in RGB space, and should not be converted to work in YV12 space, the only reason to write an avisynth plugin would be to average/fix the YV12 pixels directly, rather than having to go through the RGB conversion. Therefore, it looks to me like it wouldn't be worth the effort.

The only reason left that I might want to port it to avisynth would be the ability to look at future frames. Since it is a virtualdub plugin, it can't see into the future, and thus it only looks at the current and two previous frames. Motion detection might be improved by looking ahead as well. But since I'm pretty happy with the current level of motion related artifact avoidance, I'm not sure if this would be worth it or not.

I briefly describe how the filter works in the docs. Anyone interested in further details can look at the src and/or email me. I think that the stationary pixel detection and edge detection methods might be useful in other filters, so I'd be especially interested in the opinions of other filter authors on these. I look forward to any feedback anyone wishes to give, whether positive or negative. Feedback is good for improving the filter further :)

-Eric

High Speed Dubb

12th January 2003, 09:32

blivit,

Could you describe your motion detection? That’s the interesting — and difficult — part of any temporal smoother. (And it’s useful stuff for comb filtering, deinterlacing, and pulldown detection, too.)

This claim
What does YATS do differently than other temporal smoothers? It does not process any pixels that have moved since the previous frame(s).
is silly. All the rest of the temporal smoothers are doing the same thing. But each has a different motion detection method.

About edge detection in chroma — Don’t do it. You can use the Y for edge detection, and apply that to the U and V planes.

Regarding YUV versus RGB for motion detection — There is a strong argument for using Y rather than RGB. The signal is encoded as Y, which means that noise is normally distributed around the Y value. That means you can safely use the same criteria for distinguishing between noise and motion without needing to scale for the Y value. The same isn’t quite true for R, G, or B. (Nor is it really true for U or V, though.) Another argument for YUV is that it is supposedly closer to perceptual color space than RGB. So YUV distances and averages should be more accurate.

Looking at future frames can significantly improve noise reduction. That said, I’ve always stuck with looking at just the immediately preceding frame, since that allows the filters to run much faster.

High Speed Dubb

12th January 2003, 09:58

Hi again,

I just read the docs for your filter. (By the way, there’s a typo in your link.) You should probably just post a summary of it to this thread. Here’s a quote from the docs:

I recently came across a movie that had some serious flickering problems in certain scenes, especially around high contrast edges.

and later

Original pixel through time:
dark, light, dark, light, dark, ...

That’s a clear description of (NTSC) color crosstalk. You don’t need a temporal smoother — You need a comb filter. At the moment, Guava Comb is the only one I know of. [P.S. There’s also Kurosu’s Antiblink.] It has its flaws — I should really post my current build, which improves the motion detection — But it does work to get rid of shimmering at high contrast edges. Run that first and you can probably skip the edge detection stuff.

The rest of what you describe looks like a good design for a temporal smoother.

Taking a further look at the docs, it looks like YATS is in the same algorithmic group as TemporalCleaner, GrapeSmoother, and PeachSmoother. All three of those likewise use differences between the current frame and the (already processed) previous frame to detect motion. Of those three, your motion detection method is most similar to the one in Peach Smoother, though there are significant differences between them. It would be worth having a “how to use feedback for motion detection” thread sometime.

blivit

12th January 2003, 12:48

Originally posted by High Speed Dubb
blivit,

Could you describe your motion detection? That’s the interesting — and difficult — part of any temporal smoother. (And it’s useful stuff for comb filtering, deinterlacing, and pulldown detection, too.)

Pasted from yats.txt:

Pass 1: Flag pixels as stationary if <= threshold.

Pass 2: Look at each stationary pixel from Pass 1, check all the surrounding pixels in a diamond of radius N. If every pixel in the diamond search is stationary, then the center pixel is marked as stationary in Pass 2. You can also think of this as masking out a diamond radius around every moving pixel, so that all pixels surrounding it are marked as having moved too. YATS actually does it both ways (same end result) for speed reasons, depending on the number of stationary pixels from Pass 1.

Pass 3: A pixel must have been flagged in Pass 2 of the previous frame as well. This lets us be even more certain that the pixel really is stationary.

This claim
What does YATS do differently than other temporal smoothers? It does not process any pixels that have moved since the previous frame(s).
is silly. All the rest of the temporal smoothers are doing the same thing. But each has a different motion detection method.

I agree that this quote, taken by itself, is a silly thing to say, for the reason you give. I realized this at the time I wrote it, but hoped the sentence immediately following it, "Motion detection (or, more accurately, stationary pixel detection) is done a bit differently than any of the other filters I have used or read about.", would elaborate on the previous sentence and avoid such silliness. I guess I should have written things a little better :)

About edge detection in chroma — Don’t do it. You can use the Y for edge detection, and apply that to the U and V planes.

Edge detection in luma space misses out on detecting edges between strong colors of equal or nearly equal brightness. Edges will be missed if edge detection is done only in luma space, instead of RGB space. In RGB space, each plane is done separately, then the results are combined intelligently (opinions vary on how this should be done) into a combined edge vector. This captures more edge information than the luma plane alone. Edge detection in luma space would thus not be as accurate as in RGB space.

Regarding YUV versus RGB for motion detection — There is a strong argument for using Y rather than RGB. The signal is encoded as Y, which means that noise is normally distributed around the Y value. That means you can safely use the same criteria for distinguishing between noise and motion without needing to scale for the Y value. The same isn’t quite true for R, G, or B. (Nor is it really true for U or V, though.) Another argument for YUV is that it is supposedly closer to perceptual color space than RGB. So YUV distances and averages should be more accurate.

I've heard people argue that RGB is closer to perceptual color as well, so I'm not going to take a stand on that one way or the other. What I do know is that if I used Y alone, I got more motion artifacts. Adding a chroma threshold reduced the artifacts, hopefully because the extra chroma information let me detect more "motion" as I had defined it. Using an RGB threshold at *roughly* the equivalent luma threshold gave even better results. I can't give any good theory as to why it works better for me, I just know that I have lots of empirical testing to support my choice. Fewer artifacts at the same effective smoothing level. Other, hopefully better, methods of motion detection might work better in luma/chroma space. I just know that, for this filter, RGB space works better.

Looking at future frames can significantly improve noise reduction. That said, I’ve always stuck with looking at just the immediately preceding frame, since that allows the filters to run much faster.

Two preceeding frames is even better, to be REALLY sure that said pixels have not "moved". But future frames might not be as useful when the filter only averages over previous frames. If it only averages over previous frames, does it really matter if the pixel is GOING to move in the future? Maybe, but not as much as looking at previous frames. If the average is taken over previous AND future frames, then yes, motion detection should definately be done in the future as well, to avoid the "Ghost of Christmas Future" that visits so many filters that include future frames in the averaging :) But if I were to average over the previous, current, and future frames, then I wouldn't be able to smooth out the kind of flicker noise I mentioned in the documentation. I'm still left thinking that being able to see into the future might not be enough of a benefit, by itself, to justify me porting it to avisynth. Maybe if I had two modes, average over previous+current, and average over previous+current+future. Motion detection in the new mode would basicly work the same way, but look at the current, last, and next frames instead of the current and two previous frames, to avoid ghosts from future frames. Or maybe go out two frames in both directions instead of just one. Hmm.... The two modes would let you choose different averaging methods to deal with different types of noise. I'll have to think more about that.

-Eric

High Speed Dubb

12th January 2003, 18:58

Taking a closer look at the YATS, I think there may be a significant difference between it and the other feedback-based temporal filters. All of the feedback filters use differences between the current frame and the previously processed frame to figure out where there has been motion. But YATS uses the previous unprocessed frame when determining the final color — The other filters use the processed frame when doing so.

If that’s right, then that means pixel locking really is a good thing for this filter — It’s only a short term lock, which would avoid the usual problems with disallowing small color changes.

If you weight the previous unprocessed pixel and current pixel equally, then your averaging is appropriate for NTSC comb filtering. I suspect that’s how you came up with the scheme, since you were trying to get rid of crosstalk with the same filter you’re using to get rid of noise. Beware that this won’t work for PAL comb filtering.

Edge detection in luma space misses out on detecting edges between strong colors of equal or nearly equal brightness.

Yep — I was suggesting the use of Y alone just to keep things fast. A better edge detector would use Y, U, and V jointly. That’s what I do in Peach Smoother — The spatial difference measure is 2*|Y difference| + |U difference| + |V difference|. It’s also a part of the reason the filter is unlikely to be converted to YV12.

I've heard people argue that RGB is closer to perceptual color as well, so I'm not going to take a stand on that one way or the other.

Figuring that out would take some careful lab work, testing color perception in lots of different people. Fortunately, this stuff has been done already. Unfortunately I wasn’t able to find a link to it, so I have to rely on my memory of this; Distances in RGB correspond somewhat less well to perceptual distance than those in YUV. (Though neither one is anywhere near perceptually uniform.) Also, I’m not sure how R'G'B' and Y'U'V' compare for perceptual uniformity — gamma correction certainly will affect this. Here are the best links I came across in my search
http://kiptron.psyc.virginia.edu/steve_boker/ColorVision2/ColorVision2.html
and
http://ourworld.compuserve.com/homepages/compuphase/cmetric.htm

But future frames might not be as useful when the filter only averages over previous frames.

Technically there is a little information in the future frame’s colors. But if you’re going to make use of that information, you might as well make use of the future color for averaging, too. Doing that correctly for a feedback-based filter would be tricky, though.

scmccarthy

12th January 2003, 20:55

Also, I’m not sure how R'G'B' and Y'U'V' compare for perceptual uniformity — gamma correction certainly will affect this.
What do you mean by 'gamma correction'. It is a very technical term and an important term and one where a lot of confusion exists.

I assume that you know what it means really, but I am more concerned with what you mean when you say it whether that term is being used correctly by you anyway, just for the sake to communication.

In particular I want to know what the practical difference between gamma and contrast is. We started to discuss this via email a few months ago and this is the one part of our correspondence that still interests me.
Beware that this won’t work for PAL comb filtering.
Isn't combing due to the two fields within a frame coming from two difference time periods? How can PAL and NTSC combing be different?
That’s what I do in Peach Smoother — The spatial difference measure is 2*|Y difference| + |U difference| + |V difference|. It’s also a part of the reason the filter is unlikely to be converted to YV12.
The best way to refute this would be to actually do the conversion, so I won't argue, but I don't agree. If you are right, that is then an argument for converting PeachSmoother to avisynth 2.5 YUY2. If there are things that can only be done in YUY2 or RGB, then we need filters that work in all three color spaces for AviSynth 2.5.

Stephen

blivit

12th January 2003, 22:16

Originally posted by High Speed Dubb
Hi again,

I just read the docs for your filter. (By the way, there’s a typo in your link.) You should probably just post a summary of it to this thread. Here’s a quote from the docs:

I recently came across a movie that had some serious flickering problems in certain scenes, especially around high contrast edges.

and later

Original pixel through time:
dark, light, dark, light, dark, ...

That’s a clear description of (NTSC) color crosstalk. You don’t need a temporal smoother — You need a comb filter.

I thought that only happened on tilted edges? I was seeing this on horizontal and vertical edges too. It turns out it was an issue with mpeg2dec (all flavors). The latest mpeg2dec3, which I installed after I got back from Christmas break, fixes all the flickering problems. But before the mpeg2dec3 fix, YATS _was_ able to remove all the problematic flickering just fine. It may not have been the optimal solution to the problem (I have not tried GuavaComb), but it was a still a pretty good solution all the same.

Run that first and you can probably skip the edge detection stuff.

Even if I don't have major flickering problems anymore, edge detection is still useful for anyone trying to squeeze out higher compression without hurting quality. A lot of people use temporal smoothers for this :) Besides, removing edge detection would get rid of Line Art mode :)

-Eric

blivit

12th January 2003, 22:56

Originally posted by High Speed Dubb
Taking a closer look at the YATS, I think there may be a significant difference between it and the other feedback-based temporal filters. All of the feedback filters use differences between the current frame and the previously processed frame to figure out where there has been motion. But YATS uses the previous unprocessed frame when determining the final color — The other filters use the processed frame when doing so.

It doesn't make sense, to me, to use the previously processed frame in motion detection. We want to know which pixels in the original video have moved, not which ones have moved relative to the processing we did on them earlier. I don't think using the previously processed frame for motion detection is such a good idea (and I didn't realize the other filters did that either).

If that’s right, then that means pixel locking really is a good thing for this filter — It’s only a short term lock, which would avoid the usual problems with disallowing small color changes.

Pixel fixing is actually a bit ugly and more complicated than I briefly described in the docs. The pixel fixing threshold _IS_ done against the previously processed frame, but only on pixels that have gone through the regular motion passes (not using the previously processed frame). Thresholding against the previously processed frame is important, because pixel fixing is trying to feed relatively stable pixels to the codec, and thus it matters how different they are from the ones the codec has seen in the previous frame. Unfortunately, this leads to the usual problems of stuck pixels looking out of place in future frames. Actually, comparing to the previous unprocessed frame has the same problems, I've tried it both ways, and many other odd combinations and rules. I finally did come up with with a method that works (pseudo-code below):

rgbdist = distance between current and previously processed
if (rgbdist > fix_threshold) continue;
rgbdist2 = distance between previous and previously processed
if (!(rgb_dist2 && rgb_dist >= rgb_dist2)) fix the pixel

Unchanged pixels get fixed. Pixels that have changed must change less relative to the previously processed frame than the difference between the previous and previously processed frames.

This got rid of a lot of the pixel fixing artifacts I was seeing, while still reducing the amount of macroblock jitter that was being produced when the filtered video was compressed. The method is a bit of a kludge, but it seems to work well.

If you weight the previous unprocessed pixel and current pixel equally, then your averaging is appropriate for NTSC comb filtering. I suspect that’s how you came up with the scheme, since you were trying to get rid of crosstalk with the same filter you’re using to get rid of noise. Beware that this won’t work for PAL comb filtering.

The pixel fixing doesn't really help much with the flicker noise, since the flicker noise was massive enough to be well outside any sane pixel fixing threshold. I acutally hadn't thought about comb filtering or crosstalk at all when I wrote the filter. I just looked at the flicker, and tried to come up with a way to get rid of it.

Edge detection in luma space misses out on detecting edges between strong colors of equal or nearly equal brightness.

Yep — I was suggesting the use of Y alone just to keep things fast. A better edge detector would use Y, U, and V jointly.

Unfortunately, I don't know of and can't think of any methods for jointly using Y, U, and V edge detection. Most of the edge detection literature is in luma space. Very little of it is in RGB space or any color space at all. I came up with my RGB space methodology on my own, since I couldn't find any good references on color edge detection anywhere. If you can find or think of a good way to incorporate U and V into edge detection, let me know :)

But future frames might not be as useful when the filter only averages over previous frames.

Technically there is a little information in the future frame’s colors. But if you’re going to make use of that information, you might as well make use of the future color for averaging, too. Doing that correctly for a feedback-based filter would be tricky, though.

But, since my motion detection isn't based on feedback, that wouldn't be as much of a problem, right?

-Eric

High Speed Dubb

13th January 2003, 02:49

scmccarthy wrote
What do you mean by 'gamma correction'...
I’ll answer that in a new thread.
http://forum.doom9.org/showthread.php?s=&threadid=42854

Isn't combing due to the two fields within a frame coming from two difference time periods?...
Combing and comb filtering are unrelated. It’s a nasty terminology gotcha. Combing refers to deinterlacing artifacts, while comb filtering refers to the process of removing color crosstalk artifacts. This goes in my basket of peeves along with gamma correction (which is a power function, not a gamma function) and ghosting (which either means temporal blurring or an offset lighter image).

The best way to refute this would be to actually do the conversion...
I didn’t understand that sentence. What is being refuted?

If there are things that can only be done in YUY2 or RGB, then we need filters that work in all three color spaces for AviSynth 2.5.
Gamma correction should be done in RGB, so there’s certainly a need for that. And if you have the computing power, it’s better to do broadcast video processing in YUY2, since YV12 is tossing out some information.

Guest

13th January 2003, 03:02

Originally posted by blivit
Unfortunately, I don't know of and can't think of any methods for jointly using Y, U, and V edge detection. Most of the edge detection literature is in luma space. Very little of it is in RGB space or any color space at all. I came up with my RGB space methodology on my own, since I couldn't find any good references on color edge detection anywhere. If you can find or think of a good way to incorporate U and V into edge detection, let me know :)Why can't you just do edge detection on each plane separately, making binary maps, and then OR the maps together into one map? This is similar to what I do in FieldDeinterlace() for calculating the motion map. The UV maps are scaled up to the Y size (using nearest neighbour) before the OR'ing operation.

High Speed Dubb

13th January 2003, 03:14

blivit wrote
I thought that [color crosstalk] only happened on tilted edges?

Nope, it’s primarily a problem with horizontal changes. (Maybe you were confused by the combing / comb filtering terminology similarity? Combing can be especially bad at tilted edges.) If the capture card had a low quality 2D comb filter, there can be crosstalk problems at vertical boundaries, too.

It turns out it was an issue with mpeg2dec (all flavors). The latest mpeg2dec3, which I installed after I got back from Christmas break, fixes all the flickering problems.

You gave a pixel perfect description of NTSC color crosstalk, so it would be surprising (but not impossible) that the mpeg decoding was introducing the problem. Could you put up a small screenshot of the effect? That might make things easier to figure out. Since changing decoders fixed the problem, I suspect that either A) The new decoder has a buillt-in comb filter, or B) That you’re using the decoder on different source material, and the more recent material was taken from a better source.

Even if I don't have major flickering problems anymore, edge detection is still useful for anyone trying to squeeze out higher compression without hurting quality.

Thinking some more about this, I agree with you. In terms of noise, local edges don’t matter at all. But an area with big color differences (and therefore edges) will usually have very big changes if there is any motion. That means that it does make sense to change the averaging method, there.

It would also make sense to put the edge detector stuff into a comb filter, if you feel like writing one.

A lot of people use temporal smoothers for this :)
But it isn’t a good idea to use temporal smoothing to get rid of crosstalk. If you do the steps separately then you can make both of them more specific.

blivit

13th January 2003, 03:42

I use a compass Sobel edge detector. I make four passes on each plane, for the W, NW, N, NE directions. Compass Sobel is more accurate than simply doing regular horizontal/vertical Sobel. Then, for each of the four directions, I look at the absolute values across all three planes. The largest is kept as the final value for that direction (negative values are ok, since they simply mean the opposite direction). I map the final values of all four directions into x,y space, sum the vectors, and calculate the final magnitude and angle.

Then I do a pruning pass where I throw away weak edge pixels. I keep only edge pixels >= edge_mag (155 default), surrounded by >= 2 pixels that are both >= edge_mag and have angles within ANGLE_RANGE (22.5 degrees) of the center pixel. This leaves only the most well defined edges. Then I do a final pass where I kill off all lone edge pixels that are not surrounded by any other edge pixels.

Why can't you just do edge detection on each plane separately, making binary maps, and then OR the maps together into one map? This is similar to what I do in FieldDeinterlace() for calculating the motion map. The UV maps are scaled up to the Y size (using nearest neighbour) before the OR'ing operation.

I turn everything into binary after I have already combined the results of the three planes together and pruned weak edges. If I were to do the same sort of edge pruning on each plane individually, THEN combine them together, I don't think I'd get the sort of result I want. Combining first, then pruning, gets rid of more "noisy" edge points than pruning each plane separately then OR'ing together. Or, at least it seems like it should work that way to me. Making each plane binary, then OR'ing looks like it would be throwing away too much useful information that could be used to get rid of "noisy" edge pixels that shouldn't really be marked as edges.

If I were to apply the same algorithm I am already using to Y, U, and V planes, then I would have to sacle U and V to Y, like you said. But is this really an ok thing to do? In RGB space, each plane has the same scale to begin with, so taking the highest absolute value across them is a reasonable thing to do. But doing the same thing with luma and scaled chroma planes, I just don't know if that is a reasonble thing to do or not. How much do you scale the chroma by? What is the largest distance that two points in U or V can have from each other?

-Eric

Guest

13th January 2003, 04:36

By scaling U and V I meant scaling the plane spatially so it is the same size as the Y plane. That is just so that the correct locations are written on the combined map.

You can use different thresholds for the UV planes if you are worried about differing gradient magnitudes.

blivit

13th January 2003, 04:41

Originally posted by High Speed Dubb
blivit wrote
If the capture card had a low quality 2D comb filter, there can be crosstalk problems at vertical boundaries, too.

No capture card involved. This is from a VOB, via a .d2v file.

You gave a pixel perfect description of NTSC color crosstalk, so it would be surprising (but not impossible) that the mpeg decoding was introducing the problem.

I am not very familiar with crosstalk at all. I need to read up on that....

Could you put up a small screenshot of the effect? That might make things easier to figure out. Since changing decoders fixed the problem, I suspect that either A) The new decoder has a buillt-in comb filter, or B) That you’re using the decoder on different source material, and the more recent material was taken from a better source.

A) may very well be possible. I have no idea.
B) is right out. Same file.

It's a little difficult to see it in static screenshots, because the flicker is between frames, so you'd need to flip back and forth between screenshots to see it well. I've made two quant 3 XVID encodes, one of the original decoding setup (I had saved a huffy-yuv of it), and the other of my current updated setup. The flicker is less pronounced in the quant 3 encodes than in the original video stream, but it is still quite visible. In the original, the entire background of (M)'s was pulsating quite annoyingly, which isn't as pronounced in the quant 3 encode. The new non-flicker decode is pretty much rock solid everywhere in the image, but the quant 3 encode introduces a little wavering. The difference is most noticable around the lamp and the green bulbs along the bottom of the screen. I figured lower quality samples were an easier download than the 19 meg huffy-yuvs :) Also, YATS doesn't work as well on the quant 3 encode as on the original, probably because the compression smears the flicker out spatially too much for it to handle? YATS does a good job on the original, though. If it *is* a crosstalk problem, a crosstalk specific method would probably be superior.

You can download them, at least for the time being, from:

www.artsci.wustl.edu/~eawelsh/flicker.avi
www.artsci.wustl.edu/~eawelsh/flicker_fixed.avi

And here is an encode of the original flickering source, after having passed through YATS, so that you can compare it to the correctly decoded version:

www.artsci.wustl.edu/~eawelsh/flicker_yats.avi

A lot of people use temporal smoothers for this :)
But it isn’t a good idea to use temporal smoothing to get rid of crosstalk. If you do the steps separately then you can make both of them more specific.

I'll have to learn more about crosstalk.... But even on relatively "clean" videos, YATS can gain a good bit of compressibility, especially using higher edge RGB thresholds, without saccrificing image quality. I may have designed it to get rid of crosstalk (although I did not know it was crosstalk), but it works well on regular stuff too. I use it on all my clean videos to gain compressibility, with the default settings, and I have yet to notice any bad artifacts being produced. A temporal smoother might not be the best way to get rid of crosstalk, but if it works and doesn't hurt anything else, then hey, no problem :) I'll look into seeing if there is any anti-crosstalk stuff I could add in before the temporal smoother stuff, or maybe create a separate anti-crosstalk filter to be used before YATS.

-Eric

blivit

13th January 2003, 05:03

Originally posted by neuron2
By scaling U and V I meant scaling the plane spatially so it is the same size as the Y plane. That is just so that the correct locations are written on the combined map.

You can use different thresholds for the UV planes if you are worried about differing gradient magnitudes.

I hadn't even gotten to thinking about problems with the packed formats. Several pixels in a row sharing the same luma and/or chroma values is a bit problematic for many types of image processing.... Maybe this is why some virtualdub filters look better than their avisynth YUV2/YV12 ports? I think this is all the more reason to do things in RGB space, or at least in non-packed YUV space, where each pixel has its own set of bitplanes and doesn't share any values with neighboring pixels. Increased speed is not worth decreased accuracy and image quality, IMHO.

-Eric

Guest

13th January 2003, 05:14

You're exaggerating the difficulties. Anyway, converting the subsampled formats to fully sampled RGB doesn't invent new information. Why did you ask for a working approach if you just want to reject it on dubious grounds? :)

You wrote:

>since I couldn't find any good references on color edge detection
>anywhere

Search for "color edge detection" at Google. It's quite a well-developed area.

BTW, Avisynth can do RGB; you're not limited to YV12 and so your stated reason for not porting YATS to Avisynth is misguided.

scmccarthy

13th January 2003, 05:22

The best way to refute this would be to actually do the conversion... I mean that I still intend to investigate the PeachSmoother sources themselves and verify whether a YV12 version of PeachSmoother is possible.it’s better to do broadcast video processing in YUY2, since YV12 is tossing out some information. For the benefit of anyone listening, stay in the color space you capture in,if you capture in a 4:2:0 color space, using YUY2 would interpolate the data.The UV maps are scaled up to the Y size (using nearest neighbour) before the OR'ing operation. For FrameBased YV12, doubling each chroma line and then averaging the lines together after processing is simple and would guarantee proper siting of the chroma samples. The chroma lines should be sited between each luma line to prevent drift.

Stephen

High Speed Dubb

13th January 2003, 05:37

It’s time for an extra long post... :)

Earlier in the the thread, blivit wrote
But if I were to average over the previous, current, and future frames, then I wouldn't be able to smooth out the kind of flicker noise I mentioned in the documentation.

I just caught this line — It isn’t true. You can get rid of crosstalk using future frames. A 1:2:1 previous:current:next frame weighting would work, for example. More generally, any scheme with equal weight for odd and even frames will be okay (for NTSC).

And from a later post
It doesn't make sense, to me, to use the previously processed frame in motion detection.

There are some strong arguments for doing so — and some strong arguments against it. The overall reason to use the previously processed frame is that it is a better representation of the true image than the unprocessed previous frame. That’s because it has had its noise reduced by the previous temporal averaging. On the down side, using the previously processed frame means that it’s possible to propagate an error through time, if you’re not careful.

In more technical terms, this is an argument between IIR and FIR filtering — You can look up lots of stuff on this distinction on the web. The general advantage of IIR (infinite impulse response -- i.e., feedback) filtering is that it tends to allow a very fast algorithm to consider lots of data at once. Peach Smoother is an impressive example of this — It’s able to use averaging of just two pixels to reduce the noise by a factor of up to about 3.6 — For a non-feedback filter to do that, it would need to average together at least 13 pixels. The main advantage of FIR (finite impulse response) filtering is that the behavior is much easier to predict and control. It’s also much easier to write a bidirectional FIR filter.

(I’ll skip commenting on the pixel fixing stuff — I need to read the code pretty carefully to understand exactly what is being done, there.)

Unfortunately, I don't know of and can't think of any methods for jointly using Y, U, and V edge detection.
That’s not so hard. For example, you could use your Sobel RGB edge detector but use YUV instead. In other words, you’d take the maximum of the {Y, U, V} edge evidence in each direction. My hunch is that you’d be better off adding the evidence, instead of using a maximum — Y differences tend to be much larger than U or V differences. But otherwise the formulas could be the same.

But, since my motion detection isn't based on feedback, that wouldn't be as much of a problem, right?
Right — If the colors you’re averaging are only taken from the input colors (rather than the processed colors), then this is much easier. (You can see how feedback would be difficult, here — If the output color at time t=0 is based on the color at t=1, and the output color at time t=1 is also based on the output color at t=0, then it’s possible if you’re not careful for the estimate to get into a loop and run off to some illegal value.)

No capture card involved. This is from a VOB, via a .d2v file.

So that means it’s a direct digital transfer from a DVD? Even then it’s possible to get crosstalk if the original material was transmitted as a composite signal at any point in processing. But your clip looks like it might be computer graphics (or is it clay animation? It’s hard to tell these days.), in which case that’s not very likely.

Anyhow, looking at the clip, it is a little weird for crosstalk — there’s none of the characteristic spatial pattern I’d expect. (That why I asked about a screenshot — even a single frame would show the spatial pattern.) Most of the flickering area is at vertical boundaries. That still leaves the possibility of crosstalk — If this was transferred as a composite signal at some point, a 2D comb filter was used to remove crosstalk, and some kind of 2D smoother was used, then you _might_ see this kind of dark flicker at vertical edges. But it might also just be a glitch in the decoder.

That said, Guava Comb does seem to work perfectly at removing the flicker.

scmccarthy

13th January 2003, 05:40

@neuron2converting the subsampled formats to fully sampled I know I am not telling you anything you don't know already, except I think you should make the point even stronger.

There is no such thing as converting to fully sampled formats. A format is fully sampled only if it's been captured that way, otherwise it's called interpolation. No need to further the myth that you gain anything by converting to RGB. Let those who are more comfortable working in the RGB color space do that for you.

Stephen

blivit

13th January 2003, 05:52

Originally posted by neuron2
You're exaggerating the difficulties. Anyway, converting the subsampled formats to fully sampled RGB doesn't invent new information. Why did you ask for a working approach if you just want to reject it on dubious grounds? :)

*Sigh*. I didn't write the original post thinking I was going to reject it on the basis of packed formats. I hadn't really thought much about the difficulties of packed formats at the time. I'm not saying that conversion invents new information, I'm saying that more information is preserved during/after the processing. Lets say a pixel gets detected as an edge and the neighboring pixel is not detected as an edge. What do we do about that? Do we call them both edges? Neither? What if one pixel has moved and the neighbor has not, because one of the pixels moved more in the plane that is not shared between them? When the average is taken, one of those pixels shouldn't be averaged, because it has moved, but since they share a value, they wind up both being averaged in the plane they share. So processing packed images looks to me like it has some accuracy problems. Wouldn't higher image quality result from using a non-packed intermediate, then converting back into packed again?

You wrote:

>since I couldn't find any good references on color edge detection
>anywhere

Search for "color edge detection" at Google. It's quite a well-developed area.

Been there, done that. I didn't find anything useful :( All the stuff I found was either too vague, cited articles in journals I have no access to, or said that noone really knew what way worked better than any other, so that rolling your own is as good a way as any.

BTW, Avisynth can do RGB; you're not limited to YV12.

I know. But one of the motivations to port to avisynth is to gain speed by working in packed color spaces and not doing color conversions. This is probably the most cited reason people desire avisynth ports. If I wind up working in RGB space, there goes the speedup. So I'm just left with the advantage that avisynth can see into the future. I think I might try porting it just to play around with that.

Where might I find good documentation on writing filters for Avisynth 2.5? I haven't downloaded the src yet, so I don't know if there are docs there are not. If there are, I admit ahead of time that I should have looked before asking :)

-Eric

Guest

13th January 2003, 05:53

Originally posted by scmccarthy
@neuron2 I know I am not telling you anything you don't know already, except I think you should make the point even stronger.

There is no such thing as converting to fully sampled formats. A format is fully sampled only if it's been captured that way, otherwise it's called interpolation. No need to further the myth that you gain anything by converting to RGB. Let those who are more comfortable working in the RGB color space do that for you.
I'm not furthering any myths. It is quite reasonable to speak of color space CONVERSIONS. So one may convert YV12 to RGB, and I explicitly stated that it doesn't add information.

Are you just in a pedantic mood? :)

Guest

13th January 2003, 06:01

>Wouldn't higher image quality result from using a non-packed
>intermediate, then converting back into packed again?

No. What was that you said about avoiding color space conversions?!

scmccarthy

13th January 2003, 06:18

@Neuron2

I am a little pendantic. It's the term 'fully sampled' that got to me. It makes it sound like you're getting more samples than you started with. As I said, I know *you* know better, but a lot of people seem to use that logic to say that RGB is better.

@blivit
packed format ?

Umm, isn't RGB a packed format? Red follows green which follows blue consecutively. You have to look at three different bytes for each pixel? And for YUV formats, you have to look ( or can look ) at three bytes per pixel, a Y and a Cb and s Cr. So, it might take a little getting used to, but it's all the same. Ergo:That’s not so hard. For example, you could use your Sobel RGB edge detector but use YUV instead. In other words, you’d take the maximum of the {Y, U, V} edge evidence in each direction. My hunch is that you’d be better off adding the evidence, instead of using a maximum — Y differences tend to be much larger than U or V differences. But otherwise the formulas could be the same. The point here should be encouraging: a lot of what you learn about RGB applies to YUV.

Stephen
[edit]Here is a site on edge detection:The Generalized Compass Operator (http://robotics.stanford.edu/~ruzon/compass/index.html)

High Speed Dubb

13th January 2003, 06:40

scmccarthy wrote
I mean that I still intend to investigate the PeachSmoother sources themselves and verify whether a YV12 version of PeachSmoother is possible.

It’s certainly possible — I just don’t think it will be worthwhile.

For the benefit of anyone listening, stay in the color space you capture in.

True enough. And for DVDs, capturing in YUY2 is silly. But broadcast (NTSC or PAL) video has a higher vertical U and V resolution than is available in YV12, so YUY2 capture does make some sense.

High Speed Dubb

13th January 2003, 07:11

Originally posted by neuron2
>Wouldn't higher image quality result from using a non-packed
>intermediate, then converting back into packed again?

No. What was that you said about avoiding color space conversions?! The answer to this isn’t so clear. Packed versus unpacked formats don’t inherently affect image quality at all. And switching between packed and unpacked (or vice-versa) isn’t necessarily lossy. (But YUY2 <-> YV12 is, since YUY2 has a higher U and V resolution than YV12.)

blivit

13th January 2003, 07:11

Originally posted by neuron2
>Wouldn't higher image quality result from using a non-packed
>intermediate, then converting back into packed again?

No. What was that you said about avoiding color space conversions?!

Sure, some accuracy gets lost during the conversion due to integer round off error. But that's at most probably off by 1. But how do you treat cases like I stated before where one of a set of pixels sharing a packed value should be classified as a different type of pixel (motion, edge, fixable, averagable, etc.) than the others? Then how do you process that pixel separate from its partner without messing up the partner? Converting to a "fully sampled" format would be sort of like doing your intermediate math in floating point rather than integer. I see it sort of like spatial round off error. Do all your pixel by pixel processing in fully sampled space, where you don't have to worry about the values of one pixel being linked to the values of its neighbors. Then convert back to the packed format (I'm using packed to mean two or more pixels share a value in at least one of their planes) after you're done processing. This would be analogous to, with just regular math, taking ints, doing math in float to avoid roundoff error in the intermediates, then rounding back to ints again when you are done.

I don't see how processing in a fully sampled intermediate space won't gain you some increased accuracy in the processing. What am I missing here?

I just caught this line — It isn’t true. You can get rid of crosstalk using future frames. A 1:2:1 previous:current:next frame weighting would work, for example. More generally, any scheme with equal weight for odd and even frames will be okay (for NTSC).

I was assuming equal weights to all frames. Do I really want to weight the current frame twice as much as the surrouding frames? I can give it a try once I get an avisynth port working.

On the down side, using the previously processed frame means that it’s possible to propagate an error through time, if you’re not careful.

I'm more worried about propagating errors. It's just too easy to generate artifacts :)

That’s not so hard. For example, you could use your Sobel RGB edge detector but use YUV instead. In other words, you’d take the maximum of the {Y, U, V} edge evidence in each direction. My hunch is that you’d be better off adding the evidence, instead of using a maximum — Y differences tend to be much larger than U or V differences. But otherwise the formulas could be the same.

Luma and chroma planes aren't on the same scale. I'd want to scale them to be roughly equal before trying to combine them together in any way. After scaling, I could try combining them just like I already do. I just don't know how "correct" this would be, using values from planes with different scales and different distance metrics.

Anyhow, looking at the clip, it is a little weird for crosstalk — there’s none of the characteristic spatial pattern I’d expect.

That said, Guava Comb does seem to work perfectly at removing the flicker.

From screenshots I'd seen in the past, I didn't think it looked like crosstalk either (no checkerboarding, no rainbows, no Moire patterns). Just plain flicker. It was taken from an interlaced NTSC DVD (Monsters Inc., computer graphics), after running decomb on it to deinterlace it. The pre-deinterlaced fields suffered from the flicker just as much as after deinterlacing, so I don't think decomb was at fault here. In fact, I just tried my current setup without deinterlacing, and there is no flicker. I'm pretty sure it was some sort of problem with the mpeg2 decoder. It only happened in a few places in the movie, and all of it happened in scenes up to and including the one I sent. It did not happen after that scene. It's like they messed up on the first part of the encode. On the same DVD, "Mike's New Car" is just horrible. It can not be deinterlaced by any means known to man, lots of interlaced motion banding/blur/ghosting problems. I think Disney did some bad things in the encodes....

I'll have to try GuavaComb later, since I'll have to swap back to avisynth 2.0 (right?). Virtualdub is running some long jobs, and I can't swap out the avisynth.dll because virtualdub has it open.

-Eric

Steady

13th January 2003, 07:29

I have a module for YUV 4:4:4 planar(one Y,U,V per pixel) if you would like to try it. It has (fast) functions to convert to/from YUY2/RGB24/RGB32.

YUV 4:4:4 planar module (http://home.midwest.net/~steady/yuvp91.zip)

Guest

13th January 2003, 07:31

Originally posted by blivit
But how do you treat cases like I stated before where one of a set of pixels sharing a packed value should be classified as a different type of pixel (motion, edge, fixable, averagable, etc.) than the others? Then how do you process that pixel separate from its partner without messing up the partner?Your error is in supposing that these problems vanish when you upsample to RGB and then downsample later. The downsampling causes pixels to "mess up" their partners.

blivit

13th January 2003, 08:12

Originally posted by neuron2
Your error is in supposing that these problems vanish when you upsample to RGB and then downsample later. The downsampling causes pixels to "mess up" their partners.

I'm still looking at this as a round off error analogy. I know that converting back "messes up" the partners again, which, to follow the integer/float analogy, is like rounding back to int. But in a filter that has many stages of processing, this "mess up" should only be done at the very end, rather than at each intermediate step. Like floating point math instead of integer.

Detect motion from current frame to previous. Detect motion from previous to one before that. Average stationary pixels. That's 3 stages right there. If each one of these stages was done in a packed space, each one would have the associated "round off error" of working in that lower sampled space directly, like doing integer math. Errors in the first stage propagate through the later stages, adding more errors at each stage. By doing intermediate processing in a higher sampled space, the intermediate round off errors are avoided, all the intermediates are not "messed up". The final answer at the end, after conversion back into the lower sampled space, should be different than if each stage had been done in the lower sampled space directly. Like the difference between floating point and integer math. Doing multi-stage processing in higher ordered space should be a little more accurate at the end of the day.

For simple filters, where you just do a single processing pass, I'll agree that you don't gain anything.

-Eric

High Speed Dubb

13th January 2003, 08:17

You know, it’s really hard to keep up with this thread!

blivit wrote
I'm more worried about propagating errors. It's just too easy to generate artifacts
Fair enough. There are ways to avoid problems — never pixel lock in a feedback-based filter, for example — but it can be tricky.

Luma and chroma planes aren't on the same scale. I'd want to scale them to be roughly equal before trying to combine them together in any way.
That was another of my approximate but fast ideas. I agree that it would be better to determine the Y, U, and V variances and weight them appropriately. Empirically the 2:1:1 weighting isn’t too far off — but that does make the effectiveness of the filter dependent on the user’s video input settings. (For example if they have their color settings too high, then U and V will get more weight than is really optimal.) [P.S. Beware that this is true of RGB differences, too!]

It [the clip] was taken from an interlaced NTSC DVD (Monsters Inc., computer graphics)...
Okay, that convinces me that it isn’t crosstalk. They’d be nuts to run that through any composite steps. Unless this is from a cheap bootleg, then I’m sure the flickering was due to a decoder bug.