How to avoid "moving edges"?

[dbzgundam]

In some broadcasts, and unfortunately, some very grainy DVDs, filtering usually ends up (I can understand why of course) killing some excessive grain, but at the same time, produces a NASTY effect on edges that some grain was over.

This effect can only be described, literally, as moving edges. It's almost like what happens when you warpsharp too much and the edges start waving a bit, but the effect with grain is so much worse.

(Deen, undot, SmoothD, and Convolution3D were tested... Better add RemoveGrain/Dirt to that list)

[Manao]

What you describe seems to be ringing. So you have de deringing. Try mf's script ( HQDering, iirc ). If it's not that, remember that a screenshot is worth a thousand descriptions

[dbzgundam]

Hmm, I'll try it first, if I get poor results, I'll start posting screenshots.

[Kagura]

The problem occurs due to filters/codec causing a pixel to go "over" the threshold for filtering on one frame and "under" the next. The infamous "flickering line" effect is the result.

Having seen this effect many times on episodes (both raw and subtitled), I've come to the conclusion that it cannot be solved. Think about it. How advanced would a filter have to be to detect changes in thickness of the lines (waving effect) without going into the horrors of motion compensation?

Having said that, this effect can be avoided very easily. Tone down the number of filters you're using. IMO, all the filters you've named are too strong even at lowest settings (let's not even get into detail killing) and likely to cause flickering. Remember, the more you filter, the more likely a pixel will fluctuate between frames.

Another approach is to make a mask of the lines you want to preserve in the source. Then, apply your filter chain maskedmerged to mask.invert()

EDIT: I've seen this effect; it is definitely not ringing. Actually, the mf series of filters are notorious for causing some of these flickerings.

EDIT2: I just remembered. Sangnom, when used at the right super-sampling, can help with "some" of the lighter flickers. However, it is not a foolproof solution as it is merely spatial and can only approximate how the line is supposed to look.

[Seed]

Originally posted by Kagura
The problem occurs due to filters/codec causing a pixel to go "over" the threshold for filtering on one frame and "under" the next. The infamous "flickering line" effect is the result.

... fluctuate between frames.

If this is the case, and I believe you are right, I'd think temporal filters or spatio-temporal filters that are temporal-dominant should help to minimise this effects. So cutting down the use/strength of spatial filters (e.g. eDeen) and putting some temporal filters at the end of filter chain may help.

[AS]

/me waves at Kagura

Originally posted by Seed
I'd think temporal filters or spatio-temporal filters that are temporal-dominant should help to minimise this effects.

Imo, that wouldn't work. Here is why:

Temporal threshold - too low -> No effect, or potentially exaggerate the effect by locking the edges under threshold while doing nothing on those over the threshold.

Temporal threshold - equal to the difference of the two -> Ineffective, as it will stablise those under-threshold and yet again nothing on those over-thresholded because the "base" threshold is different due to noise.

Temporal threshold - equal or higher to highest "base" (of the over- and under-thresholded edge) + threshold equal to the difference ( the sum will be equal or most likely to be greater than the threshold of those over-thresholded): now both under-and over-thresholded edges are hit, but given the non-uniformity of the edges, only part of the edges will be locked on and risk destroying some of the original intent.

When the spatial component kicks in as the temporal part fails(changes greater than threshold)... Well, edges are just going to be eaten away (even though it shouldn't be, in principle).

As you can see, whether we can fix this or not depends on whether we can separate the two, the ones under and ones over a certain "threshold" and classification and detection of that threshold. Once we get pass that, pick out that particular set of frames (make a mask or something). then we need a method of reliably determining the relative thickness we need to add (or subtract) to the set of those under threshold (or over threshold). After that, it's simply applying a edge thickener (with clamping so it's more uniform)and then mask (or a pixel look-up) it back to the opposite set based on the mask/threshold criteria of the opposite set.

This is all imo and I probably have some misunderstanding of how they really work. Do correct me ^^;

Still, easier said than done though. This problem, in some ways, is similar to the creation of real sharpness mask or a sharpness mask (or sharpener) that enables sharpening of the background alone, specifically. Deciding which variable(s) is(are) to be used as the criteria (set) is likely to drive people up the wall.

I agree that it need motion compensation to work properly, to be honest.

[Kagura]

Quickie post for now. I have to do hmwk.

AS: Temporal filtering, especially a temporal averager (a la temporalsoften) will help on static scenes. That's because they smooth out the differences in the temporal domain and should ameliorate (not completely solve) light flickers. Moving scenes become a whole new boatload, however. Not even mo comp can solve this. See attempts in dust and xvid me. After detecting the fluctuations, which frame does the filter use as the "base"? The first frame? What about other subtle changes that occur that are not flickers? Will those be smoothed out and averaged? In short, such aggressive mo comp filtering will only cause more and different problems.

As of now, this stands with flickering luma blocks (working on it) and chroma edge bleeds (getting close on this one though) as one of the 3 great unsolved "artifacts" in encoding, imo. =/

[Kagura]

I just discovered a solution to this while experimenting with some techniques.

Do something like:

input.utoy().lanczosresize(640,480)
deen("a3d",2,3,0,8,0)

u = last.lanczosresize(320,240)

input.vtoy().lanczosresize(640,480)
deen("a3d",2,3,0,8,0)