Filter to remove rainbows from edges - for AVS2.0x (YUY2) and AVS2.5 (YV12/YUY2)

[Kurosu]

File attached out of date and therefore deleted. Please wait for the moderator's decision about the new file attached later.


After quite a long time, I'm back with a new filter. I kept it named AntiBlink, though 2nd generation. Source is released under the LGPL license and is included in the archive.

It checks U plan on 2 or 4 frames (depending on colorspaces/ version of Avisynth) to see if the chroma is oscillating (that's the shimmer/rainbow seen by everybody) and try to reduce that oscillating.
It has almost nothing to see with 1st generation of the filter, and is quite alike GuavaComb, though less powerfull. Guavacomb has those advantages:
- Also fix dot crawling (luma oscillations)
- can be setup to avoid ghosting
- can be used before deinterlacing (some little ghosting in AntiBlink2 may cause deinterlacing errors)
The idea originated from Lindsey, and it seems my plugin is worth releasing.

My filter has those advantages:
- YV12 support (80% faster)
- different parameters (see included antiblink2.txt file)
The speed has increased much:
AVS 2.5 tests: MarcFD's Mpeg2dec3 0.94 (idct=2) was used. ConvertToYUY2 is used for YUY2 data.
AVS 2.07 test: Nic's Mpegdecoder 1.32 (Dividee's MPEG2DEC plain achieves 45fps)
fps
| None | filtered
-----+------+----------
YV12 | 56 | 47 (AVS2.5)
-----+------+----------
YUY2 | 46 | 26 (AVS2.5)
-----+------+----------
YUY2 | 56 | 25 (AVS2.5)

This DLL uses some MMX code. It's my first attempt at MMX and x86 assembly in general. I expect it to be heavily plagued by typical lack of some optimizations (alignement, pairing, ...). I would greatly appreciate any feedback/help on that matter. For that purpose, the MMX YUY2 code is heavily commented.

The DLL also include the C version of the algorithms, and this version is only slower than the MMX version by 2fps (rather pitifull result).

[seewen]

Sorry, but there's no "AntiBlink2.txt" in the Zip.

It's verry difficult to guess the name of the parameter I can tweak..

Could you please copy it in this thread ?

[Kurosu]

Sorry for the .txt missing, attached (it may take a while to get approved now). I hope there is no other problem.

Therefore, I paste the anitblink2.txt:

How it works
------------

The rainbow artifacts are considered in this plugin as chroma oscillations. The filter therefore detects them by measuring chroma variation between frames for each pixel. It filters those pixels by averaging the 'blinking' pixels with their relative(s) in the next picture(s).

There is another filter out there much more efficient, namely GuavaComb by Lindsey Dubb. Its advantages are:
- Also fix dot crawling (luma oscillations)
- can be setup to avoid ghosting
- can be used before deinterlacing (ghosting in AntiBlink2 may cause deinterlacing errors)

This filter is able to work in both YUY2 colorspaces and, contrary to GuavaComb, YV12 (great speed increase).

Syntax
------
AntiBlink(minvar=[0...255],debug=true/false,MMX=true/false)
Ex: Antiblink(15,false,false)

No default, so set all options.

. Minvar
Minimum value of the amplitude of the variation. A good default is 15. Must be an integer
between 0 (enormous ghosting) and 255 (no effect, slow noop filter). Going over 30 is likely
to be useless.

. debug=true/false
Display the matches in pink if set to true

. MMX=true/false (true has no effect)
Enforce the non-use of MMX code (for speed tests), falling back to plain C code.

Code used
---------
In YUY2, the number of pixels must be MOD4 (dividable by 4 without reminder) for MMX code to
be used. 4 pixels are used for averaging.
In YV12, the number of pixels must be MOD32 for the same reason.
Otherwise, the plain C fonctions will be used. 2 pixels are used for averaging. This is a limitation brought by the MMX code and put in the C code to ensure fair speed comparison.
Those limitations might be removed depending on other coders' feedback.

[esby]

attachment is too old,
any url to get the supposed v2 ( maybe i'm wrong) of antiblink?

esby

thx in advance

[Kurosu]

I haven't updated my site (for what they're worth...

But on your request, here is a direct link:
http://kurosu.inforezo.org/avs/AntiBlink2.zip (case sensitive)

I'm trying to take into account motion, but it's a pain to use in MMX...

[seewen]

Thank you.

It's much easier so

[esby]

this links works better
http://kurosu.inforezo.org/AntiBlink2.zip

[sh0dan]

Some of your assumptions in the filter are wrong. Basicly, you only process width*height bytes from the top of the images, but that cannot be safely assumed.

All images are given a pitch. The pitch is basicly what can be described as "length of a line". What's funny is that the pitch does not have to be equal to the width of the image.
If you for instance crop something of your image, the only thing that changes is the width of your image, the pitch and the actual byte-count of a line remains the same.

The image is then layed out like this:

rrrrrrrrrrrrrrrrpppp
rrrrrrrrrrrrrrrrpppp
rrrrrrrrrrrrrrrrpppp

Where 'r' are the pixels inside the image used, and 'p' is the padding inserted after each line.
In YV12 the Y-plane always has a byte-count that is a multiple of 16, but it is still possible to have mod2 width images, because the rowsize is different from the pitch. Similar the UV-planes always have a pitch that is a multiple of 8.

When you request frame->GetRowSize(PLANAR_Y) you will get the rowsize of the pixels _inside_ the image. When you request frame->GetRowSize(PLANAR_Y_ALIGNED) you will ALWAYS get a rowsize that is a multiple of 16 (multiple of 8 on UV planes).

The reason for you to use pitch to get from one line to another is that pitch can be even larger than the aligned rowsize - for instance after a crop.

So what you should do is:
- Get a rowsize.
- For each line in the image:
- Process (aligned) rowsize pixels.
- Skip to next line, by adding pitch for the current plane.

A quick'n'dirty solution would be for you yo process picth*height pixels, but you may end up processing a lot more pixels than you have to, if your filter is placed after a crop.

[Kurosu]

@Sh0dan
Thanks for the tips.
frame->GetRowSize(PLANAR_(component)_ALIGNED) should be used for code that would fetch 16 bytes by iteration on any plan (ie, using MMX/SSE), I guess.
Considering this, the loop processing YV12 data was therefore incorrect.

@Anybody
A new version is attached, with the above change, and the possibility to check for motion (antiblink2.txt updated . I've almost dropped the MMX code as it wasn't satisfying compared to plain C. The speed is still satisfying even with the new additions:
in YV12/mpegdec3 0.94: 39fps filtered vs 52 unfiltered
in YUY2/mpegdecoder 1.32: 28 vs 45 fps.

The syntax has therefore changed.

I don't expect many updates now, unless people point some necessary/ very usefull changes.

[High Speed Dubb]

Cool — It’s good to see another comb filter become available.

Guavacomb has those advantages:
...
- can be setup to avoid ghosting

Well, it tries to. Guava Comb doesn’t manage this as well as I’d like.

- can be used before deinterlacing (some little ghosting in AntiBlink2 may cause deinterlacing errors)

Actually, Guava _must_ be used before deinterlacing. (Technically, there are some special cases in which you can use it after — But it isn’t easy to figure those out.)
[P.S. I accidentally wrote ”before“ instead of ”after“ in that last sentence.]

[Kurosu]

- can be setup to avoid ghosting
Well, it tries to. Guava Comb doesn’t manage this as well as I’d like.

The last setup I've made (not released) use the previous and the next frame. It only checks for Y variation (above a threshold) between those 2 frames and the processed frame (only checks one in the available filter), but that's still better than none at all.

My intent is to keep it relatively fast, although checking neighbour(s) could be more efficient. Yet such bad videos (even from DVDs) also have noisy edges, so it may prune too many pixels from the processing.

Actually, Guava _must_ be used before deinterlacing. (Technically, there are some special cases in which you can use it before — But it isn’t easy to figure those out.)

In term of frames (not fields), I guess GuavaComb checks the 2nd frame after the one processed (in NTSC)? In static scenes, this produces an important delay (20 frames with GuavaComb("PAL",70,30)).

Then, how many pixels do you use to 'average'? I have a test clip (the one used above, maybe you already have it) where the pattern '121' (weight of pixels across neighbour frames) doesn't filter properly the clip. Though, using 1111 manages to get rid of it, but leaves very colourfull dots. So does GuavaComb("PAL",70,30) or GuavaComb("NTSC",70,30), so I suspect this clip to have aliasing other than simple crosstalk.

Running the '111' setup twice on the source removes it, while using twice GuavaComb("PAL",10,90) (or any setup in fact) leaves it.

Last thing, could you build it for Avisynth 2.5. You just have to replace avisynth.h by the one here (a bit outdated?). And change AvisynthPluginInit(...) to AvisynthPluginInit2(...) to please Sh0dan

the right parameters are: integer, integer, boolean (read from your txt file) aren't they ? But only integer, integer, boolean works (YV12 avisynth 10/01/03) and so i can't set de max value (the min value is set to 0, the only efficient value)

feedback with "Evangelion Death & Rebirth PAL DVD"
the rainbow is over an interlaced source. deinterlacing the video with non-interlaced rainbow creates ugly artefacts, which are almost removed by antiblink in fast motion. But in low motion it creates new artefacts ;_(

with non-interlaced frame the rainbow effect are nice removed, but sometimes chroma loss (red and dark frames) are too high.

[Kurosu]

@Sigmatador
You're right, I tested the posted version, although the source files are correct... New version 2.2 attached (older ones won't be deleted, and web file isn't updated, as I'd like to keep a track of the number of downloads).

Anyway, using minvar=0 is very dangerous, as it will check every pixel for variation. It's rather the maxvar=true (that is, 1) that was causing the problem (almost no pixel could be filtered). I hope this version fixes the problem. Sorry for the inconvenience.

I'm now wondering if the use of 4 frames to average (much more efficient at cleaning frames in the design) is worth the possible ghost artifacts. I'm planning to reduce the averaging to 3 frames.

[High Speed Dubb]

Kurosu wrote
In term of frames (not fields), I guess GuavaComb checks the 2nd frame after the one processed (in NTSC)? In static scenes, this produces an important delay (20 frames with GuavaComb("PAL",70,30)).

Nope — It just checks the t=-1 and t=-2 frames for NTSC. (My current private build uses t=-1 and t=+1, instead.) But it combines this check with a past record of shimmering at that location, and only triggers the comb filtering if the shimmering has been happening for a while. Just how long a while it checks depends on the recall setting — with a recall of ~85%, it effectively looks back something like 20 frames. (With 85% recall, the weight given to past shimmering is 0.85 for the previous frame, 0.85*0.85 for the one before that, etc.)

Then, how many pixels do you use to 'average'?

The currently available version uses two pixels, weighting them equally. My current private build uses three pixels, weighted 1:2:1.

I have a test clip (the one used above, maybe you already have it)...

Could you post or send it to me? This sounds like an odd one. 121 should work fine for stationary areas in NTSC sources. (And 10201 should work for PAL.)

Last thing, could you build it for Avisynth 2.5....

One of these days... The Guava might actually work better in YV12 than in YUY2.

[Kurosu]

Nope — It just checks the t=-1 and t=-2 frames for NTSC. (My current private build uses t=-1 and t=+1, instead.)

That's really fun, as I was too not using centered temporal window
And as you might have seen, it's following the opposite trend to your filter: I started with 4 frames used, and now I think I'll use 3 frames.

[...] (With 85% recall, the weight given to past shimmering is 0.85 for the previous frame, 0.85*0.85 for the one before that, etc.)

Maybe (much later), you could add such a description to your help file, it really helps understanding how it works, and how to setup it.

I have a test clip (the one used above, maybe you already have it)...
Could you post or send it to me? This sounds like an odd one. 121 should work fine for stationary areas in NTSC sources. (And 10201 should work for PAL.)

I'll leave you a pm with the url, I guess tomorrow. But this one is PAL (from Ronny, if you remember the old thread). At a time I had a funny pattern, but private builds filter it worse. Your weighing (reminds me a lot of the old thread) indeed needs to be checked against that clip (and some other 25fps material, by the same way).

Last thing, could you build it for Avisynth 2.5...
One of these days... The Guava might actually work better in YV12 than in YUY2.

YV12 wasn't the matter for me. It's just that I like to check how the filter performs against different clips, at quite different scopes. And as I'm alos testing YV12, I need AVS2.5 for it and AVS2.0x for your filter. I have to spend some time renaming files and editing scripts
Plus what I ask you is a matter of minutes, but I understand rebuilding for AVS2.5 your filter can still bother you.

[High Speed Dubb]

And as you might have seen, it's following the opposite trend to your filter: I started with 4 frames used, and now I think I'll use 3 frames.
Well, specificity is really important for these comb filters. Averaging at the wrong place/time can look really ugly. My solution was to go overboard in looking for long term evidence, but there are better approaches.

Maybe (much later), you could add such a description to your help file, it really helps understanding how it works, and how to setup it.

There’s an explanation of it in the source comments. The user docs are meant as a practical guide for choosing settings — I want to leave the nuts and bolts stuff in the source files. Besides, if I gave a technical explanation of Recall, I’d also need to give a technical explanation of Activation, and that would get very confusing. (Check the comments in the RESCALING_PROCEDURE stuff in FLT_TemporalComb.asm and you'll see what I mean.)
http://cvs.sourceforge.net/cgi-bin/v...viewcvs-markup (This has the main filter explanation)
http://cvs.sourceforge.net/cgi-bin/v...viewcvs-markup (And this has the explanation of the Activation setting)

The main problem with the Guava docs at the moment is that they haven’t been changed enough from the DScaler version. The last sentence describing the Recall setting doesn’t really make any sense for an AVISynth filter. (I don’t think anyone will be changing their script in the middle of a run.)
http://students.washington.edu/ldubb...Guava_Comb.htm

But this one is PAL (from Ronny, if you remember the old thread). At a time I had a funny pattern, but private builds filter it worse.
Is this the clip with the skyscraper in it? I still have that one, so there’s no need to resend it. Yeah, I was wondering about the speckles in that one, myself. For that one, 10201 averaging works just fine — except in the speckles, where it isn’t working at all. I think your guess about aliasing effects could be right.

The private build is much more conservative about averaging, so I wouldn’t be surprised if it gave more speckles than version 0.9.

Plus what I ask you is a matter of minutes, but I understand rebuilding for AVS2.5 your filter can still bother you.
It’ll take a little more than that. There will need to be some more documenation to explain about the versions (both in the docs and online, in the Guava Comb thread), and the filter should properly report if it’s being used with the wrong version of AVISynth, which I don’t know how to do. I’m hoping to wait a little longer on this stuff.

[Kurosu]

Well, specificity is really important for these comb filters.

It seems our method are converging, though the way we prune some pixels is different. Yours sounds better, so I wonder if I should continue on this filter (plus I'm still learning assembly language)

There’s an explanation of it in the source comments.

Indeed... I had only looked at the last one and, being mainly ASM, I simply had dropped it at the time.

Is this the clip with the skyscraper in it?

Yes. In fact it's the way the shimmering is detected that doesn't work here. When using the older antiblink (which was only bluntly looking at spatail data), those pixels were filtered by using 3/5 pixels (seeing how the estimator was biased by this mean, I wonder how it worked fine) to average, and shimmering seemed to have been properly filtered.

The private build is much more conservative about averaging, so I wouldn’t be surprised if it gave more speckles than version 0.9.

It does with mine, and running 2 instances of the filter doesn't help.

It’ll take a little more than that.

I got your point. Next reply is just an explaination, therefore.

and the filter should properly report if it’s being used with the wrong version of AVISynth

Rename the Init function to AvisynthPluginInit2. Avisynth 2.5 will check that exported function, while Aviynth 2.0x/1.0x won't and will report on loading that it's not a plugin for that version (2.0x/1.0x), as AvisynthPluginInit isn't exported.

[Dali Lama]

Kurosu,

I havn't been able to get any luck with this filter removing bright color noise around sharp contrast edges in anime. Just using a regular temporal filter seems to do the job better. Is there anything you suggest. I've tried the highest settings but the artifacts remain and ghosting appears. I am not sure how your filter actually determines which artifacts it should remove?

I hope you improve more on this filter. Its a good idea.

Take Care,

Dali

[Kurosu]

@Dali Lama
I know this filter isn't as effective as it would need. In order to test its efficiency, you need to test by setting last parameter to true (debug activated):
pixels considered as having shimmer are either violet (will be filtered) or green (pruned due to motion). Note that motion is determined in a rather quick way and that noise may be considered as motion.

My advice would be to use AntiBlink(10,20,true) to start, or maybe AntiBlink(5,20,true). Why it ghosts so much or doesn't filter that good is that there's only a 2-frames delay even after motion for the filter to kick in. If you want less ghosting, maybe better filtering but that comes more slowly, GuavaComb is a good choice in YUY2.

Most often, the detection works okay. That's the cleaning that isn't performed that well, and unfortunately, I have yet to figure the best away (and if possible, a good one) to do it...

[gigah72]

thanx Kurosu,

i use your filter on every capture i do. i noticed, that if i change to YV12 before calling my filterchain, the debugmode shows me much less violet pixels than if i stay in YUY2. i capture in YUY2, avisynth 2.5beta, usual script:
..
Crop
FluxSmooth
AntiBlink
Convolution3d
Resize
AddBorders
..

what does this mean, less correction in YV12 ?

and no, i won't use guavacomb

g.