Is there a YV12 filter for removing rainbowing?

[Bulletproof]

I know there are many YUV2 filters for removing rainbow artifacts, but I have not found a YV12 one yet, does anyone know if there is one?

[OUTPinged_]

There aren't "many".

There are only two of them(if you would list only the ones that work).

The one is "antiblink", the other is "guava comb". Go beg their developers to release yv12 versions :-)

[jang0]

you could try cnr2 by marcfd.
get it here: http://ziquash.chez.tiscali.fr/

[OUTPinged_]

cnr2 doesnt remove rainbowing.

[jang0]

From MarcFD's Page:

Cnr2
(Chroma Noise Reducer v2)
This is the YUY2 and YV12 adaptation of CNR, a VDub Filter.
A very fast chroma denoiser. Usefull to filter VHS/TV caps
and against rainbows

At least it claims to remove rainbows, but I can't tell if it is really effective or not as I don't have any appropriate sources to test. Perhaps I also have a different definition of "rainbow artifacts".

[Bulletproof]

I tried CNR2 today, it seems to work adequetly, except I think I found a new problem. CNR2 and FluxSmooth don't seem to work together properly. In my script, if I put FluxSmooth after CNR2, Avisynth reports an access violation. However if I switch the lines in the script and put FluxSmooth before CNR2, it works properly.

[OUTPinged_]

Ok ok, it does reduce it somewhat, but ie AntiBlink _removes_ it. Go check appropriate ropic.

[MaTTeR]

@Bulletproof

For clarity, do you have screenshot you could post of this artifact? I was going to post a thread about the color stepping/contour problem but we might be talking about the same problem. If we are, I think you'll find the problem is _much_ worse in YV12 compared to YUY2.

Edit- The access violations _might_ be something SansGrip is already looking into on FluxSmooth. Is your source by chance interlaced or non-progressive?

[Bulletproof]

Actually, yes my source is PAL interlaced, I didn't add any deinterlacing to it becuase the source material is such a mess. About the stair stepping, I had not mentioned anything about that, but now that you mention that, I have noticed that color around edges of objects seem to look very blocky and look like stair stepping, and also some color leaks also, but nothing blurry, just sharp stair stepping. I'm not positive if that was in the source or not, I'll take a look again to see if its the filter causing that.

[Koepi]

in YV12, color is saved only with quarter resolution of luma.

i.e. 768x576 -> 384x288.

Thus colours are "huge" pixels of 2x2 (4 pixels) which can be seen as stair effects.

This is within the nature of that colour space.

Use the search, we had many threads where this issue has been talked about, and read up about colour spaces in general for getting an overview of what's going on before reporting such things as bugs.

Regards
Koepi

[Bulletproof]

Well at first I did start this thread just to find out what I could use for removing rainbowing in YV12. Then I was getting access violations and I didn't know which filter in particular was causing it, so I didn't start a new thread. But Matter has informed me that FluxSmooth does have some problems with interlaced sources.

About the chroma stairstepping problem, on my side it was related to Nic's decoding filter for XviD, I don't know if Matter was expiercing the same thing, but clearly there is chroma stairstepping which should not be there. I will add a new reply in the XviD forum on the latest version thread and put some new screenshots there to show you what I mean. Maybe Matter was expierencing the same thing.

[unimatrixzer0]

is there anything besides antiblink or cnr2 for YV12? I try both, antiblink gave me errors so i couldn't use it and cnr2 didn't remove any of the rainbowing.

I can't possibly be the only person here who uses Avisynth 2.5x w/ YV12 that has rainbowing in their video... how do the rest of you deal with it? None of the filters I've tried work or aren't designed for YV12.

[Schlumpf]

In the last clip I encoded there were tons of rainbowing artifacts, but I also failed to remove them.
After I saw that CNR2 and Antiblink (from Warpenterprises' page) both failed on my source (It looked like it wasn't filtered at all, at least the ranbowing was as strong as before.), I resorted to some older Filters that still work in YUY2.
Thanks to LoadPluginEx() I could then test GuavaComb and an older Version of Antiblink that I found in Kurosu's thread. GuavaComb was as bad as the previous filters: Didn't touch the rainbows at all. But when I saw the old Antiblink in action my eyes widened suddenly: It got perfectly rid of almost all the rainbows.
However, it revealed its major drawback when I played it in Virtual Dub Mod: It creates severe and ugly chroma ghosting and the scene change detection could need some improvements.
You see...I'm still searching for the holy grail myself, so post when you think you found a solution ;)
Hmm, it's weird though that the older version of AntiBlink was more effective than the newer one. :confused:

[Didée]

Oh yeah, those rainbows ...

I have them in almost all of my TV captures, sometimes more, sometimes less.
And me too, I could never get something useful out of AntiBlink or GuavaComb. I assume I am too dumb to use them correctly.

Instead, the following is always part of my scripts for encoding TV captures:


xx = 576 # desired
yy = 272 # output resolution

AviSource( Whatever )

chro = BicubicResize(xx/2, yy/2, 1.0, .0).BicubicResize(xx, yy, -.6, .8)
lum = BicubicResize(xx, yy, .0, .5)
MergeChroma(lum, chro)
This reduces the rainbowing by a fair amount, while doing very little harm to the overall picture. Eventually, a little TWEAK()-ing the saturation up can be necessary after that.
I know it's brute-force ... try yourself and see if you like the result or not.

Regards

Didée

[bilu]

I've tried AntiBlink, CNR2 and GuavaComb.

They only removed rainbows on static scenes, I don't know of any filter that worked on movement.


Bilu

[unimatrixzer0]

@Didée

hey thanks! the bit of code actually made the rainbowing about 1/2 as intense!:) How do I tweak it to make it a tad bit stronger to remove even more of the rainbowing?

[mf]

Originally posted by Didée

xx = 576 # desired
yy = 272 # output resolution

AviSource( Whatever )

chro = BicubicResize(xx/2, yy/2, 1.0, .0).BicubicResize(xx, yy, -.6, .8)
lum = BicubicResize(xx, yy, .0, .5)
MergeChroma(lum, chro)
This reduces the rainbowing by a fair amount, while doing very little harm to the overall picture. Eventually, a little TWEAK()-ing the saturation up can be necessary after that.
I know it's brute-force ... try yourself and see if you like the result or not.

Regards

Didée
Oh so you're one of those guys that totally screws up chroma by linearly blurring/downsampling. Do you work on anime by any chance? ;)

[unimatrixzer0]

@mf

...are you saying you have a better way :confused:

[mf]

Originally posted by unimatrixzer0
@mf

...are you saying you have a better way :confused:
Try MergeChroma(MSmooth()) for fun.

[unimatrixzer0]

Originally posted by mf
Try MergeChroma(MSmooth()) for fun.

tried for fun, unfortunately it didn't remove any of the rainbowing.

[mf]

Originally posted by unimatrixzer0
tried for fun, unfortunately it didn't remove any of the rainbowing.
Weird source then, works pretty well on anime, especially to clean already derainbowed sources (Narue no Sekai).

[unimatrixzer0]

Originally posted by Didée
Oh yeah, those rainbows ...

I have them in almost all of my TV captures, sometimes more, sometimes less.
And me too, I could never get something useful out of AntiBlink or GuavaComb. I assume I am too dumb to use them correctly.

Instead, the following is always part of my scripts for encoding TV captures:


xx = 576 # desired
yy = 272 # output resolution

AviSource( Whatever )

chro = BicubicResize(xx/2, yy/2, 1.0, .0).BicubicResize(xx, yy, -.6, .8)
lum = BicubicResize(xx, yy, .0, .5)
MergeChroma(lum, chro)
This reduces the rainbowing by a fair amount, while doing very little harm to the overall picture. Eventually, a little TWEAK()-ing the saturation up can be necessary after that.
I know it's brute-force ... try yourself and see if you like the result or not.

Regards

Didée

lol, thanks. For a moment I was happy until I had someone explain to me what exactly that script did. :( Don't like the idea of my video res being chopped in half then resize big again. It works but just not the methiod I was looking for =/ Thanks for the suggestion though.

[neuron2]

It's only the chroma that is resized by half and up again. I bet you couldn't tell the difference, except for the reduction of rainbowing. :)

[unimatrixzer0]

oooo, you made my day neuron2, Thanks! Didn't know it just effected the chroma.

btw... is the cat in your avatar related to Spot on TNG? :)

[mf]

Originally posted by neuron2
It's only the chroma that is resized by half and up again. I bet you couldn't tell the difference, except for the reduction of rainbowing. :)
Well, and of course the grey/colored halos, and miscolored small objects *hide* ;).

[mf]

Ok, cause I couldn't find a working 2.5 version of AntiBlink, I decided to make my own ^^'. I had it done in 20 minutes, not bad imho :D. It's slow though (as usual). It works pretty nice for me so I decided to make a function out of it (which took longer than 20 minutes btw :p).

You can get it here: http://mf.onthanet.com/avisynth/mfRainbow-0.1.avs

Parameters:

type - name - description - range - default

string mode - processing speed (and quality) - slow/slower/slowest - slowest
int strength - processing strength - 0-255 - 255
float clipstren - value clipping strength - 0.0-10.0 - 5.0
bool post - spatial postprocessing - true/false - true

"Slow" mode might cause chroma ghosting on edges. Slowest is the safest, but might not be as effective as slow.
Explanation of clipstren: this works like value boosts in mfToon, it's the gamma correction of the mask ^^'. Lower values give more variation, but less strength.

You need Deen (http://ziquash.chez.tiscali.fr/Deen%20beta%202.zip), MaskTools (http://kurosu.inforezo.org/avs/MaskTools.zip) and MSmooth (http://shelob.mordor.net/dgraft/msmooth/msmooth200b1.zip).

Like before (and probably ever again), I don't check colorspaces cause I'm too lazy, and my function always outputs YV12. Who needs another colorspace anyway. I'm still kinda sad about the fact that Layer and Mask don't work in YUV (cause of the lack of an AYUV colorspace), so alas I have to do 2 colorspace conversions, namely to RGB and back. The luma luckily doesn't have to be touched (and I do MergeChroma to ensure that), so that minimizes conversion errors.

Please test if this is any better or worse than existing derainbowers. :>

[unimatrixzer0]

how exactly do you use this? I have no idea what to do with mfRainbow-0.1.avs otherwise I'd give it a try :)

[Bulletproof]

When using YV12 colorspace, doesnt that technically already reduce chroma resolution?

[mf]

Originally posted by unimatrixzer0
how exactly do you use this? I have no idea what to do with mfRainbow-0.1.avs otherwise I'd give it a try :)
Import("mfRainbow-0.1.avs")
AVISource("file.avi")
mfRainbow()

Sorry, forgot to mention :).

[Wilbert]

You could also add it to the ShareFunctions page on avisynth.org (http://www.avisynth.org/index.php?page=ShareFunctions)!

[mf]

Originally posted by Wilbert
You could also add it to the ShareFunctions page on avisynth.org (http://www.avisynth.org/index.php?page=ShareFunctions)!
Err, how? I've looked into the creepy no-login editing system, but it seems a bit unclear to me how to add links.

[Wilbert]

I've done it for you :) Have a look at: http://www.avisynth.org/index.php?page=mfRainbow. I changed your script a bit:

function mfRainbow(clip input, string "mode", float "clipstren", \
int "strength", bool "post")

into

function mfRainbow(clip input, string mode, float clipstren,
\ int strength, bool post) {

"" means that the values have a default (which they haven't). If you want to add/change something, just go to: http://www.avisynth.org/index.php?page=mfRainbow -> check "edit document" -> change somthing -> check "preview" to see whether it is ok -> check "save".

You can add wiki-links, in two ways:

1) write two capitals in a word: for example, mfRainBow
2) put (( )) around it, for example, ((mfRandbow))

[mf]

Huh?! They do have a default! They're all defined a line lower:

mode = Default(mode, "slowest") # processing speed, slow-slower-slowest
strength = Default(strength, 255) # processing strength, 0-255
clipstren = Default(clipstren, 5.0) # value clipping strength, 0.0-10.0
post = Default(post, true) # spatial postprocessing on/off
I am absolutely positively sure that all parameters (except clip ;)) are optional.

Changed it back and also added mfToon :).

[High Speed Dubb]

It’s really important for rainbows to mention the type of video (PAL, NTSC, or SECAM) and the format (YV12 or YUY2).

Didée’s method should work well for NTSC YUY2 full resolution video — Though it would be better if it used bilinear resizing (instead of bicubic), and it would also be improved if it just halved the vertical chroma resolution, and left horizontal resolution alone. It should also work okay (but not perfectly) on PAL YV12, and passably on PAL YUY2.

[Boulder]

I tried Didée's method on my PAL YV12 capture yesterday, but it didn't work there. Unfortunately I don't have the clip anymore so I can't let you guys think of any other ways to clean it.