Attacking rainbow... with gusto!

[punistation]

Howdy ho!

First things first, I should admit I'm a poor neophyte when it comes to Avisynth. I've been using it (v2.08) to make several nice XviD encodes, but that's only using a bare-bones basic AVS script, then re-using it as a template. I mainly follow step by step "How To..." guides. This is probably because I'm terrible at programming. The only thing I can program is VCR+, and even that's pushing it.

However, when absolutely needed, I try out the occasional filter. I also read lots've guides and forum threads about how to use a particular filter... so I like to think I'm a nice newbie. I try before I cry, y'know? :p

That said, I've been learning how to use some filters to get rid of the dreaded rainbow effect in my latest anime encode attempt, Armitage III (Region 1-4). Here's a sample of what I'm up against:

CLICK ME (http://punistation.wgwashu.com/Armitage.avi)! (2.8MB AVI)

:p (I know the absence of the lead jailbait anime babe makes for a boring
few seconds, but this scene captures the yucky rainbow effect perfectly!)





Reading lots've forum threads, I decided to go with SmartsmootherIQ, and what I think is meant to be a better replacement, AntiBlink. Problem is, neither really worked that well.

AntiBlink just couldn't get rid of the rainbows that well. Maybe it was just me not fully understanding how to tweak it just right, but remember my allergy to programming. I gave it the flick after an hour or two of frustrating experimentation. :(

SmartsmootherIQ worked magnificently when used as a VirtualDub filter, but that brings the icky RGB thing into it. Of course, the worst bit is an encoding rate of 1fps. I've experienced the joys of using Fast Recompress, and I don't want to lose it. Sadly, the settings don't work the same when I use Smartsmoother in Avisynth. A perfect setting of 11-200 will yield a horrible blurry mess when used in Avisynth, and I spent ages raising and lowering the settings to get a VirtualDub equivilent. I failed. rainbows remained.




But then I had me an idea: "Why not use both?"

...and I did. Please compare and contrast this new encode with the AVI posted above. The AVS script I used is quoted below. I'm very happy with the results. :)

CLICK ME (http://punistation.wgwashu.com/Armitagee.avi)! (2.5MB AVI)



Import("C:\DVD\AviSynth-Plugins\vdub_filters.avs")
mpeg2source("C:\DVD\DVD\ARM.d2v")
BicubicResize(640,480,0,0.5)

Antiblink(15,20,false)
ConvertToRGB()
VD_SmartSmoothIQ(11, 80, true)
ConvertToYUY2()

Telecide(chroma=true,gthresh=50,guide=1,threshold=30)
Decimate(5)




It took ages to get the right sequence, let me tell you! I'd get rainbows if the resize filter was after the Smoother, or the Telecide before the Antiblink, etc. I finally tried the above, and I reckon it worked great!

Sadly, the problem is that the encoding process itself (on Fast Recompress) is an average of 1fps. I get a spurt of 2-3fps, then a few seconds of 0fps. It stays like that the whole way, so on average it's about 1fps. Someone like me may ask "why not just use SmartsmootherIQ in VirtualDub?" Well, I compared it's results to my AVS script above, and mine looks much better. It really does.

So, my question: is there anything I can do to speed things up? :(



1fps really grips my bum. All that forum reading and filter experimenting, and I finally get a nice AVS script... only it encodes at a snail's pace. I really don't want to have to leave my PC alone to itself for over 48hrs just get a nice 40min anime episode if I don't have to. Can anything be done?

Thankee kindly in advance for any help!


Kisses XXOOXX
Jen

[mf]

I'd recommend GuavaComb before telecide.

[Kurosu]

GuavaComb is indeed a lot better than AntiBlink. It kicks in less quickly, but is cleaner, and is also able to filter horrible "dot crawling". But both are only suited for almost still scenes.

is there anything I can do to speed things up?
Yes there is. Look around where you found Antiblink :p

[punistation]

That would be HERE (http://forum.doom9.org/showthread.php?threadid=33319), which I read a whole bunch of times to figure out some optimal settings (Optimal = what everyone else uses), but sadly most of the tech-speak just flew by me. I'm humble enough to know when I'm out've my depth. :D

"Do a DCT/FFT on the 16x16 block and see if frequency values are more or less regularly distributed..."

Yup-yup. Quite a learning curve I've got in front of me.

*Holds breath. Waits for FixThisCrappyDVD(5) filter* :D



Kisses XXOOXX
Jen

[Zarxrax]

Deen works really well. In fact I just used it yesterday for cleanng rainbows.
Try something like Deen("a2d",5,5,30)
Now the 30 is likely a little overkill, so I'd suggest tweaking that value. The higher it is, the better it will remove rainbows, but it also makes the color less vibrant/accurate.

[mf]

30 isn't overkill, I've done 50. :D And 20's my default :p.

Originally posted by punistation
"Do a DCT/FFT on the 16x16 block and see if frequency values are more or less regularly distributed..."
Yup-yup. Quite a learning curve I've got in front of me.
DCT (Discrete Cosine Transform, Discrete means it's digital) is a mathematical transformation that transforms spatial (image) information into frequencies (like sounds). Space is saved by cutting out frequencies, however doing that on the entire image would look very bad, so the image is cut into small pieces (8x8, 16x16 pixels). That is the "16x16 block" your quote is about, and is also the reason why JPEG/MPEG contains block artifacts. As frequencies are being taken out, the image inside the blocks change, and one block may change in a different way than the other, causing a difference between blocks, which causes the edges to become visible. :D

[Kurosu]

Actually cutting particular frequencies in the whole transformed image is the way to go. Those rainbows are a case of signal aliasing: the way Y and C (chroma = a sort of vector made of U and V) analogical signals are modulated make them sometimes hardly separable. When you apply a cheap filter to separate them, you end up with a lot of Y signal into C signal (rainbow) and C signal into Y one (dot crawling). As you have probably noticed, that happens where high frequencies are located in the image (edges in the Y or U/V planes), and affects higher frequencies of the signals.

That's the reason why Lindsey is pushing for a filter that works in the frequency domain. The suggestion I made was based on the material I had (where surprisingly it was limited to mpeg block - maybe a cause of higher quant steps in that part of the frame). How to effectively build that filter, and doing the transformation are extremely costly.

[MasterYoshidino]

currently the main downfall of using ssiq is the desaturation it causes on the IQ blurred parts.

if you try and resaturate, those are partially repaired but the rest of the picture is over saturated.

[BBQmyNUTZ]

is there anything I can do to speed things up?

Yes there is. Look around where you found Antiblink

@Kurosu - What did you mean by this? Is there a faster version of SmartsmootherIQ or a native avisynth filter based on it that will produce the same results? I took a look on your page but didn't see anything to that effect.

Kai