Looking for good rainbowing removal plugin

[ariga]

Is there any good filter to eliminate rainbowing effects in interlaced video ? I tried GuavaComb and CnR but neither seemed to make any difference to the video.

[Revgen]

Go here. (http://forum.doom9.org/showthread.php?t=102162)

[Video Dude]

Agreed. SmartSSIQ and Bifrost used in combination seem to remove any rainbow I throw at it. But it is slow ..... very slow.

[ariga]

Thanks for the tip. The source is a low quality music video with lots of motion. Only SSIQ 0.6 (avisynth plugin) seemed to do it !! Some residual rainbowing was cleaned up by FFT3dFilter (which I have started using recently on all captures.).

SSIQ(11,200,true)
LeakKernelDeint(order=0,sharp=true)
#FFT3dFilter(plane=0, measure=false)
FFT3dFilter(plane=1, sigma=3.0, measure=false)
FFT3dFilter(plane=2, sigma=3.0, measure=false)
#MipSmooth(preset="MovieLQ")

SSIQ worked better when it was run before deinterlacing. I'm trying to adapt SmartSSIQ to use SSIQ and eliminate the RGB conversion. (BiFrost or MipSmooth(preset="rainbow"), like other filters didn't make any noticable difference to the rainbowing.)

BTW, the FFT3dFilter example is picked up from one of the threads, yet to tweak the params. Frankly I don't know how to play with it. But I found that the first (plane 0) filter is not needed to eliminate rainbows.

Inspite of this there is still minor rainbowing left in some frames, but it's hardly noticable with so much of motion and scene changes. :)

Oh yes, each of these filters is slooooooow...

[Fizick]

http://www.avisynth.org/DeRainbow

[Chainmax]

ariga, SSIQ for Avisynth is a port from SSIQ for Virtualdub, which is what SmartSSIQ uses. There's no need to adapt anything. By the way, SmartSSIQ might not even be needed: read Bifrost's instructions and apply it correctly, it might be all you need (it was for me).

[jarthel]

smoothuv(7,100,false) seems to work great on anime. you may want to reduce 7(diameter) to something smaller to speed it up

[ariga]

Thanks for the replies. I'll try DeRainbow and smoothuv. (How could I overlook such an obvious name "DeRainbow" !)

Chainmax,
Could you post the link to SmartSSIQ ? The one I found here (http://forum.doom9.org/showthread.php?t=98267) seems to use VDub filter. (I assumed I could simply delete the RGB conversions in the SmartSSIQ script if I used the avisynth plugin, apparently I was wrong.)
BiFrost with the settings suggested in the readme (in combination with SSIQ), didn't help. So I switched to just SSIQ and it seemed to be very effective (but 3 fps on 720x576 :rolleyes: )

[Chainmax]

The SmartSSIQ thread is the one you linked to. You also need the VDub_Filters function and the VDub version of SSIQ (http://forum.doom9.org/showthread.php?t=100980&highlight=SmartSSIQ). Read the VDub_Filters function embedded readme in order to know how to use it.

About Bifrost, the crucial part is to know wether you have type1 or type2 rainbows, i.é if you apply Bifrost before or after IVTC. If you use it on wrong part it won't have much effect at all.

By the way, if you have a GeForce 6xxx 3D card, you migh want to take a look at FFT3DGPU, as it's supposed to be much faster and can smooth both chroma planes with a single call (unlike FFt3DFilter).

[foxyshadis]

Ngg, using virtualdub plugins is so 2002.

-> code removed, see later post

This is translated into Masktools 2.0 alpha, if you'd rather have 1.0 the translation is simple enough. I'm pretty sure it only takes YV12 input, but try with YUY2 if you want. Also I replaced the sharpen call with MSharpen, if you don't have it replace with your favorite sharpener.

I just translated it, I didn't try to improve it. Maybe it could be, but it did work very well (in concert with fft3d) at full strength on a show with utterly horrific rainbowing.

[Chainmax]

Well, LB decided to use the VDub version because he thought it performed better than the Avisynth one. Could you also post the version for MaskTools v1.5.x? By the way, what is a good way to remove color splotches in VHS captures like the red and yellow ones here:

http://img202.imageshack.us/img202/7497/splotches2zz.png (http://imageshack.us)

?

[Boulder]

Have you tried FFT3DFilter on the chroma planes with a high sigma? CNR2 might also be worth trying out.

[Chainmax]

I already tried FFT3DFilter on both chroma planes at sigma=3 and found that to be identical to Avisynth SSIQ+CNR2 (here's (http://forum.doom9.org/showthread.php?t=100001&page=2&pp=20) the thread in which it's mentioned). By the way, I seem to recall that CNR2 could cause some artifacting, does that happen with v2.6.1 too?

[foxyshadis]

Hey hey hey, where'd my post go. Grr, that must have been the tab I accidentally closed last night.

-> code removed, see later post

[Chainmax]

Got it. I wonder how well LimitedSharpen would work instead of MSharpen...
By the way, since this function is sing the Avisynth version of SSIQ, couldn't the default strength be set to 300 now?

[ariga]

smoothuv(7,100,false) is as effective as SSIQ(11,200,false) but smears colours badly and image looks darker.
smoothuv(2,100,false) is good for minor rainbowing. Fast enough to be used during playback ! In combination with FFT3Dfilter (planes 1&2) does a fairly good job.

Yet to try DeRainbow...

[Chainmax]

Boulder, foxyshadis: here's the result of FFt3DFilter at sigma 4 on both chroma planes plus CNR2("xxx",4,5,255) and some extra filtering

http://img392.imageshack.us/img392/3107/rzlt0js.png (http://imageshack.us)

As you can see, the red and yellow smudges aren't completely gone. ANy ideas on how to further remove them?

[Didée]

What's the temporal property of those splotches? Does a plain "Clense()" get them out?

[Chainmax]

You mean how they appear in succesive frames? I can upload a 5 seconds sample with this scene on it if you want. In the meantime, I'll try Clense (what does it need, Repair.dll or RemoveGrain.dll?).

[Fizick]

Chainmax,
Your clip sample has some chroma artefactes, but no rainbowing.
It is offtopic in this thread.

[Chainmax]

Actually, it does have slight rainbowing, look closely around the lines in his white collar. And isn't rainbowing a subset of chroma artifacting anyway?

[foxyshadis]

I'm not sure limitedsharpen would be any better, in fact, I'm not sure any sharpen at all is useful since the only purpose is to bring out the edges for better mask creation. I think better tuning of the mask parameters and possibly using yv12lut would be a better solution.

Oh, and the strength is already at 300, the "strength" parameter of the function is really just a blend-weight 0-255 between the original and derainbowed.

As for yours, SSIQ would never have an effect since it's just a specialized spacial smoother and yours are too big to be smoothed away; maaaaaaaybe you could try dup() with blend=true, which will blend all luma-identical frames, hopefully merging the back-and-forth "rainbow" crosstalks (if that is what they are) back into the original color, but it'd only really work if you had equal # of frames of each. And it's just an untested theory.

[ariga]

DeRainbow is truly a "RainbowKiller [tm]" AND it's quite fast. (Though I wish someone could make it work even faster :D ) Two thumbs up!!
Just one parameter to be tweak, so makes it easy to use. Default threshold is quite strong. However some light surfaces require lowering the threshold.
Thanks to Fizick for pointing it out and sh0dan for sharing it.

OT: Replacing MaskedMerge with mt_merge in DeRainbow made it completely useless. :confused:

[Chainmax]

I'm not sure limitedsharpen would be any better, in fact, I'm not sure any sharpen at all is useful since the only purpose is to bring out the edges for better mask creation. I think better tuning of the mask parameters and possibly using yv12lut would be a better solution.

Pretty please? :):D

[Chainmax]

foxyshadis?

[foxyshadis]

All right, I had a little time tonight (and found that the sharpen step is almost entirely superfluous), but I might need a clip or three to test on. Note, requires Masktools 2 and SSIQ.


function SmartSSIQ(clip input, int "strength") {
strength = default(strength, 200) #processing strength, 0-255

# apply rainbow remover to the entire video
derainbow = input.SSIQ(11, 300, false)

# create an edge mask from the rainbow video
lutexpr = "x .2 ^ 110 *"
# uncomment to use masktools 1
#edgemask = input.edgemask(thY1=1, thY2=255, type="sobel").inflate().yv12lut(yexpr=lutexpr).GreyScale()
edgemask = input.mt_edge(thY1=1, thY2=255, mode="sobel").mt_inflate().mt_lut(yexpr=lutexpr).GreyScale()

# apply ssiq only to masked area, using "strength"
result = overlay(input, derainbow, mode="blend", mask=edgemask, opacity=strength/255.0).Limiter()

return result
}
This is a 1:1 translation, but something tells me that the original levels was wrong, and the highlighted term should actually be higher. (Otherwise the full strength is virtually never used.) I'd say 100-110, with the default strength lowered to 200-215 to compensate, thus allowing heavier filtering. (I also think strength should be normalized to 1-100 but that's just me being picky.)

However, I need something to test with first. The clips I used to use are all fixed and removed now. ;_;

Edit: modified default strength (some time ago really)

[Chainmax]

What clips were you using for testing?

[Chainmax]

Was it one of the Trigun samples I posted some time ago?

[foxyshadis]

Oops, you posted right after me and I missed the thread update. ;_; Nope, it was a few things I had lying around, and since then a dvd and s'more, mostly comparing the masks and the output. I've made a couple of tweaks to see if I could improve the quality, but they really didn't make any noticable difference at all. (Except the strength adjustment.) I never did test it on anyone else's stuff, but I can.

[Chainmax]

Maybe you can talk to Didée and brainstorm to see if there's anything extra to be added to this excellent modification you made.