Derainbow.... only "exactly" where needed

[Lothar]

Looking for a reinbow removing solution i found the sh0dan's DeRainbow script:

function DeRainbow(clip org)
{
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}

This works GREAT in my tests in removing the rainbows, but there's quite a big loss in chroma: even in the areas not affected by rainbows the filtered colors are less bright and vivid.

So I tryed to limit the effects of this filter to to areas around edges only, using something like:

MaskedMerge(original, derainbow, edgemask, Y=1, U=3, V=3)

The result whit this combination is a little better in keeping the original chroma information, but it also keeps a little more of the original rainbows...

I'm looking for a way to completely overlay the edges (and the area around them) of the original picture with the ones filtered with derainbow, but (have i misunderstood something of "makedmerge"? ) something like "MaskedMerge(original, derainbow, edgemask)" simply returns the original picture...

Thanks.

[Didée]

Your 2nd example command "MaskedMerge(original, derainbow, edgemask)" has "no effect" because MaskedMerge's default for the U/V plane operators are U=V=1 (keep chroma from 1st clip).
Your 1st example with U=V=3 is the correct call to actually process chroma. But then, the edgemask probably wasn't how it should be ...
Best way would be to create the edgemask on the Y channel, and use that for the chroma planes:

MaskedMerge(original, derainbow, edgemask.FitY2UV(), U=3,V=3)

Depending on how you actually create the edgemask (there's a buch of possibilities, where a simple "EdgeMask()" isn't a too powerful one), you might want to firstly feather the mask out by one or more "expand.blur(1.58)" commands.

[Lothar]

Thx Didée, I've created the mask using your hint:

edgemask= Edgemask(type = "cartoon", thy1=6 , thy2=6, thc1=6, thc2=6).expand().blur(1.58).blur(1.58).FitY2UV()

And then

MaskedMerge(original, derainbow, edgemask, Y=1, U=3, V=3)

It works pretty good

The result image has about the same "deraibowing" as the simply derainbowed one, but the colors are much closer to the original.

This is so far the best result I've achieved in my tests on Cowboy Bebop.

Any idea to improve it or suggestions on other methods?

Thanks.

[Chainmax]

Well, you could also try loading the VDub filter SSIQ. By the way, what is the correct procedure to load VDub filters on Avisynth? The most I could find via the search was that an avs function had to be loaded, but I don't know where to find it...

In any case, I've seen the same problems that you experienced on other clips, Lothar. Could you post in here the final function once you're satisfied with the results?

[Lothar]

Well, I'm not fully satisfied, but here's my try:

function DeRainbow2(clip org, int "thresh")
{

#Based on DeRainbow by Sh0dan

assert(org.isYV12(),"DeRainbow2() requires YV12 input!")
thresh = default(thresh, 10)

org_u = utoy(org)
org_v = vtoy(org)

msharpen(org, threshold = thresh, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).levels(50,2.0,255,0,255, coring=false)

filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)

filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)


u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

derainbowed = ytouv(u_final, v_final, org)

edgem = org.Edgemask(type = "cartoon", thy1=6 , thy2=6, thc1=6, thc2=6).expand().blur(1.58).FitY2UV()

return MaskedMerge(org, derainbowed, edgem, Y=1, U=3, V=3)

}

@ Chainmax: SSIQ to remove rainbows?

[Nicholi]

@Chainmax

LoadVirtualDubPlugin("filename","filtername",preroll)

Filtername being the name you will use to call the filter later in the AVS.
Preroll determines how many frames to buffer before processing so the filter can corectly work. Filters of any temporal nature would need some frames prerolled.
The plugin though still requires RGB32 colorspace, so you will need to make the conversion for that before it's called.

As for rainbows, SSIQ nicely enough has been ported to avisynth, in yv12 and yuy2 no less. http://yatta.mellbin.org/misc/

However it's only a small portion of a much greater picture at removing rainbows. Bifrost works extremely well at removing most light-medium strength rainbows with little discoloration and bleeding. For the stronger types of rainbow, I see no solution other then accepting a small amount of color loss and bleeding using the usual tools.

Bifrost interestingly enough places rainbows in two groups: sources that apparently have had rainbows added after the Telecine process (type1) and those that have been rainbowed before (type2). Consequently it seems type1 rainbows are removed extremely efficiently, while type2's are the usual problematic ones.

[Mug Funky]

for rainbows in interlaced footage caused by composite cables (assume the rainbows are combed, and each field is the opposite end of the chroma plane), a simple thresholded blend-deinterlace on the chroma channels is enough to pretty much wipe them out without having to nuke the rest of the chroma.

subtle rainbowing caused by aliasing (you can get this even with component connections - it's more a matter of poor bandwidth filtering in the A/D conversion i think) is very difficult to remove, though. for this kind of thing we need to somehow teach the computer what a rainbow is as distinct from a colourful picture. this is beyond my expertise...

[Lothar]

Originally posted by Mug Funky
for rainbows in interlaced footage caused by composite cables (assume the rainbows are combed, and each field is the opposite end of the chroma plane), a simple thresholded blend-deinterlace on the chroma channels is enough to pretty much wipe them out without having to nuke the rest of the chroma.

Don't know if that's the case, but how would you do that?

Thx

[Mug Funky]

function ChubbyRain(clip c, int "th", int "radius", bool "show")
{
	
	th = default(th,10)
	radius = default(radius,3)
	show = default(show,false)
	
	u = c.utoy()
	v = c.vtoy()
	
	uc = u.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
	vc = v.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
	
	cc = c.yv12convolution(horizontal="1",vertical="1 2 1",Y=2,U=3,V=3,usemmx=true).temporalsoften(radius,0,255,2,2)
	
	rainbow=yv12lutxy(uc,vc,Yexpr=string("x y + "+string(th)+" > 256 0 ?")).pointresize(c.width,c.height).expand(y=3,u=-128,v=-128)
	
	overlay(c,cc,mask=rainbow)
	
	show==true? rainbow : last
	
}

that's the one i use for sources like Fawlty Towers, or Lain (PAL and NTSC respectively). it doesn't appear to touch the chroma all that much, though theoretically it will mess up moving brightly coloured objects (guy-in-red-jacket-walking-past-camera). i haven't seen this effect so far, and i consider it worth it.

one thing - apply this first, don't run anything before it - it needs raw, unadulterated interlacing to work properly. even telecide will throw it off, so run everything after it.

also, it'll only really work on stuff that is encoded as pure interlaced (be it film or video), and only works on crosstalk-type rainbows (the really strong combed ones on edges).

default settings should do.

[edit] oh, you need masktools for this to work. a version with the YV12LUTxy functions

[Lothar]

Thx Mug Funky, here's my new try:

Function ChubbyDeRain(clip c, int "th", int "radius", bool "show")
{
#based on Sh0dan's DeRainbow & Mug Funky's ChubbyRain

th = default(th,5)
radius = default(radius,3)
show = default(show,false)

u = c.utoy()
v = c.vtoy()

uc = u.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0) #,usemmx=true
vc = v.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0)

cc2 = c.yv12convolution(horizontal="1",vertical="1 2 1",Y=2,U=3,V=3).mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)

rainbow=yv12lutxy(uc,vc,Yexpr=string("x y + "+string(th)+" > 256 0 ?")).bicubicresize(c.width,c.height).expand(y=3,u=-128,v=-128)

ms = MaskedMerge(c,cc2,rainbow, Y=1, U=3, V=3)

show==true? rainbow : ms

}

Tested on progressive.

I don't understand fully how it works, but the result is really good to me

[Mug Funky]

cool. with the binary mask there, you might get some artefacts at edges in some cases, but apart from that it looks pretty good.

btw, with that mipsmooth in there, you no longer need the "radius" parameter in the function, so you can drop that out.

i'd really love a foolproof way to detect rainbows, but unfortunately once aliasing is introduced it becomes just more frequency content and can't be removed using traditional methods. maybe a neural-network could handle it, and after some training it might be pretty fast, too. but that's for a few years from now, or until someone who knows neural networks starts coding for avisynth...

[Chainmax]

Originally posted by Nicholi
@Chainmax

LoadVirtualDubPlugin("filename","filtername",preroll)

Filtername being the name you will use to call the filter later in the AVS.
Preroll determines how many frames to buffer before processing so the filter can corectly work. Filters of any temporal nature would need some frames prerolled.
The plugin though still requires RGB32 colorspace, so you will need to make the conversion for that before it's called.

As for rainbows, SSIQ nicely enough has been ported to avisynth, in yv12 and yuy2 no less. http://yatta.mellbin.org/misc/

I thought you also needed to load some kind of avs before using that line, is that not needed anymore or was it never needed at all? BTW, I know that SSIQ was ported to Avisynth, but I recall reading somewhere on the net that the Avisynth version wasn't as good as the VDub version, is that true?

[Lothar]

Here's my latest try:

function ChubbyRain2(clip c, int "th", int "radius", bool "show", int "sft")
{
#based on Mug Funky's ChubbyRain

th = default(th,10)
radius = default(radius,10)
show = default(show,false)
sft = default (sft, 10)

u = c.utoy()
v = c.vtoy()

uc = u.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0)
vc = v.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0)

cc = c.yv12convolution(horizontal="1",vertical="1 2 1",Y=2,U=3,V=3).bifrost(interlaced=false).cnr2().temporalsoften(radius,0,sft,2,2)

rainbow=yv12lutxy(uc,vc,Yexpr=string("x y + "+string(th)+" > 256 0 ?")).pointresize(c.width,c.height).expand(y=3,u=-128,v=-128)#.blur(1.5)

overlay(c,cc,mask=rainbow)

show==true? rainbow : last

}

In short: ChubbyRain works really good on removing rainbows, but the side effect is given by the use of temporalsoften high values of threshond on chroma, causing ghosts.
So i tryed a way to lower that values by reducing rainbows before applying temporalsoften.

This gives to me best results I've seen so far in my tests, so I hope it works (maybe by tweaking some parameters) on other sources.

Thx to Mug Funky and Sh0dan for the scripts I've used

[Avisynth_challenged]

Originally Posted by Mug Funky

function ChubbyRain(clip c, int "th", int "radius", bool "show")
{
	
	th = default(th,10)
	radius = default(radius,3)
	show = default(show,false)
	
	u = c.utoy()
	v = c.vtoy()
	
	uc = u.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
	vc = v.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
	
	cc = c.yv12convolution(horizontal="1",vertical="1 2 1",Y=2,U=3,V=3,usemmx=true).temporalsoften(radius,0,255,2,2)
	
	rainbow=yv12lutxy(uc,vc,Yexpr=string("x y + "+string(th)+" > 256 0 ?")).pointresize(c.width,c.height).expand(y=3,u=-128,v=-128)
	
	overlay(c,cc,mask=rainbow)
	
	show==true? rainbow : last
	
}

that's the one i use for sources like Fawlty Towers, or Lain (PAL and NTSC respectively). it doesn't appear to touch the chroma all that much, though theoretically it will mess up moving brightly coloured objects (guy-in-red-jacket-walking-past-camera). i haven't seen this effect so far, and i consider it worth it.

one thing - apply this first, don't run anything before it - it needs raw, unadulterated interlacing to work properly. even telecide will throw it off, so run everything after it.

also, it'll only really work on stuff that is encoded as pure interlaced (be it film or video), and only works on crosstalk-type rainbows (the really strong combed ones on edges).

default settings should do.

[edit] oh, you need masktools for this to work. a version with the YV12LUTxy functions

I'm experimenting with this function to attack rainbows in my (pre-IVTC'd) source. I'd like to tweak your script, because the default settings are not quite strong enough for certain instances of rainbow. But I'm not sure what to do. If you don't mind, could you tell us what to modify in the original script in order to achieve stronger derainbowing than the defaults provide? Thank you very much in advance!

[JKyle]

I slightly modded Lothar's ChubbyRain2 as follows:

# ChubbyRain2.avsi
#
# http://avisynth.nl/index.php/ChubbyRain2
#
# - based on Mug Funky's ChubbyRain
# - ChubbyRain2 by Lothar on Doom9's forum
# https://forum.doom9.org/showthread.php?p=589885#post589885
#
# mod by JKyle
# 2021-06-12
#
# --- CHANGES ---
#
# - replaced cnr2 with vsCnr2
#
# --- REQUIREMENTS ---
#
# Bifrost(https://github.com/Asd-g/AviSynth-bifrost)
# MaskTools2(https://github.com/pinterf/masktools)
# vsCnr2(https://github.com/Asd-g/AviSynth-vsCnr2)
#
# Supported input colorspace: Planar 8-bit (YV12, YV16, YV24)
# HBD is not supported yet.
# 


function ChubbyRain2(clip c, int "th", int "radius", bool "show", int "sft", bool "interlaced")
{

th          = Default(th, 10) # Masking threshold; lower values consider more pixels as rainbows.
radius      = Default(radius, 10) # radius for TemporalSoften.
show        = Default(show, false) # If true, rainbow mask will be shown. This can be helpful for adjusting th.
sft         = Default(sft, 10) # chroma_threshold for TemporalSoften.
interlaced  = Default(interlaced, false) # false means the input clip is progressive.

c = (interlaced == true) ? c.SeparateFields() : c

u = c.UToY()
v = c.VToY()

uc = u.mt_convolution(horizontal="1", vertical="1 -2 1", Y=3, U=0, V=0)
vc = v.mt_convolution(horizontal="1", vertical="1 -2 1", Y=3, U=0, V=0)

# replaced cnr2 with vsCnr2 (JKyle)
cc = c.mt_convolution(horizontal="1", vertical="1 2 1", Y=2, U=3, V=3).Bifrost(interlaced=false).vsCnr2().TemporalSoften(radius, 0, sft, 2, 2)

rainbow = mt_lutxy(uc, vc, yExpr=String("x y + "+String(th)+" > 256 0 ?")).PointResize(c.width, c.height).mt_expand(y=3, u=-128, v=-128)#.blur(1.5)

Overlay(c, cc, mask=rainbow)

output = (show == true) ? rainbow : (interlaced == true) ? last.Weave() : last

return output

}

[kedautinh12]

Asd-g was moded Chubbyrain2 and was supported HBD
https://github.com/Asd-g/AviSynthPlu...bbyRain2_.avsi

[JKyle]

Originally Posted by kedautinh12

Asd-g was moded Chubbyrain2 and was supported HBD
https://github.com/Asd-g/AviSynthPlu...bbyRain2_.avsi

for the info.

In the end, I didn't have to reinvent the wheel.

He solved the HBD problem in a very easy way. Lesson learned.

[GMJCZP]

To speed up, you can replace Overlay with:

Mt_merge(c, cc, rainbow)

Edit: asd-g already changed it.

[kedautinh12]

Originally Posted by JKyle

for the info.

In the end, I didn't have to reinvent the wheel.

He solved the HBD problem in a very easy way. Lesson learned.

More scripts was moded and backported from vapoursynth. You can check here:
https://github.com/Asd-g/AviSynthPlus-Scripts

[GMJCZP]

Collecting and ordering all the information, here the complete script:

 # ChubbyRain2.avsi updated by Asd-g
 #
 # http://avisynth.nl/index.php/ChubbyRain2
 #
 # - based on Mug Funky's ChubbyRain
 # - ChubbyRain2 by Lothar on Doom9's forum
 # https://forum.doom9.org/showthread.php?p=589885#post589885

 ### Requirements ###
 #------------------- 
 # Bifrost (https://github.com/Asd-g/AviSynth-bifrost)
 # MaskTools2 (https://github.com/pinterf/masktools)
 # vsCnr2 (https://github.com/Asd-g/AviSynth-vsCnr2)
 # Avsresize (http://avisynth.nl/index.php/Avsresize)
  
 ### Changelog ### 
 #--------------- 
 # Changed requirements: replaced Cnr2 with vsCnr2. 
 # Added support for 10..16-bit clips. 
   
  
 function ChubbyRain2(clip c, int "th", int "radius", bool "show", int "sft", bool "interlaced") 
 { 
 #based on Mug Funky's ChubbyRain 
  
     Assert(Is420(c), "ChubbyRain2: only clips with chroma subsampling 420 are supported.") 
  
     th = default(th,10) 
     radius = default(radius,10) 
     show = default(show,false) 
     sft = default (sft, 10) 
     interlaced = default(interlaced, false) 
      
     c = (interlaced) ? c.separatefields() : c 
      
     uc = mt_convolution(c, horizontal="1",vertical="1 -2 1",Y=1,U=3,V=3) 
      
     mt_convolution(c, horizontal="1",vertical="1 2 1",Y=2,U=3,V=3) 
     bifrost(interlaced=false) 
     vsCnr2() 
     cc = temporalsoften(radius,0,sft,2,2) 
      
     mt_lutxy(ExtractU(uc), ExtractV(uc), "x y + "+string(th)+" > 255 0 ?", scale_inputs="allf", use_expr=1) 
     z_pointresize(c.width,c.height) 
     rainbow = mt_expand(y=3,u=-128,v=-128)#.blur(1.5) 
      
     mt_merge(c, cc, rainbow, y=2, luma=true) 
      
     return (show) ? rainbow : (interlaced) ? weave() : last 
 }