Problem with DerainbowYUY2() Function

[Avisynth_challenged]

I recently inquired (http://forum.doom9.org/showthread.php?p=1114986#post1114986) about Sh0dan's DerainbowYUY2() function, asking if it should be used only on interlaced sources. Since no one answered that post, I decided to run some tests.

Having read the forum for some time now, I figured it would be best to start with the assumption that derainbow and decrawl filters should be applied prior to IVTC.

I wrote an Avisynth script which used the DerainbowYUY2() function (as well as Checkmate to filter out dot crawl), and ran it on a video clip (720x480, 29.97fps, interlaced, YUY2) just prior to IVTC.

This is the Avisynth script I used:

####################
### BEGIN SCRIPT ###
####################
LoadPlugin("C:\program files\avisynth 2.5\plugins\mipsmooth.dll")
LoadPlugin("C:\program files\avisynth 2.5\plugins\mt_masktools.dll")
LoadPlugin("C:\program files\avisynth 2.5\plugins\msharpen.dll")
LoadPlugin("C:\program files\avisynth 2.5\plugins\checkmate.dll")
LoadPlugin("C:\program files\avisynth 2.5\plugins\dfttest.dll")


############
function DeRainbow(clip org, int "thresh")
{
assert(org.isYV12(),"DeRainbow() requires YV12 input!")
thresh = default(thresh, 10)

org_u = utoy(org)
org_v = vtoy(org)

msharpen(org, threshold = thresh, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)
filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="strong", scalefactor=0.5)

u_final = mt_Merge(org_u, filtered_u, uv)
v_final = mt_Merge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}


function DeRainbowYUY2(clip org, int "thresh")
{
assert(org.isyuy2(),"DeRainbowYUY2() requires YUY2 input!")
thresh = default(thresh, 10)

org_yv12 = org.converttoyv12()
org_u = utoy(org).converttoyv12()
org_v = vtoy(org).converttoyv12()

msharpen(org_yv12, threshold = thresh, mask=true)
bilinearresize(last.width/2, last.height)
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=0.5)
filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=0.5)

u_final = mt_Merge(org_u, filtered_u, uv).converttoyuy2()
v_final = mt_Merge(org_v, filtered_v, uv).converttoyuy2()

return ytouv(u_final, v_final, org)
}
############


# decode with ffdshow
source=directshowsource("e:\preivtc.avi")
############


# AntiDotcrawl
out1=source.checkmate()
out2=source.checkmate(tthr2=0)
out3=out2.mergechroma(out1)

final=out3.DeRainbowYUY2()

return final
##################
### END SCRIPT ###
##################


I looked at the output (which was rendered with ffdshow's internal Huffyuv codec), and it looked great. Rainbows were eliminated, and dot crawl was greatly reduced.

Then I applied IVTC, and noticed that chroma bleeding was occuring wherever a IVTC'd progressive frame was recovered from a blended interlaced one. To help show what I mean, I've provided some screenshots.


Screenshot 1:
Blended video frame, prior to IVTC...
http://i252.photobucket.com/albums/hh18/FoolsTalk/blendpic.jpg

Screenshot 2:
Progressive frame after IVTC, but with chroma bleeding from the blended image...
http://i252.photobucket.com/albums/hh18/FoolsTalk/bleedpic.jpg


I redid the video, and this time I only applied the dot crawl filter (Checkmate), and commented out DeRainbowYUY2(). When I ran IVTC again, the chroma bleeding was gone.


So it seems reasonable to conclude that DeRainbowYUY2() should not be applied to interlaced video. I guess my options at this point are:


1. Take the interlaced source (pre-IVTC) and apply Checkmate, then separatefields(), then DeRainbowYUY2(), then weave(). Finally, apply IVTC.

2. Take the interlaced source (pre-IVTC) and apply Checkmate only, then apply IVTC, then apply DeRainbowYUY2().


Comments from the community would be greatly appreciated! Thanks :)

[TSchniede]

I usually did the first option seperatefields().derainbow().weave().

since my rainbows only appeared on strong vertical edges, I made my own script based on Sh0dan's.
I haven't tested it recently, so I can't say if there are better ways to smoothen the chroma, his script destroys sharp chroma edges. Perhaps vertical-spatial with temporal soften is better?


function hedgemask(clip cInput, int the1, int the2, int w, int n){
cx=cInput.mt_edge(mode = "-2 -1 3 -2 -1 3 -2 -1 3 "+string(n), thY1=0, thY2=255, thC1=255, thC2=255,u=1,v=1)
cy=cInput.mt_edge(mode = " 3 -1 -2 3 -1 -2 3 -1 -2 "+string(n), thY1=0, thY2=255, thC1=255, thC2=255,u=1,v=1)

mergeluma(cx,cy,0.5)
mt_lut(yexpr="x "+string(the1)+" < 0 x ?",u=1,v=1)
mt_lut(yexpr="x "+string(the2)+" > 255 x ?",u=1,v=1)

(w > 4) ? mt_expand(mode="horizontal",u=1,v=1) : last
(w > 3) ? mt_expand(mode="horizontal",u=1,v=1) : last
(w > 2) ? mt_expand(mode="horizontal",u=1,v=1) : last
(w > 1) ? mt_expand(mode="horizontal",u=1,v=1) : last
(w > 0) ? mt_expand(mode="horizontal",u=1,v=1) : last
return last
}

#show shows the edgemask
#th threshold for the mask, default 15 for old mask, 100 for new (minimum for edges to be found), strength of mask is edgedependent
#th2 threshold for full strength mask (new mask only)
#mr width of the edge region (we don't look for rainbows, but for strong vertical edges)
#norm divisor for edgededection, th=100&&norm=1 equals th=50&&norm=2 but edgemap strength is different
#interlaced process fields seperate?
#r scalefactor - stength of the blur (only where the mask == 255 visible)
#oldmask msharpen based edgemask or own edgemask
function DeRainbow2(clip org,bool "show", int "th", int "th2", int "mr", int "norm", bool "interlaced", float "r", bool "oldmask")
{
assert((!org.isRGB()),"DeRainbow() requires YUY2 or YV12 input!")
show=default(show,false)
oldmask=default(oldmask,false)
mr=default(mr,4)
norm=default(norm,1)
interlaced=default(interlaced,false)
r=default(r,0.5)
thn=default(th,100/norm)
th2n=default(th2,255/norm)

org = (interlaced) ? org.separatefields() : org

org_yv12 = isyv12(org) ? org : org.converttoyv12()
org_u = isyv12(org) ? utoy(org) : utoy(org).converttoyv12()
org_v = isyv12(org) ? vtoy(org) : vtoy(org).converttoyv12()

mask= (oldmask) ? msharpen(org_yv12, threshold = default(th, 15), mask=true).blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)
\ : hedgemask(org_yv12,thn,th2n,mr,norm) #edge detection

uv = isyv12(org) ? mask.reduceby2() : mask.HorizontalReduceBy2()

filtered_u = org_u.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=r)
filtered_v = org_v.mipsmooth(spatial=255, temporal=255, scenechange=3, show=false, method="superstrong", scalefactor=r)

u_final = isyv12(org) ? mt_Merge(org_u, filtered_u, uv, u=1, v=1) : mt_Merge(org_u, filtered_u, uv, u=1, v=1).converttoyuy2()
v_final = isyv12(org) ? mt_Merge(org_v, filtered_v, uv, u=1, v=1) : mt_Merge(org_v, filtered_v, uv, u=1, v=1).converttoyuy2()

show ? (isyuy2(org) ? mask.converttoyuy2() : mask).greyscale().merge(org,0.5) : ytouv(u_final, v_final, org)
return (interlaced) ? last.weave() : last
}
filterpath="filter\"
loadplugin(filterpath+"mt_masktools.dll")
loadplugin(filterpath+"MipSmooth.dll")
loadplugin(filterpath+"MSharpen.dll")