FastLineDarken issues

[Chainmax]

I am trying to encode a pretty messy clip, and the filtering has blurred some black lines too much, so I am trying to use FLD in order to counter that. When I load the function and step through VDubMod I see that everything is green-colored now. Any ideas as to why this happens?

[Didée]

Changed behaviour of the U|V = 1|2 (but not "3") paramters in the latest MaskTools.

"Older" scripts have to be adjusted accordingly. It's fun :)

Until the script is updated, perhaps you could simply copy chroma back from the clip before FLD was applied, like

x=last
FLD()
mergechroma(x)

?


edit:


Hm, looks like Vectrangle isn't around so much anymore - made his latest post two months ago.

Looking at the script, I'd say it's really the best to merge chroma back at the very end. That is only one [two] plane copying operation, instead of doing it again and again in each of the LUT operations.

Is that script OK after all? I'm a little intrigued by all those "hard" inpand's & expand's he is using - and reading things like

> yv12convolution("1 1 1","1 1 1",y=3,u=0,v=0) <

makes me wonder, too ... although I haven't tried the script yet, admitted.

[Chainmax]

Originally posted by Didée
Changed behaviour of the U|V = 1|2 (but not "3") paramters in the latest MaskTools.

Switching back to MaskTools v1.5.1 would solve the problem then?

[Manao]

I'd rather say : "Changed undocumented behavior in MaskTools". Defaults are y = 3, u = 1, and v = 1 and the doc states clearly that u = 1 and v = 1 leads to unpredictable results, which may or may not be those wanted.

On fastlinedarken, the modification is really easy : each times yv12lut and yv12lutxy are called, add u = 2, v = 2 as additionnal parameters.

[Chainmax]

You're right, those modifications did the trick. Thanks :).

[TNM]

Sorry to dig up this post but can any1 pls show me in more detail how to apply modifications to fix the all-green-colored problem? Thks...

[Manao]

function FastLineDarken( clip c, int "strength", int "luma_cap", int "threshold", int "thinning") {
str = string(default(strength, 48) /128.)
lum = string(default(luma_cap, 191))
thr = string(default(threshold, 4))
thinning = default(thinning,24)
thn = string(thinning /16.)
exin=c.expand().inpand()
diff = yv12lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? 127 +")
linemask = yv12lut(diff.inpand(),"x 127 - "+thn+" * 255 +")\
.yv12convolution("1 1 1","1 1 1",y=3)
thick = yv12lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +", u = 2, v = 2)
thin = yv12lutxy(c.expand(),diff,yexpr="x y 127 - "+str+" 1 + * +")
return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)
}

[Chainmax]

Originally posted by Manao
diff = yv12lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? 127 +")
linemask = yv12lut(diff.inpand(),"x 127 - "+thn+" * 255 +")\
.yv12convolution("1 1 1","1 1 1",y=3)
thick = yv12lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +", u = 2, v = 2)
thin = yv12lutxy(c.expand(),diff,yexpr="x y 127 - "+str+" 1 + * +")
return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)
}

You forgot those.

[Manao]

No, I didn't : it rather tricky, but : the final maskedmerge is u = v = 2, which means the chroma of the second clip is used, and the chroma of the first clip is useless. The second clip is thick. Its chroma is built only with the line I modified. hence there's no need for other modifications.