Strange result on FastLineDarken()

[zlab]

Hi,
I just testing use of FastLineDarken.AVS with following script:

Mpeg2source(DVD_Project, cpu=6, info=3)
ColorMatrix(hints=true, interlaced=true)

Crop(30,16,-30,-16)
TFM(d2v=DVD_Project).TDecimate().UnDot()

FFT3dFilter(sigma=2.0, bt=3, plane=4, bh=48, bw=48, degrid=1)

ConvertToYV12().FastLineDarken()
DeHalo_alpha(rx=2.25)
Deen()

It give greendish color, any idea?
comment out the "FastLineDarken()" just fine.
or any FastLineDarken.AVS replacement?

[Lil' Jer]

Your cropping values are causing FastLineDarken to do that.

[zlab]

Hi,
The Crop hv been remove and the result screen still greendish, The crop was suppose crop at multiple of 16 should not affect the result, also the ConvertToYV12 is not needed since the output of Mpeg2source() is already in YV12 but I just want to make sure that.

Mpeg2source(DVD_Project, cpu=6, info=3)
ColorMatrix(hints=true, interlaced=true)

#Crop(30,16,-30,-16)
TFM(d2v=DVD_Project).TDecimate().UnDot()

FFT3dFilter(sigma=2.0, bt=3, plane=4, bh=48, bw=48, degrid=1)

ConvertToYV12().FastLineDarken()
DeHalo_alpha(rx=2.25)
Deen()

[Lil' Jer]

Hi,
The Crop hv been remove and the result screen still greendish, The crop was suppose crop at multiple of 16 should not affect the result

But you weren't cropping to a mod16 resolution so would have been causing your problems.

also the ConvertToYV12 is not needed since the output of Mpeg2source() is already in YV12 but I just want to make sure that.

Mpeg2source(DVD_Project, cpu=6, info=3)
ColorMatrix(hints=true, interlaced=true)

#Crop(30,16,-30,-16)
TFM(d2v=DVD_Project).TDecimate().UnDot()

FFT3dFilter(sigma=2.0, bt=3, plane=4, bh=48, bw=48, degrid=1)

ConvertToYV12().FastLineDarken()
DeHalo_alpha(rx=2.25)
Deen()

If the cropping isn't causing the problems then I don't know. I loaded a number of different DVDs with this exact same script and can't reproduce any greenish tint. The only version that gave greenish colors was the original with the crop left in.

[foxyshadis]

Can you post fastlinedarken btw? I can probably fix it so that you can crop to your heart's content beforehand, it's probably some minor thing unfinished.

[Shocku]

Do you get green just with this script or every time you try to use FLD? If it always happens it could be a problem with the version of masktools you are using.

[Dreassica]

I think it's indeed the wrong version Masktools that u are using.

[zlab]

Thanks for all yr kindly info.

Following is the FastLineDraken.AVS i use:

######################
# FastLineDarken 1.3 #
######################
#
# Written by Vectrangle, last update 12 Sept 04
#
# * requires masktools 1.5.1 -- http://jourdan.madism.org/~manao/
# * requires yv12 input
#
# Usage is FastLineDarken(strength, luma_cap, threshold, thinning),
# named parameters are supported eg FastLineDarken(thinning=0)
#
# Note that you must import this avs into your script using import("...\FastLineDarken 1.3.avs")
#
# Parameters are:
# strength (integer) - Line darkening amount, 0-256. Default 48. Represents the _maximum_ amount
# that the luma will be reduced by, weaker lines will be reduced by
# proportionately less.
# luma_cap (integer) - value from 0 (black) to 255 (white), used to stop the darkening
# determination from being 'blinded' by bright pixels, and to stop grey
# lines on white backgrounds being darkened. Any pixels brighter than
# luma_cap are treated as only being as bright as luma_cap. Lowering
# luma_cap tends to reduce line darkening. 255 disables capping. Default 191.
# threshold (integer) - any pixels that were going to be darkened by an amount less than
# threshold will not be touched. setting this to 0 will disable it, setting
# it to 4 (default) is recommended, since often a lot of random pixels are
# marked for very slight darkening and a threshold of about 4 should fix
# them. Note if you set threshold too high, some lines will not be darkened
# thinning (integer) - optional line thinning amount, 0-256. Setting this to 0 will disable it,
# which is gives a _big_ speed increase. Note that thinning the lines will
# inherently darken the remaining pixels in each line a little. Default 24.
#
# Changelog:
# 1.3 - added ability to thin lines, now runs much slower unless thinning=0. Changed the defaults (again)
# 1.2 - huge speed increase using yv12lutxy =)
# - weird darkening issues gone (they were caused by yv12layer)
# - show option no longer available due to optimizations. Use subtract() instead
# 1.1 - added luma_cap option
# 1.0 - initial release
#


function FastLineDarken( clip c, int "strength", int "luma_cap", int "threshold", int "thinning") {
str = string(default(strength, 48) /128.)
lum = string(default(luma_cap, 191))
thr = string(default(threshold, 4))
thinning = default(thinning,24)
thn = string(thinning /16.)
exin=c.expand().inpand()
diff = yv12lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? 127 +",uexpr="x",vexpr="x")
linemask = yv12lut(diff.inpand(),"x 127 - "+thn+" * 255 +")\
.yv12convolution("1 1 1","1 1 1",y=3,u=0,v=0)
thick = yv12lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "\
+lum+" < y "+lum+" ? - 0 ? "+str+" * x +",uexpr="x",vexpr="x")
thin = yv12lutxy(c.expand(),diff,yexpr="x y 127 - "+str+" 1 + * +")
return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)
}

Hum...I though I was using masktools version 1.5.10 and if that was not the proper version would anyone kind enough to post a link for the correct version?
I also confirm the greendish color would happen on any script/any DVD combination.

[demi_alucard]

I had this problem before. The only version that worked for me was the edited version of fastlinedarken by cypher_soundz, http://forum.doom9.org/showpost.php?p=616362&postcount=19
the original one by Vectrangle found on post http://forum.doom9.org/showpost.php?p=542991&postcount=1 gives me the same problem you are having...

[zlab]

O... that's work great, thanks for yr kindly hints that the original one doesn't work (may be only on my 6 test: Intel, AMD, Single/Double Core, PIII, XP, P4, P4D, x2, Sempron, machines). U r my hero and save my day(s).

[foxyshadis]

lol, yeah, he totally forgot to deal with the chroma, which can work for a few frames and then poof, green. The same effect could be achieved by using:
mergechroma(fastlinedarken(),last)
I don't believe that copying chroma right through is fully the correct way to deal with it, but at least it works, eh?

[acrespo]

I thing there is a problem in all FastLineDarken scripts. The line

return (thinning == 0) ? thick : maskedmerge(thin,thick,linemask,y=3,u=2,v=2)

Would be:
return (thinning == 0) ? thick : maskedmerge(thick,thin,linemask,y=3,u=2,v=2)

But for me, the line thinning feature is not good and I always pass this parameter = 0 (and this maskedmerge will not execute).

[demi_alucard]

yeah, the same happens here... can someone make a masktools 2 version of fastlinedarken? I tried here, but I got some issues like the one acrespo has.

[foxyshadis]

I would say you'd be better off going with vmtoon, with no supersampling and no sharpening (unless you want sharpening, of course). FastLineDarken is basically an older iteration; the update was combined with mfToon to create vmToon, which is masktools2. (Keep the supersampling to 1.5-2 in any case, the default of 4 is overkill.)

[acrespo]

I try vmToon included in masktools 2.0a28 and I see the same problem. The thinning parameter is not function. If I change the line:

(thinning == 0) ? thick : mt_merge(thin, thick, linemask, Y=3, U=1, V=1)

to:
(thinning == 0) ? thick : mt_merge(thick,thin, linemask, Y=3, U=1, V=1)
everthing the thinning parameter is function OK again, but, as I said I don't like this parameter.

Other thing is: ssw and ssh parameters don't accept 1.5 as value. Only integer values are accept. If I change this parameters to float I receive a error in line 71 ("Invalid arguments to function LanczosResize") and I don't know why.

[foxyshadis]

Oh, I've been working with a modified version for a while, having fixed those errors, sorry. ^^;;

fixed ver. (http://foxyshadis.slightlydark.com/random/vmToon.avsi)