How to make a mask only on the black lines?

[Vitaliy Gorbatenko]

If possible - how to do it? All that I try or too sensitive and mask not only the dark lines and color transitions, or not enough sensitive and did not see the black lines on dark colors.

The best was:
mt_edge(mode="cartoon", thY1=8,thY2=8,thC1=8,thC2=8)
But it mask color transitions too(blue to yellow 3.jpg)

:confused:

[Sagekilla]

Add U=3, V=3 to that mt_edge line and I think it should be fine.

[Vitaliy Gorbatenko]

mt_edge(mode="cartoon", thY1=8,thY2=8,thC1=8,thC2=8, U=3, V=3)
Thank you, but this did not help at all T_T

[Comatose]

AFAIK FastLineDarkenMod attempts to mask only black lines, so take a look at it and see if you can steal that part of the script.

[Vitaliy Gorbatenko]

I have looked in his direction, but the principle of FastLineMask other. Mask on the line in the right form, it is not built.

function GreyCenteredToMask(clip input) {
input.Levels(128, 1, 255, 0, 255, false)
one = last
input.Levels(0, 1, 128, 255, 0, false)
two = last
Overlay(one, two, mode="lighten")
}

function Camembert(clip input) {

input
SmoothUV(3,200,false)
Overlay(last, Blur(1).Blur(1), mode="difference")
Greycenteredtomask().greyscale()
Levels(0, 1, 115, 0, 255, false)
mask1 = last
Blur(1)
Levels(0, 0.4, 64, 0, 255, false)
mask = last
Blur(1).Blur(1)
Levels(0, 1, 30, 0, 255, false).Blur(1)
mask2 = last
#MaskedMerge(input, input.Unfilter(-20, -20), mask)
#MaskedMerge(last, input.Deen("a3d", 4, 15, 15, 20), mask)
#Overlay(last, input, mask=mask1, mode="darken", opacity=0.5)
#deblock ? MaskedMerge(Unsharpmask(100, 2, 0).SluttyBlindPP(quant=15, cpu2="xoxoxo"), last, mask2) \
# : MaskedMerge(Unsharpmask(50, 2, 0), last, mask2)
#cleanup ? Deen("a3d", 4, 3, 5, 10) : last
#MaskedMerge(Unsharpmask(50, 2, 0), last, mask2)
return (mask2)


I need something like the mask as the attached picture, but that would mask the color transitions not covered - the moon should not be masked. This mask comes with a script Camembert.

[TheRyuu]

#Typical sobel mask:
#LoadPlugin("mt_masktools.dll")
mt_edge("sobel",7,7,5,5).mt_inflate()

#alternative one, not using sobel:
mt_edge("roberts",0,4,0,4).mt_inflate()

Both taken from AA script(s). Their purpose is to only mask the lines so they'll work for your purpose.

Another alternative way of making a mask, sometimes a better mask, I believe it's taken from LSF or something like that idk.
But in reality, when used in an AA script, the final result looks identical to a regular sobel mask (above) in nearly all cases.

logic(a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale().levels(0,0.8,128,0,255,false)

[Vitaliy Gorbatenko]

They are all loosely related to color.
1.jpg - mt_edge ( "sobel", 7,7,5,5). Mt_inflate ()
2.jpg - mt_edge ( "roberts", 0,4,0,4). Mt_inflate ()
3.jpg - logic (clip.DEdgeMask (0,255 ...
The stronger the inverted color - the stronger will be processed.
Maybe I want too much?? :confused:

[FuPP]

Here is what I get using HybridFupp's algo (edges appear in red) : seems to be less sensitive to color transitions...

FuPP

[Vitaliy Gorbatenko]

Thank you for your advice. I have tested this mask too, but it has a serious flaw - a small sensitivity. The increase in sensitivity will lead to the capture of colors, too.
mask is very simple ..
mt_makediff(i.blur(1.58).blur(1.58), i).mt_binarize(threshold = trh).mt_inflate().mt_inflate()

[Sagekilla]

You could try thresholding it against the luminance levels. You can say that, if luma < 30 then it's a black line: mt_lutxy(source, mask, "x 128 - abs "+string(thr)+" < y 0 ?")

That's quite a bit slower than just doing a simple edge detect mask, but it could give you the results you need.

@Vitaliy: Please upload your images to somewhere like imageshack. That way we can see them immediately without having to wait for a moderator to approve the attachments. I know neuron2 is usually very good with this, but it helps if we can see it right away instead of having any delay.

[FuPP]

you can try something like this (same idea than Sagekilla).

Tested (on... 1 frame :)) with e_th1 = 131, e_th2 = 135, e_th3 = 62


Function EM(clip i, int e_th1, int e_th2, int e_th3)
{
diff = mt_makediff(i.blur(1.58).blur(1.58),i)
a=diff.mt_binarize(threshold = e_th1).mt_inflate().mt_inflate()
b=diff.mt_binarize(threshold = e_th2).mt_inflate().mt_inflate()
c=i.mt_binarize(e_th3, upper=true)

return mt_logic(b,mt_logic(a,c,"and"),"or")
}

FuPP

[Vitaliy Gorbatenko]

Sagekilla: I am not much concerned about the speed of processing - the most important quality.

I do not know why, can severely distort the color jpg .. But the moon surrounded by little red. Inversion of the mask is well shown. Look at 60783903.jpg.
This code:
mt_lutxy(c, mask, "x 128 - abs "+string(threshold)+" < y 0 ?")
greatly reduces the sensitivity and do not protect against color transitions.


function LineMask( clip c, clip mask, int "threshold")
{
threshold = Default(threshold, 30)
linemask = mt_lutxy(c, mask, "x 128 - abs "+string(threshold)+" < y 0 ?")
return(linemask)
}


Function EM_FuPP(clip i, int "e_th1", int "e_th2", int "e_th3")
{
e_th1 = Default(e_th1, 131)
e_th2 = Default(e_th2, 135)
e_th3 = Default(e_th3, 62)

diff = mt_makediff(i.blur(1.58).blur(1.58),i)
a=diff.mt_binarize(threshold = e_th1).mt_inflate().mt_inflate()
b=diff.mt_binarize(threshold = e_th2).mt_inflate().mt_inflate()
c=i.mt_binarize(e_th3, upper=true)

return mt_logic(b,mt_logic(a,c,"and"),"or")
}

Function DR_Radius(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : DR_Radius(i.mt_expand(), dr_rad, count+1)
}

Function DeHaloH(clip input, int "DR_Radius", int "DR_Str", bool "Maska")
{
Maska = Default(Maska, false)
EM=input.EM_FuPP()
# EM=LineMask(input, EM)
RM=DR_Radius(EM,DR_Radius,0)
DeRinging = input.Deen("a2d", 2, DR_Str , 0, 0, 0, 0, 0)
DeRinging = Maska ? DeRinging.Invert() : DeRinging
return mt_Merge(input, DeRinging, RM)
}

Call:
DeHaloH(2, 15, Maska=true)

[TheRyuu]

I'm not sure if this was said earlier, but what are you going to do with this mask?

[Vitaliy Gorbatenko]

I want to clear the picture only in black lines. Distortion (wave effect) are visible only around them. And I do not want to touch the rest of the picture. To save the basic details.

Therefore, the logic of using mt_binarize not suitable. Too heavy processing removes detail.

[Vitaliy Gorbatenko]

It seems there is no proper solution to this problem. I stopped at this version of the script. (attached file) The script cleans well distortion (wave effect) on a light background, and not touch the dark parts (leaves the illusion of detail).
The best result for the use Tsukuyomi, I received the following call:

import ( "DeHaloH.avsi")

DeHaloH (2, 10, Maska = false)
LSFmod (ss_x = 1.5, ss_y = 1.5, smode = 3, strength = 125, overshoot = 1, undershoot = 1)
FastLineDarkenMOD (thinning = 0)
DeHaloH (2, 10, Maska = false)

Yes, the call made on two occasions. Perhaps there is a better option, but I do not know.

I used parts of the script: by mf - Camembert-v0.1.avs, HQDering-v0.1.avs; by FuPP - HybridFuPP_0.984a.avs.