How to make a mask only on the black lines?

[Vitaliy Gorbatenko]

If possible - how to do it? All that I try or too sensitive and mask not only the dark lines and color transitions, or not enough sensitive and did not see the black lines on dark colors.

The best was:
mt_edge(mode="cartoon", thY1=8,thY2=8,thC1=8,thC2=8)
But it mask color transitions too(blue to yellow 3.jpg)

:confused:

[Sagekilla]

Add U=3, V=3 to that mt_edge line and I think it should be fine.

[Vitaliy Gorbatenko]

mt_edge(mode="cartoon", thY1=8,thY2=8,thC1=8,thC2=8, U=3, V=3)
Thank you, but this did not help at all T_T

[Comatose]

AFAIK FastLineDarkenMod attempts to mask only black lines, so take a look at it and see if you can steal that part of the script.

[Vitaliy Gorbatenko]

I have looked in his direction, but the principle of FastLineMask other. Mask on the line in the right form, it is not built.

function GreyCenteredToMask(clip input) {
input.Levels(128, 1, 255, 0, 255, false)
one = last
input.Levels(0, 1, 128, 255, 0, false)
two = last
Overlay(one, two, mode="lighten")
}

function Camembert(clip input) {

input
SmoothUV(3,200,false)
Overlay(last, Blur(1).Blur(1), mode="difference")
Greycenteredtomask().greyscale()
Levels(0, 1, 115, 0, 255, false)
mask1 = last
Blur(1)
Levels(0, 0.4, 64, 0, 255, false)
mask = last
Blur(1).Blur(1)
Levels(0, 1, 30, 0, 255, false).Blur(1)
mask2 = last
#MaskedMerge(input, input.Unfilter(-20, -20), mask)
#MaskedMerge(last, input.Deen("a3d", 4, 15, 15, 20), mask)
#Overlay(last, input, mask=mask1, mode="darken", opacity=0.5)
#deblock ? MaskedMerge(Unsharpmask(100, 2, 0).SluttyBlindPP(quant=15, cpu2="xoxoxo"), last, mask2) \
# : MaskedMerge(Unsharpmask(50, 2, 0), last, mask2)
#cleanup ? Deen("a3d", 4, 3, 5, 10) : last
#MaskedMerge(Unsharpmask(50, 2, 0), last, mask2)
return (mask2)


I need something like the mask as the attached picture, but that would mask the color transitions not covered - the moon should not be masked. This mask comes with a script Camembert.

[TheRyuu]

#Typical sobel mask:
#LoadPlugin("mt_masktools.dll")
mt_edge("sobel",7,7,5,5).mt_inflate()

#alternative one, not using sobel:
mt_edge("roberts",0,4,0,4).mt_inflate()

Both taken from AA script(s). Their purpose is to only mask the lines so they'll work for your purpose.

Another alternative way of making a mask, sometimes a better mask, I believe it's taken from LSF or something like that idk.
But in reality, when used in an AA script, the final result looks identical to a regular sobel mask (above) in nearly all cases.

logic(a.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=4,Y=3,U=3,V=3)
\ ,a.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=4,Y=3,U=3,V=3)
\ ,"max").greyscale().levels(0,0.8,128,0,255,false)

[Vitaliy Gorbatenko]

They are all loosely related to color.
1.jpg - mt_edge ( "sobel", 7,7,5,5). Mt_inflate ()
2.jpg - mt_edge ( "roberts", 0,4,0,4). Mt_inflate ()
3.jpg - logic (clip.DEdgeMask (0,255 ...
The stronger the inverted color - the stronger will be processed.
Maybe I want too much?? :confused:

[FuPP]

Here is what I get using HybridFupp's algo (edges appear in red) : seems to be less sensitive to color transitions...

FuPP

[Vitaliy Gorbatenko]

Thank you for your advice. I have tested this mask too, but it has a serious flaw - a small sensitivity. The increase in sensitivity will lead to the capture of colors, too.
mask is very simple ..
mt_makediff(i.blur(1.58).blur(1.58), i).mt_binarize(threshold = trh).mt_inflate().mt_inflate()

[Sagekilla]

You could try thresholding it against the luminance levels. You can say that, if luma < 30 then it's a black line: mt_lutxy(source, mask, "x 128 - abs "+string(thr)+" < y 0 ?")

That's quite a bit slower than just doing a simple edge detect mask, but it could give you the results you need.

@Vitaliy: Please upload your images to somewhere like imageshack. That way we can see them immediately without having to wait for a moderator to approve the attachments. I know neuron2 is usually very good with this, but it helps if we can see it right away instead of having any delay.

[FuPP]

you can try something like this (same idea than Sagekilla).

Tested (on... 1 frame :)) with e_th1 = 131, e_th2 = 135, e_th3 = 62


Function EM(clip i, int e_th1, int e_th2, int e_th3)
{
diff = mt_makediff(i.blur(1.58).blur(1.58),i)
a=diff.mt_binarize(threshold = e_th1).mt_inflate().mt_inflate()
b=diff.mt_binarize(threshold = e_th2).mt_inflate().mt_inflate()
c=i.mt_binarize(e_th3, upper=true)

return mt_logic(b,mt_logic(a,c,"and"),"or")
}

FuPP

[Vitaliy Gorbatenko]

Sagekilla: I am not much concerned about the speed of processing - the most important quality.

I do not know why, can severely distort the color jpg .. But the moon surrounded by little red. Inversion of the mask is well shown. Look at 60783903.jpg.
This code:
mt_lutxy(c, mask, "x 128 - abs "+string(threshold)+" < y 0 ?")
greatly reduces the sensitivity and do not protect against color transitions.


function LineMask( clip c, clip mask, int "threshold")
{
threshold = Default(threshold, 30)
linemask = mt_lutxy(c, mask, "x 128 - abs "+string(threshold)+" < y 0 ?")
return(linemask)
}


Function EM_FuPP(clip i, int "e_th1", int "e_th2", int "e_th3")
{
e_th1 = Default(e_th1, 131)
e_th2 = Default(e_th2, 135)
e_th3 = Default(e_th3, 62)

diff = mt_makediff(i.blur(1.58).blur(1.58),i)
a=diff.mt_binarize(threshold = e_th1).mt_inflate().mt_inflate()
b=diff.mt_binarize(threshold = e_th2).mt_inflate().mt_inflate()
c=i.mt_binarize(e_th3, upper=true)

return mt_logic(b,mt_logic(a,c,"and"),"or")
}

Function DR_Radius(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : DR_Radius(i.mt_expand(), dr_rad, count+1)
}

Function DeHaloH(clip input, int "DR_Radius", int "DR_Str", bool "Maska")
{
Maska = Default(Maska, false)
EM=input.EM_FuPP()
# EM=LineMask(input, EM)
RM=DR_Radius(EM,DR_Radius,0)
DeRinging = input.Deen("a2d", 2, DR_Str , 0, 0, 0, 0, 0)
DeRinging = Maska ? DeRinging.Invert() : DeRinging
return mt_Merge(input, DeRinging, RM)
}

Call:
DeHaloH(2, 15, Maska=true)

[TheRyuu]

I'm not sure if this was said earlier, but what are you going to do with this mask?

[Vitaliy Gorbatenko]

I want to clear the picture only in black lines. Distortion (wave effect) are visible only around them. And I do not want to touch the rest of the picture. To save the basic details.

Therefore, the logic of using mt_binarize not suitable. Too heavy processing removes detail.

[Vitaliy Gorbatenko]

It seems there is no proper solution to this problem. I stopped at this version of the script. (attached file) The script cleans well distortion (wave effect) on a light background, and not touch the dark parts (leaves the illusion of detail).
The best result for the use Tsukuyomi, I received the following call:

import ( "DeHaloH.avsi")

DeHaloH (2, 10, Maska = false)
LSFmod (ss_x = 1.5, ss_y = 1.5, smode = 3, strength = 125, overshoot = 1, undershoot = 1)
FastLineDarkenMOD (thinning = 0)
DeHaloH (2, 10, Maska = false)

Yes, the call made on two occasions. Perhaps there is a better option, but I do not know.

I used parts of the script: by mf - Camembert-v0.1.avs, HQDering-v0.1.avs; by FuPP - HybridFuPP_0.984a.avs.

[real.finder]

hi, I make some modifications to the DeHaloH

1 - Make mask works on black line as much as possible
and apply mask once more in the opposite way to keep the lines as they are ,and applied it only around them

2 - TBilateral is used instead of deen, DeHaloH function stay as it, and new one with these change is DeHaloHmod

3 - DR_Radius and other setting changed in name, and Radius now 4 which roughly equivalent 2 in old DeHaloH

may there are others, but I don't remember them now

-------

edit: for 22 10 2015 you can use it with an external dehalo filter like this DeHaloHmod(exdehalo="BlindDeHalo3(4,4)")

download (those old versions, go to this post (https://forum.doom9.org/showthread.php?p=1883288#post1883288) for the last version)

DeHaloHmod 30 10 2014 http://pastebin.com/GhVpsids

and old one in Attachments

DeHaloHmod 22 10 2015 http://pastebin.com/D6M00iYG and in Attachments

DeHaloHmod 09 05 2016 http://pastebin.com/KkuiDtiK and in Attachments

DeHaloHmod 22 06 2016 http://pastebin.com/g3mjezVA

DeHaloHmod 07 10 2016 http://pastebin.com/4xv7u9te

DeHaloHmod 11 11 2016 http://pastebin.com/tA7aHtYP

DeHaloHmod 30 01 2017 (2.3) http://pastebin.com/bMQuP8v8 and in Attachments

DeHaloHmod 31 03 2017 (2.35) https://pastebin.com/gAFwi5kN and in Attachments

[luquinhas0021]

I'm sorry if I didn't see, but you've tried to convert the original image into black and white, before you apply edge mask? If not, try it and post the results, please. Is edge mask the same of edge detection?

[real.finder]

new ver.

added extlmask and RadInflate

DeHaloHmod 2.37 https://pastebin.com/uSmZqapf and in Attachments

some edit in mask

DeHaloHmod 2.39 https://pastebin.com/nQr7trcn

DeHaloHmod 2.40 https://pastebin.com/pCXGRNGh and in Attachments

DeHaloHmod 2.41 https://pastebin.com/03mpGr70 and in Attachments

DeHaloHmod 2.43 https://pastebin.com/whb85Vyi and in Attachments

[gaak]

@real.finder

Could you please tell me what the options are for mode? I can't find them documented anywhere.

Thanks.

[real.finder]

@real.finder

Could you please tell me what the options are for mode? I can't find them documented anywhere.

Thanks.

sorry for late, cuz I didn't note your post until now

but it's hard to explained mode options as I am not good in English

anyway mode for masking method, mode 0 mean it will use either Camembert_dhh() or Camembert_dhhmod() depending on strong parameter

from mode 1 and up see EMask_dhh Function