Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th July 2017, 14:27   #1  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 509
Khameleon: bidirectional masking cleaner

For some time I have had frustrations trying to prevent certain denoisers from getting more details of an image, especially with animé/cartoon.
I came to use masks, but I noticed that even mt_edge(mode="sobel") did not fulfill for me what he wanted to achieve.
One day it came to my hands DeHaloH, and I loved the way it takes the edges of the images, thanks mainly to the Camembert function, and there was no turning back.

Recently I thought if you could not reverse the process, that is, only clean the inside of what delimits the edges and in this way simplify the annoying configuration
of cleaning and try to retain as much detail.
So I came up with a two-way (bidirectional) function, Khameleon, for its chameleon effect.

Code:
# Khameleon: bidirectional masking cleaner
# version 1.1, 16/02/2018
# Based in DeHaloH, thanks to the authors!
# Developed by GMJCZP
#
# Requirements: TCannyMod, RGTools, MaskTools2, TUnsharp

# SUGGESTIONS FOR CLEANING/RESTORING

# - Suggestions for restoring edges:
# denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0) (used by default by DeHaloH)
# denoised = input.frfun7(1.01,3.0,0).frfun3b(s=8,T=3,tuv=7).Deen("a2d", 2, 5  , 0, 0, 0, 0, 0)
# denoised = input.frfun7(1.01,14,0).fft3dfilter(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen("a2d", 2, 10  , 0, 0, 0, 0, 0)
# denoised = input.fft3dfilter(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen("a2d", 2, 15  , 0, 0, 0, 0, 0)

# - Suggestions for inner cleaning:
# denoised = input.Temporalsoften(2,3,3,mode=2,scenechange=6)
# denoised = input.dfttest(sigma=2, tbsize=1)

Function Khameleon(clip denoised, clip input, int "DR_Radius", bool "Maska", int "mode",
\ bool "edge", bool "focus", int "str_focus", float "gamma", float "cont", bool "sobel")
{
 edge       = Default(edge, true)
 mode       = Default(mode, edge ? 0 : 1)
 Maska      = Default(Maska, false)
 DR_Radius  = Default(DR_Radius, 0)
 focus      = Default(focus, false)
 str_focus  = Default(str_focus, 40)
 gamma      = Default(gamma, 1.0) # 2.0 for inner cleaning is interesting
 cont       = Default(cont, 1.0)
 sobel      = Default(sobel, false)

 Assert(!(edge == true && gamma <> 1.0), "Gamma values can only be varied for inner cleaning")
 Assert((mode == 0 || mode == 1), "Wrong value for mode")
 Assert(!(focus == true && edge == false), "Focus = true: only valid for edge=true")

 EM = mode == 0 ? input.Camembert2() : input.tweak(cont=cont).TCannyMod(sobel=sobel)

 input_low    = 1
 gamma        = gamma
 input_high   = 128
 output_low   = 255
 output_high  = 0

 lmask = input.mt_lut(Yexpr = "x " +string(input_low)+ " - " +string(input_high)+ " " +string(input_low)+ " - / 1 " +string(gamma)+
    \        " / ^ " +string(output_high)+ " " +string(output_low)+ " - * " +string(output_low)+ " +", U=2,V=2).invert().RemoveGrain(12)#.blur(0.5)

 RM = DR_Radius(EM,DR_Radius,0)
 full = mt_logic(RM, lmask, "and")
 full = edge ? full :  full.Invert().mt_deflate().mt_binarize(threshold=192)
 
 # For blurring edges
 denoised = (edge == true && focus == true) ? denoised.spline16Resize(int(width(input)*1.5),int(height(input)*1.5)).tunsharp(str_focus,type=5).spline16Resize(width(input),height(input)) 
    \ : denoised

 denoised = Maska ? denoised.Invert() : denoised
return mt_merge(input, denoised, full) 
}

Function DR_Radius(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : DR_Radius(i.mt_expand(), dr_rad, count+1)
}

function Camembert2(clip input)
{
input
a=RemoveGrain(12,-1).RemoveGrain(12,-1)
mt_makediff(a,input)
#input
#a=RemoveGrain(12).RemoveGrain(12)
#mt_adddiff(input,mt_makediff(a,input))
mt_lut("x 128 - abs 2 *")
greyscale()
mt_lut(Yexpr = "X 0 - 115 0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 64  0 - / 1 0.4 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 30  0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
return (last)
}
* Requirements

TCannyMod, RGTools, MaskTools2, TUnsharp


* Syntax and Parameters


- clip denoised

Clip to which the filtering is going to apply, for example:

denoised = Deen ("a2d", 2, 10, 0, 0, 0, 0) (used by default by DeHaloH)

denoised = dfttest (sigma = 2, tbsize = 1) (for inner cleaning)


- clip input

Original clip


- int "DR_Radius" = 0

Analogous to DeHaloH, it represents the radius or detection range.


- bool "Maska" = false

If true, the detection mask will be displayed.


- bool "edge" = true

Defines whether to clean the edge or the inside of the images.


- int "mode" = 0

The detection mode. By default (mode = 0) the Camembert function is used. Mode = 1 uses TcannyMod. I recommend mode = 0 for cleaning edges (edge =true) and mode = 1 for inner cleaning (edge = false),
but the decision is at the taste of each person.


Now let's look at other additional parameters:


a) For edge = "true"

- bool "focus" = false

If true, it will try to reverse the blur caused by edge cleaning.


- int "str_focus" = 40

Defines the focusing force.


b) For mode = 1

- float "gamma" = 1.00

By increasing this value, it can help improve edge detection to protect.


- float "cont" = 1.0

As with "gamma", increasing this value may improve edge detection.


- bool "sobel" = false

If true, it improves edge detection.


* Examples

For cleaning edges:

orig = last
denoised = Deen ("a2d", 2, 10, 0, 0, 0, 0)
Khameleon(denoised, orig, DR_Radius=0, edge=true, focus=true, str_focus=40)


For inner cleaning:

orig = last
denoised = dfttest(sigma=2, tbsize=1)
Khameleon(denoised, orig, DR_Radius=1, mode=1, edge=false, focus=false, gamma=2.0, cont=2.0, sobel=true)


* Tips for cleaning edges (for edge=true, use your favourite combo):

deen: is good for halos and mosquito noise.
frfun7: good for general cleaning and for light aliasing.
fft3dfilter: it's great for rebuilding borders and general cleaning.
frfun3b: for general cleaning.
dfttest: for general cleaning.
degrainMedian: may be good for random dots and mosquito noise.
__________________
By law and justice!

Flea Market

Last edited by GMJCZP; 17th February 2018 at 17:07. Reason: Changed to version 1.1
GMJCZP is offline   Reply With Quote
Old 27th July 2017, 15:05   #2  |  Link
SaurusX
Registered User
 
Join Date: Feb 2017
Posts: 81
Always looking for a good edge cleaner. I just discovered DeHaloHMod myself and was impressed at how well it removed mosquito noise and dark halos while preserving animation lines. I'll try out your solution as well.

edit: I should add that using DehaloHMod(smooth=true) inverts the mask and only cleans on the inside toward the lines. Similar to what you're trying to achieve.

Last edited by SaurusX; 27th July 2017 at 16:32.
SaurusX is offline   Reply With Quote
Old 16th February 2018, 02:57   #3  |  Link
Heaud
Registered User
 
Join Date: Apr 2008
Posts: 55
Thanks for creating this script! It has been helpful with smoothing out mpeg2 artifacts without destroying too much detail.

Can I make a suggestion to have mode set to 1 automatically when edge is set to false?
Heaud is offline   Reply With Quote
Old 16th February 2018, 11:35   #4  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 509
Khameleon 1.1:
- Now by default mode = 0 when edge = true, otherwise mode = 1
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 16th February 2018, 13:41   #5  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,473
Is there x64 build of TUnsharp?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 16th February 2018, 23:36   #6  |  Link
Heaud
Registered User
 
Join Date: Apr 2008
Posts: 55
There are spacing issues on the script returning syntax errors upon loading.
Heaud is offline   Reply With Quote
Old 17th February 2018, 04:18   #7  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 509
Quote:
Originally Posted by Heaud View Post
There are spacing issues on the script returning syntax errors upon loading.
You could be more specific, since I tried the script and I'm doing fine.

About Tunsharp x64 Groucho2004 only uploaded the x86 version.
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 17th February 2018, 04:32   #8  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 4,994
Quote:
Originally Posted by GMJCZP View Post
About Tunsharp x64 Groucho2004 only uploaded the x86 version.
I'll post a 64 bit version tomorrow.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 17th February 2018, 04:37   #9  |  Link
Heaud
Registered User
 
Join Date: Apr 2008
Posts: 55
Quote:
Originally Posted by GMJCZP View Post
You could be more specific, since I tried the script and I'm doing fine.
One example:
Code:
# denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0) (used by default by

DeHaloH)
The code box in the OP has lines broken similar to above. I assume it is meant to be like this:
Code:
# denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0) (used by default by DeHaloH)
Heaud is offline   Reply With Quote
Old 17th February 2018, 13:19   #10  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 509
I do not know if I understood you, but the example would use it like this:

Code:
denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0)
The rest are just comments, remember that the symbol # is present at the beginning of the line.
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 17th February 2018, 13:45   #11  |  Link
Heaud
Registered User
 
Join Date: Apr 2008
Posts: 55
Code in the OP is experiencing syntax error. I have highlighted the parts that are causing avisynth to fail.
Code:
# Khameleon: bidirectional masking cleaner
# Version 1.1, 16/02/2018
# Based in DeHaloH, thanks to the authors!
# Developed by GMJCZP
#
# Requirements: TCannyMod, RGTools, MaskTools2, TUnsharp

# SUGGESTIONS FOR CLEANING/RESTORING

# - Suggestions for restoring edges:
# denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0) (used by default by 

DeHaloH)
# denoised = input.frfun7(1.01,3.0,0).frfun3b(s=8,T=3,tuv=7).Deen("a2d", 2, 5  , 

0, 0, 0, 0, 0)
# denoised = input.frfun7(1.01,14,0).fft3dfilter

(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen("a2d", 2, 10  , 0, 0, 0, 0, 0)
# denoised = input.fft3dfilter(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen

("a2d", 2, 15  , 0, 0, 0, 0, 0)

# - Suggestions for inner cleaning:
# denoised = input.Temporalsoften(2,3,3,mode=2,scenechange=6)
# denoised = input.dfttest(sigma=2, tbsize=1)

Function Khameleon(clip denoised, clip input, int "DR_Radius", bool "Maska", int 

"mode",
\ bool "edge", bool "focus", int "str_focus", float "gamma", float "cont", bool 

"sobel")
{
 edge       = Default(edge, true)
 mode       = Default(mode, edge ? 0 : 1)
 Maska      = Default(Maska, false)
 DR_Radius  = Default(DR_Radius, 0)
 focus      = Default(focus, false)
 str_focus  = Default(str_focus, 40)
 gamma      = Default(gamma, 1.0) # 2.0 for inner cleaning is interesting
 cont       = Default(cont, 1.0)
 sobel      = Default(sobel, false)

 Assert(!(edge == true && gamma <> 1.0), "Gamma values can only be varied for 

inner cleaning")
 Assert((mode == 0 || mode == 1), "Wrong value for mode")
 Assert(!(focus == true && edge == false), "Focus = true: only valid for 

edge=true")

 EM = mode == 0 ? input.Camembert2() : input.tweak(cont=cont).TCannyMod

(sobel=sobel)

 input_low    = 1
 gamma        = gamma
 input_high   = 128
 output_low   = 255
 output_high  = 0

 lmask = input.mt_lut(Yexpr = "x " +string(input_low)+ " - " +string

(input_high)+ " " +string(input_low)+ " - / 1 " +string(gamma)+
    \        " / ^ " +string(output_high)+ " " +string(output_low)+ " - * " 

+string(output_low)+ " +", U=2,V=2).invert().RemoveGrain(12)#.blur(0.5)

 RM = DR_Radius(EM,DR_Radius,0)
 full = mt_logic(RM, lmask, "and")
 full = edge ? full :  full.Invert().mt_deflate().mt_binarize(threshold=192)
 
 # For blurring edges
 denoised = (edge == true && focus == true) ? denoised.spline16Resize(int(width

(input)*1.5),int(height(input)*1.5)).tunsharp(str_focus,type=5).spline16Resize

(width(input),height(input)) 
    \ : denoised

 denoised = Maska ? denoised.Invert() : denoised
return mt_merge(input, denoised, full) 
}

Function DR_Radius(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : DR_Radius(i.mt_expand(), dr_rad, count+1)
}

function Camembert2(clip input)
{
input
a=RemoveGrain(12,-1).RemoveGrain(12,-1)
mt_makediff(a,input)
#input
#a=RemoveGrain(12).RemoveGrain(12)
#mt_adddiff(input,mt_makediff(a,input))
mt_lut("x 128 - abs 2 *")
greyscale()
mt_lut(Yexpr = "X 0 - 115 0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 64  0 - / 1 0.4 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 30  0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
return (last)
}



Here is the corrected script:
Code:
# Khameleon: bidirectional masking cleaner
# Version 1.1, 16/02/2018
# Based in DeHaloH, thanks to the authors!
# Developed by GMJCZP
#
# Requirements: TCannyMod, RGTools, MaskTools2, TUnsharp

# SUGGESTIONS FOR CLEANING/RESTORING

# - Suggestions for restoring edges:
# denoised = input.Deen("a2d", 2, 10  , 0, 0, 0, 0, 0) (used by default by DeHaloH)
# denoised = input.frfun7(1.01,3.0,0).frfun3b(s=8,T=3,tuv=7).Deen("a2d", 2, 5  , 0, 0, 0, 0, 0)
# denoised = input.frfun7(1.01,14,0).fft3dfilter(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen("a2d", 2, 10  , 0, 0, 0, 0, 0)
# denoised = input.fft3dfilter(sigma=3,sigma2=3.25,sigma3=3.5,sigma4=6).Deen("a2d", 2, 15  , 0, 0, 0, 0, 0)

# - Suggestions for inner cleaning:
# denoised = input.Temporalsoften(2,3,3,mode=2,scenechange=6)
# denoised = input.dfttest(sigma=2, tbsize=1)

Function Khameleon(clip denoised, clip input, int "DR_Radius", bool "Maska", int "mode",
\ bool "edge", bool "focus", int "str_focus", float "gamma", float "cont", bool "sobel")
{
 edge       = Default(edge, true)
 mode       = Default(mode, edge ? 0 : 1)
 Maska      = Default(Maska, false)
 DR_Radius  = Default(DR_Radius, 0)
 focus      = Default(focus, false)
 str_focus  = Default(str_focus, 40)
 gamma      = Default(gamma, 1.0) # 2.0 for inner cleaning is interesting
 cont       = Default(cont, 1.0)
 sobel      = Default(sobel, false)

 Assert(!(edge == true && gamma <> 1.0), "Gamma values can only be varied for inner cleaning")
 Assert((mode == 0 || mode == 1), "Wrong value for mode")
 Assert(!(focus == true && edge == false), "Focus = true: only valid for edge=true")

 EM = mode == 0 ? input.Camembert2() : input.tweak(cont=cont).TCannyMod(sobel=sobel)

 input_low    = 1
 gamma        = gamma
 input_high   = 128
 output_low   = 255
 output_high  = 0

 lmask = input.mt_lut(Yexpr = "x " +string(input_low)+ " - " +string(input_high)+ " " +string(input_low)+ " - / 1 " +string(gamma)+
    \        " / ^ " +string(output_high)+ " " +string(output_low)+ " - * " +string(output_low)+ " +", U=2,V=2).invert().RemoveGrain(12)#.blur(0.5)

 RM = DR_Radius(EM,DR_Radius,0)
 full = mt_logic(RM, lmask, "and")
 full = edge ? full :  full.Invert().mt_deflate().mt_binarize(threshold=192)
 
 # For blurring edges
 denoised = (edge == true && focus == true) ? denoised.spline16Resize(int(width(input)*1.5),int(height(input)*1.5)).tunsharp(str_focus,type=5).spline16Resize(width(input),height(input)) 
    \ : denoised

 denoised = Maska ? denoised.Invert() : denoised
return mt_merge(input, denoised, full) 
}

Function DR_Radius(clip i, int dr_rad, int count)
{
return count > dr_rad ? i : DR_Radius(i.mt_expand(), dr_rad, count+1)
}

function Camembert2(clip input)
{
input
a=RemoveGrain(12,-1).RemoveGrain(12,-1)
mt_makediff(a,input)
#input
#a=RemoveGrain(12).RemoveGrain(12)
#mt_adddiff(input,mt_makediff(a,input))
mt_lut("x 128 - abs 2 *")
greyscale()
mt_lut(Yexpr = "X 0 - 115 0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 64  0 - / 1 0.4 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
RemoveGrain(12)
mt_lut(Yexpr = "X 0 - 30  0 - / 1 1.0 / ^ 255 0 - * 0 +", U=2,V=2)
RemoveGrain(12)
return (last)
}
Heaud is offline   Reply With Quote
Old 17th February 2018, 17:10   #12  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 509
Try now, the problem was with my Notepad and the line feeds.
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Reply

Tags
edge detection, khameleon, masking cleaner, restoring

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.