Hadien
2nd February 2012, 09:03
I wanted to try and see if I could make a powerful sharpener that can run in realtime and this is what I came up with.
#SuperToon is my attempt to optimize/speed up the previous versions of mfToon, vmToon, etc.
#
# requires MaskTools V2.0 (at the time of writing this function the used version was V2.0a35)
# Removegrain(), Unfilter(), and Degrainmedian(), Repair() and Medianblur
# tested only in YV12, although I 'think' it could do other colorspaces it is reccomended to
# do this in YV12 or YUV2 due to the math involed in this filter.
# thanks to:
# mastrboy for suggestion to use Removegrain() instead of Undot()
# theProfileth for optimizations to use mode=3
# Chikuzen for error checking
# Didée for suggestion for mode =4.
function SuperToon(clip input, float "power", int "mode",
\ int "Nthr", int "Nstr", int "Ncap",
\ int "Lthr", int "Hthr", int "Lcap",
\ int "cont", bool "show")
{
power = default(power,.75) #values -255.0-255.0 will have impact
# however a range from 0.2-1.5 is usually
# the desired range. Positive values darken lines,
# negative values brighten them. 0 won't sharpen
# the video however SuperToon will still consume CPU.
# when setting the Nthr, NCap, Lcap, and pretty much any other
# setting, you can set this to an extreme value like 100, this way
# you can easily see (as if the edge mask was overlayed on the original)
# what supertoon will be darkening (whether it'd be the lines you want it
# to darken or the noise you didn't want it you detect)
# for mode = 3, Power works differently. it ranges from -100 to 100 and its no
# longer the "strength of darkening" but instead the "strength of smoothing +
# sharpening"
mode = default(mode,0) # ranges from 0-4; tells SuperToon to try different methods
# no mode is always better than the other, its just that one mode
# can fare better for one source and not another. so the user
# has the option of which mode is best for their source. if you
# choose a number outside of the range it will default to 0.
# mode =0: just applies a very simple edge detection mask with
# no effective noise detection.
# mode =1: creates an edge mask and filters out noise by finding
# the average of all neighboring pixels and determines
# if the current pixel is an "outlier" and not noise, meaning
# its an edge to darken.
# mode =2: silmilar to mode 1 but does things differently, it runs a sieve
# on all the neighboring pixels and then resets the edge mask
# to be the median of the results from that sieve.
# this method is alot slower than the others, but sometimes its
# the only one that can seem to pull it off that the others can't
# mode =3: does things completely differently in that it performs a "filtered"
# unfilter on the source. by filtered I mean it sharpens the lines while
# smoothing everything else. very effective at removing noise and thinning
# lines, and is very fast. its just not as "strong" at sharpening as I
# had hoped as the other modes.
# mode =4: works to ignore non-edge lines like shadow lines so it doesn't darken lines
# that weren't meant to be darkened. if you're getting something like black
# outlines around stars or snow, this mode is very good at preventing that.
# this mode doesn't run realtime for me (~20 fps for a 2.01 GHz single core)
Lthr = default(Lthr,0) # can range from 0-255, raising this value makes darkened lines thicker and helps
# with edge detection, but it'll usually give very very thick lines if you set this
# too high. 9 times out of 10 you will likey not have to change this from the default.
Hthr = default(Hthr,255) # can range from 0-255, lowering of this value has a stronger
# impact on line detection than the low threshold.
# most of the time the default is the best setting.
Lcap = default(Lcap,255) # brightness cap which tells the function to only darken pixels under a certain luma value
# this is mostly intended to give more control to modes 0-2. mode 3 doesn't use it, and mode 4
# shouldn't ever need it. setting it to 255 basiaclly disables this.
cont = default(cont,180) # pixels are darkened by an amount relative to their spatial luma change. pixels with very low luma
# change (like noise) is darkened very very slightly. pixels with large luma change (like a black line)
# are greatly darkened. while moderate change (like a shadowline) is partially darkened.
# this value controls the contrast of these changes. lowering this values makes it so that both low luma
# and high luma changes are subtracted by near the same amount. if you lower this value "power" will have
# to be raised to show a visible difference when used with Ncap, its iw very effective at removing noise.
Nthr = default(Nthr,4) # threshold to try to detect noise around the lines to be sharpened set this too high and
# it'll begin to confuse the lines you want sharpened as noise. this parameter works
# differently depending on the mode so certain in modes it works better in certain ranges.
# in nearly all modes its used to clip lower values from the generated mask (as noise). for
# modes 1 and 2 it used more to clean the noise from the lines. for 3 its used to filter between
# smoothing or sharpening. for mode 4 its used to overlay mode 0's mask onto mode 4's mask. lower numbers
# add more weight to Mode 0's mask.
Nstr = default(Nstr,0) # can range from -100 to 100. 0 will make this do nothing. This is
# usually used to blur the edgemask to try to smooth out residual
# noise. basically its a parameter that uses Unfliter() on the
# treated mask. negative values work great for reducing noise
# its fast and effective but values near the limits can make demolish lines
# or produce a small halo effect. positive values and help thin some
# lines which works great if the source doesn't present alot of noise.
Ncap = default(Ncap,30) # ranges 0-255 increasing it is the best (as far as filter
# speed) at preventing noise from being sharpened, however
# setting it too high and you won't be darkening any lines
# this setting helps determine how "sensitive" it should be when looking
# at the edge mask. if it looks like the entire picture is darker,
# then its likely this value it too low. 18 is a good starting point
# for most sources I've tried.
show = default(show,false) # if you want the edge mask returned instead of the sharpened
# result
powerF = min(100,max(1,Round(abs(power))))
Spow = string(power)
SHthr = string(Hthr/4.0)
SNthr = string(Nthr)
SNcap = string(Ncap)
SLcap = string(Lcap)
mask = mt_makediff(input.mt_expand(Hthr),input.mt_inpand(Lthr))
#mode=1 -> abs(y-x)/x>n/255.0? y: 128
#mode=2 -> expr= abs(y-x)<n && x>y ? 128: y - x + 128
mask = (mode==2) ? mask.mt_luts(mask,pixels=mt_square( 4 ), mode="med", expr="y x - abs "+SNthr+" < x y > & 128 y x + 128 - ?")
\ : (mode==1) ? mask.mt_lutf(mask,mode="avg",expr="y x - abs x / "+SNthr+" 255.0 / > y 128 ?")
\ : mask
#now this tries a completely different mechanic working more like a "filtered" sharpener AND smoother.
# very very fast but is still experimental...not effective
hard = mode==3 ?input.degrainmedian().unfilter(powerF,powerF).Removegrain(mode=1):input
soft = mode==3 ?input.degrainmedian().unfilter(-powerF,-powerF).Removegrain(mode=1):input
#"abs(x-y)>n?y:0"
#"x==0?y:x"
hard = mode==3 ?input.mt_lutxy(hard,"x y - abs "+SNthr+" > y 0 ?"):hard
hard = mode==3 ?hard.mt_lutxy(soft,"x 0 = y x ?"):hard
hard = mode==3 && Nstr!=0?hard.unfilter(Nstr,Nstr):hard
#mode 4 works to avoid shadow lines. since the current mask it generates is far too weak, I have it add
# a portion of the previously generated mask. as a result shadow lines aren't completely removed from
# this mask, however their strength is so weak Nthr, Ncap, Nstr, and cont can easily remove it.
#x - (y-128)/n + 128
mask = (mode==4) ?input.medianblurcw(2,0,0,0).mt_logic(input,"max").mt_makediff(input).mt_lutxy(mask,"x y 128 - "+SNthr+" / +")
\ :mask
#these 2 lines workd to clean all/most of the noise caught in the mask
#as well and reset it's levels so its white to black and not white to grey.
mask = ((Nstr!=0)?mask.unfilter(Nstr,Nstr):mask).Removegrain(mode=1)
mask = mask.levels(128+Nthr,1,255,0,cont).Removegrain(mode=1)
#residual Noise reduction regardless of mode. the Unfilter before doesn't
# actually remove the noise... but it makes it simpler for the sharp LUT below to see it and
# filter it out with NCap
#"y<n||x>L?x:x-(y-n)*p"
sharp = mode==3 ? hard:input.mt_lutxy(mask,"y "+SNcap+" < x "+SLcap+" > | x x y "+SNcap+" - "+Spow+" * - ?",u=2,v=2)
#if show is true and the mode is not 3 (since it doesn't use an edge mask) return the edge mask
# else return the sharpened video.
return show && mode !=3 ?mask.greyscale():sharp
}
here are some screen caps of before/after of it at work:
http://screenshotcomparison.com/comparison/106060/
and the parameters I used:
#1 Gurren Lagann Supertoon(power=.5,mode=0,nthr=8,ncap=40,nstr=10,cont=250)
#2 Soul Eater Supertoon(power=1,nthr=16,ncap=32,nstr=-70,cont=200)
#3 Full Metal Panic Supertoon(power=.75,Lthr=0,Nthr=4,Ncap=40,cont=230,nstr=20)
#4 & 5 Fooly Cooly Supertoon(power=.5,Nthr=24,Ncap=48,cont=230)
#4 and 5 where to show how well it can work with sources that had scenes with strong noise and weak noise
here's some new screen caps showing off the new mode=4
http://screenshotcomparison.com/comparison/107324
and the parameters I used for those screencaps:
#1 Soul Eater Supertoon(power=.75,mode=4,Lthr=0,nthr=3,nstr=-15,ncap=32,cont=200)
#2 Gurren Lagann Supertoon(power=.5,mode=4,nthr=4,ncap=40,nstr=-10,cont=200)
all the modes I used would play in realtime when I tested them, except for mode 2 which would run around 10 fps on my PC (2.01 Ghz single-core) and mode 4 (~20fps). Theres still alot of things I want to add and some problems I want to fix. but I wanted to go ahead and post what I've got so far. see if I could get some critiques.
Edit: added more screencaps and updated code.
#SuperToon is my attempt to optimize/speed up the previous versions of mfToon, vmToon, etc.
#
# requires MaskTools V2.0 (at the time of writing this function the used version was V2.0a35)
# Removegrain(), Unfilter(), and Degrainmedian(), Repair() and Medianblur
# tested only in YV12, although I 'think' it could do other colorspaces it is reccomended to
# do this in YV12 or YUV2 due to the math involed in this filter.
# thanks to:
# mastrboy for suggestion to use Removegrain() instead of Undot()
# theProfileth for optimizations to use mode=3
# Chikuzen for error checking
# Didée for suggestion for mode =4.
function SuperToon(clip input, float "power", int "mode",
\ int "Nthr", int "Nstr", int "Ncap",
\ int "Lthr", int "Hthr", int "Lcap",
\ int "cont", bool "show")
{
power = default(power,.75) #values -255.0-255.0 will have impact
# however a range from 0.2-1.5 is usually
# the desired range. Positive values darken lines,
# negative values brighten them. 0 won't sharpen
# the video however SuperToon will still consume CPU.
# when setting the Nthr, NCap, Lcap, and pretty much any other
# setting, you can set this to an extreme value like 100, this way
# you can easily see (as if the edge mask was overlayed on the original)
# what supertoon will be darkening (whether it'd be the lines you want it
# to darken or the noise you didn't want it you detect)
# for mode = 3, Power works differently. it ranges from -100 to 100 and its no
# longer the "strength of darkening" but instead the "strength of smoothing +
# sharpening"
mode = default(mode,0) # ranges from 0-4; tells SuperToon to try different methods
# no mode is always better than the other, its just that one mode
# can fare better for one source and not another. so the user
# has the option of which mode is best for their source. if you
# choose a number outside of the range it will default to 0.
# mode =0: just applies a very simple edge detection mask with
# no effective noise detection.
# mode =1: creates an edge mask and filters out noise by finding
# the average of all neighboring pixels and determines
# if the current pixel is an "outlier" and not noise, meaning
# its an edge to darken.
# mode =2: silmilar to mode 1 but does things differently, it runs a sieve
# on all the neighboring pixels and then resets the edge mask
# to be the median of the results from that sieve.
# this method is alot slower than the others, but sometimes its
# the only one that can seem to pull it off that the others can't
# mode =3: does things completely differently in that it performs a "filtered"
# unfilter on the source. by filtered I mean it sharpens the lines while
# smoothing everything else. very effective at removing noise and thinning
# lines, and is very fast. its just not as "strong" at sharpening as I
# had hoped as the other modes.
# mode =4: works to ignore non-edge lines like shadow lines so it doesn't darken lines
# that weren't meant to be darkened. if you're getting something like black
# outlines around stars or snow, this mode is very good at preventing that.
# this mode doesn't run realtime for me (~20 fps for a 2.01 GHz single core)
Lthr = default(Lthr,0) # can range from 0-255, raising this value makes darkened lines thicker and helps
# with edge detection, but it'll usually give very very thick lines if you set this
# too high. 9 times out of 10 you will likey not have to change this from the default.
Hthr = default(Hthr,255) # can range from 0-255, lowering of this value has a stronger
# impact on line detection than the low threshold.
# most of the time the default is the best setting.
Lcap = default(Lcap,255) # brightness cap which tells the function to only darken pixels under a certain luma value
# this is mostly intended to give more control to modes 0-2. mode 3 doesn't use it, and mode 4
# shouldn't ever need it. setting it to 255 basiaclly disables this.
cont = default(cont,180) # pixels are darkened by an amount relative to their spatial luma change. pixels with very low luma
# change (like noise) is darkened very very slightly. pixels with large luma change (like a black line)
# are greatly darkened. while moderate change (like a shadowline) is partially darkened.
# this value controls the contrast of these changes. lowering this values makes it so that both low luma
# and high luma changes are subtracted by near the same amount. if you lower this value "power" will have
# to be raised to show a visible difference when used with Ncap, its iw very effective at removing noise.
Nthr = default(Nthr,4) # threshold to try to detect noise around the lines to be sharpened set this too high and
# it'll begin to confuse the lines you want sharpened as noise. this parameter works
# differently depending on the mode so certain in modes it works better in certain ranges.
# in nearly all modes its used to clip lower values from the generated mask (as noise). for
# modes 1 and 2 it used more to clean the noise from the lines. for 3 its used to filter between
# smoothing or sharpening. for mode 4 its used to overlay mode 0's mask onto mode 4's mask. lower numbers
# add more weight to Mode 0's mask.
Nstr = default(Nstr,0) # can range from -100 to 100. 0 will make this do nothing. This is
# usually used to blur the edgemask to try to smooth out residual
# noise. basically its a parameter that uses Unfliter() on the
# treated mask. negative values work great for reducing noise
# its fast and effective but values near the limits can make demolish lines
# or produce a small halo effect. positive values and help thin some
# lines which works great if the source doesn't present alot of noise.
Ncap = default(Ncap,30) # ranges 0-255 increasing it is the best (as far as filter
# speed) at preventing noise from being sharpened, however
# setting it too high and you won't be darkening any lines
# this setting helps determine how "sensitive" it should be when looking
# at the edge mask. if it looks like the entire picture is darker,
# then its likely this value it too low. 18 is a good starting point
# for most sources I've tried.
show = default(show,false) # if you want the edge mask returned instead of the sharpened
# result
powerF = min(100,max(1,Round(abs(power))))
Spow = string(power)
SHthr = string(Hthr/4.0)
SNthr = string(Nthr)
SNcap = string(Ncap)
SLcap = string(Lcap)
mask = mt_makediff(input.mt_expand(Hthr),input.mt_inpand(Lthr))
#mode=1 -> abs(y-x)/x>n/255.0? y: 128
#mode=2 -> expr= abs(y-x)<n && x>y ? 128: y - x + 128
mask = (mode==2) ? mask.mt_luts(mask,pixels=mt_square( 4 ), mode="med", expr="y x - abs "+SNthr+" < x y > & 128 y x + 128 - ?")
\ : (mode==1) ? mask.mt_lutf(mask,mode="avg",expr="y x - abs x / "+SNthr+" 255.0 / > y 128 ?")
\ : mask
#now this tries a completely different mechanic working more like a "filtered" sharpener AND smoother.
# very very fast but is still experimental...not effective
hard = mode==3 ?input.degrainmedian().unfilter(powerF,powerF).Removegrain(mode=1):input
soft = mode==3 ?input.degrainmedian().unfilter(-powerF,-powerF).Removegrain(mode=1):input
#"abs(x-y)>n?y:0"
#"x==0?y:x"
hard = mode==3 ?input.mt_lutxy(hard,"x y - abs "+SNthr+" > y 0 ?"):hard
hard = mode==3 ?hard.mt_lutxy(soft,"x 0 = y x ?"):hard
hard = mode==3 && Nstr!=0?hard.unfilter(Nstr,Nstr):hard
#mode 4 works to avoid shadow lines. since the current mask it generates is far too weak, I have it add
# a portion of the previously generated mask. as a result shadow lines aren't completely removed from
# this mask, however their strength is so weak Nthr, Ncap, Nstr, and cont can easily remove it.
#x - (y-128)/n + 128
mask = (mode==4) ?input.medianblurcw(2,0,0,0).mt_logic(input,"max").mt_makediff(input).mt_lutxy(mask,"x y 128 - "+SNthr+" / +")
\ :mask
#these 2 lines workd to clean all/most of the noise caught in the mask
#as well and reset it's levels so its white to black and not white to grey.
mask = ((Nstr!=0)?mask.unfilter(Nstr,Nstr):mask).Removegrain(mode=1)
mask = mask.levels(128+Nthr,1,255,0,cont).Removegrain(mode=1)
#residual Noise reduction regardless of mode. the Unfilter before doesn't
# actually remove the noise... but it makes it simpler for the sharp LUT below to see it and
# filter it out with NCap
#"y<n||x>L?x:x-(y-n)*p"
sharp = mode==3 ? hard:input.mt_lutxy(mask,"y "+SNcap+" < x "+SLcap+" > | x x y "+SNcap+" - "+Spow+" * - ?",u=2,v=2)
#if show is true and the mode is not 3 (since it doesn't use an edge mask) return the edge mask
# else return the sharpened video.
return show && mode !=3 ?mask.greyscale():sharp
}
here are some screen caps of before/after of it at work:
http://screenshotcomparison.com/comparison/106060/
and the parameters I used:
#1 Gurren Lagann Supertoon(power=.5,mode=0,nthr=8,ncap=40,nstr=10,cont=250)
#2 Soul Eater Supertoon(power=1,nthr=16,ncap=32,nstr=-70,cont=200)
#3 Full Metal Panic Supertoon(power=.75,Lthr=0,Nthr=4,Ncap=40,cont=230,nstr=20)
#4 & 5 Fooly Cooly Supertoon(power=.5,Nthr=24,Ncap=48,cont=230)
#4 and 5 where to show how well it can work with sources that had scenes with strong noise and weak noise
here's some new screen caps showing off the new mode=4
http://screenshotcomparison.com/comparison/107324
and the parameters I used for those screencaps:
#1 Soul Eater Supertoon(power=.75,mode=4,Lthr=0,nthr=3,nstr=-15,ncap=32,cont=200)
#2 Gurren Lagann Supertoon(power=.5,mode=4,nthr=4,ncap=40,nstr=-10,cont=200)
all the modes I used would play in realtime when I tested them, except for mode 2 which would run around 10 fps on my PC (2.01 Ghz single-core) and mode 4 (~20fps). Theres still alot of things I want to add and some problems I want to fix. but I wanted to go ahead and post what I've got so far. see if I could get some critiques.
Edit: added more screencaps and updated code.