Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
2nd February 2012, 09:03 | #1 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
My attempt at writing a new shapener - SuperToon
I wanted to try and see if I could make a powerful sharpener that can run in realtime and this is what I came up with.
Code:
#SuperToon is my attempt to optimize/speed up the previous versions of mfToon, vmToon, etc. # # requires MaskTools V2.0 (at the time of writing this function the used version was V2.0a35) # Removegrain(), Unfilter(), and Degrainmedian(), Repair() and Medianblur # tested only in YV12, although I 'think' it could do other colorspaces it is reccomended to # do this in YV12 or YUV2 due to the math involed in this filter. # thanks to: # mastrboy for suggestion to use Removegrain() instead of Undot() # theProfileth for optimizations to use mode=3 # Chikuzen for error checking # Didée for suggestion for mode =4. function SuperToon(clip input, float "power", int "mode", \ int "Nthr", int "Nstr", int "Ncap", \ int "Lthr", int "Hthr", int "Lcap", \ int "cont", bool "show") { power = default(power,.75) #values -255.0-255.0 will have impact # however a range from 0.2-1.5 is usually # the desired range. Positive values darken lines, # negative values brighten them. 0 won't sharpen # the video however SuperToon will still consume CPU. # when setting the Nthr, NCap, Lcap, and pretty much any other # setting, you can set this to an extreme value like 100, this way # you can easily see (as if the edge mask was overlayed on the original) # what supertoon will be darkening (whether it'd be the lines you want it # to darken or the noise you didn't want it you detect) # for mode = 3, Power works differently. it ranges from -100 to 100 and its no # longer the "strength of darkening" but instead the "strength of smoothing + # sharpening" mode = default(mode,0) # ranges from 0-4; tells SuperToon to try different methods # no mode is always better than the other, its just that one mode # can fare better for one source and not another. so the user # has the option of which mode is best for their source. if you # choose a number outside of the range it will default to 0. # mode =0: just applies a very simple edge detection mask with # no effective noise detection. # mode =1: creates an edge mask and filters out noise by finding # the average of all neighboring pixels and determines # if the current pixel is an "outlier" and not noise, meaning # its an edge to darken. # mode =2: silmilar to mode 1 but does things differently, it runs a sieve # on all the neighboring pixels and then resets the edge mask # to be the median of the results from that sieve. # this method is alot slower than the others, but sometimes its # the only one that can seem to pull it off that the others can't # mode =3: does things completely differently in that it performs a "filtered" # unfilter on the source. by filtered I mean it sharpens the lines while # smoothing everything else. very effective at removing noise and thinning # lines, and is very fast. its just not as "strong" at sharpening as I # had hoped as the other modes. # mode =4: works to ignore non-edge lines like shadow lines so it doesn't darken lines # that weren't meant to be darkened. if you're getting something like black # outlines around stars or snow, this mode is very good at preventing that. # this mode doesn't run realtime for me (~20 fps for a 2.01 GHz single core) Lthr = default(Lthr,0) # can range from 0-255, raising this value makes darkened lines thicker and helps # with edge detection, but it'll usually give very very thick lines if you set this # too high. 9 times out of 10 you will likey not have to change this from the default. Hthr = default(Hthr,255) # can range from 0-255, lowering of this value has a stronger # impact on line detection than the low threshold. # most of the time the default is the best setting. Lcap = default(Lcap,255) # brightness cap which tells the function to only darken pixels under a certain luma value # this is mostly intended to give more control to modes 0-2. mode 3 doesn't use it, and mode 4 # shouldn't ever need it. setting it to 255 basiaclly disables this. cont = default(cont,180) # pixels are darkened by an amount relative to their spatial luma change. pixels with very low luma # change (like noise) is darkened very very slightly. pixels with large luma change (like a black line) # are greatly darkened. while moderate change (like a shadowline) is partially darkened. # this value controls the contrast of these changes. lowering this values makes it so that both low luma # and high luma changes are subtracted by near the same amount. if you lower this value "power" will have # to be raised to show a visible difference when used with Ncap, its iw very effective at removing noise. Nthr = default(Nthr,4) # threshold to try to detect noise around the lines to be sharpened set this too high and # it'll begin to confuse the lines you want sharpened as noise. this parameter works # differently depending on the mode so certain in modes it works better in certain ranges. # in nearly all modes its used to clip lower values from the generated mask (as noise). for # modes 1 and 2 it used more to clean the noise from the lines. for 3 its used to filter between # smoothing or sharpening. for mode 4 its used to overlay mode 0's mask onto mode 4's mask. lower numbers # add more weight to Mode 0's mask. Nstr = default(Nstr,0) # can range from -100 to 100. 0 will make this do nothing. This is # usually used to blur the edgemask to try to smooth out residual # noise. basically its a parameter that uses Unfliter() on the # treated mask. negative values work great for reducing noise # its fast and effective but values near the limits can make demolish lines # or produce a small halo effect. positive values and help thin some # lines which works great if the source doesn't present alot of noise. Ncap = default(Ncap,30) # ranges 0-255 increasing it is the best (as far as filter # speed) at preventing noise from being sharpened, however # setting it too high and you won't be darkening any lines # this setting helps determine how "sensitive" it should be when looking # at the edge mask. if it looks like the entire picture is darker, # then its likely this value it too low. 18 is a good starting point # for most sources I've tried. show = default(show,false) # if you want the edge mask returned instead of the sharpened # result powerF = min(100,max(1,Round(abs(power)))) Spow = string(power) SHthr = string(Hthr/4.0) SNthr = string(Nthr) SNcap = string(Ncap) SLcap = string(Lcap) mask = mt_makediff(input.mt_expand(Hthr),input.mt_inpand(Lthr)) #mode=1 -> abs(y-x)/x>n/255.0? y: 128 #mode=2 -> expr= abs(y-x)<n && x>y ? 128: y - x + 128 mask = (mode==2) ? mask.mt_luts(mask,pixels=mt_square( 4 ), mode="med", expr="y x - abs "+SNthr+" < x y > & 128 y x + 128 - ?") \ : (mode==1) ? mask.mt_lutf(mask,mode="avg",expr="y x - abs x / "+SNthr+" 255.0 / > y 128 ?") \ : mask #now this tries a completely different mechanic working more like a "filtered" sharpener AND smoother. # very very fast but is still experimental...not effective hard = mode==3 ?input.degrainmedian().unfilter(powerF,powerF).Removegrain(mode=1):input soft = mode==3 ?input.degrainmedian().unfilter(-powerF,-powerF).Removegrain(mode=1):input #"abs(x-y)>n?y:0" #"x==0?y:x" hard = mode==3 ?input.mt_lutxy(hard,"x y - abs "+SNthr+" > y 0 ?"):hard hard = mode==3 ?hard.mt_lutxy(soft,"x 0 = y x ?"):hard hard = mode==3 && Nstr!=0?hard.unfilter(Nstr,Nstr):hard #mode 4 works to avoid shadow lines. since the current mask it generates is far too weak, I have it add # a portion of the previously generated mask. as a result shadow lines aren't completely removed from # this mask, however their strength is so weak Nthr, Ncap, Nstr, and cont can easily remove it. #x - (y-128)/n + 128 mask = (mode==4) ?input.medianblurcw(2,0,0,0).mt_logic(input,"max").mt_makediff(input).mt_lutxy(mask,"x y 128 - "+SNthr+" / +") \ :mask #these 2 lines workd to clean all/most of the noise caught in the mask #as well and reset it's levels so its white to black and not white to grey. mask = ((Nstr!=0)?mask.unfilter(Nstr,Nstr):mask).Removegrain(mode=1) mask = mask.levels(128+Nthr,1,255,0,cont).Removegrain(mode=1) #residual Noise reduction regardless of mode. the Unfilter before doesn't # actually remove the noise... but it makes it simpler for the sharp LUT below to see it and # filter it out with NCap #"y<n||x>L?x:x-(y-n)*p" sharp = mode==3 ? hard:input.mt_lutxy(mask,"y "+SNcap+" < x "+SLcap+" > | x x y "+SNcap+" - "+Spow+" * - ?",u=2,v=2) #if show is true and the mode is not 3 (since it doesn't use an edge mask) return the edge mask # else return the sharpened video. return show && mode !=3 ?mask.greyscale():sharp } http://screenshotcomparison.com/comparison/106060/ and the parameters I used: Code:
#1 Gurren Lagann Supertoon(power=.5,mode=0,nthr=8,ncap=40,nstr=10,cont=250) #2 Soul Eater Supertoon(power=1,nthr=16,ncap=32,nstr=-70,cont=200) #3 Full Metal Panic Supertoon(power=.75,Lthr=0,Nthr=4,Ncap=40,cont=230,nstr=20) #4 & 5 Fooly Cooly Supertoon(power=.5,Nthr=24,Ncap=48,cont=230) here's some new screen caps showing off the new mode=4 http://screenshotcomparison.com/comparison/107324 and the parameters I used for those screencaps: Code:
#1 Soul Eater Supertoon(power=.75,mode=4,Lthr=0,nthr=3,nstr=-15,ncap=32,cont=200) #2 Gurren Lagann Supertoon(power=.5,mode=4,nthr=4,ncap=40,nstr=-10,cont=200) Edit: added more screencaps and updated code. Last edited by Hadien; 11th February 2012 at 16:30. Reason: code & screencap update |
2nd February 2012, 23:28 | #4 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Since this is not a plugin or a new compiled version of avisynth I think this fits more in the avisynth usage section rather than the avisynth development section.
Just saying
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
2nd February 2012, 23:40 | #5 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
well then I was confused. the description for this forum seemed to imply the creation of new filters, functions and such
while the other forum gave me the impression for learning how to code in avisynth, troubleshooting, etc. |
2nd February 2012, 23:45 | #6 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Ah well they mean things like dll filters and such but yeah no big deal just make a new thread in the other forum and maybe use real screenshots too lol but yeah I will look forward to testing and giving feedback on this filter in there.
Even better still use this site to compare http://screenshotcomparison.com/ should be better for everyone and more convenient too
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
7th February 2012, 10:57 | #10 | Link |
typo lover
Join Date: May 2009
Posts: 595
|
I got an error "Script error: expected `:'"
maybe, this will fix. Code:
--- supertoon.orig.avs 2012-02-07 18:52:06.108082300 +0900 +++ supertoon.fix.avs 2012-02-07 18:51:09.442841300 +0900 @@ -114,7 +114,7 @@ #"x==0?y:x" hard = mode==3 ?input.mt_lutxy(hard,"x y - abs "+SNthr+" > y 0 ?"):hard hard = mode==3 ?hard.mt_lutxy(soft,"x 0 = y x ?"):hard -hard = mode==3 ?(Nstr!=0)?hard.unfilter(Nstr,Nstr):hard +hard = mode==3 && (Nstr!=0)?hard.unfilter(Nstr,Nstr):hard #"y<n||x>L?x:x-(y-n)*p" sharp = mode==3 ? hard:input.mt_lutxy(mask,"y "+SNcap+" < x "+SLcap+" > | x x y "+SNcap+" - "+Spow+" * - ?",u=2,v=2)
__________________
my repositories |
7th February 2012, 13:50 | #11 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
yeah I noticed that error, and thats the same fix I did. I haven't been able to post the proper fix lately due to my PC crashing a lot recently (a capacitor blew on my mobo).
I've been trying to wrap my head around a way to subtract the shadow lines from my edgemasks or have it so that the edgemasks don't pick them up in the first place. I don't think shadow lines should be darkened however the way I have my edge detection set up it'll confuse them as lines that should be darkened. thinking of writing a mask using lut or convolution that detects only shadow lines and then subtract that mask from the edgemask. but not exactly sure on how to write it up. |
10th February 2012, 03:21 | #12 | Link | |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Quote:
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
|
10th February 2012, 08:27 | #13 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
ugh I hate the fact that if you type a long post and quick reply it suddenly forgets who you are since you are signed out. so you press the back button to find your whole post you wrote up is lost.
the blurring it mostly meant to reduce/prevent noise from being detected and darkened, its never strong enough to effect shadowlines. infact if you check Supertoon's Mask (via show=true) you can see that it can detect shadow lines just as clearly as edge lines. to supertoon, it sees edges and shadowlines as actual lines, and it'll darken everything it confirms as a line (for now, I want to make it more specific). I think the biggest difference between an edge line and a shadow line is that an edge line is usually one unique, darker color (within a threshold of course) that lies as a boundary between 2 other usually brighter colors. shadow lines are usually 2 different colors pushed together without a darker line splitting the two. so what I would want to do is tell my edge mask to look at its neighboring pixels in the input clip which weren't masked and determine if they were the same color as the pixel that were masked (again within a certain threshold) what I have tried is overlay the edgemask onto the input (making edges white), then perform an inpand or two. afterwards I'd make a comparision between this overlay and the input stating that if the input pixel is darker than the same pixel in the overlay, its an edge. I'd turn that result into my new mask and tried applying it. While it did mostly demolish the shadow lines, it also broke up much of the actual edges, and somehow made certain other areas darken which weren't edges (notably i motion areas where things got motion blurred). I feel I'm on the right track here, But I think I need to update Masktools (current version is V2.0a35) and see if theres a better function to use here. mt_luts is the only neighboring pixel lut I have and it performs the expression on each pixel before doing the mode (I want it to do that in reverse) Last edited by Hadien; 10th February 2012 at 08:31. Reason: grammar fixes |
10th February 2012, 17:43 | #14 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
A median filter is another possibility to "clamp out" lines, while not reacting to boundary-edges of uniform areas.
> vid.repair(vid.medianblur(2,0,0),1).mt_logic(vid,"max",U=2,V=2) or maybe FastLineDarken-style? > vid.repair(vid.mt_expand().mt_expand().mt_inpand(),1,0) # number of ex/inpands acc. to line thickness that shall be catched
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
11th February 2012, 00:48 | #15 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Didee would the dithertools box filter be of any use here? As it is certainly much faster than any median filter (as far as I know)
Cause here is a application of it I came up with a while ago after seeing someone ask for a mean filter Code:
Function vmean (clip v,int "r",int "rc") { #variable mean r = default(r, 4) rc= default(rc, r*2) v.Dither_convert_8_to_16 () Dither_box_filter16 (radius=r, y=3, u=2, v=2) Dither_box_filter16 (radius=rc, y=2, u=3, v=3) # for 4:2:0 chroma subsampling DitherPost (mode=-1,stacked=true) }
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
11th February 2012, 04:45 | #16 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,389
|
A box filter is a mean filter. For the problem posed, mean filtering is exactly what needs to be avoided.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
11th February 2012, 05:41 | #17 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
Ok that is what I thought, I just wanted to make sure
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon |
11th February 2012, 11:17 | #18 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
@Didée:
thanks for that fantastic "clamping" suggestion. I tried it out and while the mask it gave me was still kinda weak (it prevented those shadow lines but would sometimes demolish other lines) I instead played with it some more and I believe I came across a pretty good solution. the mask itself was still weak so what I did was add mode=0's mask back onto this weak mask (mode 0's mask values were greatly reduced to help hide its shadow lines). Together with the noise reductions I already have in place (Nthr, Ncap, Nstr, and cont parameters) the shadow lines in mode 4's mask (mode 4 is the new mode to remove these shadow lines) were made so weak that Supertoon could pass them off as simple noise to filter out. speed-wise mode 4 runs about 20 fps (again on a single core 2.01 Ghz). It feels like I could make some speed improvements to it but I'll worry about that tomorrow since it's getting late/early here I post a full code update and new comparison screen-caps when I wake up in a few hours. |
11th February 2012, 23:56 | #19 | Link |
Leader of Dual-Duality
Join Date: Aug 2010
Location: America
Posts: 134
|
You should be using medianblur(2,0,0) rather than medianblurcw(2,0,0,0) as it is (about) twice as fast and does the same thing in this situation.
That hopefully should allow you to get realtime playback.
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood My New filter is in the works, and will be out soon Last edited by TheProfileth; 11th February 2012 at 23:58. |
12th February 2012, 00:55 | #20 | Link |
Infected Mushroom Fan
Join Date: Dec 2007
Location: Atlanta,GA
Posts: 17
|
thanks for the tip and you're right its does greatly speed up that mode to play real-time. I initially thought the center weight version would prove more roboust if I would have it not count the center pixel by setting it to 0. when I changed it back I saw no visual change (though I do remember a change when I first tried it, maybe in the end after adding the difference mask and overlaying the previous mask that got balanced out) but yeah, definite speed boost.
I'm pondering if I should add a hook to the radius variable in median blur cause it might be that a radius of 2 won't always help for every instance. Besides that, what is everyone's opinion of this sharpener? would you ever see yourself using it? do you like where its going? I'd like to hear some feedback on this. |
Thread Tools | Search this Thread |
Display Modes | |
|
|