Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st July 2005, 00:29   #1  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
Idea for a sharpening filter

Hi all,
Yesterday I had some ideas about a new filter, but before I start coding I would like to share my thoughts to have some suggestions and to know if anything like it has ever been created. I'm quite good in c and c++ programming, but I'm quite a noob about filtering and video...
anyway, as everybody (I think) here I'm amazed by the quality of Didée's LimitedSharpen filter, but I also think that if it would be faster, I would be happier... so I was thinking: is there any way to create a simple (and fast) filter, not so rich of features, but which uses its main principle?
what we would like is to enhance the steepyness of an edge without create overshooting. for example, in a one-dimensional world, if a sequence of three values is:
1 4 10
I could think that the central value is a blurred edge, and so I could try to enhance its steepyness moving it towards the nearest surrounding value, which is the 1, and so the processed output sequence could be:
1 1 10
or better:
1 2 10
but if the input sequence is:
1 10 4
I would not perform any modification because it would cause just 'overshooting', and so it would be useless to the sharpening of the image.

Trying to apply this in a two-dimensional environment, I think that two ways could be followed:

1 - use a simple 3*3 sharpening matrix like:
-1 -2 -1
-2 13 -2
-1 -2 -1
or any other (maybe customizable), but only when the central value is contained between the highest and the lowest of the surrounding values. maybe this happens too often, and maybe the mask could be applied just when the value is between the second or third highest and lowest of the surrounding pixels, or also this could be a customizable parameter. I also think that probably it's also quite easy to write it as a function (maybe using some masking), before trying developing it as a compiled filter...

2 - in a 3*3 window, I could look in all the 4 directions (horizontal, vertical and diagonals). in every direction I have a one-dimensional situation similar to the one showed above. so if the value of the central pixel is contained between the two neighbors it could be clipped to the nearest value of the neighbors, otherwise it would be unchanged. in this way, I obtain 4 new possible values for the pixel, and I could make a weighted average (also customizable) between the 4 new values and the old value. so the strength of the filter could be higher or lower choosing the values apllied for the average. don't know if the result could be nice-looking...

Both this ways could be developed using a variable radius, with the possibility of using more information than the one supplied by a 3*3 window, for example using a 5*5 one or a bigger one...
That's all!
So, what do you think? Comments, critics and suggestions are welcome!
Thanks,
Pippo
ilpippo80 is offline   Reply With Quote
Old 1st July 2005, 03:32   #2  |  Link
ambrotos
Evil tweaker...
 
ambrotos's Avatar
 
Join Date: Sep 2002
Posts: 33
Quote:
Originally Posted by ilpippo80
1 - use a simple 3*3 sharpening matrix like:
-1 -2 -1
-2 13 -2
-1 -2 -1
This kind of filter already exists and it's called Laplacian filter
http://homepages.inf.ed.ac.uk/rbf/HIPR2/unsharp.htm

beside the fact that the examples of Didée were 1-D, I think that very similar 2D kernel is already used in most sharpening filters and so in Limitedunsharpen().
So, what you propose is a new adaptative sharpener ? but what's the difference with the didée's limitedsharpen() ?

btw if you want to play with kernels : http://homepages.inf.ed.ac.uk/rbf/HI...lutiondemo.htm
(I think you can load your own monochromatic pictures)

Quote:
Originally Posted by ilpippo80
2 - in a 3*3 window, I could look in all the 4 directions (horizontal, vertical and diagonals). in every direction I have a one-dimensional situation similar to the one showed above. so if the value of the central pixel is contained between the two neighbors it could be clipped to the nearest value of the neighbors, otherwise it would be unchanged.
Interesting variant of the min/max directional blur ^^ but for sharpening, it worth a try
ambrotos is offline   Reply With Quote
Old 1st July 2005, 09:13   #3  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Finally someone comes up with that

In the wake of LimitedSharpen, I also built a function "SlopeBend()", which does something very similar.

Where LS is - basically - doing a simple trick (do normal sharpening, and just cut off the overshoot where it occurs ... hence the need for supersampling to avoid jaggy edges), SlopeBend shifts values inbetween local min/max values towards the closer-in-range extremum.

Main drawback of SlopeBend() is, that I never liked the results, and thus posted LS instead ...

To put the problem in simple words: when going to sharpen a pixel that's inbetween the local min. & max., it's hard to tell if the pixel actually

a) is part of a "blurry edge" (-> should be sharpened)
b) is part of a gradient (-> should be left alone, to avoid banding/posterization)
c) is part of the "antialiasing" of a hard edge (-> must not be sharpened, to not produce aliasing)

One could, of course, do more probability checks to process only the a)-pixels and bypass the b)- and c)-pixels. But then you leave the fast lane of 3x3 kernel operation. At least you'd have to look at the local 3x3 and 5x5 kernel (better even 7x7), and evaluate & compare their properties.
Read: slow operation

If you're interested, I'll post the function here this weekend.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 1st July 2005, 10:21   #4  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
@ambrotos
That is a simple 2-D sharpening filter, there's nothing new with it, the only thing added would be that I would apply it just when there is an edge, which is also the main principle of LimitedSharpen.
Thanks for the links, I'll have a look!

@Didée
Quote:
c) is part of the "antialiasing" of a hard edge (-> must not be sharpened, to not produce aliasing)
I think I don't understand this fully, could you please explain it better?
and I would appreciate much if you could post that function, that would be a very good starting point...

Thanks,
Pippo
ilpippo80 is offline   Reply With Quote
Old 1st July 2005, 10:51   #5  |  Link
mg262
Clouded
 
mg262's Avatar
 
Join Date: Jul 2003
Location: Cambridge, UK
Posts: 1,148
I've been meaning for some time to point people at this link on upsampling and retaining sharpening, and this seems like a decent place to do it:

http://www.avsforum.com/avs-vb/showt...&&#post4767386

In fact the whole thread is IMO very useful reading, especially the posts by dmunsil. Hope someone finds it interesting!

Last edited by mg262; 1st July 2005 at 10:53.
mg262 is offline   Reply With Quote
Old 1st July 2005, 11:17   #6  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
@ ilpippo80

Look closely at this picture: (Primitive showcase, but hey...)



In the middle there's the "blurry edge" for which shifting the pixels with inbetween values is OK.
On the left there's the "hard edge" where shifting the inbetween values produces aliasing.
On the right there's a smooth gradient getting destroyed by shifting inbetween values.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 1st July 2005, 12:26   #7  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Oh, surprise. Found that once I had copied the script to this machine here, too.

SlopeBend_v01a

Warning: Fully experimental, do not use! Development version with more code commented out than code actually doing something ...

There're not even explaining comments in it (except for the header), but methinks you can figure the story from the variable names. If not, just ask.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 1st July 2005, 13:20   #8  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
@Didée
many thanks for the clear examples and the code, this weekend I'll try experimenting with it...
and many thanks to everybody for your help! it's a big relief to be helped by so experienced filter programmers!
Pippo
ilpippo80 is offline   Reply With Quote
Old 1st July 2005, 13:34   #9  |  Link
kassandro
Registered User
 
Join Date: May 2003
Location: Germany
Posts: 502
Quote:
Originally Posted by ilpippo80
1 - use a simple 3*3 sharpening matrix like:
-1 -2 -1
-2 13 -2
-1 -2 -1
As already mentioned by ambrotos, this is a very standard way of sharpening. It easily leads to oversharpening, i.e. the edge halos, which are nicely displayed in a picture at the very beginning of the LimitedShapen thread.

Quote:
2 - in a 3*3 window, I could look in all the 4 directions (horizontal, vertical and diagonals). in every direction I have a one-dimensional situation similar to the one showed above. so if the value of the central pixel is contained between the two neighbors it could be clipped to the nearest value of the neighbors, otherwise it would be unchanged. in this way, I obtain 4 new possible values for the pixel, and I could make a weighted average (also customizable) between the 4 new values and the old value. so the strength of the filter could be higher or lower choosing the values apllied for the average. don't know if the result could be nice-looking...
This idea is more interesting and I will probably implement two versions of it in the forthcoming version 1.0 of RemoveGrain. The first one will sharpen along the line, where the difference between the end points is minimal. This version is very conservative and doesn't destroy thin lines like most sharpeners. The second will choose the line with maximal difference. This is very aggressive and may easily destroy thin lines. Actually I have already implemented the first one. In fact, from any denoise mode of RemoveGrain one can derive a sharpening mode and I have already done this (the plugin is called RSharpen). Then RShapen(mode=9) is exactly the first very conservative version. Didee's SlopeBend corresponds to RSharpen(mode=1). However, my one dimensional sharpening method is more conservative much more sophisticated. The basic mistake in your sharpening technique can be shown by looking at four instead of three values 10, 30, 40, 60. Then your sharpening technique yields something like 10, 40, 30 , 60, which is terrible. Any good sharpener has to cope with this problem, that sharpening may occur from both sides. If it dose not it will not only be a noise amplifier but even a noise creator. Perhaps Didee made a similar mistake in his SlopeBend. My approach in RemoveGrain 1.0 takes care of this problem in a SSE friendly way.

Edit: My method of sharpening also tries to avoid aliasing as much as possible.

As I was locked out from this forum for 30 days because of my contribution to the Avisynth 2.6.6 beta thread, I will reduce my participation in this forum substantially. In particular, I will only announce new versions of my plugins here, but will no more support these filters. If you want support you have to go to my support forum (see the RemoveDirt, RemoveGrain web sites).
However, I will continue to critise neuron2, whenever this is justified. That is the last thing I will stop in this forum.

Last edited by kassandro; 1st July 2005 at 14:08.
kassandro is offline   Reply With Quote
Old 1st July 2005, 14:09   #10  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Hi, kassandro - may I say "welcome back" ...

No, Didée did not make a similar mistake in his script. He just takes the absolute maxima and minima from a neighborhood of given radius, compares center pixel's value to (max+min)/2, and then does a normalized multiplication.
So no noise is ever generated. But the function can achieve very strong posterization, if 'strength' and 'radius' are set high enough

BTW, the topmost link on the RemoveGrain page never worked for me - always gives "Whoops - Seite nicht gefunden!" ...
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 1st July 2005, 14:43   #11  |  Link
kassandro
Registered User
 
Join Date: May 2003
Location: Germany
Posts: 502
Quote:
Originally Posted by Didée
Hi, kassandro - may I say "welcome back" ...
Probably I will be out again soon. As an immediate response I got the following PM
Quote:
Warning
We have discovered that you have violated forum rule #4 in one of your posts. If you violate the forum rules 1 more times you will be suspended for 30 days.
Quote:
BTW, the topmost link on the RemoveGrain page never worked for me - always gives "Whoops - Seite nicht gefunden!" ...
Fortunately mg262 just did point out this problem to me in a PM. It will be fixed soon.
kassandro is offline   Reply With Quote
Old 1st July 2005, 15:23   #12  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
@ ilpippo80

I just remembered that by that time, I had correspondence with a collegue (foreign, hence in English) about the operation, because I wasn't sure how to do some things. Have a look at this PDF, could make things somewhat clearer. Perhaps.
BTW, right now I realize I never implemented that weigthened strength thingie - just forgot about it ...)

(kassandro - *don't* you look at that! You'll die for laughter ...)
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 1st July 2005 at 15:28.
Didée is offline   Reply With Quote
Old 1st July 2005, 16:32   #13  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
@mg262
thanks for the link! now I've understood something more, for example why we have to deal with an aliasing problem. indeed, it's not very natural to think about Nyquist frequency when you're watching an image...

by the way, some more ideas: using the values of the 8 surrounding pixels, maybe it would be possible to calculate (or probably just estimate) the minimum and maximum value that we could give to the central pixel without producing aliasing...
if this is possible, we could compare the central pixel values with this limits, and if it's contained we could push its value towards the nearest limit, otherwise we would leave it unchanged. so we would solve two problems:
1 - find a good way to apply the original 1-D idea in a 2-D world
2 - how not to create aliasing

now two questions:
Q: how could I calculate min and max?
A: dunnow. maybe some googling will help...

Q: what about gradients?
A: ...
...
(five minutes thinking)
ehm...
(five minutes thinking)
...
(five minutes thinking)
doh!
...
well, I think it's not good to try to solve too problems in the same time, is it?

Bye, Pippo
ilpippo80 is offline   Reply With Quote
Old 1st July 2005, 18:04   #14  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
@kassandro
About point 1 - I know that it's a very simple filter, but I was thinking that I could apply it just where needed to enhance blurred edges and not to apply it where it would produce just artifacts (easy to say, but difficult to do!)...
About point 2 - The truth is that the I 'stole' the basic idea from your way of denoising with RemoveGrain. BTW, great denoiser, and it's SO FAST...
Anyway, I know that you're completely right when you wrote that the use of such a way of sharpening could produce such problems, because a pixel value is not changed by its own, also its surroundings will be changed, I already knew it. Anyway, if the user is careful enough with the strength of the filter (see below), it should not produce relevant artifacts...

@Didée
I've read the pdf. I was thinking to something easier, just a weighted average between the actual value and the nearest limit, with a strength parameter used for the weight. if the strength is str and it's a value between 0 and 32, the actual value is act and the nearest limit is lim, the new value could be obtained in this way:
(lim*str+act*(32-str))/32
But I think that I'm thinking too much. Now I'd better start coding for a while!
You're really really really helping me a lot, Didée! Thanks again!
ilpippo80 is offline   Reply With Quote
Old 1st July 2005, 18:17   #15  |  Link
kassandro
Registered User
 
Join Date: May 2003
Location: Germany
Posts: 502
Quote:
Originally Posted by ilpippo80
@kassandro
Anyway, if the user is careful enough with the strength of the filter (see below), it should not produce relevant artifacts...
Exactly, if you at least halfen the strength, then the unpleasant phenomenon decribed above cannot occur.
By the way, I have corrected the links, but the old web pages may still be in Firefox cache. In my support forum I have discussed the problem in more detail and gave a precise description, how I sharpen between to points.

@mg262:
thanks for the interesting link to the avsforum. I didn't even know that there exists another advanced a/v forum besides that of doom9.

Last edited by kassandro; 1st July 2005 at 18:20.
kassandro is offline   Reply With Quote
Old 2nd July 2005, 11:38   #16  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
Here's some code.

Code:
function pSharpen(clip clp, int "strength", int "threshold")
{
    strength=default(strength, 16)
    strength=strength<0? 0 : strength>32? 32 : strength
    threshold=default(threshold, 4)
    threshold=threshold<4? 4 : threshold

    temp=clp.greyscale()

    # calculating the max and min between 9 pixels for every pixel
    max=temp.expand()
    min=temp.inpand()
    
    # low-pass filtering max and min to avoid aliasing
    conv=temp.dEdgeMask(0,255,0,255,"1 2 1 2 0 2 1 2 1",12)
    th=string(threshold)
    max=yv12lutxy(conv,max,"x 12 * y "+th+" * + 12 "+th+" + /")
    min=yv12lutxy(conv,min,"x 12 * y "+th+" * + 12 "+th+" + /")
    
    # where mask_mod is 0, the original value must be kept
    mask_mod=logic (yv12lutxy(temp,max,"x y > 0 255 ?"),
                   \yv12lutxy(temp,min,"x y < 0 255 ?"),
                   \"and")
    
    # where mask_hi is 0, the pixel must be rounded towards the max
    avg=yv12lutxy(max,min,"x y + 2 /")
    mask_hi=yv12lutxy(temp,avg,"x y > 0 255 ?")
    
    # calculates the values for the rounding towards max or min
    str=string(strength)
    clp_hi=yv12lutxy(temp,max,"y "+str+" * x 32 "+str+" - * + 32 /")
    clp_lo=yv12lutxy(temp,min,"y "+str+" * x 32 "+str+" - * + 32 /")

    # merges the clips obtained
    temp=MaskedMerge    (temp,
                        \maskedMerge(clp_hi,clp_lo,mask_hi),
                        \mask_mod)
    
    clp=clp.MergeLuma(temp)
    clp
}
The strength is the wheight given to the min or max value averaged with the old pixel value. The threshold is how much the aliasing is allowed, the lower is this value and the higher is the low-pass filtering (a simple average filter) performed on the min and max values to avoid the aliasing but also the lower will be the effect of the strength.
The standard value do not produce artifacts but the sharpening is very (VERY) weak. You may try something like:
psharpen(20,6)
to have some more visible effect. anyway, the aliasing problem starts to show. it seems that there's no other choice than oversampling+filtering+subsampling to avoid it...
BTW, I'll make some more experimenting...
ilpippo80 is offline   Reply With Quote
Old 2nd July 2005, 13:26   #17  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
version with the upsampling and downsampling.
the threshold is disabled, as it seems it does not improves the quality and just slows down the filter...

Code:
function pSharpen(clip clp, int "strength", int "threshold", float "ss_x", float "ss_y")
{
    strength = default(strength, 16)
    strength = strength<0? 0 : strength>32? 32 : strength
    threshold = default(threshold, 4)
    threshold = threshold<4? 4 : threshold
    ss_x = default(ss_x, 1.5)
    ss_y = default(ss_y, 1.5)
    
    temp = clp.greyscale()
    
    ox = clp.width
    oy = clp.height
    
    temp =  ss_x!=1.0 || ss_y!=1.0 ?
            \temp.lanczosresize(round(ox*ss_x/8)*8,round(oy*ss_y/8)*8) :
            \temp
    
    # calculating the max and min between 9 pixels for every pixel
    max = temp.expand()
    min = temp.inpand()
    
    # low-pass filtering max and min to avoid aliasing
    #conv = temp.dEdgeMask(0,255,0,255,"1 2 1 2 0 2 1 2 1",12)
    #th = string(threshold)
    #max = yv12lutxy(conv,max,"x 12 * y "+th+" * + 12 "+th+" + /")
    #min = yv12lutxy(conv,min,"x 12 * y "+th+" * + 12 "+th+" + /")
    
    # where mask_mod is 0, the original value must be kept
    #mask_mod = logic(yv12lutxy(temp,max,"x y > 0 255 ?"),
    #                \yv12lutxy(temp,min,"x y < 0 255 ?"),
    #                \"and")
    
    # where mask_hi is 0, the pixel must be rounded towards the max
    avg = yv12lutxy(max,min,"x y + 2 /")
    mask_hi = yv12lutxy(temp,avg,"x y > 0 255 ?")
    
    # calculates the values pushed towards min or max
    str = string(strength)
    clp_hi = yv12lutxy(temp,max,"y "+str+" * x 32 "+str+" - * + 32 /")
    clp_lo = yv12lutxy(temp,min,"y "+str+" * x 32 "+str+" - * + 32 /")
    clp_flt = maskedMerge(clp_hi,clp_lo,mask_hi)
    
    # low-pass filtering the adjusted values to avoid the aliasing (?)
    #conv = temp.dEdgeMask(0,255,0,255,"1 2 1 2 0 2 1 2 1",12)
    #th = string(threshold)
    #clp_flt = yv12lutxy(conv,clp_flt,"x 12 * y "+th+" * + 12 "+th+" + /")
    
    # merges the clips obtained
    #temp = MaskedMerge(temp, clp_flt, mask_mod)
    temp = clp_flt
    
    temp = ss_x!=1.0 || ss_y!=1.0 ? temp.lanczosresize(ox,oy) : temp
    
    return clp.MergeLuma(temp)
}
Edit: I've added the return in the end, it's more elegant...

Last edited by ilpippo80; 2nd July 2005 at 14:26.
ilpippo80 is offline   Reply With Quote
Old 2nd July 2005, 13:45   #18  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by ilpippo80
Code:
function pSharpen
{
    [...]
    
    temp = clp.greyscale()
    
    [...]
    
    clp = clp.MergeLuma(temp)
    clp
}
ermh ...
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 2nd July 2005, 13:50   #19  |  Link
scharfis_brain
brainless
 
scharfis_brain's Avatar
 
Join Date: Mar 2003
Location: Germany
Posts: 3,653
what is wrong with that, Didèe?


temp = clp.greyscale()

return clp.MergeLuma(temp)


equals to:

temp = clp.greyscale()

return temp.Mergechroma(clp)
__________________
Don't forget the 'c'!

Don't PM me for technical support, please.
scharfis_brain is offline   Reply With Quote
Old 2nd July 2005, 15:45   #20  |  Link
ilpippo80
Registered User
 
Join Date: May 2005
Location: Neverland
Posts: 79
@Didée
agree with scharfis_brain, it should work correctly...

anyway, I've edited the last lines, the previous version:
clp=clp.MergeLuma(temp)
clp
was a bit awful to see... much better:
return clp.MergeLuma(temp)

has anybody tried the filter? what do you think about it?
it's quite slow, but it should not be a problem to use the same algorithm to write a plugin... the only thing I don't like is that resizing, I have to find some other anti-aliasing method...
ilpippo80 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.