How to do thresholded sharpening?

[Didée]

Searching for ideas ...

As part for some special image enhancement, I'd need to perform thresholded sharpening - but have no (good) idea how to achieve that.

In theory, it's simple: In the very same way like a thresholded softener considers only pixels within the specified threshold for smoothing, the sharpening equivalent should only consider pixels within the threshold for sharpening.

But AFAIK there is no such filter around. (Or did I miss one?)

What one can do is to go the way mf went in his "UnSmooth" script: apply a thresholded smoother, calculate the difference to the original, multiply that difference with the original.
This basically works - but this way too much precision is lost: after the final multiplication, the clip has only half of the dynamics left. That means, instead of a max. of 255 different Y tonalities (0-255, for 16~235 it's 220), there are only max. 128 (110) different Y tonalities left, since the multiplication brings the whole clip down to roughly 0~128 range.

That's not good.

Next thing I could try is splitting the difference image up into the <128 and >128 part, perform two different operation chains, and merge back afterwards ...
But without even trying this, it is clear that things will get slow to a crawl - for an operation that seems so easy! That can't be the right way.

Does someone see another way to do such an operation?


- Didée

[Guest]

Your goal is not well defined but here's an idea.

For each 3x3 area, calculate some function of pixel distribution. Possible functions:

* difference between brightest and lightest pixel
* standard deviation of brightness

The function is trying to assess the "sharpness" of the area. Then if the metric falls below your threshold it is not sharp enough and standard sharpening should be applied to all the pixels of the area. You can try different area sizes too.

This will probably produce something like a thresholded version of windowed histogram equalization (see my VirtualDub filter).

[Soulhunter]

Originally posted by Didée
But without even trying this, it is clear that things will get slow to a crawl...Do you really care about speed... :rolleyes:

Anyway, nice to see you working on this stuff !!!


Bye

[AS]

Nice idea. I suppose if a video/picture has real soft features which were intended, this thresholded sharpener wouldn't work as well because it sharpens 'unproportionally' with respect to the whole picture compare to the traditional sharpener?

[Mug Funky]

i could easily reverse my denoiser script - make it overlay the sharp bits instead of the soft bits.

should run ~ 8fps on a decent machine with a decent frame size

we're talking about an unsharp mask technique aren't we? that's what i understood from your post.

[Didée]

In order of responses:



neuron2:

A better definition will follow ;)

Your suggestions make much sense, of course. I could even think of much more things to do in that manner -
if I could only access and work with single pixels. But such things like "examine the pixels in neighborhood of the current one, then do something with the result" are hardly possible to do through an AviSynth script :( - exept for doing convolutions. But using convolutions only is much too limited to do the nifty stuff.

I should go ahead and learn some programming language, yeah, I know.


Soulhunter:

Believe it or not - I do care about speed! It's just so that my understanding of "acceptable speed" differs a little from the common sense. My goal is not to do half a dozen of mediocre rips per day. I'm aiming to do one or two pearls per week. ;)


AS:

Yep, you got the idea, and the potential problem as well. But I would take care of that problem, of course.


Mug Funky:

Ahm, no, it's not really about unsharp masking, although some very similar operation is involved, amongst others.

And BTW, would you please stop with your denoising script(s) :D
You are getting dangerously close to my not-yet published script WickedDenoiser() - it's waiting for nothing more than a little polishing, and it's waiting for some months now ...

[Didée]

Okay, let's spoil the fun then.

The idea is about a more "real" image sharpening. The traditional sharping techniques are not really *sharpening* any image features, but do only enhance the per-pixel-contrast of each pixel in relation to its neighbors. This enhances the perceptual sharpness very well - but it is prone to produce artefacts, enhances noise, badly hurts the clip's compressability, etc. pp.
"Real" sharpening is more similar to "gradient thinning", if one can say so.

I'm not sure if the folowing concept would work out in practice, since too many steps are out of grip yet.


However, the basic procedure I'm thinking of is as follows:


A. The easy version

Perform some sort of traditional sharpening, PLUS: Make sure that the sharpened pixel does not exceed the highest and the lowest pixel of the (not sharpened) neighborhood.
This should avoid most of the usual "oversharpening" artefacts. For noisy clips, it might even be a good idea to do some blurring before checking for the highest and lowest pixel in the neighborhood.

Though this method seems quite easy, I already fail to see how to do it through a script: no loops, no chance to access and evaluate single pixels.



B. The complicated version

Preambles:

- a single pixel solely does never contain important information on its own. Even the sharpest detail to be found in any sources should be present in some anti-aliased form, otherwise the source would not appear smooth.

Furthermore: If we look at the clip on the pixel-level, we will find:

- (flat areas are flat areas, except for noise)

- in detail areas, every pixel belongs to either side of the represented detail. Aha!

Okay then ...


1.
Subtract a blurred version of the clip from the plain input clip. This gives us a mask, where all "bright sides" of detail will become < 127 (they were darkend by the blurring), and all "dark sides" of detail will become > 127 (they were brightened by the blurring).

So, when making two binarize()'s of that subtraction, we get two masks, where one covers the "bright side" of all the image's detail, and the other mask covers all "dark sides".

Having these two masks, different operations become possible:

2.
Make two clips, where each will contain only either the bright-side or dark-side of detail, the rest is made black by the mask.
Then, apply a thresholded smoother on these two "half" clips.
As result, the edges of detail will be smoothed "along the edge", without the danger of accidentially blurring *across* the edge. This will straighten out the detail borders, and - apart from perhaps being a good filter on its own - will be a good preparation for the next step:

3.
Do either of the following:

3a.
- Inflate or expand the clip that contains only the bright side of detail.
- Deflate or Inpand the clip that contains the dark side of detail only.
- Merge both halves back.

3b.
- Apply the thresholded sharpening I mentioned in my initial post.
- Afterwards, merge both halves of the clip back together.


These operations should make the gradients at the dark-bright-transition of detail thinner, speak: sharper.


One could think of much more operations that could be done. I like especially the idea of breaking the clip up into the two "half" clips that contain only the dark/bright side of detail, and processing that. Just because it totally avoids the danger of doing any processing "across" the border of edges/detail.

All methods would probably benefit from supersampling, to get some sort of subpel accuracy, and to make sure we get a smooth result.


Uff - much text here, without any pictures ...

Any thoughts?


- Didée


Lession of the day: "Try to speak in full sentences" ;)

[mf]

I only got a little more than half of that, Didee.
Warning to other thread readers: --GENIUS AT WORK-- Didee is on to something here :D.

[Mug Funky]

i heed your warning, mf. i can't keep it all in my head at any one time... my brain's becoming interlaced (like scharfi's avatar, i guess)

didee: sorry about posting my scripts :rolleyes:
i'm sure yours will treat edges better.

[edit]

hmm, i'm playing with the idea as i understand it (which is very little), and i've hit a conundrum of sorts: how does one smooth each detail part separately without creating false edges from the masking used? i have overlayed the original clip onto blank grey using the light or dark detail as a mask, but it occurs to me that when this is smoothed, there will be false edges created across the clip and the grey (or whatever colour is used... haven't got far enough to decide what's best here).

just throwing thoughts out there...

[Didée]

Mug Funky -

I hope you reckognized the joke as a joke! Your scripts are really good, fast, and effective. Mine are not bad, ultra-slow (mfps - hihi, still laughing about that, mf!), and often causing more confusion than providing any help. ;)
I'll stick to that style, though :D

-Didée


[edit]

Try it this way:

overlay the parts >128 resp. <127 with black. Then one could use e.g. convolution3d(1,15,15,0,0,0,0) as a spatial smoother: assuming we have a YUV clip within 16~235, the plack parts will not be considered from C3D for the smoothing.

Would require drawing a lot of pictures to visualize all of that ...

[Mug Funky]

of course i did :) i forgot to put "LMAO" in my post... sorry :)

i'm currently busting my gulliver trying to get your idea into a script. there's merit in it, though i've only got 0.3465 of your idea actually scripted. i've got to sort out this masking thing.

[edit]

here's my cobbled together script... not fully done of course, but gives some extra sharpness already, not unlike an unsharp-mask.

it's coded the standard way - changing bits and whacking f5 in VirtualDubMod.


function dideeSharp(clip clip, int "rad")
{

rad=default(rad,3)
blursome=clip.gauss(rad,conv=true)

detailmask=subtract(clip,blursome)
highmask=detailmask.levels(129,1,255,0,255,coring=false).binarize().greyscale()
lowmask=detailmask.levels(0,1,129,255,0,coring=false).binarize().greyscale()

high=clip.expand(Y=3,U=2,V=2).gauss(3,conv=true)
low=clip.inpand(Y=3,U=2,V=2).gauss(3,conv=true)

sharp=overlay(high,low,mask=highmask)

fullmask=overlay(highmask,lowmask,opacity=.5).levels(254,1,255,255,0,coring=false)
overlay(clip,sharp,mask=fullmask)

}


[edit edit]

ooops.. that doesn't stand up well at all. i've done something wrong.

[edit 3]

updated above to something that works... kinda. it aliases like a beyatch.

[edit 4]

simplified it, got less aliasing. vary value of "rad" to experiment

[edit 5]

made it into a function... dideesharp(clip clip,int rad)
default 3

[Dreassica]

I get an error stating theres no such function as "gauss"

[Mug Funky]

ah, sorry. that's my blurrer script. you need masktools to make it work as well (oh what a tangled web we weave..)

put this in with it:


function gauss (clip c,int "radius",bool "conv",int "precision") {

radius=default(radius,4)
conv=default(conv,false)
precision=default(precision,8)
mul=int(pow(2,precision))

Function siney(string s,int stop,int len,int mul)
{
eqn = string(round(mul*pow(sin(pi*stop/len),2)))
return (stop == 0) ? s : siney(s,stop-1,len,mul) + " " + eqn
}

matrix=siney("",radius*2,radius*2,mul)
matrix=matrix.midstr(2,(matrix.strlen()-3))

(conv==false)?c.bilinearresize(4*(c.width/(radius*2)),4*(c.height/(radius*2)))
\.bicubicresize(c.width,c.height,1,0):
\c.yv12convolution(horizontal=matrix,vertical=matrix,Y=3,U=3,V=3,usemmx=true)

}


[edit]

code changed veeeery slightly to allow for manao's new versions of masktools... if nothing happens when trying my scripts, replace your "gauss" with the above (i use this script a LOT)...

[Mug Funky]

okay, i feel like i've hijacked this thread somewhat, but i'm on a bit of a roll...

here's a couple pictures - downsizing from a blurry 720x480 to a sharp 512x288 (sizes are kinda arbitrary, mainly for purposes of demo'ing this nicely).

with didee's sharpening (as i understand it...):

http://www.sharemation.com/mugfunky/chi_dideesharp.jpg

without (straight resize):

http://www.sharemation.com/mugfunky/chi_no_dideesharp.jpg

my host is being finicky at the mome, but i think that's a problem at my end.

[Soulhunter]

Originally posted by Mug Funky
my host is being finicky at the mome, but i think that's a problem at my end. Dont know what finicky means, but if you have problems to upload pics go here... (http://www.imageshack.us/) ;)

Btw, this demo looks already quite promising !!!

But, could you add a non anime sample too ???


Bye

[Mug Funky]

okay..

gotta feel sorry for the poor bugger in these pics :cool:

with:
http://www.sharemation.com/mugfunky/me_dideesharp.jpg

without:
http://www.sharemation.com/mugfunky/me_no_dideesharp.jpg

please remember this is a proof of concept, and having not heard back from didee, i'm not even sure it's the right concept...
also, the sharpen is applied before resizing, so there's a fair amount of stair-stepping going on.

right now it works best supersampled, but unfortunately this makes it run at mfps...

[mf]

Mug, you look like my ex-classmate stoner friend Guus! (Yes, that's a compliment)

[Didée]

Back on track.

Mug Funky, your script looks basically right (must do some examination on the pixel-level though, to be sure what exactly is going on). Same goes for the aliasing, that's indeed a problem. Supersampling gets rid of it, but is sooo slow ...

But I'd say, *if* the concept turns out useful, *and* someone made a plugin of it, then it could run much faster.

Myself, I didn't proceed with the "complicated" version yesterday.
Instead, I implemented the "easy" version: limited sharpening.
This works quite good, too. But it also needs supersampling, otherwise aliasing on near-to-vertical and near-to-horizontal edges may occur.

For the moment no function, only a linear script:

ss_x = 1.0 # 1.0 - fast, 1.5 - fair, 2.0 - good, 3.0 - fine
ss_y = 1.0 #
overshoot = 1 # allowed amount of oversharpening
radius = 2 # blur radius for unsharpmask
strength = 255 # strength for unsharpmask. May go up to 4096 and above, technically
useedgemask = false
compare =false

ox = last.width
oy = last.height

orig0=last

# recommended for noisy clips!
#converttoyuy2().pixiedust(2).converttoyv12()

lanczosresize( int(ox*ss_x/16+.5)*16, int(416*ss_y/16+.5)*16 )

orig=last

edge = overlay(
\ DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=2)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=2)
\ ,mode="lighten").greyscale.levels(0,1.0,160,0,255,false)

#bright_limit = orig.expand() # normal
#dark_limit = orig.inpand() # operation
bright_limit = orig.blur(1.0).expand() # less
dark_limit = orig.blur(1.0).inpand() # aliasing

plain = orig.unsharpmask(strength,radius,0)

too_bright = YV12subtract(
\ overlay(plain,bright_limit,mode="lighten")
\ ,bright_limit
\ ).YV12LUT(Yexpr="x 128 - abs")
\ .binarize(overshoot,upper=false)

too_dark = YV12subtract(
\ overlay(plain,dark_limit,mode="darken")
\ ,dark_limit
\ ).YV12LUT(Yexpr="x 128 - abs")
\ .binarize(overshoot,upper=false)

MaskedMerge(plain,bright_limit, too_bright, Y=3,U=1,V=1, useMMX=true)
MaskedMerge(last,dark_limit, too_dark, Y=3,U=1,V=1, useMMX=true)

useedgemask == true ? MaskedMerge(orig,last, edge, Y=3,U=1,V=1) : NOP
lanczosresize(ox,oy)

compare == true ? stackvertical( last,orig0) : NOP

return last
The "parameters" should speak for themselves. "UseEdgemask" might be worth a try for animee together with radius = 5|7. Without supersampling, a radius of 2 is okay, with supersampling, take a radius of 3.
Through "overshoot" a small amount of oversharpening may be allowed. "0" is stringent, no oversharpening will take place. "1" or "2" is fine especially when the limiting clips are blurred (the un/commented part in the middle of the script). With much bigger values for overshoot, it will act like normal sharpening, but very much slower.

Needed are MaskTools >= 1.4.15.2, and the japanese WarpSharp package, providing the UnsharpMask function. I found that a limited UnsharpMask seems to be much more effective than to limit a normal sharpen() - but try on your own.

For the following test of Soulhunter's link, I took LOTR-TTT, resized to 640*272 with soft bicubic, and then sharpened that with above script.

Normal:
http://img10.imageshack.us/img10/2090/gollum_plain.jpg

LimitedSharpen:
http://img9.imageshack.us/img9/2241/gollum_limitedsharpen.jpg

It gave me also very nice results on some DVB captures that come along very soft. But these are on another PC, can't show them now.

Keep on the experimenting!

- Didée

[Soulhunter]

Looks good !!!

How much was the speed drop in your example ???


Bye

[Didée]

On my poor old XP1800+, the above script renders a 720*432 clip at ~8 fps.

As soon as supersampling gets used, speed drops noticeably: ~4 fps for SS=1.5, 2~2.5 fps for SS=2.0

That is a general problem that can not at all be avoided: Supersampling is *always* slow, and will *ever* be.
One must understand that, say supersampling to 2* original size will give you 4* more pixels that are to process.
Furthermore, and even more bad, if a script is using (several) masking and/or plane copying operations, then the internal amount of data to process is simple HUGE.

For these very reasons, things should be implemented as plugins, if anyhow possible. There you can do the same operations with an overhead *magnitudes* lower: simply look at how fast Sh0dan's MipSmooth has become, compared to the very first versions of it.

The point is: in a script, we use masking and plane copying operations because its the only way to do things, but it is not an optimal way, at all.

Example: say we want to blur a clip, but not the edges.

script:

<clip>
<clip - edgemask>
<clip - blurred>
copy plane(s) <clip> <blurclip> by <edgemask>

plugin:

<clip>
for each <pixel>:
-- IsEdge ? <blur> : <DoNothing>


Very strange pseudo-code, but you get the idea. The script does X-times the work that a plugin would do:
What *has* to be done is the comparing of pixels for the edgemasking, and the blurring itself. In a plugin, you can do all of that in one loop, more or less (I guess).
In a script, every operation is discrete, consumes memory and overhead not necessarily needed, - and in the end, you have to copy pixels from one plane to another, by comparing with the pixels from the edgemask to decide what shall be copied, and what not.

The same is (even more) true for supersampling: The memory consumtion of the script gets more and more bad, because the framesizes go up in a quadratic manner - and you need two additional clips for the given example. All operations get slower because of the amount of data, and so will the plane copying.
What comes into play here also, is the fact that we're probably overload the internal cache memory, get cache misses, and everything slows down even more.
In a plugin, it were possible to do the supersampling in small windowed blocks, thus avoiding most of these memory/cache limitations.

Sorry if I produced some technical inaccurcies here - since I am NOT a programmer, I don't know enough details about all this to be precise. But in general, the above should be somewhat right.

Final note: It will get at least a little faster. Manao just posted a new version of the MaskTools, along with some tips ... lets see how much improvement is possible.

But not now - I have to leave for a birthday party.


- Didée


edit: clarifications

[Soulhunter]

Originally posted by Didée
On my poor old XP1800+, the above script renders a 720*432 clip at ~8 fps.Nah, meant in contrast to the "unfiltered" equivalent... ;)

Originally posted by Didée
Very strange pseudo-code, but you get the idea.
Yes, much easier to understand than some AviSynth functions... :D


Some sort of "intelligent" GC supersampling would be cool !!!

So that only the "edges" get supersampled (dreaming)... :rolleyes:


Bye

[SoonUDie]

Uh, forgive me for possible ignorance here, but how is this different from the concepts of Msharpen(threshold, strength) or XSharpen(strength, threshold)?

Personally, I've been toying with a very simple (in concept) thresholding sharpen filter for a while. I simply use a dEdgeMask (from mask tools) to find the edges (threshold is controlled by the divisor setting), and overlay a sharpened version of the input clip on top of a smoothed version. The results have been better than MSharpen (but only barely so) for quality, but the advantage is being able to smooth things internally. Since it's not an optimized pluging, however, it's much slower!

Her are some comparisons (mine was made using the "original" images and ImageReader):
Orig:
http://www.sharemation.com/mugfunky/chi_no_dideesharp.jpg

DideeSharp:
http://www.sharemation.com/mugfunky/chi_dideesharp.jpg

Mine:
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/Soon_chi.jpg
SoonSharp(8,"-2 7 -2")

Orig:
http://img10.imageshack.us/img10/2090/gollum_plain.jpg

DideeSharp:
http://img9.imageshack.us/img9/2241/gollum_limitedsharpen.jpg

Mine:
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/Soon_gollum.jpg
SoonSharp(5)


EDIT: I will re-optimize this shortly

Function (requires masktools):

function SoonSharp (clip "c", int "domain", string "sMatrix")
{
domain = default(domain, 7)
#domain = edge detection threshold. Lower values = more edges.

sMatrix = default(smatrix, "-1 4 -1")
#sMatrix = sharpening matrix. Edges always negative; should have positive total.
#The closer the middle is to the sides, the more sharpening will take place.

edges = c.YV12Convolution(
\ "1 4 1", "1 4 1", 6, true, Y=3, U=1, V=1).YV12Convolution(
\ sMatrix, sMatrix, automatic=true, usemmx=true, Y=3, U=1, V=1)

emask = c.DEdgeMask(3, 3, 255, 255, matrix="1 1 1 1 -8 1 1 1 1", Y=3, U=1, V=1,
\ setdivisor=true, divisor=domain, usemmx=true).YV12Convolution(
\ "1 2 1", "1 2 1", 6, true, Y=3, U=1, V=1).YV12LUT(
\ "x 20 - 200 20 - / 1 1 / ^ 255 0 - * 0 +", "x", "x").Undot()


#use a spatial smoother on "c" to smooth non-edge area

smooth = c

#You can also process the edges further. I suggest using a spatial smoother to
#prevent haloing from the sharpening. However, if you have a very clean source,
#you may not need one. Another good idea would be something like WarpSharp().

edges = edges.deen("a2d", 2, 10, 10)


return MaskedMerge(edges, smooth, emask.Invert(), Y=3, U=2, V=2, usemmx=true)
}


What do you think? I like the fact that didee's sharpener works somewhat like xsharpen (and can look really good with super-sampling!), but at the same time I appreciate the contrast enhancement of mine. It would be great if someone chould do a version that has SSXsharpen-like "detail" and good contrast without haloing :)

[Mug Funky]

hmm... from the looks of those images (haven't checked the code yet, i'm incredibly badly hung over, it's a miracle i can type at all) your method works well as a regular sharpener by didee's definition (increasing contrast on a per pixel level), but there are subtle differences in didee's version.

if you look at the texture on Gollum's skin, you'll see there's more "crispness" in didee's version, and also there's none of the ringing artefacts one normally encounters with regular sharpeners. as i understand unsharp-masking and convolution based approaches, they are rather like turning up the treble knob on a stereo - there's nothing new created, but the high frequencies are just amped up more.

i can't think of a similar analogy for didee's method unfortunately... it doesn't create anything new, but moves edges closer together rather than just making them brighter. of course, no new detail is being created (it simply gives the blur an edge to speak of).

i wont be doing any scripting tonight... too ill. but if i get a sudden flash of inspiration i'll get it into code ASAP.

[edit]
there's noticably more ringing on Chii as well... that's okay. your method doesn't hurt compressibility as much as mine - just look at the size difference of those jpegs which were compressed at the same "quality"

[edit edit]

i've hacked didee's script into a funktion, and tried it out a tad (i found that i didn't have an unsharp-mask filter installed, so i wrote one that performs the same way)

it works quite well, and thankfully faster than mine (less aliasing too) :)

i've modded it slightly - radius scales with supersampling now.

[Didée]

Mug Funky,

regarding the fact that you're actually half-dead, you observations (and posting as well) are still very precise.

Wish you a speedy recovery :D

Still had no time to dive into your script: sorry. But currently I've so much correspodence to work out, I can't script even one single line.

Lord, give us more time ;)


SoonUDie:

It's just as Mug Funk said. Your way of sharpening is good and OK, but it's still the old story of traditional sharpening, along with the danger of the usual artefacting through over-sharpening and per-pixel contrast enhancement.

MSharpen is also "only" a traditional sharpener, that is restricted to process only sharp edges, to achieve a sharpening of image features (by enhancement), but without amplifying the noise.

My usual strategy OTOH, that I especially follow in within the iiP script, is the exact contrary: Leave the already-sharp edges alone, and sharpen only the weak features. The aim is to enhance small/weak detail, that otherwise would get lost in the encoding process. But it also requires some rather good pre-denoising of the source. (My clear favourite for that is PixieDust).

Regarding the concept that I proposed in this thread:

Well, at first there was the plain idea about how to process edges. The thinking about what the effect really is about came later ;)

In fact, it is somewhat similar to XSharpen and WarpSharp, a little closer to XSharpen than to WarpSharp.

For what YOU want to achive - contrast enhancement without haloing - that is a difficult task. It can be done, but regardless of how it is done, it requires to do spatial searches + evaluations with very large radii, and thus will be VERY slow also.

CruNcher posted a link to a filter that does exactly that: Browse through Soulhunters XviD-thread about Reloaded HQ-encoding.


- Didée

[Didée]

Help! A little math problem, and right now I can't see the right solution:


We have:

- input clip
- a clip "upper_limit" (U)
- a clip "lower limit" (L)
- a clip "normal sharpened"


We construct:

- (A) difference clips "normal sharpened" from both "_limit" clips
- (B) difference clip "input clip" from both "_limit" clips


Wanted:

From these two difference clips, construct a mask by combining (A) and (B) in such a manner that:

- On the location of the peak spot in (A), the result of f{ A, B } = { U && L } (don't exceed neither U nor L)

- Scale the rest somehow down, so that at the spots where {A} == { U && L } we get 128 in the wanted mask (half strength for final copying)

- Scale the rest down so that 100% strenght is achieved


Ouch, my brain starts hurting.

In other words, instead of the "hard limiting" that currently is done in the LimitedSharpen-snipplet, I want to do copy the normal-sharpened frame onto the input, weightened by a mask, so that still no oversharpening takes place.

Erh, isn't this similar to performing a *division* between two clips?

+3 in one clip, -5 in the other. On paper, this results in a factor of -(3/5).

How do I divide two clips, putting the result into a mask?



Forget it - supposely figured it out. Needs a night of sleeping over it, though.


- Didée

[Mug Funky]

@ didee:

i don't remember posting that! :confused:

hmm... your idea seems good, but would require a lot of fancyness...

in the meantime here's your code "snippet" turned into a script rather ad-hocly.


function limitedsharpen(clip clip, float "ss_x", float "ss_y", int "overshoot", int "radius", int "strength", bool "useedgemask", bool "compare")

{

ss_x = default(ss_x,1.0) # 1.0 - fast, 1.5 - fair, 2.0 - good, 3.0 - fine
ss_y = default(ss_y,1.0) #
overshoot = default(overshoot,1) # allowed amount of oversharpening
radius = default(radius,2) # blur radius for unsharpmask
strength = default(strength,256) # strength for unsharpmask. May go up to 4096 and above, technically
useedgemask = default(useedgemask,false)
compare = default(compare,false)

radius = round(radius * (ss_x + ss_y)/2)

ox = clip.width
oy = clip.height

orig0=clip

# recommended for noisy clips!
#converttoyuy2().pixiedust(2).converttoyv12()

clip
lanczosresize( int(ox*ss_x/16+.5)*16, int(416*ss_y/16+.5)*16 )

orig=last

edge = overlay(
\ DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=2)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=2)
\ ,mode="lighten").greyscale.levels(0,1.0,160,0,255,false)

#bright_limit = orig.expand() # normal
#dark_limit = orig.inpand() # operation
bright_limit = orig.blur(1.0).expand() # less
dark_limit = orig.blur(1.0).inpand() # aliasing

plain = orig.unsharpmask(strength,radius,0)

too_bright = YV12subtract(
\ overlay(plain,bright_limit,mode="lighten")
\ ,bright_limit
\ ).YV12LUT(Yexpr="x 128 - abs")
\ .binarize(overshoot,upper=false)

too_dark = YV12subtract(
\ overlay(plain,dark_limit,mode="darken")
\ ,dark_limit
\ ).YV12LUT(Yexpr="x 128 - abs")
\ .binarize(overshoot,upper=false)

MaskedMerge(plain,bright_limit, too_bright, Y=3,U=1,V=1, useMMX=true)
MaskedMerge(last,dark_limit, too_dark, Y=3,U=1,V=1, useMMX=true)

useedgemask == true ? MaskedMerge(orig,last, edge, Y=3,U=1,V=1) : NOP
#lanczosresize(ox,oy)
bicubicresize(ox,oy,0,.5)

compare == true ? stackvertical( last,orig0) : NOP

return last

}

hope this helps

it works faster than my script, and gives less aliasing.
i've modded radius so that it scales up with supersampling (otherwise there's not much sharpening done)

[edit]

one thing to note: this sharpening concept works beautifully for resizing. you can get zero stairstepping, and no awkward blurs associated with kernel based interpolation (bilinear, bicubic, lanczos) if you simply upsize with something sharp (bicubic or lanczos) and then apply this sharpener at an appropriate radius (4 works well for me, but it depends on how much you're upsizing).

[Manao]

Just to show you how to use the MaskTools to get something fast.

MugFunky, in your first script : detailmask=subtract(clip,blursome)
highmask=detailmask.levels(129,1,255,0,255,coring=false).binarize().greyscale()
lowmask=detailmask.levels(0,1,129,255,0,coring=false).binarize().greyscale()

high=clip.expand(Y=3,U=2,V=2).gauss(3,conv=true)
low=clip.inpand(Y=3,U=2,V=2).gauss(3,conv=true)

sharp=overlay(high,low,mask=highmask)

fullmask=overlay(highmask,lowmask,opacity=.5).levels(254,1,255,255,0,coring=false)
overlay(clip,sharp,mask=fullmask)Can be replaced bydetailmask = YV12Subtract(clip, blursome)
highmask = detailmask.YV12LUT(Yexpr = "x 139 < 255 0 ?")
lowmask = detailmask.YV12LUT(Yexpr = "x 119 > 255 0 ?")

high = clip.expand(Y=3,U=2,V=2).gauss(3,conv=true)
low = clip.inpand(Y=3,U=2,V=2).gauss(3,conv=true)

sharp = maskedmerge(high, low, highmask, Y=3,U=1,V=1)

fullmask = detailmask.YV12LUT(Yexpr = "x 119 > x 139 < 255 0 ? 0 ?", Y=3, u=1, V=1)
maskedmerge(sharp, clip, fullmask,y=3,u=1,v=1)
In the last script you posted :edge = overlay(
\ DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=2)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=2)
\ ,mode="lighten").greyscale.levels(0,1.0,160,0,255,false)should be replaced byedge = logic(
\ DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=2)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=2)
\ ,mode="max").YV12LUT(Yexpr = "x 255 160 / *",y=3, u=1, v=1)AndYV12LUT(Yexpr="x 128 - abs").binarize(overshoot,upper=false)byYV12LUT(Yexpr = "x 128 - abs " + string(overshoot) + " <= 0 255 ?", y=3,u=1,v=1)

Edit : I forgot also to mention that the only delimiter for any expression in the masktools ( for example, in YV12Convolution & YV12LUT ) is a single space ( ' ' ), so you'll have to rewrite your gauss function.

[SoonUDie]

Manao, while we're talking about optimizing with MaskTools, which is faster:

Blur(1)
or
YV12Convolution("1 2 1", "1 2 1", y=3, u=1, v=1)
?

I don't think YV12Convolution is ASM optimized, and I don't know if Blur is or not (that could make the difference - YV12Con only processing luma vs. blur processing both with ASM?)

[Manao]

Blur is faster. YV12Convolution isn't optimized for 3x3 matrices, and doesn't use ASM at all. When I chain 10 blur, or 10 YV12Convolution, I get respectively 17 and 4.5 fps.

[Mug Funky]

manao:
I forgot also to mention that the only delimiter for any expression in the masktools ( for example, in YV12Convolution & YV12LUT ) is a single space ( ' ' ), so you'll have to rewrite your gauss function.

i'm not sure i understand... but certainly speeding things up with masktools is a good idea, and i thank you for taking the time to do this (and indeed to write masktools to begin with).

yes, my gauss function is a bit of a hack, and i'd much rather a plugin do that stuff for me (if it works faster than the resize approach, that'd be just perfect, as i'm not a fan of the slight inaccuracies the resize approach brings).

man, i really have to learn some C/C++ some day.

[Manao]

I didn't make myself clear on the delimiter issue : I changed the behavior of YV12Convolution regarding which delimiter could be used in the expression of 'horizontal' and 'vertical'. With the latest version of the MaskTools, your gauss function won't work. I'm the culprit here ( I forgot to mention the change ).

However, concerning the pluginization of gauss, you would not gain much ( if not nothing at all ). But you should accept the 'slight' innacuracies of the resize approach, because it should not matter for the final result.

BTW, my rewritting of your first script doesn't give exactly the same output : the overlay & maskedmerge behavior differs slightly, though I don't know why yet ( formulas in the source code are the same ).

[Mug Funky]

aaah. i understood when i installed te new masktools just now.

just a matter of changing "," to " " in gauss... all peachy now.

[edit]

i'll change it in my post earlier in this thread a well, so people using my stuff wont be disappointed at the lack of any difference :)

[edit edit]

had to tweak your changes slightly to get what i want, but here's an improved version of my first script.

WARNING: this WILL introduce aliasing - no doubt about it, but supersampling helps it, and there's an option for it (default none for speed).


function dideeSharp(clip c, int "rad", int "damp", float "ss")
{

ss=default(ss,1.0)
rad=default(rad,3)
damp=default(damp,0)

rad = round(rad*ss)

clip = (ss > 1)? c.bicubicresize(round(c.width*ss),round(c.height*ss),0,.5) : c

blursome=clip.gauss(rad,conv=true)

detailmask = YV12Subtract(clip, blursome)
highmask = detailmask.YV12LUT(Yexpr = "x 128 < 255 0 ?")
lowmask = detailmask.YV12LUT(Yexpr = "x 128 > 255 0 ?")

high = (damp > 0)? clip.gauss(damp,conv=true).expand(Y=3,U=2,V=2) : clip.expand(Y=3,U=2,V=2)
low = (damp > 0)? clip.gauss(damp,conv=true).inpand(Y=3,U=2,V=2) : clip.inpand(Y=3,U=2,V=2)

sharp = maskedmerge(high, low, highmask, Y=3,U=1,V=1)

fullmask=detailmask.YV12LUT(Yexpr = "x 128 - abs").binarize(1,upper=true)
maskedmerge(sharp, clip, fullmask,y=3,u=1,v=1)

(ss > 1)? last.bicubicresize(c.width,c.height,0,.5) : last

}

[SoonUDie]

Bah, you beat me to it Mug Funky. But I seem to be having problems with yours.

Chi is sad at default settings :( :
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/sad_chi.jpg

The problem seems to go away at ss=2, damp=1 :) :
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/ok_chi.jpg
but now it's a bit blurry? :confused:


Anyway, here's my version (default settings):
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/happy_chi.jpg

You can use the darken parameter to change how dark the lines are.
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/dark_chi.jpg
dstr = 0.3 :devil:

The previous 2 shots at ss=4 (slow):
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/happy_chi_ss4.jpg
http://students.oxy.edu/tcoldwell/forum_uploads/Doom9/Sharpen/dark_chi_ss4.jpg

It would be nice if someone could figure out a way to anti-alias this without having to do so much supersampling...

The function:

function SSdideeSharp(clip clip, int "rad", bool "darken", float "dstr", int "ss")
{
ox = clip.width()
oy = clip.height()
xm16 = int(((ox/16)+.5)*16)
ym16 = int(((oy/16)+.5)*16)
darken = default(darken,true)
dstr = default(dstr,0.9)
ss = default(ss,2)

rad=default(rad,3)
blursome=clip.gauss(rad,conv=true)

detailmask = YV12Subtract(clip, blursome, y=3,u=1,v=1)

highmask=detailmask.levels(129,1,255,0,255,coring=false).BicubicResize(xm16*ss, ym16*ss).binarize().greyscale()
lowmask=detailmask.levels(0,1,129,255,0,coring=false).BicubicResize(xm16*ss, ym16*ss).binarize().greyscale()

high=clip.expand(Y=3,U=2,V=2).gauss(3,conv=true).BicubicResize(xm16*ss, ym16*ss,0,0.5)
low=clip.inpand(Y=3,U=2,V=2).gauss(3,conv=true)
low= (darken) ? low.levels(15,dstr,255,0,255) : low
low = low.BicubicResize(xm16*ss, ym16*ss,0,0.5)

sharp = MaskedMerge(high, low, highmask,y=3,u=1,v=1,usemmx=true)

fullmask=YV12layer(highmask,lowmask,"add",level=128, y=3,u=1,v=1).levels(254,1,255,255,0,coring=false)

MaskedMerge(clip.BicubicResize(xm16*ss, ym16*ss, 0, 0.5),sharp,fullmask, y=3,u=1,v=1,usemmx=true).BicubicResize(ox,oy, 0, 0.5)

}

[Mug Funky]

regarding that first picture: update to the latest masktools, then update to the "gauss" function on the last page. syntax has changed, leading to edge removal rather than edge enhancement :)

isn't chii kawaii?

[SoonUDie]

Originally posted by Mug Funky
regarding that first picture: update to the latest masktools, then update to the "gauss" function on the last page. syntax has changed, leading to edge removal rather than edge enhancement :)

isn't chii kawaii?
1. EDIT: Actually, I am using the new version.
2. Yes, yes she is.

[Mug Funky]

edited my script above, fixing a stupid bug (dampen was being ignored, using a radius of 3 at all times. now it uses whatever is set for damp)

i don't think your script will run any differently with the new syntax = it only seemed to affect YV12convolution - matrices are now "1 1 1" instead of the old "1,1,1"

i don't see any of that in your script, so there should be no probs updating masktools.

@ mods: this thread is mostly scripts... should it be moved to usage?

[SoonUDie]

Re: Masktools:
I thought I was using the old version, but I wasn't.

Re: your script:
Still having the same problem. Can you reproduce it? I also have a similar (but opposite) problem with didee's function (edge bloating).

[Mug Funky]

hmmm... interesting that you're still getting that prob. sure you updated "gauss"? i ask because if gauss is feeding YV12convolution a matrix with commas in it, then there'll be no blurring - just an identical clip.

under these conditions, no masking will take place, and so the expanded clip will simply be overlayed onto the inpanded clip with no masking, hence the lack of outlines.

anyhoo, you wanted a line-darken that didn't need supersampling? here's one:

function lineDarken (clip c, float "strength")
{
strength = default(strength,4)
eq = string("x 130 - " + string(strength) + " *")
outline = YV12subtract(c.gauss(4),c).YV12lut(Yexpr=eq).invert()
yv12layer(c,outline,op="mul")

}


you need updated gauss for this one too, but it'll work without it, as it doesn't use YV12convolution by default like the sharpener does (will change this in a minute)

[Didée]

Originally posted by SoonUDie
It would be nice if someone could figure out a way to anti-alias this without having to do so much supersampling...

Well, that's what I want to try with the above posted fiddling-together of the different masks: to obtain a final mask that requires none, or at least less, supersampling.

But I can't start before today evening ... the possible results are as follows:

- it won't work at all
- it will work, but the wicked creation of that mask requires more time than simple supersampling
- it indeed works out

OTOH: the more I think about it, I conclude that the whole process would be better described as edge re-sampling, instead of sharpening.

Seen under this aspect, I fear that supersampling will not be avoidable. Without going into lenghty detail, such re-sampling simply needs sub-pixel accuracy.

For example, those of you that have worked with ray-tracers, might know that these generally need some option like "subsampling" activated to give a smooth representation of image borders. And technically, subsampling or supersampling are (almost) the very same thing.

But it's really fun to play with all of this ;)

- Didée

[AS]

I think you can get a good edge re-sampler out of sangnom if you somehow managed to avoid the artifacts while using high aa values. Unfortunately I don't have a clue how to go about it.

[SoonUDie]

@Mug Funky
What I actually meant was a way to antialias the processed lines which are being overlayed onto the clip; my version of the script already has darkening built in (and it ends up looking better because it doesn't create dark halos). Thanks anyway, though; I'm definately saving that.

@Didée
You've probably already seen this, but if you haven't, take a look at the "resize" section of this page (from the SAR image processor manual):
http://www.general-cathexis.com/manual2/index.html

[Mug Funky]

well, soft thresholding is no help AFAIK - i tried it with my script, and it basically was too soft, producing almost the same as the input, but at mfps.

if this were in plugin form, there could be hope for speed - as you said with raytracers, they need supersampling. however, a raytracer doesn't know what it's nyquist is, whereas we know exactly what it is as we're working with source images of fixed size.

therefore a smart enough plugin will only supersample (subsample) the needed parts. this could easily be done using the detail mask (in my script it's called "fullmask" and is binary) as a guide for what pixels to subsample.

this would be a HELL of a lot faster for exactly the same result.

[mf]

What the hell! You guys are researching line darkening in a way I haven't before? Wait for me!!

Edit: Hmm. False alarm. This SSDideesharp thing can't cope with my dreaded NARUTO.14.avi, and it's way slower than mfToon with sharpen=false. Pfew, and I thought the revolution was starting without me! :)

[SoonUDie]

Ah, if only we had a MaskedSuperSampleProcess(your functions here) :D

[Mug Funky]

that fractal resizing technique looks really interesting (and kinda similar in output to what we're doing, considering the totally different approaches used).

thanks for the article, soon.

[SoonUDie]

Originally posted by Mug Funky
thanks for the article, soon. No problem, and thanks for calling me Soon. 'Tis my preferred nickname. :D

One thing I've been playing around with recently is encorporating FastEDIUpsizer() into some image processing scripts (seems a bit too slow for general video usage!). It was posted here a while back and I believe it uses the NEDI algorithm. It's not perfect, but it's definately better than Lanczos. It has a fixed usize value of 2x, though, and it doesn't sharpen edges (which is the topic of the thread)...

[Didée]

Does this sound familiar to anyone: you start out to do something particular, but end up in doing something completely different ...

For the moment, I stick to LimitedSharpen, as it is much easier to control (I think).
Yesterday, I didn't manage to do do the intended "smart scaling" of the masks. Instead, now something like "soft thresholding" for the oversharpening problem is implemented.

The previous method of hard limiting through the overshoot parameter may still be used.
New default however is soft thresholding through the parameter power. Overshoot then is simply ignored.


So what's the difference?


With the hard-limiting method, LimitedSharpen was only refining the edges in a frame, but was hardly doing any contrast enhancement. This was the intended behaviour, since we want to avoid oversharpening artefacts as far as possible, especially edge halos.

However, SoonUDie was not satisfied! ;)


Now, with the soft-limited version, there is more room to adjust the sharpening process to personal taste and needs.
The idea is: the more oversharpened a pixel appears to be, the less it will be weightened into the output clip, and vice versa.

The key is the new parameter power:

- power = 1.0 will make the function behave like normal, not-protected sharpening.
- Power = 8.0 is (almost) the same as the old hard-limiting method, just without the possibility to allow any "overshoot".
- therefore, power = 0.0 disables the new method. With power=0.0, one can use the overshoot parameter just as in the old version. Otherwise, overshoot is simply ignored.
- the interesting range for power´ is 1.5 < power < 4.0. One can achieve a pretty fair amount of contrast enhancement, but mostly without visible edge haloing. On the way to "MSU Smart Sharpen" ... not yet there, but on the way ;)

Notes:

- With soft limiting, there *IS* oversharpening and edge-haloing going on! But these effects are somewhat "soft", then ... difficult to put in words: try and see for yourself.

- alas, the soft limiting method is more sensible to noise. The smaller power is, the more noise will be enhanced, too. In cases, it will be a good idea to denoise the source beforehand.


Updated function:

MaskTools >= v1.4.16 is strictly required!

function LimitedSharpen( clip clp, float "ss_x", float "ss_y", int "strength", int "radius",
\ float "overshoot", float "power", bool "soft")

{
ss_x = default( ss_x, 1.5 )
ss_y = default( ss_x, 1.5 )
strength = default( strength, 160 )
radius = default( radius, 2 )
radius = round( radius * (ss_x + ss_y)/2 )
overshoot= default( overshoot, 1 )
power = default( power, 3.0)
soft = default( soft, true )
ox = clp.width
oy = clp.height

ss_x == 1.0 && ss_y == 1.0
\ ? clp : clp.lanczosresize( int(ox*ss_x/16+.5)*16, int(oy*ss_y/16+.5)*16 )

edge = logic( DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=1)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=1)
\ ,"max").YV12LUT(Yexpr = "x 255 160 / *",y=3, u=1, v=1) # levels(0,1.0,160,0,255,false)

bright_limit = (soft == true) ? last.blur(1.0) : last
dark_limit = bright_limit.inpand()
bright_limit = bright_limit.expand()

normsharp = unsharpmask(strength,radius,0)

too_bright = YV12subtract( logic(normsharp,bright_limit,"max")
\ ,bright_limit )

too_bright = (power == 0.0)
\ ? too_bright.YV12LUT(Yexpr="x 128 - abs " +string(overshoot)+" <= 0 255 ?", y=3,u=1,v=1)
\ : too_bright.YV12LUT(Yexpr="x 128 - abs "+string(power)+" ^", y=3,u=1,v=1)

too_dark = YV12subtract( logic(normsharp,dark_limit,"min")
\ ,dark_limit )

too_dark = (power == 0.0)
\ ? too_dark.YV12LUT(Yexpr = "x 128 - abs " + string(overshoot) + " <= 0 255 ?", y=3,u=1,v=1)
\ : too_dark.YV12LUT(Yexpr="x 128 - abs "+string(power)+" ^", y=3,u=1,v=1)

MaskedMerge(normsharp,bright_limit, too_bright, Y=3,U=1,V=1, useMMX=true)
MaskedMerge(last,dark_limit, too_dark, Y=3,U=1,V=1, useMMX=true)

ss_x == 1.0 && ss_y == 1.0 ? NOP : lanczosresize( ox, oy )
return last
}



The unfunny part:
parameter description

ss_x, ss_y [float],[float]
The usual factors for oversampling the clip. Default: 1.5 / 1.5

strength, radius [int],[int]
Strength and radius for the unsharp masking command. Default: 160 / 2

overshoot [int]
How much a pixel is allowed to get oversharpened, i.e. to exceed the highest or lowest of its neighbors.
Default: 1

This parameter is ignored when power != 0.0. By default, overshoot is not used.

power [float]
The power parameter somehow controls the soft limiting - it's hard to explain in a few words. Just try values from 1.5 to 4.0, and see for yourself ;)
8.0 makes the function almost work the same as hard-limiting, but without any "overshoot".
1.0 effectively disables all the smart stuff, giving a plain, unprotected sharpening. Not useful for encoding, but nice to see the difference.
0.0 disables the soft limiting mode. When power is set to 0.0, then the hard-limiting mode will be used, and can be controlled by overshoot

soft [bool]
When soft is set TRUE, then the clip will be softened when searching for the high/low limits.
Intentions:
- in case of a noisy clip, to not get distorted high/low limits, caused by the noise
- perhaps give less aliasing when running with little (or completely without) supersampling

Attention: when supersampling is set to 1.0, then soft=true endangers the loss of some very fine details.

I'm not yet sure how useful the soft parameter is at all.


For the moment, the "UseEdgeMask" function is not implemented, since I feel it doesn't make much sense together with soft thresholding. I'll put it back on request, though ... it's just one line.


Did I forget something? Surely I forgot something. But I can't remember what it was :)


- Didée


edit: linebreaks ...
edit 2: memory kicked in again

[Mug Funky]

cool. i'll try it soonish.

still runs at mfps?

(btw, i've been trying my method on images, but feeding 300dpi scans through avisynth is hell, and supersampling leads to the computer gibbering like GWB in a press conference, then crashing)

[Soulhunter]

Yay, next weekend isn't far away... ;)


Bye

[malkion]

i'm trying to test these various scripts.

on some which has the function gauss, i can not get those to run.
can someone point me in the right direction on what i need.

/edit/

thanks mug. the handy search revealed gauss to me in another thread.
then i reread this thread as well, and hopefully have your latest version of gauss. i believe i found it in 3 other threads as well.

[Didée]

Originally posted by Didée
Did I forget something? Surely I forgot something. But I can't remember what it was :)
Ah, my memory started working again.

For the above posted version of LimitedSharpen(), it is absolutely necessary to use the latest version of the MaskTools! (Currently: v1.4.16)

With a lower version of MaskTools, the function will work without throwing an error, and without having any useful effect ;)
It's because a bug (multiple-instances-bug?) in the Logic() function was fixed in v1.4.16.

BTW, yesterday I applied the following to a DVB capure (eXistenZ):

ConvertToYUY2().PixieDust(2).ConvertToYV12()

LimitedSharpen(ss_x=1.75,ss_y=1.75,strength=127,radius=1,power=2.5,soft=false)Yummy!

- Didée

[Manao]

It's because a bug (multiple-instances-bug?)No it was the stupid copy & paste bug while programming the mode "min" and "max" which was making them totally uneffective ( result = ( src1 < src2 ) ? src1 : src1; ... ).

The funny thing was that the bug wasn't present in the optimized version of the filter, but another bug prevented it from being used ( I love such combo bugs :p )

[Soulhunter]

Ok, here is my first small playaround with LimitedSharpen... ;)


Source:

The Matrix - Lobby Shootout / 3:07 min. @ 25fps / 1024x576 pix.


Settings:

XviD v1 RC4 (Koepis)

Fixed Quant 2 (VHQ1)

I-frame interval = 250

Matrix = Soulhunter v3

BVOP's = 1 / 1 / 1


Results:

No filters (resized only)
Size = 154 MB / Time = 05 min.

SharpResize (defaults)
Size = 162 MB / Time = 65 min.

LimitedSharpen (defaults)
Size = 246 MB / Time = 23 min.

LimitedSharpen (strength, 100)
Size = 215 MB / Time = 23 min.

LimitedSharpen (strength, 50)
Size = 187 MB / Time = 23 min.

LimitedSharpen (radius, 1)
Size = 238 MB / Time = 23 min.

LimitedSharpen (overshoot, 2)
Size = 246 MB / Time = 23 min.

LimitedSharpen (power, 2.0)
Size = 285 MB / Time = 23 min.

LimitedSharpen (soft, false)
Size = 272 MB / Time = 23 min.


Note: For all encodes except SharpResize one, Lanczos was used for the resizing !!!

Note: Resizing was always done after the filtering !!!

Note: I take no responsibility for the trueness of the results !!!


More tests n' comments on the visual impression will follow... ;)


Bye

[Soulhunter]

Ok, visual impression now...

I really hate it to say, but after watching the encodes I was disappointed !!!

Nah, maybe a bit too harsh... :p

It simply does not what I thought it would do !!!

Thought it would sharpen detail/edges without amplifying noise too much... :rolleyes:

But it does amplify noise very much !!!

Yeah sure, de-noising would help with this problem... ;)



Idea:

Sounds maybe a bit dumb, but could you tweak the script to amplify noise even more ???

Coz we could use it in a complete other way then !!!

UnSmooth(Smoother="LimitedSharpen") already gives some sort of (poor) de-noising... :D

Or even a combo with some sort of edge-remover instead of amplifier ???


Bye

[Didée]

Originally posted by Soulhunter
Ok, visual impression now...

I really hate it to say, but after watching the encodes I was disappointed !!!

Nah, maybe a bit too harsh...

It simply does not what I thought it would do !!!Ok, visual impression now...

[/start copy'n'paste]

Ok, let's answer this now...

I really hate it to say, but after reading your comments I was disappointed !!!

Nah, maybe a bit too harsh...

I simply thought my explanations have been clear enough !!! But obviously, they were not !!!

[/end of copy'n'paste] ;)


I really start thinking about doing the following for my 'experimental' scripts:

- If the function is called with default values, this gets detected, and the function will print, in big orange letters,

"ALERT! Usage of default values detected! DONT EVER DO THAT!"

onto the clip ... :D


Okay, enough joking now. Lets get serious.

What you were experiencing, was exactly the expected behaviour, Soulhunter. Well, more or less.
You missed the fact that LimitedSharpen() provides two totally different sharpening effects, and *tries* to also enable a usage somewhere inbetween these two modes.

I'll try it once more:


- power = 0.0

This is the mode that will do edge-condensing only, and nothing else. Like shown in the screenshot of Gollum. ;)


- power >= 1.0 (1.0 <= power <= 8.0)

This mode delivers traditional sharpening, with some additional twists. The effect of edge condensing will be much weaker - it will do normal sharpening/contrast enhancing with as little oversharpening (edge halos) as possible. But this has nothing to do with noise! Noise will be enhanced, too, just like any normal sharpening will do!!

power = 1.0 is exactly normal sharpening (but slower)
power = 8.0 is edge-condensing only (almost), no contrast enhancement
In-between values are in between.


Bottom line(s):

Sharpening a.k.a. contrast enhancement *without* noise reduction is always a bad idea, of course. The above script is a script for sharpening, not more. It doesn't try to be sort of an integrated solution doing *all* the work for you! For that purpose, we have iiP() ...


My posts always tend to get long, I know. Sorry for that. I also understand that some might be bothered too much with reading all that stuff.

The thing is that I try to explain what is going on in my scripts. It's a pity then, after doing thoroughly explanations, to find that the explanations didn't reach the audience, people instead through any stuff at the defaults, afterwards stating "blah, that's crap", totally missing the possibilities of adapting the parameters.

EDIT: the last sentence refers not to Soulhunter!


- Didée

[Dali Lama]

Originally posted by Didée
a pity then, after doing thoroughly explanations, to find that the explanations didn't reach the audience, people instead through any stuff at the defaults, afterwards stating "blah, that's crap", totally missing the possibilities of adapting the parameters.


Didée, I understood you very well the first time. Consider the possibility that you notice people who don't understand you, because they usually post about it in hope for clarification. Those that do understand won't feel as compelled to say, "Yes, I do understand" in a post. Perhaps more people understand than not?

Best Wishes,

Dali

[Soulhunter]

Originally posted by Didée
I really start thinking about doing the following for my 'experimental' scripts:

- If the function is called with default values, this gets detected, and the function will print, in big orange letters,

"ALERT! Usage of default values detected! DONT EVER DO THAT!"

onto the clip ... :D
But I used the default settings only 1x time, all other tests where done with other settings... :(

But yes, this edge-enhancement seems to work more or less !!!


Just thought the sharpening should work in a more "intelligent" way... :confused:

Would it be somehow possible to prevent noise amplification (without de-noising) ???


Anyhow, I will stay with IIP for this kind of stuff... ;)


Bye

[koszopal]

@dideee
maybe little [OT] but will be any new version of iip ?
koszopal
PS i've thought that limitedsharp script its kinda a repleacement for iip :)

[Mug Funky]

i believe i found it in 3 other threads as well.

hehe.. just call me PIMP. i might do that same search and update them all.

a question occurs to me: could this filtering concept be used to remove haloes? considering they're present on so much stuff, it would be useful to throw another technique at it.

i've got R4 Akira, and it's a great copy, but it's got haloing and thus it has amplified noise. would it be possible to adapt this edge-limiting to something like this?

i might see if i can come up with a proof-of-concept for it, but don't be holding your breath. i'll be out of town a while starting next week, and i doubt there'll be much of an internet connection (to say nothing of avisynth availability). there's not even mobile/cellphone reception where i'm going (arguably a very good thing).

[Didée]

That's already an old story (http://forum.doom9.org/showthread.php?s=&threadid=74003&highlight=blinddehalo), Mug Funky ;)

The only thing that needs to be adjusted there is the brightness of the edgemask. With the values I used there (they were tweaked for a particular source), the overall effect is too weak. One or two numbers to change, that's all.

I thought I'd wait for anyone saying "hey, that script does nothing, what's up here?" before correcting it. Well, I'm patient ...


koszopal:

Something like that is what I have in mind, sure. But it's not so easy as it may seem.
In the meantime, feel free to implement it yourself :D


I am still at this stuff, but actually I have also some real encoding work to do. One can only do so much a day.


- Didée

[pieter1976]

Didée

Do you implement filters yourself?
If you do Could you implement my code in a plugin.
It is very small.
I can't do it because i don't know much about C++.
What do you think about the code?

code (http://home.zonnet.nl/pieterboots/code.htm)

[Didée]

Unfortunately not. My knowledge about compilers comes to its end with spelling the word - and approx. 12 years ago, for some specific reasons I decided to stop with coding, forever.

- Didée

[Si]

You know - we really could do with a parser that works at the pixel/byte level so people with ideas can try them out without having to learn C(++)

Any super coder available?

Simon

[scharfis_brain]

I can mind had read about a virtualdub filter that was able being directly programmed with something imperative like C on the per pixel level....

maybe someone could dig this thing out of the net and convert it for usage with AVISynth ?!?

[Didée]

LimitedSharpen is not dead.

It is alive, bursting with health, and better than ever.

There have been a lots of changes, but I'm currently too lazy to describe them all. For now, the comments in the header must suffice ;)

But one fact should be pointed out: Through the usage of MaskTool's new yv12LUTxy function (:D), LimitedSharpen is now pretty fast!

The following function call will give an output that is almost identical to the good old 4*-supersampled-XSharpen() treatment, except for that it runs 2.0~2.5 times faster: :eek:

LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=2,strength=100,Lmode=1,overshoot=1,wide=false,soft=false,radius=1,edgemode=0)
I think I'll make a thread in AviSynth Usage for this one.


The script: (MaskTools >= v1.5.1 required)
# LimitedSharpen()
#
# Multi-purpose sharpener for
# - edge condensing
# - detail contrast enhancement without oversharpening or edge halos
#
# 04.05.2004 - 'edgemode' re-implemented, now all three options (on edge [1], on NOT edge [2], disable [0])
# 05.08.2004 - Big speed-up by implementation with MaskTool's new function "yv12LUTxy()" (my proposal)
# - removed the "power" limiting. Never liked it much.
# - no more scaling of "radius" with the ss_x|y values. I like it more the 'direct' way.
# - "mode" Parameter added: mode=1 == uses UnsharpMask() for sharpening
# mode=2 == uses Sharpen() for sharpening. "strength" then is internally
# divided by 100, and "radius" then has no effect.
# 11.08.2004 - added "wide" parameter (boolean): "true" takes the min|max from a 5x5 neighborhood for limiting.
# "false" works, as formerly, on a 3x3 neighborhood
# - added "Lmode" parameter: "1" does hard limiting as formerly:
# if [sharpen] is within [min-OS,max+OS], take it. Else, use (min|max).
# "2" will replace sharpen's effective overshoot with sqrt(overshoot).
# - renamed "mode" to "Smode": "1" = UnsharpMask / "2" = Sharpen / "3" = MinMaxSharpen
# Smode=3 will, instead of sharpening the current pixel against the average
# of its 8 neighbors, sharpen it against average(min,max). It's less exact
# in theory (blurred corners!), but results in stronger overall sharpening.
# Also, it is less prone to enhancing pre-existing crap like DCT noise

function LimitedSharpen( clip clp,
\ float "ss_x", float "ss_y",
\ int "Smode" , int "strength", int "radius",
\ int "Lmode", bool "wide", int "overshoot",
\ bool "soft", int "edgemode" )
{
ss_x = default( ss_x, 1.5 )
ss_y = default( ss_y, 1.5 )
Smode = default( Smode, 1 )
strength = Smode==1 ? default( strength, 160 ) : default( strength, 100 )
strength = Smode==2&&strength>100 ? 100 : strength
radius = default( radius, 2 )
Lmode = default( Lmode, 1 )
wide = default( wide, false )
overshoot= default( overshoot, 1)
overshoot= overshoot<0 ? 0 : overshoot
soft = default( soft, false )
edgemode = default( edgemode,0)
#radius = round( radius*(ss_x+ss_y)/2)
ox = clp.width
oy = clp.height

ss_x != 1.0 || ss_y != 1.0 ? clp.lanczosresize(round(ox*ss_x/8)*8,round(oy*ss_y/8)*8) : clp
tmp = last

edge = logic( DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=2)
\ ,DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=2)
\ ,"max").greyscale.levels(0,0.86,128,0,255,false) #.levels(0,2.0,208,0,255,false)

bright_limit = (soft == true) ? last.blur(1.0) : last
dark_limit = bright_limit.inpand() #.inflate.deflate
bright_limit = bright_limit.expand() #.deflate.inflate
dark_limit = (wide==false) ? dark_limit : dark_limit .inflate.deflate.inpand().inflate.deflate
bright_limit = (wide==false) ? bright_limit : bright_limit.deflate.inflate.expand().deflate.inflate
minmaxavg = yv12lutxy(bright_limit,dark_limit,yexpr="x y + 2 /")

Str=string(float(strength)/100.0)
normsharp = Smode==1 ? unsharpmask(strength,radius,0)
\ : Smode==2 ? sharpen(float(strength)/100.0)
\ : yv12lutxy(tmp,minmaxavg,yexpr="x x y - "+Str+" * +")

OS = string(overshoot)
Lmode == 1 ? yv12lutxy( bright_limit, normsharp, yexpr="y x - 0 "+OS+" + < y x ?")
\ : yv12lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x y x - 1 2 / ^ + ?")
Lmode == 1 ? yv12lutxy( dark_limit, last, yexpr="y x - 0 "+OS+" - > y x ?")
\ : yv12lutxy( dark_limit, last, yexpr="y x "+OS+" - > y x x y - 1 2 / ^ - ?")

edgemode==0 ? NOP
\ : edgemode==1
\ ? MaskedMerge(tmp,last,edge,Y=3,U=1,V=1,useMMX=true)
\ : MaskedMerge(last,tmp,edge.inflate.inflate.inflate.blur(1.0),Y=3,U=1,V=1,useMMX=true)

ss_x != 1.0 || ss_y != 1.0 ? lanczosresize(ox,oy) : last

clp.mergeluma(last)
return last
}
#

Have fun

- Didée

[gamr]

well it works like advertised, much faster than ssxsharpen and it looks almost exactly the same :devil: 1 comment so far, where is wide used? i dont see it anywhere . . . :rolleyes:

[Soulhunter]

So, again...

Here (http://mf.onthanet.nl/forum/thread.php?threadid=100&boardid=4&sid=f4e4e523d639a6321ea2f40976f3cac8&page=1#9) are some samples !!!


Bye

[Chainmax]

So what would be the main diferences between LimitedSharpen and SSXSharpen regarding output? BTW, the japanese warpsharp is the "warpsharp_2003_1103.cab" one, right?

[Soulhunter]

Originally posted by Chainmax
So what would be the main differences between LimitedSharpen and SSXSharpen regarding output?
The difference is minimal !!!

Considering the speed factor then, and... :p


Bye

[Chainmax]

I was going to try it on a very awful VHS capture and received the following error when trying to load the script in VDubMod:

Avisynth open failure:
Script error: LimitedSharpen does not have a named argument "mode"

I copied the function from here and I called it with the following line (from the thread SoulHunter linked to):

LimitedSharpen (ss_x=2.0,ss_y=2.0,mode=2,strength=127,radius=2,power=0.0,overshoot=1,soft=false,edgemode=0)

[Didée]

the function from here and I called it with the .. line from the thread SoulHunter linked toVery clever.

I posted an actual function call together with the actual script in this thread.
# [...]
# 11.08.2004 - added "wide" parameter (boolean): "true" takes the min|max from a 5x5 neighborhood for limiting.
# "false" works, as formerly, on a 3x3 neighborhood
# - added "Lmode" parameter: "1" does hard limiting as formerly:
# if [sharpen] is within [min-OS,max+OS], take it. Else, use (min|max).
# "2" will replace sharpen's effective overshoot with sqrt(overshoot).
# - renamed "mode" to "Smode": "1" = UnsharpMask / "2" = Sharpen / "3" = MinMaxSharpen
# Smode=3 will, instead of sharpening the current pixel against the average
# of its 8 neighbors, sharpen it against average(min,max). It's less exact
# in theory (blurred corners!), but results in stronger overall sharpening.
# Also, it is less prone to enhancing pre-existing crap like DCT noise

[Chainmax]

:o :o

[Chainmax]

Ok, here are the pictures:

Before:
http://members.arstechnica.com/x/master_burrfoot/Frame100_NoS.png
After:
http://members.arstechnica.com/x/master_burrfoot/Frame100_S.png


Before:
http://members.arstechnica.com/x/master_burrfoot/Frame430_NoS.png
After:
http://members.arstechnica.com/x/master_burrfoot/Frame430_S.png

[morsa]

Chainmax, You won´t tell me you are near your thirty´s ;)
Verano Azul, hahahah.I liked it so much!!!!! (When I was 10) :D

[Chainmax]

¿Así que sos argentino? Yo soy uruguayo :).
Tengo 24 años, así que no estás muy lejos :sly::(. Veía verano Azul cuando lo pasaban hace unos cuantos años (no recuerdo cuántos exactamente), pero estoy haciendo ésto para mi futura cuñada.

[morsa]

Good luck then!!!!
Anyway I see your source is really messed up....:(

[Chainmax]

Actually, the "before" shots are from the filtered source. :(++

[morsa]

If that's the situation I guess you are filtering too much!!!!!!!!!!
Too much median cut.
Could you post an original frame to compare with?

[Chainmax]

Will do. I'll start a new thread in Avisynth Usage so this one won't be further derailed. I'll post screenshots and a clip from the unfiltered source. You won't believe how shitty it is.

[Boulder]

Could someone please point me to the correct dlls? I've got the latest MaskTools but the WarpSharp package (20031103) doesn't seem to work. VirtualDubMod is unable to open the script when I try to use LimitedSharpen(). I suppose that warpsharp.dll includes UnsharpMask()?

[Didée]

The missing culprit is belong:

msvcr70.dll
msvcr71.dll
msvcp70.dll
msvcp71.dll

I can never remember which of these dll's is needed by which version ... or which is needed by WarpSharp, and which by some Avisynth C-plugins ... big confusion ;)

Just get all of them, place'em in /system32, and be done.

[Boulder]

Nope, got all of them in %WINDIR%\System32. I still get an error message in VDubMod saying "Avisynth open failure: Avisynth: script open failed!"

God I hate the different WarpSharp packages:D

[Didée]

I hate this game, too :)

Well, on this machine here, I've available the WarpSharp packages 06-2003 and 11-2003. (I'm actually using the 06 version).

When loading scripts with either DLL, I get the following DLL usage (MSxxx libraries only):

WarpSharp 06-2003
msvcp60
msvcp70
msvcr70
msvcrt


WarpSharp 11-03
msvcp60
msvcp71
msvcr71
msvcrt


Hope this helps, I've to leave now.

[Boulder]

I got it working, I forgot that warpsharp.dll in the plugins folder causes weird behaviour. I moved it elsewhere and loaded it manually in the script and now it works.

[kassandro]

Originally posted by Didée
The missing culprit is belong:

msvcr70.dll
msvcr71.dll
msvcp70.dll
msvcp71.dll

I can never remember which of these dll's is needed by which version ... or which is needed by WarpSharp, and which by some Avisynth C-plugins ... big confusion ;)

Just get all of them, place'em in /system32, and be done.
This is an unpleasant dynamic linking problem, which partially destroys the advantage of dynamic linking. For each compiler version you need one version of msvcpXX.dll msvcrXX.dll. In the Linux world the corresponding libraries are nearly always backward compatible, whence you need only the most recent one.