LimitedSharpness Fix for Jaggedness?

[Ditto666]

I'm sure this has been discussed before, and this could be considered opinion based I guess, but what I'm about to ask isn't for "the best" but for exactly the same really without the irksome issue.

Both LimitedSharpen and LimitedSharpenFaster operate the same way from my understanding, just, LimitedSharpenFaster is.. faster, lol. My problem with LimitedSharpen in general is that I've noticed it makes distinct edges significantly jagged, and this obviously applies more heavily for SD content. I understand why this happens as before when it looked more smooth, it was due to the blurred distinction between the pixels when after it becomes more patent. Otherwise, I actually really like the way this works, and I feel it's very fitting when working with the x264 codec especially. I couldn't really find anything else that does things quite as nicely though admittedly I'm sure there's much more than what I've tried.

My question is, is there anything else that works very similarly to LimitedSharpen, or, I guess more likely and rational, is there any combination that someone has found to work well that sorta neutralizes this whole jagged scenario? XD Thanks guys :)

My thoughts were to use WarpSharp, as it actually bends the edges to make it appear smooth, which I actually don't like the concept of, but it didn't work anyway so doesn't matter, lol...

[leeperry]

LaTo told me that w/ ss1.0, the only way to avoid jaggies was "Lmode=-1" but it only works w/ LSFmod I think.

[Ditto666]

LaTo told me that w/ ss1.0, the only way to avoid jaggies was "Lmode=-1" but it only works w/ LSFmod I think.

Sorry, what is LimitedSharpenFasterMod? Is it another version, or rather, some modded version of LSF? Has it something to do with Lmode in regular LSF basically just 1-4, -1 being some additional value?

[poisondeathray]

Sorry, what is LimitedSharpenFasterMod? Is it another version, or rather, some modded version of LSF? Has it something to do with Lmode in regular LSF basically just 1-4, -1 being some additional value?


http://forum.doom9.org/showthread.php?t=142706
http://avisynth.org/mediawiki/LSFmod

[Ditto666]

http://forum.doom9.org/showthread.php?t=142706
http://avisynth.org/mediawiki/LSFmod

Cool, thanks! Is there a downside to this? Meaning, if I set it to -1, there's no more "jaggies" but is it less effective? I hope the downside is just that it's slower XD

[leeperry]

if I set it to -1, there's no more "jaggies" but is it less effective?
you try it, you tell us :)

I use LSF in realtime and LSFmod is substantially slower so I didn't bother trying...but yes, those jaggies also annoy me.

[Ditto666]

you try it, you tell us :)

I use LSF in realtime and LSFmod is substantially slower so I didn't bother trying...but yes, those jaggies also annoy me.

You never use LSF for encoding? Really? Just out of curiosity, what do you use for image sharpening then? I'm interested :)

And me telling you doesn't work too well on here. Too many people here look at everything from a technical front only, so if I describe how there might be some deterioration in another aspect as effect, somebody else would say they did whatever test that shows otherwise. I would've suspected that this forum would be filled with video and audiophiles but as it turns out, it isn't, unfortunately... I say that in the end, your eyes and ears are what process the data. Whoever might be listening or watching wouldn't care about what the tests might've shown.

[leeperry]

I don't encode my own stuff, I just use LSF and a few other scripts in ffdshow to improve playback PQ :)

[Ditto666]

I don't encode my own stuff, I just use LSF and a few other scripts in ffdshow to improve playback PQ :)

Lol, oh, alright. Yeah, that's quite different. Thanks though.

[leeperry]

in fast mode, it's not that slower...I'll give it a shot see if that fixes the jaggies w/o drawbacks.

[Ditto666]

in fast mode, it's not that slower...I'll give it a shot see if that fixes the jaggies w/o drawbacks.

Cool. I'll probably do that today as well. I just wanna get the preview in AvsP working the way I'd like first :D

[leeperry]

I've done a bit of testning...IME, jaggies can't really be avoided :o

If you do anything to avoid them, then you are forced to increase the "strength" setting and >40 it posterizes too much to my taste. PPl faces look like they're made out of wax.

[Ditto666]

I've done a bit of testning...IME, jaggies can't really be avoided :o

If you do anything to avoid them, then you are forced to increase the "strength" setting and >40 it posterizes too much to my taste. PPl faces look like they're made out of wax.

Thanks for the followup. Agreed about the above 40 thing. You say jaggies can't be avoided though, but with other sharpeners, of course at the cost of other problems/less effectiveness, worse IMO, it doesn't happen. LimitedSharpen is with what the issue really becomes more serious hence the reason for my query. I feel it's still the best balance from what I've tried but if the fix not within the filter itself, there must be something applicable afterward to possibly douse that jagged effect, even partially if not fully.

Still didn't try the mod version myself though - haven't gotten the chance :/

[Didée]

Oh, the RealTime guys wanting to have all possible processing, but please in nanoseconds.

There are alternating sharpening concepts, like e.g. this here (http://forum.doom9.org/showthread.php?t=96690).
I've also tried different methods for "soft"-processing of LSF. They even work pretty okay, but ..... if you can't use LSF with supersampling in realtime, then most probably you also won't be able to use any of those alternatives in realtime. :)

Either you can have fast simple filters with their issues, or you can have slower filters with less issues.

If you want a superfast filter with supergood processing, you need to invent it on your own. :p

[leeperry]

so SS is a pratical proper solution to minimize jaggies? these days I upscale above my display native res in Lanczos3, use SmoothLevels>LSF>GrainF3 and let madVR downscale in Lanczos4 at the very end...I have to admit that the jaggies are far less visible :)

I love LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40) and nothing really compares to my eyes, I'll live w/ the jaggies :p

[Ditto666]

Oh, the RealTime guys wanting to have all possible processing, but please in nanoseconds.

There are alternating sharpening concepts, like e.g. this here (http://forum.doom9.org/showthread.php?t=96690).
I've also tried different methods for "soft"-processing of LSF. They even work pretty okay, but ..... if you can't use LSF with supersampling in realtime, then most probably you also won't be able to use any of those alternatives in realtime. :)

Either you can have fast simple filters with their issues, or you can have slower filters with less issues.

If you want a superfast filter with supergood processing, you need to invent it on your own. :p
I like LimitedSharpen because of the way it works. The fact that it's fast is an added bonus :) For encoding though, I really wouldn't mind if the process takes a bit longer. It would be worth fixing the jaggies. It sounds like you would know how. It's just a process that would add time which I don't mind - expected actually.
so SS is a pratical proper solution to minimize jaggies? these days I upscale above my display native res in Lanczos3, use SmoothLevels>LSF>GrainF3 and let madVR downscale in Lanczos4 at the very end...I have to admit that the jaggies are far less visible :)

I love LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40) and nothing really compares to my eyes, I'll live w/ the jaggies :p

That's pretty much what I use with LSF. Strength from 30-40 usually, depending...

Could you show your process a bit more detailed? It sounds interesting. It somehow makes sense to upscale past the native resolution and then downscale - I might've actually tried it once. I can't figure out HOW it makes sense though, lol, of course depending on how you use that too effects its usefulness...

[leeperry]

It's fairly simple, it's all done in ffdshow: upscale > ffdshow scripting using SetModeMT() and then sent to madVR via YV12, which will downscale using the GPU :)

[Ditto666]

It's fairly simple, it's all done in ffdshow: upscale > ffdshow scripting using SetModeMT() and then sent to madVR via YV12, which will downscale using the GPU :)

What's the script for it? >.< I wouldn't know how to do that even if it's simple. I'd be able to slowly figure out based on the code though :) You know, checking out what each individual thing does. Sorry if I'm prying, that's if you don't mind that is.

[leeperry]

you can just put this in ffdshow:
MT("LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40)",4)

[Ditto666]

you can just put this in ffdshow:
MT("LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40)",4)

Am I missing something here? This back and forth with uncertainty on my end is making me feel slow. Isn't that not much more than just sharpening? I'm not sure what MT or the ",4" is but I'm pretty certain simply that doesn't involve the main bulk of what you were saying that I was interested in - the upscale, the downscale, the "SmoothLevels", etc.. I have my own method currently for filtering in FFDShow but from what you described, a few things you implement might further improve on mine. That's why I'm curious. Sorry again for the persistence, and thanks again for the response >.<

[leeperry]

1) upscale in ffdshow, as high as your CPU will allow. I prefer Lanczos3 for luma and default bicubic for chroma.

2) use this script
SetMTmode(2,4)
ffdshow_source()
SmoothLevels(preset="tv2pc",multithread=0,Smode=0)
LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40)
Distributor()

uncheck "add ffdshow video source" in ffdshow's avisynth filter, otherwise it will crash when using SetMTMode().

use SetMTmode(2,4) for a quad CPU and SetMTmode(2,2) for a dual core CPU.

you can use GrainFactory2/3 on top if you have enough CPU juice.

3) set madVR to keep the levels untouched, and it will automatically downscale to your display res...using the GPU ;)

madVR and SmoothLevels() both add dithering, the PQ is stunning to my eyes. I compared them here (http://forum.doom9.org/showpost.php?p=1454344&postcount=96).

[Ditto666]

1) upscale in ffdshow, as high as your CPU will allow. I prefer Lanczos3 for luma and default bicubic for chroma.

2) use this script
SetMTmode(2,4)
ffdshow_source()
SmoothLevels(preset="tv2pc",multithread=0,Smode=0)
LimitedSharpenFaster(ss_x=1.0,ss_y=1.0,strength=40)
Distributor()

uncheck "add ffdshow video source" in ffdshow's avisynth filter, otherwise it will crash when using SetMTMode().

use SetMTmode(2,4) for a quad CPU and SetMTmode(2,2) for a dual core CPU.

you can use GrainFactory2/3 on top if you have enough CPU juice.

3) set madVR to keep the levels untouched, and it will automatically downscale to your display res...using the GPU ;)

madVR and SmoothLevels() both add dithering, the PQ is stunning to my eyes. I compared them here (http://forum.doom9.org/showpost.php?p=1454344&postcount=96).

Now that's what I call epic! Thank you very much good sir :) I will look into all of this more in depth. Sounds interesting :D