SweetFX Video Shaders

[StinDaWg]

For 720p->1080p upscaling I'm currently using LumaSharpen pre-resize with nnedi image doubling in madVR. I have it set to low levels. Just enough to take out the softness from upscaling but still being a noticeable improvement. nnedi is a really great upscaler, I highly recommend it if your card can handle it. It's very sharp with basically no ringing or aliasing.

#define sharp_strength 0.5
#define sharp_clamp 0.05
#define pattern 3

[CeeJay.dk]

Originally Posted by turbojet

CeeJay.dk: I struggle with most parts of it, no clue how convolutions work. I don't understand what the code does. I'll just stay happy with pattern=3 it's plenty good, maybe at some point trying a pattern with more taps could be an improvement for film content or it might be interesting to see if more texture fetches has any effect with many taps.

A convolution kernel (also called a convolution matrix) is a map of the center pixel and the pixels around it

For a area that is 3 x 3 pixels you could have a kernel like this :

000
010
000

The number tells you how much you should sample and from what pixels.

The above kernel samples 1 time from the center pixel and nothing from the surrounding 8 pixels. This yield the exact same picture as before.

000
100
000

The above kernel samples 1 time from the pixel to the left of the center and nothing from the others.
Since the kernel is applied to every pixel in the image, you have just moved the entire image one pixel to the left.

Easy enough but not that useful

111
111
111

The kernel above samples 1 time from each pixel. All the samples are added and then you need to divide by the number of samples you took (9 in this case) or you will make the image brighter.
Adding 9 numbers together and dividing by 9 is just averaging them together - and averaging the center pixel by it's surrounding 8 pixels like this blurs the image.

This link explains it well and comes with a tool you can play with to make your own convolution kernels

For pattern 2 i use a kernel that is :

121
141
121

This blurs the image smoothly. I then subtract the original image to get the difference (the contrast), which i then increase and add back to the original.

In the link above try the kernel :

-1, -2, -1
-1, 21, -1
-1, -2, -1

To get a similar sharpening.

This looks much too sharp which I why I limit the sharpening so it never goes above a certain amount.

I think the most important part of a sharpening filter is not so much how you sharpen, but how you decide how much to sharpen.

[Ge'in]

Many thanks for your LumaSharpen on MPC-HC.
With pattern 8 and the experimental limiter, it's just awesome.

[toniash]

Originally Posted by Ge'in

Many thanks for your LumaSharpen on MPC-HC.
With pattern 8 and the experimental limiter, it's just awesome.

How do you activate "the experimental limiter"?

[CeeJay.dk]

Originally Posted by Ge'in

Many thanks for your LumaSharpen on MPC-HC.
With pattern 8 and the experimental limiter, it's just awesome.

Originally Posted by toniash

How do you activate "the experimental limiter"?

Pattern 8 is just a slower implementation of Pattern 2. I used it with earlier versions of MPC because it seemed to have a bug where the image would be slightly stretched along the diagonal if I used the faster method of sampling several taps with one texture sample (used in pattern 2). Pattern 8 worked with older version.

However with the newer MPC builds this problem have gone away and I no longer use it, because other than speed the two patterns (2 and 8) are identical. Pattern 9 is likewise just a slower version of pattern 4.

I keep them in the code because it easy to prototype new convolution kernels when you just need to change a few numbers rather than write that code again.
Once I'm satisfied with a kernel I can always find the most optimal way to take those samples at a later time.

To activate the new limiter you need to modify the code.

Looking at it quickly it looks like you can change :

 #if 0 //New experimental limiter .. not yet finished

to

 #if 1 //New experimental limiter .. not yet finished

The idea with the new limiter is to give the pixels with very little contrast very little sharpening because they might be gradients.
Pixels with some contrast are given more sharpening because these are details we want to enhance.
And finally pixels with a lot of contrast, we don't wan't to sharpen very much or not at all because they already have plenty of contrast and giving them more makes the image look fake and over-sharpened.

Here is a graph of the sharpening with the old clamp (f4) and the new limiter (f2)

The X-axis shows difference in brightness to the original and the Y-axis is the sharpening strength.

Pixels that are brighter than their surroundings are made even brighter and pixels that are darker are made even darker.
Pixel with the same brightness (0,0 in the graph) are not touched

It's experimental because I want to find a good formula that allows the user to tweak the max sharpening, when it slopes up in strength and when it slopes down but a good tweakable formula is hard to find. The best I can currently do is tell you to change the / 16 at the end. A smaller divider gives a greater max strength and a greater gives a small max strength.

A good limiter formula should be easy to tweak and understand - this one is not (yet).

I wonder what limiter formula Didee uses for his LimitedSharpen (I can look at the code but I don't understand avisynth code).

In case you are curious - the #elif 0 //SweetFX 1.4 code and #else //SweetFX 1.5.1 code do the same thing. It clamps the sharpening. The 1.5.1 code just does it using 1 instruction less, by moving some of the math into other parts of the code to utilize the available instruction lanes better. I kept the older code around for reference because the new code while faster is also harder to comprehend for anyone trying to understand what the code does.

[CeeJay.dk]

Originally Posted by sidspyker

Well well, fancy seeing you here CeeJay

I've been a member here for over 10 years now - Welcome to the forum Sidspyker.

[turbojet]

Thanks for the info on convolutions CeeJay.dk I'll play around with it when I find some more time in the next few weeks.

After a quick initial test with experimental limiter it doesn't seem right for me, it's noticeably softer and low contrast edges (mainly faces) is what I notice most. I've noticed banding shows much more with lumasharpen because of it but debanding can solve that mostly. What about if there's very small differences in edges within a large radius don't sharpen, otherwise do?

[StinDaWg]

I've noticed that using this shader boosts contrast, and makes the picture brighter overall. It's especially noticeable on white text, there is a glowing effect. I don't like that. Is there anyway to modify the values to reduce or eliminate the contrast boost while still sharpening? It's not nearly as bad as the Sharpen Complex MPC-HC shader, or even unsharp mask, but it's something that really bothers me and is putting me off from using this full time.

[turbojet]

StinDaWg have you tried the experimental limiter? It noticeably reduces sharpening of high contrast edges. I don't notice an overall contrast increase but the high contrast edges are definitely higher contrast with the default clamping.

[StinDaWg]

I tried it, that didn't change the brightness/contrast boost for me. Like I said it's not severe like MPC Sharpen Complex, but it's still quite noticeable. I don't know if this is just a side effect of doing this method of sharpening (unsharp mask), or if there are other methods (Darbee?) that do it without the contrast boosts.

[yok833]

Hello I really would like to try the lumasharpen shader as it has so many good feedback... However I am not sure that I do it correctly as I do not see a huge difference with or without it.... For example if I select it 5 or 6 times consecutively in pre resize there is no difference and it offer the same results than if I would have selected it only once??? Is it normal? If a do the same with sharpen complex the sharpen is so strong that the picture is unwatchable....

Envoyé de mon GT-I9300 en utilisant Tapatalk

[Guest]

@yok833

You are posting this question in multiple places. That is called cross-posting and it is not allowed here, per forum rules. Please read and follow our rules to avoid strikes. Thank you for your understanding. Follow up to PM if needed, please.

[CeeJay.dk]

Originally Posted by StinDaWg

I've noticed that using this shader boosts contrast, and makes the picture brighter overall. It's especially noticeable on white text, there is a glowing effect. I don't like that. Is there anyway to modify the values to reduce or eliminate the contrast boost while still sharpening? It's not nearly as bad as the Sharpen Complex MPC-HC shader, or even unsharp mask, but it's something that really bothers me and is putting me off from using this full time.

It sounds like you are talking about haloing.

The dark or bright edges that strong edges get sometimes.

Sharpen Complex and SweetFX both do an unsharp mask filter, but SweetFX limits this using the sharp_clamp.

Try lowering the clamp until it no longer annoys you.

The idea with the experimental limiter was to reduce that even further but I don't think i have the formula right yet, judging from the feedback. More work is needed in this area.

It would also be possible to limit the sharpening in other ways.
LimitedSharpen by didee has a min/max option where the filter finds the minimum and maximum values of the surrounding pixels and uses those values as the clamp values that the sharpening is not allowed to go beyond.

This is not as fast (SweetFX is written primarily for games so speed is important to me) and it also limits the sharpening, but it should eliminate the possibility of halos.

[CeeJay.dk]

Originally Posted by yok833

Hello I really would like to try the lumasharpen shader as it has so many good feedback... However I am not sure that I do it correctly as I do not see a huge difference with or without it.... For example if I select it 5 or 6 times consecutively in pre resize there is no difference and it offer the same results than if I would have selected it only once??? Is it normal? If a do the same with sharpen complex the sharpen is so strong that the picture is unwatchable....

If you want a stronger effect edit the settings in the lumasharpen file. Either use a higher strength or a higher clamp or both.

Adding the same filter many times is not the right way to get a stronger effect.

[XRyche]

Originally Posted by CeeJay.dk

I
The idea with the experimental limiter was to reduce that even further but I don't think i have the formula right yet, judging from the feedback. More work is needed in this area.

Agreed, it looks a little surreal in both games and video. It's seems to emphasize bright whites too much. The limiter seems to be a good idea though. I always tend to think sharpeners rely too heavily on higher contrasts. It reminds me of the early SD to BD resizing attempts.

[StinDaWg]

Originally Posted by CeeJay.dk

It sounds like you are talking about haloing.

The dark or bright edges that strong edges get sometimes.

Sharpen Complex and SweetFX both do an unsharp mask filter, but SweetFX limits this using the sharp_clamp.

Try lowering the clamp until it no longer annoys you.

The idea with the experimental limiter was to reduce that even further but I don't think i have the formula right yet, judging from the feedback. More work is needed in this area.

It would also be possible to limit the sharpening in other ways.
LimitedSharpen by didee has a min/max option where the filter finds the minimum and maximum values of the surrounding pixels and uses those values as the clamp values that the sharpening is not allowed to go beyond.

This is not as fast (SweetFX is written primarily for games so speed is important to me) and it also limits the sharpening, but it should eliminate the possibility of halos.

I changed the clamp to .015. This seems to help out with the blooming while still allowing enough sharpening for my tastes. I'm sure it could be improved some more. Looking forward to what you can do with the experimental limiter. You might want to consider changing default settings for both gaming and movie watching. As we've seen pattern 3 works better for movies and the two types of video are so different it doesn't really make sense to recommend the same settings for both.

Post-resize

#define sharp_strength 1.5
#define sharp_clamp 0.015
#define pattern 3
#define offset_bias 1.0
#define show_sharpen 0

[turbojet]

Has anyone else raised offset_bias and reduced strength?

Currently using strength 1.0, clamp 0.5, pattern 3, bias 3.0 and getting good results. Raising bias seems to have a great effect on lower contrast edges with little effect on high contrast edges. Just what I've been looking for.

[yok833]

@turbojet Are you using lumasharpen pre or post resize?

Envoyé de mon GT-I9300 en utilisant Tapatalk

[turbojet]

I've been using it pre and post resize for a few months now but trying to switch to post-resize only because on sharpening dirty SD sources pre-resize sharpening creates artifacts. In order to get the sharpening I was able to get from pre+post with post only I have to turn the strength up to 8.0+ which shows a lot of high contrast artifacts until I raised offset_bias and dropped strength.

[kasper93]

I'm using post-resize
Strength 1.55
Pattern 3
Bias 1.0
experimental limiter with "12" at the end 16 was limiting to much for sources that I need sharpen.