Adaptive sharpen - Image sharpening shader

[bacondither]

Originally Posted by aufkrawall

Don't want to sound like a broken record, but maybe you could still do something regarding brightness?

Tuning it to one scenario would make it pretty crappy in other places, i have currently no idea to "solve" it with an algorithm that would work in shaders or be fast.
But the latest version 2015-12-04-E should be somewhat better at fine details in most cases. Uploaded a comparison of 2015-11-17E and 2015-12-04E with all settings at default.

I also experimented some with sharpening that test image in raw therapee with RL deconvolution, that made the image alot more brighter!

[aufkrawall]

This is a very nice progress, thanks a lot!
I think this can be very helpful in games, e.g. when strong sharpen is needed to fight temporal AA blur. May be a bit expensive in realtime, but should be great for screenshots.

[JPulowski]

In games, AdaptiveSharpen requires quite a bit of performance. In my GPU it utilizes 15 - 20% of extra computing power. In movies where the framerate is between 23.976 - 30.0 fps, the performance drop does not make a huge difference. But in recent video games, every bit of performance is important. bacondither, do you have any ideas about possible optimizations? Currently the shader uses an additional RGBA16F texture for storing the alpha value of the first pass. This is a bit too much just for one floating point value. I might use R16F or encode the value as RGBA8 and decode it in the second pass afterwards, depending on which method is faster. Other than that, it is now my favorite shader for sharpening. It really goes well with the video games.

We also have a GitHub repo. You are welcome to modify the code as you see fit.

[ts1]

I've ported some things from 2-pass shaders.

Edit: fixed a bug

[foozoor]

Originally Posted by ts1

I've ported some things from 2-pass shaders.

Edit: fixed a bug

Where can we download it? Can I use it in mpv?
Thanks I love this sharpener but I think 1.0 is too much as default value.

Edit: I found it inside the mpv wiki...
https://gist.github.com/bjin/152a2f8bdf87d83261ac

[ts1]

foozoor
1-pass version is not so strong, try it, it's on a 3rd page.

[foozoor]

Originally Posted by ts1

foozoor
1-pass version is not so strong, try it, it's on a 3rd page.

Thank you, really nice.

Is there something between lumasharpen and adaptive sharpen for mpv?
Because my gpu can‘t handle adaptive sharpen with some video files.
Could you port finesharp or lsfmod as mpv shader?

[ts1]

Sorry, but no.
Use it as prescale then maybe.

[lanzorg]

Originally Posted by ts1

foozoor
1-pass version is not so strong, try it, it's on a 3rd page.

Should I turn `linear-scaling` and `sigmoid-upscaling` off like the 2-passes shader?

[ts1]

lanzorg
Yes

[bacondither]

Originally Posted by JPulowski

Do you have any ideas about possible optimizations?

Sure, did a pull request on reshade-shaders now. Up to 45% speedup on rendering times for the shader (3.2ms to 2.2ms) at 1080p on my 7790.

[aufkrawall]

With a 980, I can note a gain. However, it's not large.
No ReShade shaders: 96fps
AS old: 81fps
AS new: 83fps

[bacondither]

Originally Posted by aufkrawall

With a 980, I can note a gain. However, it's not large.

At least that's 18% faster rendering times for the shader.
Did you have have "fast_ops = 1" in JPulowski.cfg?

I did my test on a DX11 target application.

[aufkrawall]

Yep, had it at 1 and tested DX11.
With a higher resolution, the results don't get wider:
42 vs 41 fps

[ts1]

I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

[lanzorg]

Originally Posted by ts1

I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

Thanks!

[darkasylum666]

Originally Posted by ts1

// Copyright (c) 2015, bacondither
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
//    notice, this list of conditions and the following disclaimer
//    in this position and unchanged.
// 2. Redistributions in binary form must reproduce the above copyright
//    notice, this list of conditions and the following disclaimer in the
//    documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// Adaptive sharpen - version 2015-06-25 - (requires ps >= 3.0)
// Tuned for use post resize, EXPECTS FULL RANGE GAMMA LIGHT

#pragma optionNV (unroll all)

#define curve_height    1.0                  // Main sharpening strength, POSITIVE VALUE ONLY!
                                             // 0.3 <-> 1.5 is a reasonable range of values

#define video_level_out false                // True to preserve BTB & WTW (minor summation error)
                                             // Normally it should be set to false

// Defined values under this row are "optimal" DO NOT CHANGE IF YOU DO NOT KNOW WHAT YOU ARE DOING!

#define curveslope      (curve_height*0.7)   // Sharpening curve slope, edge region
#define D_overshoot     0.001                // Max dark overshoot before max compression
#define D_comp_ratio    0.250                // Max compression ratio, dark overshoot (1/0.25=4x)
#define L_overshoot     0.003                // Max light overshoot before max compression
#define L_comp_ratio    0.167                // Max compression ratio, light overshoot (1/0.167=6x)
#define max_scale_lim   0.1                 // Abs change before max compression (1/10=±10%)

// Saturation loss reduction
#define minim_satloss  ( (c[0].rgb*(CtL(c[0].rgb + sharpdiff)/c0_Y) + (c[0].rgb + sharpdiff))/2 )

// Soft limit
#define soft_lim(v,s)  ( ((exp(2*min(abs(v), s*16)/s) - 1)/(exp(2*min(abs(v), s*16)/s) + 1))*s )

// Get destination pixel values
#define get(x,y)       ( texture(tex, pos + vec2(x*px, y*py)) )
#define sat(input)     ( vec3(clamp((input).xyz, 0.0, 1.0) ) )

// Colour to luma
#define CtL(RGB) ( pow(dot(vec3(0.256, 0.651, 0.093), clamp((RGB).rgb, 0.0, 1.0)), 1.2) )

// Center pixel diff
#define mdiff(a,b,c,d,e,f,g) ( abs(luma[g]-luma[a]) + abs(luma[g]-luma[b])           \
                             + abs(luma[g]-luma[c]) + abs(luma[g]-luma[d])           \
                             + 0.5*(abs(luma[g]-luma[e]) + abs(luma[g]-luma[f])) )

vec4 sample(sampler2D tex, vec2 pos, vec2 tex_size) {
	// Pixel "width"
	float px = 1.0 / tex_size[0];
	float py = 1.0 / tex_size[1];
	
	vec4 orig = texture(tex, pos);
	
	// Get points and saturate out of range values (BTB & WTW)
	// [                c22               ]
	// [           c24, c9,  c23          ]
	// [      c21, c1,  c2,  c3, c18      ]
	// [ c19, c10, c4,  c0,  c5, c11, c16 ]
	// [      c20, c6,  c7,  c8, c17      ]
	// [           c15, c12, c14          ]
	// [                c13               ]
	vec3 c[25] = vec3[](sat( orig), get(-1,-1), get( 0,-1), get( 1,-1), get(-1, 0),
	                    get( 1, 0), get(-1, 1), get( 0, 1), get( 1, 1), get( 0,-2),
	                    get(-2, 0), get( 2, 0), get( 0, 2), get( 0, 3), get( 1, 2),
	                    get(-1, 2), get( 3, 0), get( 2, 1), get( 2,-1), get(-3, 0),
	                    get(-2, 1), get(-2,-1), get( 0,-3), get( 1,-2), get(-1,-2));
	
	// Blur, gauss 3x3
	vec3  blur   = (2*(c[2]+c[4]+c[5]+c[7]) + (c[1]+c[3]+c[6]+c[8]) + 4*c[0])/16;
	float blur_Y = (blur.r*(1.0/3.0) + blur.g*(1.0/3.0) + blur.b*(1.0/3.0));
	
	// Contrast compression, center = 0.5, scaled to 1/3
	float c_comp = clamp(0.266666681f + 0.9*pow(2, (-7.4*blur_Y)), 0.0, 1.0);	
	
	// Edge detection
	// Matrix, relative weights
	// [           1          ]
	// [       4,  4,  4      ]
	// [   1,  4,  4,  4,  1  ]
	// [       4,  4,  4      ]
	// [           1          ]
	float edge = length( abs(blur-c[0]) + abs(blur-c[1]) + abs(blur-c[2]) + abs(blur-c[3])
					+ abs(blur-c[4]) + abs(blur-c[5]) + abs(blur-c[6]) + abs(blur-c[7]) + abs(blur-c[8])
					+ 0.25*(abs(blur-c[9]) + abs(blur-c[10]) + abs(blur-c[11]) + abs(blur-c[12])) );
	
	// Edge detect contrast compression, center = 0.5
	edge *= c_comp;
	
	// RGB to luma
	float c0_Y = CtL(c[0]);
	
	float luma[25] = float[](c0_Y, CtL(c[1]), CtL(c[2]), CtL(c[3]), CtL(c[4]), CtL(c[5]), CtL(c[6]),
	                         CtL(c[7]),  CtL(c[8]),  CtL(c[9]),  CtL(c[10]), CtL(c[11]), CtL(c[12]),
	                         CtL(c[13]), CtL(c[14]), CtL(c[15]), CtL(c[16]), CtL(c[17]), CtL(c[18]),
	                         CtL(c[19]), CtL(c[20]), CtL(c[21]), CtL(c[22]), CtL(c[23]), CtL(c[24]));

	// Precalculated default squared kernel weights
	vec3 w1 = vec3(0.5,           1.0, 1.41421356237); // 0.25, 1.0, 2.0
	vec3 w2 = vec3(0.86602540378, 1.0, 0.5477225575);  // 0.75, 1.0, 0.3

	// Transition to a smaller sharpening kernel if the center edge val is above thr
	vec3 dW = pow(mix( w1, w2, smoothstep( 0.3, 0.6, edge)), vec3(2.0));

	// Partial laplacian outer pixel weighting
	float mdiff_c0  = 0.02 + 3*( abs(luma[0]-luma[2]) + abs(luma[0]-luma[4])
	                       +     abs(luma[0]-luma[5]) + abs(luma[0]-luma[7])
	                       + 0.25*(abs(luma[0]-luma[1]) + abs(luma[0]-luma[3])
	                              +abs(luma[0]-luma[6]) + abs(luma[0]-luma[8])) );
	
	float weights[12]  = float[]( ( dW.x ), ( dW.x ), ( dW.x ), ( dW.x ), // c2, // c4, // c5, // c7
	                       ( min(mdiff_c0/mdiff(24, 21, 2,  4,  9,  10, 1),  dW.y) ),   // c1
	                       ( min(mdiff_c0/mdiff(23, 18, 5,  2,  9,  11, 3),  dW.y) ),   // c3
	                       ( min(mdiff_c0/mdiff(4,  20, 15, 7,  10, 12, 6),  dW.y) ),   // c6
	                       ( min(mdiff_c0/mdiff(5,  7,  17, 14, 12, 11, 8),  dW.y) ),   // c8
	                       ( min(mdiff_c0/mdiff(2,  24, 23, 22, 1,  3,  9),  dW.z) ),   // c9
	                       ( min(mdiff_c0/mdiff(20, 19, 21, 4,  1,  6,  10), dW.z) ),   // c10
	                       ( min(mdiff_c0/mdiff(17, 5,  18, 16, 3,  8,  11), dW.z) ),   // c11
	                       ( min(mdiff_c0/mdiff(13, 15, 7,  14, 6,  8,  12), dW.z) ) ); // c12
	
	weights[4] = (max(max((weights[8]  + weights[9])/4,  weights[4]), 0.25) + weights[4])/2;
	weights[5] = (max(max((weights[8]  + weights[10])/4, weights[5]), 0.25) + weights[5])/2;
	weights[6] = (max(max((weights[9]  + weights[11])/4, weights[6]), 0.25) + weights[6])/2;
	weights[7] = (max(max((weights[10] + weights[11])/4, weights[7]), 0.25) + weights[7])/2;
	
	// Calculate the negative part of the laplace kernel
	float weightsum   = 0;
	float neg_laplace = 0;
	
	int order[12] = int[]( 2, 4, 5, 7, 1, 3, 6, 8, 9, 10, 11, 12 );
	
	for (int pix = 0; pix < 12; ++pix)
	{
		neg_laplace  += luma[order[pix]] *(weights[pix]);
		weightsum    += weights[pix];
	}
	
	// Compute sharpening magnitude function, x = edge mag, y = laplace operator mag
	float sharpen_val =  0.01 + (curve_height/(curveslope*pow(edge, 3.5) + 0.5))
	                          - (curve_height/(8192*pow((edge*2.4), 4.5) + 0.5));
	
	// Calculate sharpening diff and scale
	float sharpdiff = (c0_Y - (neg_laplace/weightsum))*(sharpen_val*0.8);
	
	// Calculate local near min & max, partial cocktail sort (No branching!)
	for (int i = 0; i < 3; ++i)
	{
		for (int i1 = 1+i; i1 < 25-i; ++i1)
		{
			float temp   = luma[i1-1];
			luma[i1-1] = min(luma[i1-1], luma[i1]);
			luma[i1]   = max(temp, luma[i1]);
		}
	
		for (int i2 = 23-i; i2 > i; --i2)
		{
			float temp   = luma[i2-1];
			luma[i2-1] = min(luma[i2-1], luma[i2]);
			luma[i2]   = max(temp, luma[i2]);
		}
	}
	
	float nmax = max(((luma[22] + luma[23]*2 + luma[24])/4), c0_Y);
	float nmin = min(((luma[0]  + luma[1]*2  + luma[2])/4),  c0_Y);
	
	// Calculate tanh scale factor, pos/neg
	float nmax_scale = min(((nmax - c0_Y) + L_overshoot), max_scale_lim);
	float nmin_scale = min(((c0_Y - nmin) + D_overshoot), max_scale_lim);
	
	// Soft limit sharpening with tanh, lerp to control maximum compression
	sharpdiff = mix(  (soft_lim(max(sharpdiff, 0), nmax_scale)), max(sharpdiff, 0), L_comp_ratio )
			  + mix( -(soft_lim(min(sharpdiff, 0), nmin_scale)), min(sharpdiff, 0), D_comp_ratio );
	
	if (video_level_out == true)
	{
		if (sharpdiff > 0) { return vec4( orig.rgb + (minim_satloss - c[0].rgb), 1.0 ); }
	
		else { return vec4( (orig.rgb + sharpdiff), 1.0 ); }
	}
	
	// Normal path
	if (sharpdiff > 0) { return vec4( minim_satloss, 1.0 ); }
	
	else { return vec4( (c[0].rgb + sharpdiff), 1.0 ); }
}

how to convert this to hlsl as 2 pass filter doesn't work for me on mpc.

[BRM]

Originally Posted by ts1

I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

Both turned off, mpv spits this out whenever I run it with the 1-pass filter in page 3 (2-pass gist version is unusable now for some reason, slows everything down to a crawl and is overly intense).

[vo/opengl] fragment shader compile log (status=0):
[vo/opengl] 0:96(15): error: type error in array constructor: expected: vec3, found vec4
[vo/opengl] 
[vo/opengl] shader link log (status=0): error: linking with uncompiled shader
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[fpsadjust] Setting speed to 1 
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.022 DS: 2.500/0 Cache:  9s+30MB
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.006 DS: 2.250/0 Cache:  9s+40MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.005 DS: 2.200/0 Cache:  9s+47MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.005 DS: 2.167/0 Cache:  9s+53MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.003 DS: 2.286/0 Cache:  9s+58MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V:  0.001 DS: 2.250/0 Cache:  9s+67MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:02 / 00:06:29 (0%) A-V: -0.005 DS: 2.389/0 Cache:  9s+302MB

[ts1]

Post whole log.

Try updated.

[BRM]

Originally Posted by ts1

Post whole log.

Try updated.

Too strong and slows down video playback a LOT (literally cannot run at more than 3fps, using a gigabyte 280x with the radeon git driver).

The rest of the log contained the shader and that's it.