Log in

View Full Version : Adaptive sharpen - Image sharpening shader


Pages : 1 [2] 3

madshi
29th October 2015, 17:31
For example, if you blur the example image with a gaussian filter or a median filter it gets darker
Does it, though, when doing the blurring in linear light? Which is how blurring should ideally be done.

bacondither
30th October 2015, 01:35
Does it, though, when doing the blurring in linear light? Which is how blurring should ideally be done.

Yes, forgot that my image editor did blurring in gamma light. Blurred it in linear light and the apparent brightness was then about the same as the original.

Did some tests and saw that doing the summation of the negative laplace vales in "fast" linear light did reduce the brightening effect.
Have not seen any negative effects of doing it that way yet.

Added it to the current experimental version.
Experimental - 2015-10-29 (https://u.teknik.io/qy20mv.zip)

madshi
30th October 2015, 03:19
Great, thanks - any progress is welcome! :)

One other thing that I've noticed when comparing FineSharp with AdaptiveSharpen: They have a quite different look. FineSharp somehow makes lines (at least dark lines) appear thinner, while AdaptiveSharpen seems to have a certain tendency to make dark lines fatter. Personally, I like the look of thinner lines. Is that something that could maybe be improved in AdaptiveSharpen?

aufkrawall
30th October 2015, 21:02
Added it to the current experimental version.
Experimental - 2015-10-29 (https://u.teknik.io/qy20mv.zip)
Amazing, image brightness pretty much doesn't increase and it still looks as good as usual otherwise. :eek:

Yes, line thickening is noticeable, especially with cartoons. This can look nice with low strength after soft scaling algorithms, but with higher strength it gets a bit too obvious.

jewshawn2
6th November 2015, 02:19
I can't seem to get any of the new 2 pass shaders to work with potplayer. However, the early single pass shader works fine.

I use the "Shader Stacker" but no matter what I do I end up with a green screen. But MPC-HC works perfectly fine.

I would much appreciate it if someone else can someone verify?

toniash
6th November 2015, 09:05
I can't seem to get any of the new 2 pass shaders to work with potplayer. However, the early single pass shader works fine.

I use the "Shader Stacker" but no matter what I do I end up with a green screen. But MPC-HC works perfectly fine.

I would much appreciate it if someone else can someone verify?
In "trade quality for performance" you must UNTICK:
store custom pixel shader in 16 bit ...
use 10bit chroma ...
use 10bit image ...
run custom pixel shaders in video levels ...

hope it helps

CruNcher
6th November 2015, 11:47
Not sure which version is used inside of MPC-BE currently but it does a great job at relatively low shader costs (~10% above the simpler ones) :)

Very impressive results even on rather complex textures but shimmering in high frequency skylines for example still seems to be a big problem at least in the version used inside of MPC-BE.

I use it as a very decent post resize pass solution, which works out for most different compressed MPEG content just great without oversharpen it to much that it feels odd :)

http://i2.sendpic.org/t/3o/3oTpr1c09evN2V1MOXGr3fjtgzi.jpg (http://sendpic.org/view/2/i/55YOJaWChqs3D0KhddFfZEcsXUS.png)http://i2.sendpic.org/t/jl/jlZrMEjPKEbu7nW7xSogt5DcDyH.jpg (http://sendpic.org/view/2/i/j275irS8d6x0ZJRLIaChyt7SFCj.png)http://i1.sendpic.org/t/ph/phnAbVgg8co6HZvj0JiqhKQyem1.jpg (http://sendpic.org/view/1/i/gAB6i4sTRsa8nsVUA03YzGxQY1E.png)

http://i1.sendpic.org/t/95/95kgfqSdffARoOB8FIdoI7qOnhN.jpg (http://sendpic.org/view/1/i/fg1FNBcoQ0RMWjShlUrMc8fbt9B.png)http://i1.sendpic.org/t/g8/g879123LMHEmFsi2oQDCVasEBrn.jpg (http://sendpic.org/view/1/i/2K9hM9OiEJm3t5ute2FXGJY456k.png)http://i2.sendpic.org/t/wO/wO6vOXUZk61HI3XUyU6temyzK3a.jpg (http://sendpic.org/view/2/i/lesT6jOkdWdxIz4k8hmE6G3eUci.png)

http://i1.sendpic.org/t/jY/jYhDRiiQvkE318QNrfnpNXhecuk.jpg (http://sendpic.org/view/1/i/1xnoy2RuMfIldutcu1KPNWu0Lwy.png)http://i1.sendpic.org/t/jx/jxtnRthm3Ob6Fu1k3VP09FCVHf8.jpg (http://sendpic.org/view/1/i/kjPKP0Cvc9vL5vAGCIut9EJmxwr.png)http://i2.sendpic.org/t/aF/aFivjJWFLatrylVIPGpHr0y9Hp9.jpg (http://sendpic.org/view/2/i/8MlbPHAfN3JplvfiK5hfJYlJZmE.png)

http://i2.sendpic.org/t/mw/mwJUh9jvmcRQI02GwidZYKDVBlO.jpg (http://sendpic.org/view/2/i/aRdDz1T4yik8sEUywXZleXvH6Hm.png)http://i2.sendpic.org/t/yq/yqXV82LwbSAWDovpF8MQJkAIS3J.jpg (http://sendpic.org/view/2/i/1BS3XxubBZQlL8vQbkEZjKkXWTq.png)http://i2.sendpic.org/t/u0/u0L7mZYXh8UICgLfgVJ4wythyXC.jpg (http://sendpic.org/view/2/i/9DzsxLo2f2T9FZQLQwXcG9H96aK.png)

bacondither
6th November 2015, 16:08
I can't seem to get any of the new 2 pass shaders to work with potplayer. However, the early single pass shader works fine.

I use the "Shader Stacker" but no matter what I do I end up with a green screen. But MPC-HC works perfectly fine.

If you use madvr as renderer you could also try unticking the "lose BTB and WTW if it improves performance"

Not sure which version is used inside of MPC-BE currently
I think it uses the old one-pass 2015-06-25 one.

I could backport some things to the old one pass version and make it better for a very small performance cost.

bacondither
6th November 2015, 16:13
The new 2015-11-05 experimental should be a little bit faster then the 2015-10-29 one. ~1% to 2%.

vood007
6th November 2015, 18:24
I could backport some things to the old one pass version and make it better for a very small performance cost.

This would be awesome! I still use the older 1-pass version because it allows me to stay at the lowest power-state of my GFX.

Popwax
6th November 2015, 21:26
I tried the GLSL port with MPV, but the result is looking weird, could anyone post a sample screenshot (before and after sharpening) of the HLSL shader, so that I can make a comparison?

bacondither
7th November 2015, 01:08
Some test images, used the 2015-11-05 shader with default settings.

Blurred test image (http://screenshotcomparison.com/comparison/149622)

AVS HD 709 - Blu-ray & MP4 Calibration, Sharpness & Overscan (http://screenshotcomparison.com/comparison/149624)

Ghost in the shell (http://screenshotcomparison.com/comparison/149626)

Popwax
7th November 2015, 11:41
Some test images, used the 2015-11-05 shader with default settings.

Blurred test image (http://screenshotcomparison.com/comparison/149622)

AVS HD 709 - Blu-ray & MP4 Calibration, Sharpness & Overscan (http://screenshotcomparison.com/comparison/149624)

Ghost in the shell (http://screenshotcomparison.com/comparison/149626)

The GLSL port doesn't produce the exact same sharpened image. It widen the edge sigificantly for the first test image, doesn't change anything visiblely for the second one. There are also slight differences for the third test image. Is this due to version different (GLSL port is based on 2015-09 I believe)I? What's the possible cause which can lead to widen edge in the first test image?

ts1
7th November 2015, 12:28
Here is mine comparison (this is without scaling):
http://screenshotcomparison.com/comparison/149655

Popwax
7th November 2015, 13:20
Here is mine comparison (this is without scaling):
http://screenshotcomparison.com/comparison/149655

Actually what I want to compare are the sharpened images of GLSL and HLSL shader, since I notice distortion of the GLSL shader for noisy images.

bacondither: Could you post more test images for video with heavy film grains?

ts1
7th November 2015, 14:04
Yeah, output result is different. MPC-HC vs MPV:
http://screenshotcomparison.com/comparison/149670

ts1
7th November 2015, 15:05
This is only with opengl-hq preset (sigmoid-upscaling in particular). With opengl result is identical.

vood007
7th November 2015, 15:31
There is some sort of overshot in the glsl (one-pass) version. Upper is HLSL, lower GLSL.

http://thumbnails113.imagebam.com/44542/5d2888445416062.jpg (http://www.imagebam.com/image/5d2888445416062)

ts1
7th November 2015, 15:41
vood007
Try updated glsl 1 pass shader.

vood007
7th November 2015, 16:19
vood007
Try updated glsl 1 pass shader.

Yep looks fine now, thanks!

Popwax
7th November 2015, 17:59
This is only with opengl-hq preset (sigmoid-upscaling in particular). With opengl result is identical.

Oh, that make sense. With "sigmoid-upscaling" option the shader is applied on linear RGB, but the original shader explicitly stated to require gamma light.

EDIT: I can confirm that with "sigmoid-upscaling" disabled, the GLSL shader can produce exactly same sharpened image.

XMonarchY
13th November 2015, 23:37
Is there any chance the latest experimental AdaptiveSharpen (11/5/2015) will get an update soon to work properly with DirectX 11? I mainly use AdaptiveSharpen in games through ReShade and right now it says that this latest AdaptiveSharpen does not work as intended in DirectX 11. Does it create some odd artifacts in DirectX 11 or slow down performance too much?

bacondither
14th November 2015, 00:40
Is there any chance the latest experimental AdaptiveSharpen (11/5/2015) will get an update soon to work properly with DirectX 11? I mainly use AdaptiveSharpen in games through ReShade and right now it says that this latest AdaptiveSharpen does not work as intended in DirectX 11. Does it create some odd artifacts in DirectX 11 or slow down performance too much?

Hmm.... does it complain about "warning X3081: potentially unintended use of a comma expression in a variable initializer"?

If so, i have already fixed it. It did not result in any errors just the compiler complaining.

CruNcher
15th November 2015, 14:13
if i look @ the GPU usage even that small amount 10% hit is huge for games resources.

JPulowski
15th November 2015, 19:28
Hmm.... does it complain about "warning X3081: potentially unintended use of a comma expression in a variable initializer"?

If so, i have already fixed it. It did not result in any errors just the compiler complaining.No, the compiler does not give any errors. In DX11, white dot artifacts occur around very thin or very bright surfaces (e.g. stars in the sky, in-game text etc.). Will upload a screenshot when I am available. Not sure what is the problem exactly, ReShade's transcompiler might be the culprit but I don't think that is the case unless there are very tricky shader model specific instructions. Lowering the strength of the shader removes some of the artifacts, but it doesn't make any sense since in DX9 the shader works as it should.

Edit: Here it is, notice the black dots around text and stars:
https://i.imgur.com/9lmTylj.png

bacondither
18th November 2015, 19:28
No, the compiler does not give any errors. In DX11, white dot artifacts occur around very thin or very bright surfaces (e.g. stars in the sky, in-game text etc.). Will upload a screenshot when I am available. Not sure what is the problem exactly, ReShade's transcompiler might be the culprit but I don't think that is the case unless there are very tricky shader model specific instructions. Lowering the strength of the shader removes some of the artifacts, but it doesn't make any sense since in DX9 the shader works as it should.
Tested reshade with adaptive sharpen on some dx11 samples.
Found that the problem was with the tanh function that overflowed and resulted in a NaN in dx11 and a 0 in dx9.
It was not really visible in dx9 because it would return 0 when often it should be limited to a very small value near 0.

Modified the AdaptiveSharpen.h so it should work fine now.
Fixed AdaptiveSharpen.h (https://u.teknik.io/r2ASRR.zip)

bacondither
18th November 2015, 19:36
New stable version 2015-11-17 and experimental version 2015-11-17-E.

JPulowski
19th November 2015, 07:33
Tested reshade with adaptive sharpen on some dx11 samples.
Found that the problem was with the tanh function that overflowed and resulted in a NaN in dx11 and a 0 in dx9.
It was not really visible in dx9 because it would return 0 when often it should be limited to a very small value near 0.

Modified the AdaptiveSharpen.h so it should work fine now.
Fixed AdaptiveSharpen.h (https://u.teknik.io/r2ASRR.zip)Thanks, I have also uploaded the fix to ReShade's repository.

Would you mind telling me which part of the code have you modified? Or did you completely fix it, so there is no need to apply a hotfix after the code has been ported?

bacondither
19th November 2015, 11:19
Thanks, I have also uploaded the fix to ReShade's repository.

Would you mind telling me which part of the code have you modified? Or did you completely fix it, so there is no need to apply a hotfix after the code has been ported?

Updated everything to version 2015-11-17. The code that fixed the overflow can be found at line 61 and 243-244 in the new AdaptiveSharpen.h.

Diff for AdaptiveSharpen.h (https://www.diffchecker.com/q9m0ezvb)

aufkrawall
19th November 2015, 20:39
I should have looked at the image in unscaled size to judge brightness, sorry.
Don't want to sound like a broken record, but maybe you could still do something regarding brightness?
It still gets noticeably brighter with a strength of 0.5.

off:
http://abload.de/thumb/offeak8l.png (http://abload.de/image.php?img=offeak8l.png)

0.5:
http://abload.de/thumb/0.5n8k5p.png (http://abload.de/image.php?img=0.5n8k5p.png)

bacondither
7th December 2015, 08:54
Don't want to sound like a broken record, but maybe you could still do something regarding brightness?
Tuning it to one scenario would make it pretty crappy in other places, i have currently no idea to "solve" it with an algorithm that would work in shaders or be fast.
But the latest version 2015-12-04-E should be somewhat better at fine details in most cases. Uploaded a comparison of 2015-11-17E and 2015-12-04E (http://screenshotcomparison.com/comparison/153261) with all settings at default.

I also experimented some with sharpening that test image in raw therapee with RL deconvolution, that made the image alot more brighter!

aufkrawall
7th December 2015, 14:12
This is a very nice progress, thanks a lot!
I think this can be very helpful in games, e.g. when strong sharpen is needed to fight temporal AA blur. May be a bit expensive in realtime, but should be great for screenshots.

JPulowski
9th December 2015, 17:47
In games, AdaptiveSharpen requires quite a bit of performance. In my GPU it utilizes 15 - 20% of extra computing power. In movies where the framerate is between 23.976 - 30.0 fps, the performance drop does not make a huge difference. But in recent video games, every bit of performance is important. bacondither, do you have any ideas about possible optimizations? Currently the shader uses an additional RGBA16F texture for storing the alpha value of the first pass. This is a bit too much just for one floating point value. I might use R16F or encode the value as RGBA8 and decode it in the second pass afterwards, depending on which method is faster. Other than that, it is now my favorite shader for sharpening. It really goes well with the video games.

We also have a GitHub repo (https://github.com/crosire/reshade-shaders). You are welcome to modify the code as you see fit. :)

ts1
12th December 2015, 11:17
I've ported some things from 2-pass shaders.

Edit: fixed a bug

foozoor
13th December 2015, 20:50
I've ported some things from 2-pass shaders.

Edit: fixed a bug

Where can we download it? Can I use it in mpv?
Thanks I love this sharpener but I think 1.0 is too much as default value.

Edit: I found it inside the mpv wiki... :rolleyes:
https://gist.github.com/bjin/152a2f8bdf87d83261ac

ts1
14th December 2015, 14:47
foozoor
1-pass version is not so strong, try it, it's on a 3rd page.

foozoor
17th December 2015, 22:23
foozoor
1-pass version is not so strong, try it, it's on a 3rd page.
Thank you, really nice. :)

Is there something between lumasharpen and adaptive sharpen for mpv?
Because my gpu can‘t handle adaptive sharpen with some video files.
Could you port finesharp or lsfmod as mpv shader?

ts1
17th December 2015, 23:00
Sorry, but no.
Use it as prescale then maybe.

lanzorg
17th December 2015, 23:10
foozoor
1-pass version is not so strong, try it, it's on a 3rd page.
Should I turn `linear-scaling` and `sigmoid-upscaling` off like the 2-passes shader?

ts1
17th December 2015, 23:14
lanzorg
Yes

bacondither
18th December 2015, 11:54
Do you have any ideas about possible optimizations?

Sure, did a pull request on reshade-shaders now. Up to 45% speedup on rendering times for the shader (3.2ms to 2.2ms) at 1080p on my 7790. :p

aufkrawall
19th December 2015, 14:24
With a 980, I can note a gain. However, it's not large.
No ReShade shaders: 96fps
AS old: 81fps
AS new: 83fps

bacondither
19th December 2015, 14:41
With a 980, I can note a gain. However, it's not large.

At least that's 18% faster rendering times for the shader.
Did you have have "fast_ops = 1" in JPulowski.cfg?

I did my test on a DX11 target application.

aufkrawall
19th December 2015, 15:08
Yep, had it at 1 and tested DX11.
With a higher resolution, the results don't get wider:
42 vs 41 fps :)

ts1
19th December 2015, 20:21
I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

lanzorg
22nd December 2015, 03:37
I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

Thanks! :D

darkasylum666
28th December 2015, 06:11
// Copyright (c) 2015, bacondither
// All rights reserved.
//
// Redistribution and use in source and binary forms, with or without
// modification, are permitted provided that the following conditions
// are met:
// 1. Redistributions of source code must retain the above copyright
// notice, this list of conditions and the following disclaimer
// in this position and unchanged.
// 2. Redistributions in binary form must reproduce the above copyright
// notice, this list of conditions and the following disclaimer in the
// documentation and/or other materials provided with the distribution.
//
// THIS SOFTWARE IS PROVIDED BY THE AUTHORS ``AS IS'' AND ANY EXPRESS OR
// IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
// OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
// IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
// INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
// NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
// THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

// Adaptive sharpen - version 2015-06-25 - (requires ps >= 3.0)
// Tuned for use post resize, EXPECTS FULL RANGE GAMMA LIGHT

#pragma optionNV (unroll all)

#define curve_height 1.0 // Main sharpening strength, POSITIVE VALUE ONLY!
// 0.3 <-> 1.5 is a reasonable range of values

#define video_level_out false // True to preserve BTB & WTW (minor summation error)
// Normally it should be set to false

// Defined values under this row are "optimal" DO NOT CHANGE IF YOU DO NOT KNOW WHAT YOU ARE DOING!

#define curveslope (curve_height*0.7) // Sharpening curve slope, edge region
#define D_overshoot 0.001 // Max dark overshoot before max compression
#define D_comp_ratio 0.250 // Max compression ratio, dark overshoot (1/0.25=4x)
#define L_overshoot 0.003 // Max light overshoot before max compression
#define L_comp_ratio 0.167 // Max compression ratio, light overshoot (1/0.167=6x)
#define max_scale_lim 0.1 // Abs change before max compression (1/10=±10%)

// Saturation loss reduction
#define minim_satloss ( (c[0].rgb*(CtL(c[0].rgb + sharpdiff)/c0_Y) + (c[0].rgb + sharpdiff))/2 )

// Soft limit
#define soft_lim(v,s) ( ((exp(2*min(abs(v), s*16)/s) - 1)/(exp(2*min(abs(v), s*16)/s) + 1))*s )

// Get destination pixel values
#define get(x,y) ( texture(tex, pos + vec2(x*px, y*py)) )
#define sat(input) ( vec3(clamp((input).xyz, 0.0, 1.0) ) )

// Colour to luma
#define CtL(RGB) ( pow(dot(vec3(0.256, 0.651, 0.093), clamp((RGB).rgb, 0.0, 1.0)), 1.2) )

// Center pixel diff
#define mdiff(a,b,c,d,e,f,g) ( abs(luma[g]-luma[a]) + abs(luma[g]-luma[b]) \
+ abs(luma[g]-luma[c]) + abs(luma[g]-luma[d]) \
+ 0.5*(abs(luma[g]-luma[e]) + abs(luma[g]-luma[f])) )

vec4 sample(sampler2D tex, vec2 pos, vec2 tex_size) {
// Pixel "width"
float px = 1.0 / tex_size[0];
float py = 1.0 / tex_size[1];

vec4 orig = texture(tex, pos);

// Get points and saturate out of range values (BTB & WTW)
// [ c22 ]
// [ c24, c9, c23 ]
// [ c21, c1, c2, c3, c18 ]
// [ c19, c10, c4, c0, c5, c11, c16 ]
// [ c20, c6, c7, c8, c17 ]
// [ c15, c12, c14 ]
// [ c13 ]
vec3 c[25] = vec3[](sat( orig), get(-1,-1), get( 0,-1), get( 1,-1), get(-1, 0),
get( 1, 0), get(-1, 1), get( 0, 1), get( 1, 1), get( 0,-2),
get(-2, 0), get( 2, 0), get( 0, 2), get( 0, 3), get( 1, 2),
get(-1, 2), get( 3, 0), get( 2, 1), get( 2,-1), get(-3, 0),
get(-2, 1), get(-2,-1), get( 0,-3), get( 1,-2), get(-1,-2));

// Blur, gauss 3x3
vec3 blur = (2*(c[2]+c[4]+c[5]+c[7]) + (c[1]+c[3]+c[6]+c[8]) + 4*c[0])/16;
float blur_Y = (blur.r*(1.0/3.0) + blur.g*(1.0/3.0) + blur.b*(1.0/3.0));

// Contrast compression, center = 0.5, scaled to 1/3
float c_comp = clamp(0.266666681f + 0.9*pow(2, (-7.4*blur_Y)), 0.0, 1.0);

// Edge detection
// Matrix, relative weights
// [ 1 ]
// [ 4, 4, 4 ]
// [ 1, 4, 4, 4, 1 ]
// [ 4, 4, 4 ]
// [ 1 ]
float edge = length( abs(blur-c[0]) + abs(blur-c[1]) + abs(blur-c[2]) + abs(blur-c[3])
+ abs(blur-c[4]) + abs(blur-c[5]) + abs(blur-c[6]) + abs(blur-c[7]) + abs(blur-c[8])
+ 0.25*(abs(blur-c[9]) + abs(blur-c[10]) + abs(blur-c[11]) + abs(blur-c[12])) );

// Edge detect contrast compression, center = 0.5
edge *= c_comp;

// RGB to luma
float c0_Y = CtL(c[0]);

float luma[25] = float[](c0_Y, CtL(c[1]), CtL(c[2]), CtL(c[3]), CtL(c[4]), CtL(c[5]), CtL(c[6]),
CtL(c[7]), CtL(c[8]), CtL(c[9]), CtL(c[10]), CtL(c[11]), CtL(c[12]),
CtL(c[13]), CtL(c[14]), CtL(c[15]), CtL(c[16]), CtL(c[17]), CtL(c[18]),
CtL(c[19]), CtL(c[20]), CtL(c[21]), CtL(c[22]), CtL(c[23]), CtL(c[24]));

// Precalculated default squared kernel weights
vec3 w1 = vec3(0.5, 1.0, 1.41421356237); // 0.25, 1.0, 2.0
vec3 w2 = vec3(0.86602540378, 1.0, 0.5477225575); // 0.75, 1.0, 0.3

// Transition to a smaller sharpening kernel if the center edge val is above thr
vec3 dW = pow(mix( w1, w2, smoothstep( 0.3, 0.6, edge)), vec3(2.0));

// Partial laplacian outer pixel weighting
float mdiff_c0 = 0.02 + 3*( abs(luma[0]-luma[2]) + abs(luma[0]-luma[4])
+ abs(luma[0]-luma[5]) + abs(luma[0]-luma[7])
+ 0.25*(abs(luma[0]-luma[1]) + abs(luma[0]-luma[3])
+abs(luma[0]-luma[6]) + abs(luma[0]-luma[8])) );

float weights[12] = float[]( ( dW.x ), ( dW.x ), ( dW.x ), ( dW.x ), // c2, // c4, // c5, // c7
( min(mdiff_c0/mdiff(24, 21, 2, 4, 9, 10, 1), dW.y) ), // c1
( min(mdiff_c0/mdiff(23, 18, 5, 2, 9, 11, 3), dW.y) ), // c3
( min(mdiff_c0/mdiff(4, 20, 15, 7, 10, 12, 6), dW.y) ), // c6
( min(mdiff_c0/mdiff(5, 7, 17, 14, 12, 11, 8), dW.y) ), // c8
( min(mdiff_c0/mdiff(2, 24, 23, 22, 1, 3, 9), dW.z) ), // c9
( min(mdiff_c0/mdiff(20, 19, 21, 4, 1, 6, 10), dW.z) ), // c10
( min(mdiff_c0/mdiff(17, 5, 18, 16, 3, 8, 11), dW.z) ), // c11
( min(mdiff_c0/mdiff(13, 15, 7, 14, 6, 8, 12), dW.z) ) ); // c12

weights[4] = (max(max((weights[8] + weights[9])/4, weights[4]), 0.25) + weights[4])/2;
weights[5] = (max(max((weights[8] + weights[10])/4, weights[5]), 0.25) + weights[5])/2;
weights[6] = (max(max((weights[9] + weights[11])/4, weights[6]), 0.25) + weights[6])/2;
weights[7] = (max(max((weights[10] + weights[11])/4, weights[7]), 0.25) + weights[7])/2;

// Calculate the negative part of the laplace kernel
float weightsum = 0;
float neg_laplace = 0;

int order[12] = int[]( 2, 4, 5, 7, 1, 3, 6, 8, 9, 10, 11, 12 );

for (int pix = 0; pix < 12; ++pix)
{
neg_laplace += luma[order[pix]] *(weights[pix]);
weightsum += weights[pix];
}

// Compute sharpening magnitude function, x = edge mag, y = laplace operator mag
float sharpen_val = 0.01 + (curve_height/(curveslope*pow(edge, 3.5) + 0.5))
- (curve_height/(8192*pow((edge*2.4), 4.5) + 0.5));

// Calculate sharpening diff and scale
float sharpdiff = (c0_Y - (neg_laplace/weightsum))*(sharpen_val*0.8);

// Calculate local near min & max, partial cocktail sort (No branching!)
for (int i = 0; i < 3; ++i)
{
for (int i1 = 1+i; i1 < 25-i; ++i1)
{
float temp = luma[i1-1];
luma[i1-1] = min(luma[i1-1], luma[i1]);
luma[i1] = max(temp, luma[i1]);
}

for (int i2 = 23-i; i2 > i; --i2)
{
float temp = luma[i2-1];
luma[i2-1] = min(luma[i2-1], luma[i2]);
luma[i2] = max(temp, luma[i2]);
}
}

float nmax = max(((luma[22] + luma[23]*2 + luma[24])/4), c0_Y);
float nmin = min(((luma[0] + luma[1]*2 + luma[2])/4), c0_Y);

// Calculate tanh scale factor, pos/neg
float nmax_scale = min(((nmax - c0_Y) + L_overshoot), max_scale_lim);
float nmin_scale = min(((c0_Y - nmin) + D_overshoot), max_scale_lim);

// Soft limit sharpening with tanh, lerp to control maximum compression
sharpdiff = mix( (soft_lim(max(sharpdiff, 0), nmax_scale)), max(sharpdiff, 0), L_comp_ratio )
+ mix( -(soft_lim(min(sharpdiff, 0), nmin_scale)), min(sharpdiff, 0), D_comp_ratio );

if (video_level_out == true)
{
if (sharpdiff > 0) { return vec4( orig.rgb + (minim_satloss - c[0].rgb), 1.0 ); }

else { return vec4( (orig.rgb + sharpdiff), 1.0 ); }
}

// Normal path
if (sharpdiff > 0) { return vec4( minim_satloss, 1.0 ); }

else { return vec4( (c[0].rgb + sharpdiff), 1.0 ); }
}


how to convert this to hlsl as 2 pass filter doesn't work for me on mpc.

BRM
6th January 2016, 16:56
I've made my port compatible with linear-scaling/sigmoid-upscaling. Also much less distortions now (if any).

Both turned off, mpv spits this out whenever I run it with the 1-pass filter in page 3 (2-pass gist version is unusable now for some reason, slows everything down to a crawl and is overly intense).

[vo/opengl] fragment shader compile log (status=0):
[vo/opengl] 0:96(15): error: type error in array constructor: expected: vec3, found vec4
[vo/opengl]
[vo/opengl] shader link log (status=0): error: linking with uncompiled shader
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[fpsadjust] Setting speed to 1
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.022 DS: 2.500/0 Cache: 9s+30MB
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
[vo/opengl] after rendering: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.006 DS: 2.250/0 Cache: 9s+40MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.005 DS: 2.200/0 Cache: 9s+47MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.005 DS: 2.167/0 Cache: 9s+53MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.003 DS: 2.286/0 Cache: 9s+58MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:00 / 00:06:29 (0%) A-V: 0.001 DS: 2.250/0 Cache: 9s+67MB
[vo/opengl] after creating framebuffer texture: OpenGL error INVALID_OPERATION.
AV: 00:00:02 / 00:06:29 (0%) A-V: -0.005 DS: 2.389/0 Cache: 9s+302MB

ts1
6th January 2016, 18:22
Post whole log.

Try updated.

BRM
8th January 2016, 22:16
Post whole log.

Try updated.

Too strong and slows down video playback a LOT (literally cannot run at more than 3fps, using a gigabyte 280x with the radeon git driver).

The rest of the log contained the shader and that's it.