View Full Version : Video pixel shader pack
Pages :
1
2
3
4
5
6
7
8
[
9]
10
11
XRyche
1st January 2014, 12:33
@JanWillem32: I'm quite sure you won't be interested, but maybe curiosity will persuade you.
Samsung TV offers a setting called "Dynamic Contrast" in options (not "CE dimming"), which changes original image, but I find it to be "nice looking" sometimes. You have 3 settings, but "low" is the best, 2 others change image too much.
Do you think you could write a shader for this, or it would me more complicated for the GPU ?
I suppose it changes pixel like this:
(post-resized by renderer)
Old Pixel: R=50; G=40; B=230
New Pixel:
R= 50+((50-128)*5%) = 46 (rounded)
G= 40+((40-128)*5%) = 36 (rounded)
B= 230+((230-128)*5%) = 235 (rounded)
Do you think my reasoning is stupid ?:confused:
Is there a simple way to test this algorithm with MPC ?
Just as JanWillem32 suggested use his CIECAM02 shaders. If for example your movies appear too dark (to the point of even losing some detail in dark ambiance and/or night-time scenes) and lowering gamma just washes everything out than the CIECAM02 shaders using the "average""User Settings" are for you. Just try them, they will pleasantly surprise you.
pirlouy
1st January 2014, 19:40
AFAIK the dynamic contrast option works by measuring ambient light in one's room so I guess it's impossible to simulate it correctly using shaders alone.
My TV does not have any sensor like this; it really is a modification with a precise algorithm.
@JanWillem32: Your example code works well. But what are those CIECAM02 shaders you're talking about ? I tried to search but did not find/understand what it is or where to download these shaders ? :/
JanWillem32
2nd January 2014, 03:10
James Freeman, many chroma up-sampling and other resizing methods already have sharpening internally. Modifying any sharpening factors on these is easy. The sharpest chroma up-samplers (I implemented several) are not always the best-looking though. The sharpness factors are just one part of good scaling. What processing chain did you use for this test? It's possible that all you want is a different chroma upsampler.
pirlouy, "dynamic contrast" just compresses image dynamics or changes gamma without using much logic. Using "precise algorithm" to describe such an effect is quite an overstatement.
The CIECAM02 transforms were already integrated in the EVR-CP/VMR9 r. color management options I helped to integrate. For the new basic color correction option I added, there is no such option yet. I wrote a separate pixel shader before integrating it into newer builds to test the function. See for reference: http://forum.doom9.org/showthread.php?p=1658826#post1658826 .
James Freeman
2nd January 2014, 07:06
James Freeman, many chroma up-sampling and other resizing methods already have sharpening internally. Modifying any sharpening factors on these is easy. The sharpest chroma up-samplers (I implemented several) are not always the best-looking though. The sharpness factors are just one part of good scaling. What processing chain did you use for this test? It's possible that all you want is a different chroma upsampler.
Jan
I use MadVR Lanczos/Jinc for chroma upscaling but even that is not sharp enough for what I have in mind.
The goal of this experiment is to make the 4:2:0 chroma look like 4:4:4 chroma which is a lot sharper an more saturated.
I simply want to have control over the sharpness of the chroma, and "Sharpness Complex 2" is the best in my taste.
I can make a short video to demonstrate what I have done so far.
EDIT:
Here is a short video where I compare 4:4:4 vs 4:2:0 vs 4:2:0+Sharpen: Chroma Sharpen Demo (http://www.mediafire.com/watch/plaj4kh59zv8jpg/Chroma%20Sharpen%20Demo.avi)
*Its a screen capture, 28MB uncompressed rgb32.
*Please download the original video (don't watch online => quality destroyed).
The upscaler in the video is MadVR's Lanczos 3+AR which is not sharp enough.
There are two presets that I use in this video.
Chroma Only: Shows just the chroma.
Chroma Sharpen: Shows just chroma + Sharpen Complex 2 (Pre-Upscaler/resize).
*Placing the Sharpen after-upscaling does not have the same effect!!
*Note that the The Sharpen Complex 2 is carefully tweaked to look like 4:4:4 and not at it default settings.
Look at the green lines as they become more vibrant like in the original 4:4:4.
Look at the red/cyan fine lines as they become more distinct and colorful like in the original 4:4:4.
You can clearly see that 4:2:0+Sharpen looks a lot closer to the original 4:4:4.
Leaving the Luma untouched to prevent ringing and artifacts, just a better looking, more "True to the Source" chroma.
EDIT 2:
An even clearer video of this "trick"
Chroma Sharpen Demo 2 (http://www.mediafire.com/watch/yzozc6mb7ujxcbo/Chroma%20Sharpen%20Demo%202.avi)
Here you can clearly see that sharpening the chroma makes "magic" in fine details, almost recreating the missing data.
Thanks for your time.
XRyche
2nd January 2014, 09:59
My TV does not have any sensor like this; it really is a modification with a precise algorithm.
@JanWillem32: Your example code works well. But what are those CIECAM02 shaders you're talking about ? I tried to search but did not find/understand what it is or where to download these shaders ? :/
The shaders are located in 2 text documents in the compressed file (.7z) Of JanWillem32's latest build of his renderer found here: http://forum.doom9.org/showthread.php?t=161047 . At the bottom of the very first post.
toniash
2nd January 2014, 14:03
The shaders are located in 2 text documents in the compressed file (.7z) Of JanWillem32's latest build of his renderer found here: http://forum.doom9.org/showthread.php?t=161047 . At the bottom of the very first post.
What's the name of the shaders? I can't find them ...
JanWillem32
2nd January 2014, 14:07
James Freeman, what exact processing chain did you use? When feeding textures containing Y'CbCr into the "Sharpen Complex 2" shader it does not exclude the Y' channel without modifications.
Also note that Lanczos filtering is sharp, but it's not the sharpest resizer. There are other options that sharpen more, but as expected, these give heavy artifacts.
James Freeman
2nd January 2014, 15:11
Jan,
My chain in MPC-HC:
http://www.mediafire.com/convkey/b68f/n6svgk5389vsghyfg.jpg
There is no luma in those videos, just chroma.
That is why I ask you for help.
There is currently no sharpen shader only for chroma.
We need something that will divide Y from CbCr, and only apply sharpening to CbCr then combine them to YCbCr again without touching the Y.
Lanczos filtering is sharp, but it's not the sharpest resizer. There are other options that sharpen more, but as expected, these give heavy artifacts.
Lanczos is the sharpest resizer in MadVR (to my eyes), but not sharp enough.
Thats why I use Sharpen Complex 2, to carefully sharpen the chroma image without adding unwanted artifacts.
But as stated, there is no way currently to combine back my chain with the removed Y (luma) and the already sharpen chroma.
JanWillem32
2nd January 2014, 23:34
That actually explains a lot. "chroma for SD&HD input" is a test shader. It converts R'G'B' to YCbCr, discards original luma, multiplies chroma by 1.5 (to make it more visible), and converts back to R'G'B' with the Y' channel fixed at .5. The output isn't chroma, but just weird R'G'B'. To get real Y'CbCr, use a "R'G'B' to Y'CbCr"-type shader. You can then really filter chroma. I've already written Y'CbCr filtering shaders. Doing any filtering in Y'CbCr is risky. The Y'CbCr forms don't directly specify any color; all specifications are dependent on R'G'B'/RGB color space (which in turn has specifications in XYZ color space and a reference surround specification). In the category of "everyday-use effect pixel shaders" I only made chroma up-sampler shaders and the deband/denoise/sharpen effect shaders for now. (That last effect is only to deal with encoder noise and bad raw image quantization, these are really not suitable for any dramatic effects.)
When I think of effects to compensate anything of a resizer, it will probably be completely dependant as a compensation factor for a specific resizer, and also be integrated into that resizer as well. Chroma up-samplers are essentially resizers, with pretty much the same problems.
As for sharper resizers than Lanczos, there are a few bicubic variants that you might like to try. I'm not sure any of those are integrated in madVR. (I'm also not too sure that these would be really useful as well.)
James Freeman
3rd January 2014, 14:56
Jan,
If I understood you correctly, to sharpen only the chroma, this chain is necessary:
1. R'G'B' to Y'CbCr.
2. CbCr Sharpen Complex 2 (chroma only)
3. Y'CbCr to R'G'B'.
If so,
Is there any way to do #2 with any of the current shaders?
I also modified the "Chroma for SD&HD video input" shader to show only pure chroma (0.5=>0 & 1.5->1.0).
The image looks Black, Red & Blue (as expected).
Now I have tweaked the Sharpen shader even more accurately.
It looks even closer to 4:4:4 now., almost identical.
Jan,
How hard will it be to insert a "Sharpen Complex 2" to only affect the CbCr data into the above 1,2,3 chain,
and make a single Shader that does this?
JanWillem32
4th January 2014, 03:21
That chain is correct. To simplify it for other Y'CbCr-type shaders, I've merged the Y'CbCr to R'G'B' step (and onward standard color transforms) into the previous shader for some of the chains I designed earlier. The first shader is better off being separate, as the next step takes multiple samples. (If a shader takes only one sample, you can always merge it with the previous shader. Even merging several shaders is possible.)
The shaders for Y'CbCr I've already written are chroma up-samplers and the deband/denoise/sharpen effects. Both have sharpening to partiallly compensate some of the base problems of the main filter.
'Simple' sharpening filters are useful for raw, high-quality video and images straight from the camera, in a decent color space, to compensate for areas that should be more in the foreground, but seem a bit 'weak' (determined by the person that does the mastering).
I see no place for such 'simple' sharpening outside of that context. The various Y'CbCr forms are only used to encode color, and you need some filters in Y'CbCr stages to filter some of the encode's problems and later on convert it to R'G'B'. However, Y'CbCr does a terrible job at actually describing a uniform or human vision-relative color space. Every flaw in a mixer's/renderer's Y'CbCr filtering stages becomes amplified a lot in terms of visible artifacts. You can't really do much with it in filters directly.
vood007
4th January 2014, 09:31
Jan, is it possible to have a single color plane with a fixed size in the background and the actual video centered on top of it? In other words; When the player window is bigger than the video and scaling is turned off, the video is obviously surrounded by a black frame. Can we fill this frame with a color other than black?
JanWillem32
4th January 2014, 22:39
If an interface allows you to set a background color in the video renderer, it's easy. I implemented such a function a while ago for the renderers I edit (miscellaneous tab). For the cases that require a special approach, use the "letterbox" and "pillarbox" shaders.
toniash, sorry for the late response. I simply overlooked your question earlier. The CIECAM02 shaders are included along with the five tester builds at the bottom of the first post of that other thread. (It's not included with the source code archive.)
vood007
5th January 2014, 20:22
If an interface allows you to set a background color in the video renderer, it's easy. I implemented such a function a while ago for the renderers I edit (miscellaneous tab). For the cases that require a special approach, use the "letterbox" and "pillarbox" shaders.
What you added to the misc tab of your study build is exactly what i want, so how do we get this into main MPCHC branch?
JanWillem32
6th January 2014, 00:47
Sorry, but I don't edit the video renderers in the main branch anymore. The mixer classes part of the internal renderers (VMR-9 r., EVR CP, RealMedia DX9, Quicktime DX9) and the two renderer sockets were reasonably well-written, so only had to modify those. The main renderer is beyond salvation. I tried to modify it at first. Some of the simple patches I made a few years ago worked well, so these were integrated. When I tried to edit more parts, disaster on disaster followed. Once I finally had enough I decided to no longer edit it. I integrated a renderer I wrote/edited earlier that worked and got rid of the old one. It broke pretty much all compatibility with the old code parts. It took weeks for it to compile, and a bit longer to run well. I'm very satisfied with the results. I even integrated some complicated functions without too much of a hassle. The main branch renderer and the one I wrote are similar in settings and menus, and I didn't sacrifice original functionality (although I'd rather not have transplanted the original "VSync" functions). However, I still modified several megabytes of code. Integration will not be easy. Smaller parts of the code can go in the main branch without a problem as patches. The main renderer change and all of its functionality will be one huge patch.
vood007
6th January 2014, 01:55
Not a big problem, theres still good old FFDShow. Anyway thanks for answering.
turbojet
7th January 2014, 08:03
James Freeman: Defining chroma coefficients in the developer section of lumasharpen shader (http://dropcanvas.com/l46x9) to sharpen only chroma. It sharpens much like sharpen complex 2 but with less edge artifacts.
James Freeman
10th January 2014, 19:17
James Freeman: Defining chroma coefficients in the developer section of lumasharpen shader (http://dropcanvas.com/l46x9) to sharpen only chroma. It sharpens much like sharpen complex 2 but with less edge artifacts.
Thanks.
Can you please guide me how to do this?
I tested the MPC Luma Sharpen and its very nice !!
turbojet
11th January 2014, 01:04
I don't know the chroma coeffecients, maybe someone else does?
I think just this line needs to be commented and insert a new line around it.
#define CoefLuma float4(0.2126, 0.7152, 0.0722,0)
Unrelated but there's an interesting comment that might be worth trying for someone with the knowledge. // -- Clamping the maximum amount of sharpening to prevent halo artifacts --
sharp_luma = clamp(sharp_luma, -sharp_clamp, sharp_clamp); //TODO Try a curve function instead of a clamp
James Freeman
11th January 2014, 17:20
JanWillem32,
I have been using a small avisynth script to create Black Frame Insertion to see how a video looks without motion blur.
I looks fantastic (leaving the flicker aside) !!
But it takes huge amount of memory and is non fluid (stutters).
Is it possible to write a simple shader that replaces 1 or 2 black frames after each refresh rate?
for example:
72Hz monitor.
1, B, B, 4, B, B, 7, B, B, 10, B, B etc... to 72.
The goal is to show only One out of Three frames.
or
1, 2, B, 4, 5, B, 7, 6, B, 10, 11, B etc... to 72.
show only 2 out of 3 frames.
This way after each movie frame (24fps) the display will show 1 or 2 black frames to eliminate motion blur.
There is no need to take the movie frame rate into consideration, only the monitor refresh rate.
It also will be nice if the shader can automatically read the display refresh rate, or let the user select how much black frames to insert after how much normal frame/s.
Of course it should be in-sync with the monitor refresh rate.
Are you familiar with Backlight Strobing or BFI?
nevcairiel
11th January 2014, 17:36
Shaders work on a given image, I don't think they can produce new images.
JanWillem32
11th January 2014, 21:16
Correct, this is more a task for an internal renderer routine. A pixel shader can change entire frames into solid colors or gradients, and with some temporal effects using the frame and time counters, but actually presenting more output frames per second has to be done by the host renderer.
I can easily (temporarily) add this function to the 'Alternative Scheduler' renderer function (requires Vista or newer) for the renderer I'm editing if you just want to test a more lightweight solution than using Avisynth for this case.
As for these presenting techniques, some parts are easy, some are not:
-Setting a frame to black before presenting it is very easy.
-Setting the amount of refreshes for a presented frame is not supported under XP, handled by a nasty function under Vista in windowed mode and can be handled normally under 7 and newer in windowed mode, plus under Vista and newer in D3D full-screen exclusive mode.
-Frame-time, jitter and other presentation statistics requires managing a lot of code.
-The frame time stamps of incoming video source frames are unreliable. The average over a lot of frames does add up nicely to a frame-time clock, but individual time stamps will often be off by several milliseconds. That means that a renderer has to estimate for how long a frame should be presented, and then also implement scheduling for getting a matching input frame rate to display refresh rate. All of this stuff is also implemented along with the statistics parts, as you always have to compensate for jitter and imprecision of timing data in the long run.
-Getting the monitor refresh rate is difficult. The pixel clock of the video card is actually really stable, but most software interfaces (even the ones that seem to have decent precision) report its rate incorrectly. The few that do work have low precision and will fail sometimes.
-The video renderer runs its main loop for every incoming frame (and once in a while for a repeated frame in paused mode). The the display refresh rate only comes into play with presenting after the scheduling functions/constant frame interpolator correlated it with the video frame rate (with compensation). It really doesn't work the other way around.
All in all, the various presentation functions usually take a lot of code to handle. Black frame insertion is easy to add to the 'Alternative Scheduler' renderer function, because of its method. Backlight strobing has to be handled by the hardware of the display device. The data stream from the source device doesn't include such a control.
James Freeman
11th January 2014, 23:00
Thanks Jan.
I can see its a big hassle, so never mind.
I've played enough with avisynth BFI to be convinced of this effect.
I think I should buy a proper Backlight Strobing monitor.
Though, till an IPS panel with 120Hz and backlight strobing will exist, I'm waiting.
JanWillem32
17th January 2014, 02:26
I was asked to improve the quality of the sphere shader, so I did a bit of work on the version I already edited. It's a bit more complicated, so it takes a lot more instructions compared to the original, but it's worth it.
If someone can help a bit with the old cinema film grain effect shader with colorimetry data of silver grains, sepia toning and such, or with documented methods that could help (that don't use external textures), I would appreciate it. I'm a bit stuck at getting significantly better results from prototype shaders compared to the already available shaders unfortunately. I had to guess a lot of variables, so that's one of the problems. Another problem is getting multiple grain sizes on screen, with decent randomness.// (C) 2011-2014 Jan-Willem Krans (janwillem32 <at> hotmail.com)
// This file is part of Video pixel shader pack.
// This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2.
// This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
// sphere, Catmull-Rom spline4×4 interpolated
// This shader should be run as a screen space pixel shader if you are up-scaling.
// This shader should not be run as a screen space pixel shader if you are down-scaling.
// This shader requires compiling with ps_2_a, but higher is better, see http://en.wikipedia.org/wiki/Pixel_shader to look up what PS version your video card supports.
// This shader is meant to work with linear RGB input and output. Regular R'G'B' with a video gamma encoding will have to be converted with the linear gamma shaders to work properly.
// Use this shader to apply an effect that looks like projecting the video from a rotating sphere.
// fractions, either decimal or not, are allowed
// rotation speed, in rotations per second
#define rs .125
// border canvas size
#define bs 7.75
// border gamma factor
#define bf 128.
// light position
#define pl float3(6., -6., -7.)
// light intensity
#define cl .5
// light size
#define sl 64.
// camera position
#define pc float3(0., 0., -1.)
// sphere position
#define ps float3(0., 0., .75)
// sphere radius
#define ra acos(-1.)*.25
// base size constant
#define Ai 1.
sampler s0;
float4 c0 : register(c0);
float2 c1 : register(c1);
#define sp(a, b, c) float3 a; {float2 tmp = tex+c1*float2(b, c); a = tex2D(s0, frac(abs(float2(tmp.x, abs(tmp.y)*-1.+1.)))).rgb;}
float4 main(float2 tex : TEXCOORD0) : COLOR
{
float ar = c0.x*c1.y;// aspect ratio
tex.x = tex.x*ar+.5-.5*ar;// surface rectangle to square
// relate the sphere to the camera position
float3 pd = normalize(float3(tex.xy-.5, 0.)-pc);
float B = dot(pd, 2.*(pc-ps));
float C = dot(pc-ps, pc-ps)-pow(ra, 2);
float D = B*B-4.*Ai*C;// calculate the sphere
if(D < 0.) return 0.;// output black borders, only map if things are inside the sphere
// intersection data
float3 p = pc+pd*(-sqrt(D)-B)*.5/Ai;
float3 n = normalize(p-ps);
float3 l = normalize(pl-p);
float2 cd = .75*(.5-tex);// measure the distance to the image center
tex = acos(-n.xy)/acos(-1.);// mapping the image onto the sphere
tex.x = frac(tex.x+c0.w*rs);// rotation
tex.y = (tex.y-.5)*ar-.5;// aspect ratio correction
tex *= c0.xy;// normalize to texture size in pixels
float2 t = frac(tex);// calculate the difference between the output pixel and the original surrounding two pixels
tex = (tex-t+.5)*c1+float2(1., 0.);// make the sampling position line up with an exact pixel coordinate for L1, normalized to the interval (1, 2), not (0, 1) as we want texture wrapping in this case
// weights
float2 t2 = t*t, t3 = t2*t;
float4 w13 = t3.xyxy*float2(1.5, .5).xxyy+t2.xyxy*float2(-2.5, -.5).xxyy;
float4 w02 = t3.xyxy*float2(-.5, -1.5).xxyy+t2.xyxy*float2(1., 2.).xxyy+t.xyxy*float2(-.5, .5).xxyy;
w13.xw += 1.;
// original pixels
sp(L0, -1., -1.) sp(L1, -1., 0.) sp(L2, -1., 1.) sp(L3, -1., 2.)
sp(K0, 0., -1.) sp(K1, 0., 0.) sp(K2, 0., 1.) sp(K3, 0., 2.)
sp(J0, 1., -1.) sp(J1, 1., 0.) sp(J2, 1., 1.) sp(J3, 1., 2.)
sp(I0, 2., -1.) sp(I1, 2., 0.) sp(I2, 2., 1.) sp(I3, 2., 2.)
// vertical interpolation
float3 Q0 = L0*w02.y+L1*w13.y+L2*w02.w+L3*w13.w;
float3 Q1 = K0*w02.y+K1*w13.y+K2*w02.w+K3*w13.w;
float3 Q2 = J0*w02.y+J1*w13.y+J2*w02.w+J3*w13.w;
float3 Q3 = I0*w02.y+I1*w13.y+I2*w02.w+I3*w13.w;
float3 P0 = Q0*w02.x+Q1*w13.x+Q2*w02.z+Q3*w13.z;// horizontal interpolation
return ((P0+cl*pow(max(dot(l, reflect(pd, n)), 0.), sl))*min(pow(bs*dot(cd, cd), -bf), 1.)*dot(n, l)).rgbb;// add specular mapping, sphere edges, diffuse mapping and then output
}// (C) 2011-2014 Jan-Willem Krans (janwillem32 <at> hotmail.com)
// This file is part of Video pixel shader pack.
// This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2.
// This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
// You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
// sphere, Catmull-Rom spline4×4 interpolated on 16-bit integer surfaces
// This shader should be run as a screen space pixel shader if you are up-scaling.
// This shader should not be run as a screen space pixel shader if you are down-scaling.
// This shader requires compiling with ps_2_a, but higher is better, see http://en.wikipedia.org/wiki/Pixel_shader to look up what PS version your video card supports.
// This shader is meant to work with linear RGB input and output. Regular R'G'B' with a video gamma encoding will have to be converted with the linear gamma shaders to work properly.
// Use this shader to apply an effect that looks like projecting the video from a rotating sphere.
// fractions, either decimal or not, are allowed
// rotation speed, in rotations per second
#define rs .125
// border canvas size
#define bs 7.75
// border gamma factor
#define bf 128.
// light position
#define pl float3(6., -6., -7.)
// light intensity
#define cl .5
// light size
#define sl 64.
// camera position
#define pc float3(0., 0., -1.)
// sphere position
#define ps float3(0., 0., .75)
// sphere radius
#define ra acos(-1.)*.25
// base size constant
#define Ai 1.
sampler s0;
float4 c0 : register(c0);
float2 c1 : register(c1);
#define sp(a, b, c) float3 a; {float2 tmp = tex+c1*float2(b, c); a = tex2D(s0, frac(abs(float2(tmp.x, abs(tmp.y)*-1.+1.)))).rgb;}
float4 main(float2 tex : TEXCOORD0) : COLOR
{
float ar = c0.x*c1.y;// aspect ratio
tex.x = tex.x*ar+.5-.5*ar;// surface rectangle to square
// relate the sphere to the camera position
float3 pd = normalize(float3(tex.xy-.5, 0.)-pc);
float B = dot(pd, 2.*(pc-ps));
float C = dot(pc-ps, pc-ps)-pow(ra, 2);
float D = B*B-4.*Ai*C;// calculate the sphere
if(D < 0.) return 0.;// output black borders, only map if things are inside the sphere
// intersection data
float3 p = pc+pd*(-sqrt(D)-B)*.5/Ai;
float3 n = normalize(p-ps);
float3 l = normalize(pl-p);
float2 cd = .75*(.5-tex);// measure the distance to the image center
tex = acos(-n.xy)/acos(-1.);// mapping the image onto the sphere
tex.x = frac(tex.x+c0.w*rs);// rotation
tex.y = (tex.y-.5)*ar-.5;// aspect ratio correction
tex *= c0.xy;// normalize to texture size in pixels
float2 t = frac(tex);// calculate the difference between the output pixel and the original surrounding two pixels
tex = (tex-t+.5)*c1+float2(1., 0.);// make the sampling position line up with an exact pixel coordinate for L1, normalized to the interval (1, 2), not (0, 1) as we want texture wrapping in this case
// weights
float2 t2 = t*t, t3 = t2*t;
float4 w13 = t3.xyxy*float2(1.5, .5).xxyy+t2.xyxy*float2(-2.5, -.5).xxyy;
float4 w02 = t3.xyxy*float2(-.5, -1.5).xxyy+t2.xyxy*float2(1., 2.).xxyy+t.xyxy*float2(-.5, .5).xxyy;
w13.xw += 1.;
// original pixels
sp(L0, -1., -1.) sp(L1, -1., 0.) sp(L2, -1., 1.) sp(L3, -1., 2.)
sp(K0, 0., -1.) sp(K1, 0., 0.) sp(K2, 0., 1.) sp(K3, 0., 2.)
sp(J0, 1., -1.) sp(J1, 1., 0.) sp(J2, 1., 1.) sp(J3, 1., 2.)
sp(I0, 2., -1.) sp(I1, 2., 0.) sp(I2, 2., 1.) sp(I3, 2., 2.)
// vertical interpolation
float3 Q0 = L0*w02.y+L1*w13.y+L2*w02.w+L3*w13.w;
float3 Q1 = K0*w02.y+K1*w13.y+K2*w02.w+K3*w13.w;
float3 Q2 = J0*w02.y+J1*w13.y+J2*w02.w+J3*w13.w;
float3 Q3 = I0*w02.y+I1*w13.y+I2*w02.w+I3*w13.w;
float3 P0 = Q0*w02.x+Q1*w13.x+Q2*w02.z+Q3*w13.z;// horizontal interpolation
return ((P0*65535./32767.-16384./32767.+cl*pow(max(dot(l, reflect(pd, n)), 0.), sl))*min(pow(bs*dot(cd, cd), -bf), 1.)*dot(n, l)*32767./65535.+16384./65535.).rgbb;// add specular mapping, sphere edges, diffuse mapping and then output
}
romulous
31st January 2014, 13:58
Hi JanWillem32,
Blight is doing some work to integrate the rotation shader you wrote on page 20 into Zoom Player. He just had a couple of questions he wanted me to ask you if you don't mind. Some quoted text first, and then the question afterwards.
Question #1:
Note that geometry changes due to rotation usually require manual compensation, as the video renderer host usually doesn't compensate for it automatically.
Where does the manual compensation have to be performed?
Question #2:
// Note than when enabling diagonal flipping, the resizing factor has to be lowered in advance to allow the full image to be visible.
Where do you set the resize factor?
Thanks!
hadi79
31st January 2014, 18:10
Hi guys,
Could anyone help me?
Why this shader doesn't work on my MPC?
This is the error: error X3507: 'main': Not all control paths return
link (http://dl.dropboxusercontent.com/s/fh3mqtqcylhe2r4/discman.txt)
JanWillem32
1st February 2014, 02:40
hadi79, I don't see why that shader wouldn't compile. It could use a few changes, though. Change "static float" to "static const float", as it's a compile-time constant. Change "1.0f" to "1.0" or "1.", as the 'f' or 'F' suffixes are considered to be bad style when working with HLSL. Change "#define BlockCount 75" to "#define BlockCount 75.0" or "#define BlockCount 75.", as it's never used as an integer. The HLSL compiler doesn't reliably do implicit type casts when compiling. Always specify floating-point values as floating-point or use explicit type casts. (I really think all C, C++ and HLSL compilers should should warn about all implicit type casts.)
The effect itself is actually quite a funny one. It's pretty basic, though. (It doesn't pre-process with a low-pass, nor does it apply anti-aliasing on its mask.)
If PS 3.0 code isn't a problem, you can also add branching to optimize out the sampling of the zero-value output pixels:#define BlockCount 75.
#define Min .2
#define Max .45
sampler2D input : register(S0);
// static constants used as optimization
static const float BlockSize = 1./BlockCount;
float4 main(float2 uv : TEXCOORD) : COLOR
{
// Calculate block center
float2 blockPos = floor(uv*BlockCount);
float2 blockCenter = blockPos*BlockSize+BlockSize*.5;
// Round the block by testing the distance of the pixel coordinate to the center
float dist = length(uv-blockCenter)*BlockCount;
[branch] if(dist < Min) {
return 0.;}
else if(dist > Max) {
return 0.;}
else {// Sample color at the calculated coordinate
return tex2Dlod(input, float4(blockCenter, 0., 0.));}
}
romulous, setting the resizing factor is usually handled by the host player. Most video renderers do have a default setting which just resizes video to the full window size plus letterboxing/pillarboxing it to maintain the intended aspect ratio of the video source.
For standard interfaces to renderers there are these calls:
EVR/MF interface control:
IMFVideoDisplayControl::SetVideoPosition()
Standard DirectShow interface controls:
IBasicVideo::SetDefaultSourcePosition()
IBasicVideo::SetDestinationPosition()
IBasicVideo also has a lot of other calls to handle input/output video size for a video renderer
IVideoWindow::SetWindowPosition()
IVideoWindow also has a few other calls to handle the state of the presentation window
The EVR also natively supports rotation, but I've never worked with that, so I don't know how it behaves.
As for resizing factors, if you take the simple example of a 1920×1080 image on a 1920×1080 window without resizing or repositioning, your source and destination rectangles will simply be {0, 0, 1920, 1080}, {0, 0, 1920, 1080} (left, top, right, bottom).
When you use diagonal flipping and you intend to keep the image in the same aspect ratio, the destination rectangle will have to change. For this case {420, 0, 1500, 1080} would be suitable if you want a centered image. (That's a resizing factor of 9./16..)
For the case of a DirectX 9 renderer that you can edit, you can just set different vertices. (Note that when you can edit vertices, you don't need the rotation shader at all.)
Example for a very basic FVF-style (D3DFVF_XYZRHW | D3DFVF_TEX1) pre-transformed vertex statement in a top-left, top-right, bottom-left, bottom-right index order:struct CUSTOMVERTEX_TEX1 {
float x, y, z, rhw, u, v;
};
float hres = 1920.f, vres = 1080.f;
// lists for DrawIndexedPrimitive() or DrawPrimitive()
// standard
CUSTOMVERTEX_TEX1 v[4] = {
{ -.5f, -.5f, 0.f, 1.f, 0.f, 0.f},
{hres-.5f, -.5f, 0.f, 1.f, 1.f, 0.f},
{ -.5f, vres-.5f, 0.f, 1.f, 0.f, 1.f},
{hres-.5f, vres-.5f, 0.f, 1.f, 1.f, 1.f}};
// flipped horizontally
CUSTOMVERTEX_TEX1 v[4] = {
{hres-.5f, -.5f, 0.f, 1.f, 0.f, 0.f},
{ -.5f, -.5f, 0.f, 1.f, 1.f, 0.f},
{hres-.5f, vres-.5f, 0.f, 1.f, 0.f, 1.f},
{ -.5f, vres-.5f, 0.f, 1.f, 1.f, 1.f}};
// flipped vertically
CUSTOMVERTEX_TEX1 v[4] = {
{ -.5f, vres-.5f, 0.f, 1.f, 0.f, 0.f},
{hres-.5f, vres-.5f, 0.f, 1.f, 1.f, 0.f},
{ -.5f, -.5f, 0.f, 1.f, 0.f, 1.f},
{hres-.5f, -.5f, 0.f, 1.f, 1.f, 1.f}};
// flipped horizontally and vertically
CUSTOMVERTEX_TEX1 v[4] = {
{hres-.5f, vres-.5f, 0.f, 1.f, 0.f, 0.f},
{ -.5f, vres-.5f, 0.f, 1.f, 1.f, 0.f},
{hres-.5f, -.5f, 0.f, 1.f, 0.f, 1.f},
{ -.5f, -.5f, 0.f, 1.f, 1.f, 1.f}};
// note: diagonal flipping requires adaption of the input-to-output resolution
// flipped diagonally over the top-left-to-bottom-right axis
CUSTOMVERTEX_TEX1 v[4] = {
{ -.5f, -.5f, 0.f, 1.f, 0.f, 0.f},
{ -.5f, vres-.5f, 0.f, 1.f, 1.f, 0.f},
{hres-.5f, -.5f, 0.f, 1.f, 0.f, 1.f},
{hres-.5f, vres-.5f, 0.f, 1.f, 1.f, 1.f}};As a last note, the IDirect3DDevice9::StretchRect() function can do nearest neighbor and bilinear image resizing, but does not support rotation. A call to DrawIndexedPrimitive() or DrawPrimitive() on a correctly configured device, with optionally a resizing shader enabled for that stage is required for rotation.
hadi79
1st February 2014, 15:12
Thanks for your help. JanWillem32
hadi79
1st February 2014, 15:26
Thanks for your help. JanWillem32
JanWillem32
2nd February 2014, 20:07
You're welcome. If you need any help with other filtering code, such as the low-pass or anti-aliasing effects I mentioned, I'm happy to help.
romulous
7th February 2014, 09:11
Jan-Willem: Many thanks for your response to my questions from Blight - he has made some progress in integrating the script into Zoom thanks to your help :)
Regards,
romulous
romulous
12th February 2014, 07:55
Hi again Jan-Willem,
Blight had a follow-up question regarding the rotation shader. He isn't sure if you can compensate for the cropping like you described previously. He has tried, but he believes the actual content is cropped and is therefore not accessible to the video renderer. He wanted me to ask you using ibasicvideo/ivideowindow, how to make it work if the source video resolution is1920x1080 (for example, what values to pass the ibasicvideo/ivideowindow interfaces).
Regards,
romulous
JanWillem32
12th February 2014, 11:49
I just reviewed the rotation shader again. I added the comment "// This shader shoud be run as a screen space pixel shader when enabling diagonal flipping, else both modes will work.", but that's actually not entirely correct. If the resizer section clips the input video rectangle before the rotation (common when down-sizing), a part of the input is lost. In such a case, the rotation shader can be inserted before resizing.
Note that there's no proper way to compensate if you need down-sizing in one direction and up-sizing in another. Also, positioning the input on the output rectangle can cause clipping in the resizing section as well. However, if you edit the vertices of the resizing section like I described before, none of these problems occur.
romulous
12th February 2014, 12:19
Thanks - this implementation is using madVR, which does not allow access to the vertices as far as Blight knows. I think that means we are stuck until madshi adds rotation support to madVR?
JanWillem32
12th February 2014, 14:51
I think you are correct.
leeperry
26th March 2014, 18:45
Hi Jan, would that be possible to align MPEG1 to look like MPEG2/H264 chroma via a PS script?
mVR only does MPEG2/H264 chroma alignment so this would be very handy.
I was under the impression that outputting YUY2 from ffdshow would align chroma properly but I ain't too sure anymore, I guess only AYUV & Y416 would do what I want?
Anyway, PotPlayer's seamless playback feature likes to crash randomly when not using its built-in decoders so that doesn't really solve anything :o
:thanks: for your help,
JanWillem32
27th March 2014, 13:52
Do you mean chroma cositing? http://msdn.microsoft.com/en-us/library/windows/desktop/ms694252%28v=vs.85%29.aspx
It's part of the standard of how to convert Y'CbCr to R'G'B'. The pixel shader stages accessible by the user are already R'G'B' or onward, and there's no real way to convert back if it was handled in the wrong way. I think it's best to just wait for the renderer to be fixed. I implemented support for both common forms of chroma cositing a few months ago. It's really not that hard to implement.
leeperry
27th March 2014, 15:53
HI Jan, thanks for the reply.
From http://bengal.missouri.edu/~kes25c/ddcc.zip\ddcc - ReadMe.txt
cplace -
Specifies horizontal chroma placement... used when input is yuy2. Possible settings:
0 - chroma is aligned with left pixel in each pair (mpeg2, mpeg4, h264)
1 - chroma is centered between each pair of pixels (h261, h263, mpeg1)
default: 0
I do realize that it'll require some work considering that PS scripts work in RGB, but Leak wrote a script to smooth chroma (http://forum.doom9.org/showpost.php?p=1184975&postcount=32) so it still should be possible? Or not? :o
What I would need is a PS script that would align MPEG1 chroma like MPEG2 so mVR would align it properly afterwards, basically a PS script of one of those scripts: http://avisynth.nl/index.php/External_filters#Chroma_correction
ChromaShiftSP : This script can shift chroma in all directions with subpixel accuracy.
Humm, so forcing AYUV/Y416 output from ffdshow is the only way to overcome the problem?
nevcairiel
27th March 2014, 17:10
Humm, so forcing AYUV/Y416 output from ffdshow is the only way to overcome the problem?
That is assuming ffdshow even does it correctly, especially when it comes to the 16-bit formats I wouldn't really trust it.
foxyshadis
28th March 2014, 02:18
No, forcing any output won't change anything, except RGB output. (ffdshow will sample chroma correctly.)
To perform it in pixel shader, you'd have to convert it back to YUV exactly how it was converted to RGB, then convert it back to RGB correctly. The pixel shader never sees the YUV data, because it's already done by the time you can run a shader. Given the many possible ways to convert to RGB in the renderer, that's a pretty tall order, and madshi would have to help. Presumably, that workflow would also break dithering.
You have to do it in the decoding pipeline, not the rendering. Consider enabling ffdshow's avisynth processing and using that script to process the YUV data while decoding.
DarkSpace
28th March 2014, 13:00
To perform it in pixel shader, you'd have to convert it back to YUV exactly how it was converted to RGB, then convert it back to RGB correctly.
Why? From my limited understanding, shouldn't it be enough to separate Luma from Chroma using any matrix, then shift the Chroma channels (I think by 0.25 to the left?), and then convert back to RGB using the same matrix?
leeperry
28th March 2014, 17:58
OK thanks for the replies. I also thought that it would be fairly easy just like Darkspace is explaining it, I even assumed that YUY2 out of ffdshow would look properly aligned duh....I'll have to find a proper test pattern and give it a whirl.
leeperry
28th March 2014, 18:01
I'll just ask DragonQ for this sample (http://forum.doom9.org/showpost.php?p=1622754&postcount=18186) :)
JanWillem32
29th March 2014, 01:50
I have been thinking. Converting R'G'B' back to Y'CbCr is possible, but it will be in the 4:4:4 form. Given the context of video playback in this case, it is absolutely impossible to get the 4:2:2 or 4:2:0 chroma back as it was output by the decoder if any attempt of chroma-upsampling was made, with the notable exception of nearest neighbor filtering. Nearest neighbor filtering doesn't change any values nor discards any pixels. Given the 2:1 magnification factor, the sampling grid is simple as well. I already made pixel-shaded filters that exploit this trick. Both EVR and VMR-9 will use nearest neighbor chroma upsampling in certain cases (but still convert to flawed R'G'B' anyway). Fixing the internal stages of EVR or VMR-9 isn't possible. Using the renderer to convert the flawed R'G'B' back to 4:2:2 or 4:2:0 Y'CbCr, upsampling the chroma to 4:4:4 with one of the 15 filters I wrote and converting to R'G'B' again isn't ideal, but it does work. I'm fine with publicating the 15 chroma up-sampling up-sampling filters (in both common cositing modes) if people really want to use them. If anyone would like to have a sneak peak, you can open the file containg the source code for the internal filters I've written. It's the set of initial pass shaders in the bottom half of \src\filters\renderer\VideoRenderers\InternalDX9Shaders.h . However, the preferred fix for the problem is still of course to edit the renderer. As I said before, if you have access to the Y'CbCr mixing stages, implementing chroma upsampling for both common modes of cosited chroma is easy.
leeperry (first part), I advise to not give much regards to Leak's shader. None of the filtering stages do anything correctly. That shader should never be used.
As for the set of Avisynth filters, I can port a few on request. I guess that some of them could be useful.
foxyshadis, both madVR and my renderer in quality mode keep quantization high enough to generally not cause additional quantization artifacts than present in the source video. To be more specific; operations are done in single-precision floating point precision and intermediate storage of pixels in textures are formatted in half precision floating-point (lowest quality), 16-bit unsigned integer (very decent quality) or single-precision floating point (same as the processing format, mostly useful for debugging because of the high memory consumption). Between filtering stages pixels are never dithered. Dithering is only done in the very last filtering stage, as the backbuffer presented on the video adapter's output is merely 10-bit R'G'B' or worse, 8-bit R'G'B'. Breaking dithering stages is an artificial problem of all those old methods that employ poor quantization and vapid integer color processing methods of a bygone era.
In short, the video renderer stages don't have to bother with dithering at all until creating the output image for the video port.
On top of that, I'm really not a fan of moving any filters that the renderer should handle to some intermediate filtering pass attached to the decoder. Taking raw output from the decoder to the renderer is the best situation, partially because of the reasons I already noted. Sadly I'm stuck at the moment with the VMR-9, EVR, RealMedia an Quicktime mixers that are extremely limited in terms of input formats and various other capabilities. The renderer actually far exceeds the capabilities of the mixers. For madVR there are no such limitations, but that comes at the cost of the minimum processing efficiency of the mixer stages.
leeperry (second part), YUY2 (8-bit 4:2:2 Y'CbCr) has the same chroma cositing issues as any other 4:2:0 or 4:2:2 format.
As for the sample posted by DragonQ, I support the comment posted by nevcairiel following that sample. For what I've seen, the EVR and VMR-9 always upsample chroma using the MPEG1 method. (I actually don't understand why the MPEG2 chroma cositing method was ever implemented at all, as there are only disadvantages to it. Not that I really mind though, the cositing issues are just yet another fine example of what an absolutely terrible format Y'CbCr is and that it shoud be supplanted by a good implementation of color representation as soon as possible.)
yok833
30th March 2014, 07:06
Is it normal that when I apply lumasharpen several times in post or pre resize, there is no difference as if I had made it only once? I really would like to try this shader but I do not see any real difference (unlike of when I am using complex 2 or finesharp) so I am not sure to use it well...
However everything seems to be well configured (V 1.4.1) and I have also tried to rise the sharpening in the lumasharpen.hlsl file....
Maybe the sharpening is more light or subtle with this shader?
turbojet
30th March 2014, 07:25
It's not normal but there is at least one lumasharpen 1.4.1 out there that does nothing but load the gpu. Depending on settings it might not be very strong. If you open the file in a text editor what's next to strength and pattern?
http://forum.doom9.org/showthread.php?t=170357 (http://forum.doom9.org/showthread.php?t=170357) is the thread for lumasharpen, better to post your reply there.
CiNcH
28th June 2014, 09:42
Hi JanWillem32,
I am currently playing around with shaders and stuff. I am not a lot into D3D and video algorithms yet. There is one thing I wonder though. The vertices you define for resize shaders contain the texture coordinates in screenspace (so in pixels) rather than [0,1]. I wonder what difference that makes. I can't find anywhere that this is actually legal. I though D3D would clamp that to [0,1]. What happens with those texture coordinates? Are they passed to the shader?
JanWillem32
4th July 2014, 17:23
Texture coordinates are simply scaled in the space that you declare with the vertex declaration. For the resizers I just optimized the input and output parameters to suit the routines.
The vertices for the five main stages are set up like this:struct CUSTOMVERTEX_TEX1 {
float x, y, z, rhw, u, v;
};
...
// dCenterXo and dCenterYo already have .5 subtracted from them
float utlX = static_cast<float>(dTopLeftX + dCenterXo), utlY = static_cast<float>(dTopLeftY + dCenterYo);// offset to the top-left point
float utrX = static_cast<float>(dTopRightX + dCenterXo), utrY = static_cast<float>(dTopRightY + dCenterYo);// offset to the top-right point
float ublX = static_cast<float>(dBottomLeftX + dCenterXo), ublY = static_cast<float>(dBottomLeftY + dCenterYo);// offset to the bottom-left point
float ubrX = static_cast<float>(dBottomRightX + dCenterXo), ubrY = static_cast<float>(dBottomRightY + dCenterYo);// offset to the bottom-right point
float vidw = m_fVideoWidth - .5f, vidh = m_fVideoHeight - .5f, wndw = m_fWindowWidth - .5f, wndh = m_fWindowHeight - .5f;
__declspec(align(16)) CUSTOMVERTEX_TEX1 v[20] = {// lists for DrawIndexedPrimitive() with the number used for the BaseVertexIndex item
{-.5f, -.5f, 0.f, 1.f, 0.f, 0.f},// window size: 0
{wndw, -.5f, 0.f, 1.f, 1.f, 0.f},
{-.5f, wndh, 0.f, 1.f, 0.f, 1.f},
{wndw, wndh, 0.f, 1.f, 1.f, 1.f},
{-.5f, -.5f, 0.f, 1.f, 0.f, 0.f},// video size: 4
{vidw, -.5f, 0.f, 1.f, 1.f, 0.f},
{-.5f, vidh, 0.f, 1.f, 0.f, 1.f},
{vidw, vidh, 0.f, 1.f, 1.f, 1.f},
{utlX, utlY, 0.f, 1.f, -.5f, -.5f},// 1 pass resizers: 8
{utrX, utrY, 0.f, 1.f, vidw, -.5f},
{ublX, ublY, 0.f, 1.f, -.5f, vidh},
{ubrX, ubrY, 0.f, 1.f, vidw, vidh},
{utlX, -.5f, 0.f, 1.f, -.5f, 0.f},// 2 pass resizers x: 12
{utrX, -.5f, 0.f, 1.f, vidw, 0.f},
{ublX, vidh, 0.f, 1.f, -.5f, 1.f},
{ubrX, vidh, 0.f, 1.f, vidw, 1.f},
{-.5f, utlY, 0.f, 1.f, 0.f, -.5f},// 2 pass resizers y: 16
{wndw, utrY, 0.f, 1.f, 1.f, -.5f},
{-.5f, ublY, 0.f, 1.f, 0.f, vidh},
{wndw, ubrY, 0.f, 1.f, 1.f, vidh}
};
ryrynz
2nd September 2014, 09:00
Anyone else seen Samsung's auto depth enhancer at work on the U9000? It's IMO very effective, It brings about an almost 3D experience that I would hope other manufacturers apply to their TVs in future. With some excellent sharpeners already available how much work is it release something similar?
burfadel
2nd September 2014, 09:07
Anyone else seen Samsung's auto depth enhancer at work on the U9000? It's IMO very effective, It brings about an almost 3D experience that I would hope other manufacturers apply to their TVs in future. With some excellent sharpeners already available how much work is it release something similar?
That can be done, it's just a question of knowing how to do it and to do it so the performance penalty is low.
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.