Video pixel shader pack

[Deshi]

Thank you all !
:cool:

[CiNcH]

I have done some further testing concerning the chroma upsampling problem that I am experiencing with the MPC-HC custom presenter and found out the following...

Chroma seems to be interpolated when the display is 1080p50. When resolution is 720p50 (native resolution of my Sanyo PLV-Z4) or 1680x1050 (TFT) chroma is upsampled by doubling the values. (GPU is always the same HD 3650)

So maybe it is resolution dependant and not specific to < HD 4000 series. Still one has to keep in mind that this is not happening with standard EVR, but only with the custom presenters.

[JanWillem32]

Can you provide a screenshot of both resolution states with an exact magnification of 400%? (Like I did with my four pictures at the start of this topic.)
You can achieve exactly 400% by using "View", "Video Frame", "Double Size" and then "View", "Pan&Scan", "Increase Size" (a lot of times). I assigned a few F# keys of my keyboard to these functions using the "View", "Options", "Player - Keys" menu. I assigned a lot of these kinds of functions to keys, because I've been busy making screen space resizer shaders. Those require the "View", "Video Frame", "Normal Size" setting for input, by the way.
I already made "Catmull-Rom Spline resizer", "4÷2÷0 Catmull-Rom Spline chroma up-sampling for SD&HD video input" and "Lanczos2 resizer". I'm currently investigating the "ringing beasts": multi-tap Spline and Lanczos.

[CiNcH]

Can you provide a screenshot of both resolution states with an exact magnification of 400%?
Hm, they look pretty much the same then. What could be the cause of the bad chroma at lower screen resolutions (e.g. 720p)? Source is 576i.

[JanWillem32]

If the exact magnification screenshots of both resolution states show bad chroma upsampling, then the display resolution is not a factor in the origin of this problem. It just means that the regular resolution scaler is blurring everything if it stretches a picture up a lot, including the bad chroma edges.

Edit:
For those who are interested, here are my resolution shaders. These include chroma up-sampling shaders that work normally if you set them as the very first pixel shader.
The Spline-type resizers are all right, but Lanczos2 is a bit more blurry than I expected it to be. All of these shaders will need additional comments, but I will add those when I'm confident that the shaders work properly. It's too bad that the 2-tap Spline uses just a few too many registers to compile in DirectX 9. In my DirectX 10 "teapot" renderer it does a proper upscale of textures. The same effect can be achieved by upscaling 2× with the "×2" shader, and then upscale further with a second shader in the stack.
Downscaling should work if the shader is in the regular shader stack.
Upscaling should work if the shader is in the screen space shader stack.
Scaling shaders require the "View", "Video Frame", "Normal Size" setting (blending multiple scalers is generally a bad idea).
Because the video input resolution isn't passed on to the shader, you will have to set your video input resolution for the shader, or set the magnification factor.

[CiNcH]

If the exact magnification screenshots of both resolution states show bad chroma upsampling, then the display resolution is not a factor in the origin of this problem. It just means that the regular resolution scaler is blurring everything if it stretches a picture up a lot, including the bad chroma edges.
Then I don't get why it looks a lot better when being upscaled to 1080p compared to 720p or 1680x1050.

Upscaling should work if the shader is in the screen space shader stack.
What is your upscaling shader worth when the image has already been upscaled by the renderer?

[JanWillem32]

Total resolution up-scaling by normal means will always blur, alias and/or make interpolation halos over the source image, that can be so much that the bad chroma up-scaling becomes much less apparent. It is still present, however. If you use exact magnifications (preferably with nearest neighbor scaling), there's no way you can overlook bad chroma up-sampling. To make it even more visible, you can view the chroma data directly with my "chroma" shader.

Scaling shaders require the "View", "Video Frame", "Normal Size" setting. This disables resolution scaling by the renderer, the picture is then centered on the screen with either black borders or parts of the frame cut off.

[CiNcH]

Scaling shaders require the "View", "Video Frame", "Normal Size" setting. This disables resolution scaling by the renderer, the picture is then centered on the screen with either black borders or parts of the frame cut off.
Ah, I see, I will conduct a comparison tomorrow.

Can you provide a screenshot of both resolution states with an exact magnification of 400%? (Like I did with my four pictures at the start of this topic.)
You can achieve exactly 400% by using "View", "Video Frame", "Double Size" and then "View", "Pan&Scan", "Increase Size" (a lot of times).
There seems to be a problem.

What I am doing:
- I stretch the video frame to double size
- I increase the size via 'Pan&Scan' until the status bar reads 2.00

I think I am at 400% now, right?

I then try to navigate to the top right corner (with CTRL+4/8/6/2), where the ATV logo is (see my posted sample). On my 1080p screen I can go there. But on my 1680x1050 screen, the logo does not seem to be part of the rendered image any longer. I have to decrease size again until I am at about 1.88 to see at least the 'A' within the logo.

[JanWillem32]

The repositioning filter uses the internal re-sizer, and that one indeed doesn't compensate for resolution/scale/magnification changes. The screenshots I took were from the center of the image. You can try to use the "chroma for SD&HD video input" shader on a high chroma contrast image to directly view the chroma data. That will definitely pinpoint the chroma up-sampling problem, if present.

[CiNcH]

I will try it at 1080p. Is the ATV logo good enough? Shall I use magnification?

[JanWillem32]

400% nearest neighbor scaling doesn't distort the picture much. It does have a bug that creates a central horizontal and vertical line line when scaling (even by exact amounts), so avoid these when taking a crop of a screenshot.
If you can, don't re-position from the center of the image. Try to make comparison screenshots with and without the chroma up-sampling shader, like I did with my first set of pictures. You can use about every input picture if you use the "chroma for SD&HD video input" shader, as it can expose chroma blocks on any color border.

I made a nice chain of of shaders, and to allow people to compare the quality, I made three screenshots of a 400% scaled medium bitrate AVC encoded HD video.
standard; see page 1 of this thread for most of my settings, MPC-HC v1.4.2752.0, no shaders activated (bad chroma up-sampling is also visible).
flitered; pixel shaders: "4÷2÷0 Catmull-Rom Spline chroma up-sampling for SD&HD video input", "gamma conversion of BT.709 or BT.601 derived full range RGB to linear RGB", "test 12, sharpen complex v3 + deband + mild denoise" (noise detection factor .75, square root removed from the equation for sharpening) or "test 12, sharpen complex v3 + deband + minimal denoise" (noise detection factor .625, square root removed from the equation for sharpening)
screen space pixel shaders: "Catmull-Rom Spline ×2 resizer", "Catmull-Rom Spline ×2 resizer" (to get 400% magnification), "gamma conversion of linear RGB to BT.709 or BT.601 derived full range RGB"
The screenshots are 2048×1440 8-bit raw PNG format files from direct "Print Screen" screenhots saved in GIMP, 2.7 MB each.

[CiNcH]

This is what it looks like on the 1080p screen:
http://members.inode.at/762450/chroma_diff.png

Guess I am always suffering from bad chroma upsampling. Though for some reason, it is less apparent on the 1080p screen.
I first thought it may be due to the 16:10 TFT and a faulty/bad upscaling in this case. But it is also as apparent on the 720p projector (without any magnification or filtering, just 576i deinterlaced and upscaled to screen space).

[CiNcH]

Compared to the madVR Lanczos implementation yours is extremely blurry. Even a lot worse than Bilinear.

Small comparison between your Lanczos and MPC-HC Bicubic

Source: TWILIGHT DVD VTS_01_2.vob Frame 5882 (720x576)

Bicubic -0.60 (http://members.inode.at/762450/bicubic.png)
Lanczos2 (http://members.inode.at/762450/lanczos2.png)

[JanWillem32]

I already thought that the display resolution couldn't be a key factor in the lack of chroma up-scaling. It should be one of the transformations before the color format converts to RGB, while the display resolution is only passed to the renderer in the texture to screen space stage (an RGB mode).
I know that "my" Lanczos2 looks blurry. I only borrowed the code and made it a bit more organized. I didn't check the methods, however…
I did build the Catmull-Rom Spline types almost from scratch, and those are fine compared to some other single-pass resizers. It also helped a lot that the first shader I made of that type worked right away, as that doesn't happen very often to me.

[bobdynlan]

Hallo JanWillem32! One of my first attempt at shaders was a film-grain one, made "blind" but surprisingly it did work. You seem like you have the skills and the tools to do a proper one. I will not put up a request but instead a challenge :) because I still believe it's one of the most needed shader extension for MPC-HC.

I don't know about others, but when I watch a movie I like that extra feeling. It's one way closer to dreams (all dreams have some HDR applied to them). I don't need pixel perfect "reality", 5'o'clock TV News in my country beats the hell out of Paranormal Activity, Cloverfield and their likes. Just to chase away those "purists"...

This film grain thing can actually improve perceived details from the distance, and hide some of those damn color bands. Are you up to the challenge? :rolleyes:

I don't know much about directx and gpu processing, but did some basic translations/adaptations from various sources. Here it's a noise(grain) shader, good against color bands, giving some film effect, any improvement is welcomed: //ps_2_0
/* I've missed Xvid's film effect
bobdynlan's adaptation of GrainPS2 from Looki's Shader Pack V 2.0 */

sampler s0 : register(s0);
float4 p0 : register(c0);
#define clock (p0[3]) /* it was static only before I found this :D */
#define PI acos(-1)

float mccool_rand(float2 ij) {
/* Random algorithm by Sylvain Lefebvre */
const float4 a=float4(pow(PI,4),exp(5),pow(13, PI / 2.0),sqrt(1997.0));
float4 result =float4(ij,ij);
for(int i = 0; i < 3; i++) {
result.x = frac(dot(result, a));
result.y = frac(dot(result, a));
result.z = frac(dot(result, a));
result.w = frac(dot(result, a));
}
return (float)result.xy;
}


float4 main(float2 tex : TEXCOORD0) : COLOR
{
float fStrength = 0.10; /* 0.05=min; 0.1=LOW; 0.15=med; 0.2=strong */

float clkvar = 0.001; /* 0.0=static grain 0.001=fix some noise banding */
float4 c1 = tex2D(s0,tex);
float rand = mccool_rand(tex+clkvar*clock)*fStrength;
// c1.rgb=(0.5,0.5,0.5); /* display only grain */
c1.rgb *= float3(1-rand,1-rand,1-rand);

return c1*(1+fStrength/2);
}

[JanWillem32]

I already have a bunch of film and special effect shaders in storage. Those are mostly game-adapted, unfinished or distortion effect shaders. I currently have: sepia (brownish grayscale, a bit red-sensitive), film roll drive scratches (vertical scratching on film), dust (doesn't quite look realistic yet) and a few types of noise (digital static, grain, fog, smoke, rain, snow and two types of dithering).
Your shader does look interesting, but the notion of "fix some noise banding" is wrong. Banding in the digital realm is due to limited bit depth of source files and displays. The correct way to handle banding from a source is to use a type of deband filter. For output it will need temporal dithering with settings for 5-bit to 11-bit digital displays or connections. The input type of shader I made already works quite well. The output type of shader I made is difficult to analyze, as it needs to be run on the output frame texture, that's after the color correction. The temporal dithering methods don't use random noise. They are fixed dithering functions, using fixed color bit depth with sampled pixel color, coordinates and time.
For digital static grayscale noise (that's what your shader does, it's not exactly film grain), it will require conversion to linear RGB or RGB with a constant gamma (this can already be done with my experimental gamma shaders). Running this shader directly on video output will change the darkest colors differently than the lighter colors.

I will see what I can do, but I'm not the best nor the fastest programmer. I'm planning to first finish my current set of shaders, as most of them just need some comment editing. Later on, I will review the film and special effect shaders (including yours). To illustrate my programming speed: earlier in this thread, burfadel mentioned adding noise and I came up with temporal dithering. Since then, that item has been on my wish list, but the experimental shader just doesn't work exactly right yet.
My first "denoise" shader was made in September and my first "chroma up-sampling" shader was made in October. My "Hello world!" shader was made over 2 years ago.
To add variables: -My disease complicates things from time to time, so I can't always do everything I want immediately. -My programming skill certainly isn't great, so I sometimes really need help from other programmers, but it's hard to get decent programmers to do something without payment. I'm not going to pay anyone, so I'm left with the very scarce free help I can find.

For now, I will store a quickly cleaned up version of your shader, as it's a bit different from those that I already have.

By the way, the introduction of HDR (high dynamic range) rendering changed a lot of vertex, geometry and pixel shaders. As far as I know, direct noise isn't one of them. What HDR implementation did you mean?

[CiNcH]

Your shader does look interesting, but the notion of "fix some noise banding" is wrong. Banding in the digital realm is due to limited bit depth of source files and displays. The correct way to handle banding from a source is to use a type of deband filter. For output it will need temporal dithering with settings for 5-bit to 11-bit digital displays or connections. The input type of shader I made already works quite well. The output type of shader I made is difficult to analyze, as it needs to be run on the output frame texture, that's after the color correction. The temporal dithering methods don't use random noise. They are fixed dithering functions, using fixed color bit depth with sampled pixel color, coordinates and time.
For digital static grayscale noise (that's what your shader does, it's not exactly film grain), it will require conversion to linear RGB or RGB with a constant gamma (this can already be done with my experimental gamma shaders). Running this shader directly on video output will change the darkest colors differently than the lighter colors.
BTW, for those interested... here is an image of how the display controller within latest AMD GPU series handles color correction (e.g. sRGB -> wide gamut). It now converts to linear RGB first.

http://images.anandtech.com/doci/3987/ColorCorrection2.png

[JanWillem32]

That picture should only apply to color profiling unaware programs, where the standard engine from the video card completely takes over the color management. If indicated by a program, many of those functions can be disabled or changed. I tested this with MPC-HC, and indeed, many of the items listed in the control panel don't work at all when I play a video. It will however take some time before the color engine in MPC-HC matches the quality of the one in Photoshop. I'm already glad it passed the test with the R and G inverted test profile.
I do like that system with de-gamma, as you can see in my post from yesterday. I de-gamma for a lot of the RGB-mode shaders, and then I have to invert it again at the end, or else the color correction fails. Linear RGB is just an easy format to make compare and interpolation schemes for.

[CiNcH]

Short chroma upsampling comparison with my HD 3650

Sample: Bronzés.mkv (720p H.264)
Decoder: ffdshow-mt
Renderer: MPC-HC EVR Custom Presenter (Nearest Neighbour)

ffdshow NV12
http://members.inode.at/762450/chroma/nv12.pnghttp://members.inode.at/762450/chroma/nv12_chroma.png

ffdshow YUY2
http://members.inode.at/762450/chroma/yuy2.pnghttp://members.inode.at/762450/chroma/yuy2_chroma.png

ffdshow RGB32 HQ
http://members.inode.at/762450/chroma/rgb32hq.pnghttp://members.inode.at/762450/chroma/rgb32hq_chroma.png

ffdshow NV12 + Jan's 4:2:0 Chroma Upsampler
http://members.inode.at/762450/chroma/nv12_chroma_shader.png

[JanWillem32]

Thanks for those very clear samples, they even show a better representation of the two types of the problem (4:2:0 and 4:2:2).

I have some good news and some bad news.
The bad news is, that one of my experimental shaders proved that MPC-HC has a bug. I used the following shader:
sampler2D s0 = sampler_state{MipFilter = None; MinFilter = None; MagFilter = None; AddressU = Border; AddressV = Border; BorderColor = 0x111111;};

float4 main(float2 tex : TEXCOORD0) : COLOR{
return tex2D(s0,.125*tex);
}

That proved that either MPC-HC or my video card's driver ignores the given sampler states. As my reference renderer does properly project the texture on a teapot with the given sampler states, it seems likely that it's a problem with MPC-HC.
This bit of code does compile without errors within MPC-HC, and I can't see anything wrong with the compiler code output.
This shader does nothing more than magnifying 8× with nearest neighbor. The problem is, "MagFilter = None;" should disable any texture filtering (bilinear, trilinear or anisotropic), but in MPC-HC the filtering is forced. I was already wondering why my shaders didn't improve with different sampler states. I also tried down-scaling with this shader, by changing ".125" to "1.5". "AddressU = Border; AddressV = Border; BorderColor = 0x111111" should then give white borders where there once was a texture, but it doesn't work.
Changing settings for the renderer, video scaler or video card driver didn't change anything.
Can anyone test this as well? I will need to make a bug report if everyone has this problem.

The good news is, once this bug is solved, the quality of some my shaders could improve quite a bit, especially the scaling types. Until then, I will only have my reference renderer to test the shader quality. :(

[CiNcH]

How about integrating the scalers the way bicubic is integrated into MPC-HC CP? The rendering engine then handles these things..

I now tried Catmull (again with DVD content upscaled to 1080p). It is a bit more blurry compared to Bilinear but a lot better than your Lanczos2.

[JanWillem32]

I wonder if that would solve it, as it's still a shader, and it remains in the same rendering environment.
The shader is blurry because the sampler it uses is forced to filter 4 to 64 pixels per sampled pixel, depending on the depth of the forced trilinear/anisotropic filter. An unaltered input (as requested by the sampler state) would be much better for performance and accuracy.
I do want to remind everyone that the shader doesn't iterate automatically, so it can't scale up or down more than 2 times. Use the "2×" version and the regular version on top of that to be able to scale between 2 and 4 times.
Right now I'm in the process of trying to bypass the automatic filters by either trying to set environment variables or by using texture loading to memory. It's really not my cup of tea to try to bypass this problem, as I'm completely in the dark why it's happening in the first place.

Edit:
Upscaling video content with non-square pixels doesn't work with that pixel shader either. Video content from most DVD's will always look wrong when scaled by that shader. I will add a version can split scaling for width and height.

Edit:
The "test 13" batch this time, with 42 shaders. :) This includes new shaders, among those shaders is my first version of "semi-random grayscale noise". It looks quite the same as the original version, but it's a good start.
I really should start recruiting beta testers, testing all the normal and experimental shaders under various conditions is taking too much time and effort. Not to mention that English is only my third language, so I need to carefully re-check grammar and spelling for the names and comments. I rather work on the experimental types and get them to function. :rolleyes:

Edit:
I added some comments and made the scalers work properly. "Spline6" (square root of 36) now resizes with the regular sharpness, including the ringing artifacts. Be careful when filtering in combination with that shader, over-sharpening isn't pretty.

[CiNcH]

Hi Jan,

please don't edit old postings. It's really hard to follow then..

So if I want to upscale anamorphic DVD to 1080p, I have to do the following...

Stack the shaders 'Catmull-Rom Spline6 width resizer' and 'Catmull-Rom Spline6 height resizer' in screen space and edit them the following way:

#define Magnify (1920/720.)

resp.

#define Magnify (1080/576.)


Is that correct?

[EDIT]
Result seems to be too wide, horizontally stretched too much, cutting of parts of the image to the left and to the right.

[CiNcH]

#define Magnify (1920/1024.)

and

#define Magnify (1080/576.)

seem to give me correct results. Output pretty much looks like a bilinear filter.

[JanWillem32]

Don't forget that if anamorphic correction is enabled, the internal resizer will still be used. If you disable that one too, (1920/720.) and (1080/576.) are correct.
Anamorphic correction is under View, Video Frame, "Keep Aspect Ratio".
I've had some very good results with Spline6, it should really not look like bilinear filtering, even when the forced sampling filtering bug is still present. Can you compare 400% with Spline6, Spline4 and some default scalers? It might be a problem with my shaders after all.

[CiNcH]

Anamorphic correction is under View, Video Frame, "Keep Aspect Ratio".
OK, result is now comparable to Bicubic with Spline6.

Can you compare 400% with Spline6, Spline4 and some default scalers?
Doubling the size + pan&scan does not give me the same result with your scaler vs. default scaler (like Bilinear).

[JanWillem32]

It's been a while since I last posted, so here's the test 15 batch. I mostly fixed minor errors, and I finished the other old film effect shaders to accompany the noise shader. I can merge all or some of the film effect shaders later on, if that's preferred.
At the moment I'm searching for the bug in the shader environment code that forces the sampler states to use filtering. I already found the main resizers, effect shaders and color control/final shader at: http://sourceforge.net/apps/trac/mpc-hc/browser/trunk/src/apps/mplayerc/res/shaders.
Once I know a bit more about the bug I will make a support ticket, with some references of how to solve it.

I would appreciate some feedback from testers about the usual performance with my shaders in different environments. I currently only use my standard (game-orientated) shader editor and MPC-HC with EVR to host the shaders. Other projects that could use a few new shaders are definitely welcome too.

[TheElix]

semi-random grayscale noise.txt - is this the "BD grain" filter? Unfortunately it doesn't add to the detail of the picture. With its current state it seems more like an interference. Maybe it's too intense? Or maybe it'll never be able to enhance detail like the real BD grain.

As for hybrid shader here's my usual portion of comarisons:
1) On HD anime content: http://screenshotcomparison.com/comparison/12394
While it looks sharper and nicer at first you may notice that there's a huge loss of detail in the grass and in the mountains. It's like lowering the bitrate of a video considerably which is unacceptable.
2) On HD real-life content: http://screenshotcomparison.com/comparison/12402
There's a loss of detail in the shadings. Look at her palm and the area around her mouth. A lot of transient colors in that area was lost and it looks bad. Almost like a scrub, lol.
3) On SD real-life content: http://screenshotcomparison.com/comparison/12404
Again, there's huge loss of detail everywhere: on her skin, her clothes and in the background.

I hope this doesn't strike you too hard) I was only testing.

[JanWillem32]

Thank you for the comparison. First of all: what's the "BD grain" filter? The "semi-random grayscale noise" was modeled to evenly add or subtract brightness, randomized per pixel, and evenly as much on dark and bright patches. I re-designed most of the shaders that rely heavily on the RGB contrast to require linear RGB gamma, including this one.
What kind of noise generation adds detail to a picture? All of the examples with other filters that I've seen either do the same thing as my shader (grayscale per-pixel random noise) or add grain of a certain size (a bit like my dust shader with some altered settings). Neither is beneficial to the detail level.

For the pictures: all three pictures have below average noise. 1 is synthetic, this will require minimal noise filtering (maybe even a bit lower).
2 really requires pre-processing to convert the picture to linear RGB gamma, and mild noise filtering. This picture has a large contrast. Unfortunately video contrast (BT.601 and BT.709) is linear for low brightness, and has gamma scaling above a certain point of brightness. That makes direct RGB fltering with the same weights for darker and lighter areas impossible. See this website for the video gamma function (at number 9): http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html
3 is really low resolution (interlaced NTSC?), it rather needs a good scaler than sharpening. It can be sharpened by "sharpen complex v3 + deband + minimal denoise" in screenspace after scaling. It will be hard to find the right set of filters for this picture that do sharpen, but don't accentuate staircasing artifacts because of the scaling.

I hope you can test again. If required, I can change a few functions. The "semi-random grayscale noise" will probably need a bit of work, or a second shader to with a different type of noise (if I can find an example how it should look).

[Jong]

The video file to display path in Windows requires Y′CbCr data to be converted to RGB, even if data is converted to Y′CbCr for DisplayPort, HDMI or dual link HD-SDI transport. Hi Jan, only just found this thread after a while away.

Are you sure this is always true.

What about the "EVR onto overlay surface" used by the "commercial players", used mainly for Blu-ray, but also for DVD in TMT?

I ask because some months ago it became clear over @AVS Forums that this 'special renderer':

a) allows BTB/WTW for Blu-ray, when all other players end up clipping during RGB conversion, even if YCbCr is eventually output.

b) This renderer also fixes a slight "Green push" error at low lunminance, when using YCbCr that AFAIK is still present in the latest ATI drivers for 5xxx. This "green push" was assumed to be introduced during RGB -> YCbCr conversion, as it is not present with RGB output.

[JanWillem32]

As long as no DirectX surface/texture format is used, Y′CbCr can be exported directly to the video driver (with some flags set). That does disable any other blending, such as subtitles, color correction, filters, etcetera. It also can't use that many scaling modes (for both chroma and luma). Further usage of the videocard is limited, as there are not a lot of pure Y′CbCr mode filters/shaders. If any blending does happen with the picture, then it's converted to RGB in between, the video driver can force such a conversion in the end stages of video processing as well. The video card driver will always use larger processing formats than 8-bit integer, if possible. That can influence things too.
As I stated before, I don't know what method of range limiting is used in the ATi drivers. There's a difference between RGB value compressing (like the original shader) and Y′&CbCr value compressing (like my shader). If there's a green push, then it's likely the former.

[TheElix]

2 really requires pre-processing to convert the picture to linear RGB gamma, and mild noise filtering. This picture has a large contrast.
Ugh. Is this better? =\ I'm not sure. http://screenshotcomparison.com/comparison/12459
Maybe I got the wrong conversion shader.

So you're saying each video needs its own approach? It's only natural, I suppose. But from a user's point of view that'd be a nightmare - to pick up new shaders each time you watch a different video. That is assuming you want to get a better picture. It makes me wonder if you can enhance it at all using these means. Maybe it's a better idea to leave the enhancement thing to studios and watch videos they were produced? Not saying that your or others' work is meaningless. On the contrary, video rendering technologies improved substantially thanks to the likes of you. I'm only saying that these shaders thing is more for the guys who understand video processing deeper than common users.

[JanWillem32]

That filtered picture looks perfectly OK, if I view it in Photoshop with a linear RGB input setting.
You got the right conversion shader, but maybe I forgot to tell that the "gamma conversion of linear RGB to HD&SD video RGB" shader, should then be added at the end of the filter chain (can be used in screenspace). If you use a display calibrated to sRGB or wide gamut RGB, you can use a different conversion shader for output, video gamma output is more something to use in combination with the lCMS color correction function or uncalibrated Y′CbCr mode over DisplayPort or HDMI.
The simple shaders and Y′CbCr-type shaders are made to be used stand-alone. However it's bad for the performance of the Y′CbCr-type shaders, it's more convenient for the user if there's only one or two shaders in use.
It's indeed true that the more complex shaders are a bit harder to configure. In games, it's very common that a single pixel will be handled by more then 20 different shaders before output (multiply that by 10 to 100 for ray-tracing filtering). In video processing, I would have to write a manual for most people to understand how to make proper shader chains, so I try to combine functions.
At first I said that people should even enter the "NoiseLevel" value manually. Later on, I just created 5 basic profiles, with only the "NoiseLevel" value changed.

As for switching, I personally don't hate it. I use chains of up to 8 shaders, and I switch them with simple .REG registry profiles. I already stated that activating and switching shaders within MPC-HC is harder then editing the registry, so the program could use a bit of improvement in the GUI. Maybe an item for shaders in the options screen, combined with decent descriptions of what they do and how to use them would be welcome. Y′CbCr- and linear RGB-type shaders could then also be placed automatically into the right category where they receive a correct input by default. Profiles of shader chains, with some default examples would be useful, too.
Seeing that the default end-stage shader also executes the lCMS color matrix, I think that function could be added as a selection for the end-stage shaders. (The absolute last function should be the "0-256 to 16-235 for SD&HD video input" shader, if it's really required.)
On top of that, don't forget that the default scaling functions are handled by 2 stacked shaders, as well.

I agree with you, the use of shaders could be easier, but I completely disagree to let studios/distributors handle things...:
http://theabyssgazes.blogspot.com/2010/03/teal-and-orange-hollywood-please-stop.html
http://en.wikipedia.org/wiki/Loudness_war
http://www.cracked.com/article_18664_5-annoying-trends-that-make-every-movie-look-same.html

Not to mention the terrible encoding settings I've seen that distributors use for DVD/Blu-ray audio and video. Even worse, DVD's/Blu-ray's from a tape master from video that was shot and/or edited in a digital format.
I'm wondering when they start realizing that 4:2:0 sub-sampled chroma, 8-bit color, interlacing, compressing dynamics, etcetera, are all lowering the overall quality compared to the master, even before the lossless/lossy compression scheme can do it's work.

[bobdynlan]

I finished the other old film effect shaders to accompany the noise shader. I can merge all or some of the film effect shaders later on, if that's preferred.
After your initial reply I thought that you did not get it and I was thinking to recode it myself, as you've misunderstood the comment on your first look (it was about the noise banding i.e. the random noise algorithm was too limited and generated visible patterns from time to time and that's why I've added the variable at that time).
And now, Bull's Eye! Just what I asked for, a recode that's mathematically sound and outputs the same effect. And a proper name for it, too as "semi-random grayscale noise" best describes it - but not the best "selling" one. What could potentially improve it will be higher displacement and a variable size, as it is now too uniform and per-pixel like. I'm even considering replication of bad hdmi cables coupled with plasma display - you know that discreet red noise. Someday...

For the people that did not understand the purpose of this (unfinished) effect, think about it this way: a print can be outstanding, but will never match an oil paint. Why is that? - Because of the flatness and perfectness a print has vs. the 3d layered nature of the oil paint. You can move around it and discover new details, if the light changes you discover other details and so on. Most of those details may have been inherited flaws of the technique but you see them as part of the art. A parallel can be made to studio(tv) versus film. Grain was a flaw in the technique as well, but become part of the art. To hell with studio-like dull pictures wasting space on a blu-ray disk being encoded with lame noise killing parameters and being watched on a lame LCD. Praised be da' Noise, so say we all. It's not there yet, but it helps. It does not improve detail? Try watching some gaga doing telephone - you'll find out she definetly has a d*ck! Joke aside, any movement added on large static areas does just that. But you have to watch it from a suitable distance, and try not to search for it at the beginning... you will get used to it. It must be run as a screen space shader, try a higher NoiseStrength (.10625 magically matched my preference on the old shader).

About the other shaders, TheElix has some point. It's true that it cannot be perfect, because you cannot detect objects from a still alone, you need to consider motion. The only way to fix issues will be a temporal approach, that is to analyse multiple frames and that is a halt. Let's stick with what we've got, MPC-HC with customizable shaders and a one JanWillem32 that brought a consistent contribution on the matter.
Thank you.

[JanWillem32]

Well, I have to thank you too. I could not use the regular noise functions for snow, smoke, fog, dust and scratches, as these use pre-rendering and/or DirectX 10/11 functions. The randomization method was quite a useful template for the film effect shaders. Snow, smoke and fog will need a bit more work, but those are not priority (just special effects). I can take a look if I can make another version of the noise shader that has a preference for a less extreme minus to plus contrast variance distribution. I will definitely raise the default NoiseStrength to a binary rounded fraction near .10625 (7/64. maybe). I already raised the value when I demonstrated it with the old film effects, to make the effect look "authentic".
As for temporal functions, it can be done, but it does require 4 complete texture framebuffers (two past, two future, often with the native 128 bit per pixel color format) for 1 iteration (6 for 2 and 8 for 4). That's 4*128*1920*1080/sqr(1024)=1012,5 MByte of memory throughput per processed 1080p frame. It's 759,375 MByte if alpha (multi-layer translucency) blending is disabled. It also needs vertex and maybe some geometry shaders if it's meant to be handled in realtime on the shadercore, and even then the output has to be re-synchronized because of the buffering. Advanced frame interpolation is a popular temporal filter type. But as many might know, it's one of the heaviest types of video filters. To take that kind of filter to GPU processing in realtime, it will require a recent performance model video card, in most likely a DirectX 10 or 11 D3D fullscreen exclusive mode.

[CruNcher]

To take that kind of filter to GPU processing in realtime, it will require a recent performance model video card, in most likely a DirectX 10 or 11 D3D fullscreen exclusive mode.

doesn't fft3dgpu implements a temporal sharpening (dx9 shader) ?

[JanWillem32]

I looked it up, fft3dgpu uses YV12 (12-bit per pixel planar format) input and output. A bit odd, since the internal calculations can use 32-bit floating point per component. By default, it only processes luma. I wonder how it solves the hue and saturation problems I encountered when using independent luma sharpening on sharpen complex v3. I chose to use independent processing of linear R, G and B with equal weights in the end to make a uniform appearance.
I must say, using an additional .DLL to make some threads for the CPU to work on, is quite a smart way of offloading some tasks.
For my example, I was thinking about a windowed 4-frame method with motion adaptation for frame interpolation (one of the most common methods), sharpening kernels are a bit simpler. Still, the memory requirements per frame I stated are still valid. (I would not recommend to do FFT transformations in a limited 16-bit floating-point format.) Even on the default (low) settings, running fft3dgpu in realtime currently requires a fast system to handle up to 1080p video.

[G_M_C]

JanWillem;
Is there any way to strip away FFT3DGPU to it core, so it just functions as a vehicle to drop an hlsl script into an Avisynth script ?

(i.e. just leave the core so you can use whatever .hlsl you want and the core just feeds&reads frames etc. That way it would become somewhat of a universal GPU filter.)

[JanWillem32]

That's possible, but the environment settings, input registers and function calls to the (very complex) HLSL file are locked by the .DLL file. It won't be easy. If someone would like to start on a project like that, it might be more worthwhile to use the DirectX 10//11 input template and program an Avisynth link for it. That might do the trick to execute in realtime with some performance on various (layers of) complex shaders. If the environment settings, input registers and function calls are then set to generic modes and input modes from the script, it should then be capable of running all kinds of shaders.

[tetsuo55]

Janwillem, any chance of you digging into the bug mentioned here http://forum.doom9.org/showthread.php?p=1463148#post1463148 , and providing us with more detail so we can fix it?

[JanWillem32]

Sure, I've already tried to find the source code that hosts the shaders to try and point out the specific code to update. So far I haven't been very successful.
As a reference, this is what the sampler states are supposed to do:
http://drzovil.blogspot.com/2007/04/texture-filtering-modes.html
It turns out that some other programs tend to force those parameters too:
http://efreedom.com/Question/1-2700041/HLSL-Can-Set-Sampler-Min-Mag-Mip-Filters-Disable-Filtering-Anti-Aliasing
If it takes only one line to solve this problem, like in the above example, it shouldn't be too hard to do something about it.

[Deshi]

Hi everyone !

I would like to have some insight about shaders chain.
I'm corrently testing this chain :

16-235 to 0-256 for SD&HD video input
4÷2÷0 to 4÷2÷2 intermediate Catmull-Rom spline5 chroma up-sampling for SD&HD video input
SharpenComplex v2 + Deband + Medium Denoise
SuperResolution
gamma conversion of linear RGB to HD&SD video RGB

with moderate satisfaction. I'm wondering if I 'm using the shaders in the correct order ? I'm using the bicubic scaler of the renderer because I can't find where to put the Spline scalers in the chain :confused:

I totaly understand that you can't comment every chain that everyone could use but would it be possible for you just to give us a guide line ?
Something like :
these shaders (x, y, z) must go first/middle/end...
these shaders (x, y, z) must go before/after these...

I've read attentively your comment in every shader but I'm such a newbie in this that I don't understand much...

Thanks

[JanWillem32]

For regular SD&HD video input, the following shader order is correct:
-input video
~~~"for SD&HD video input"-type shaders (these use Y'CbCr mode)
~~~"gamma conversion of HD&SD video RGB to linear RGB" shader
~~~regular shaders
-screenspace renderer
~~~scaling/resizer shaders
~~~regular shaders
~~~"gamma conversion of linear RGB to"-type shader*
-video output

*Use the "gamma conversion of linear RGB to sRGB" shader for sRGB standard display devices.
*Use the "gamma conversion of linear RGB to wide gamut RGB" shader for wide gamut RGB standard display devices.
*Use the "gamma conversion of linear RGB to HD&SD video RGB" shader for display devices:
- with a calibrated .ICM display color profile installed system-wide and color management option in the program enabled.
- connected with Y'CbCr mode over DisplayPort or HDMI.
- with a very specific need for a BT.709 or BT.601 derived RGB input.

**The "0-256 to 16-235 for SD&HD video output" is an exeption, it needs HD&SD video RGB input and it has to be placed at the very end of the chain of shaders, and only without any further software color management or video card driver-based color correction.

[Deshi]

Waow !

Thanks a lot for the quick reply !

The difference you make between "input/screenspace/output" shaders...
Does it mean that I have to choose if I use them as input or screenspace ?
Does it mean that they be combined all together in the combine option for shaders in MPC-HC ?
Is there a different "place" to put the screenspace shaders ?

Thanks again...

[JanWillem32]

Shaders that run on the video input texture run at the video FPS with the same height and width as the source video.
[Currently not implemented in MPC-HC, but is a future possibility.] Screen resolution shaders run on the video FPS with same height and width as the output display resolution.
Screen space shaders run on the screen refresh rate with same height and width as the output display resolution.

In the main comment line 2 (and 3 for the scaling shaders), I indicate if a shader can, should or should not be used in screen space.
Running a shader in screen space is usually much heavier, by the way. Many displays are locked to 60 Hz. That means that the shader is then executed 60 times for every output pixel per second when used in screenspace. The shaders that run on the input video texture typically run at 24000/1001 to 30000/1001 frames per second, depending on the input video FPS.

Screen space shader slots are available under the "Combine Screen Space Shaders..." option. It can be active simultaneously with the shaders under "Combine Shaders...".

[tetsuo55]

i tried browsing the code but i too cannot find where this stuff is happenng...

[JanWillem32]

Which part of the source code even sets up the standard sampler and registers? By default, the environment registers are empty. Something is setting up the registers containing basic metrics, framecounter, clock, pixel size and texture sampler. I have been looking at far too much code, and found absolutely nothing yet.

[clsid]

A quick search reveals that most shader related code is located in the VideoRenderers part of the solution file. For example the resizer is in DX9RenderingEngine.cpp.

[JanWillem32]

Thank you, that's indeed the DirectX 9 host. It's rather far away in the folder structure... (I didn't look that far, my bad.)
Ignoring the forced sampler states for the dithermap and 3D color correction space for now, the regular sampler states start on line 1661. As far as I can see, lines 1661 to 1666 can simply be commented out. The given states (Linear interpolation and texture border Clamp) are the defaults for the DirectX 9 engine anyway, so if a shader doesn't specify any sampler states, it will simply inherit the same sampler states as it currently receives from the engine. By the way, if textures are handled in pure 2D mode, forcing mipmap generation to "None", can be very useful in preventing extra work for the GPU. (Mipmaps are used for anisotropic filtering on textures that are displayed with a z-angle.)

While I'm at it anyway, why is "full floating point processing" specified as fp16 RGBA? In my book, a full floating point is fp32, a half is fp16 and a double is fp64. On top of that, shaders will use fp32 mode on textures/calculations anyway, unless the "partial precision" flag is set. (Even if that flag is set, it's ignored at least in my case: http://forum.doom9.org/showthread.php?p=1461777#post1461777 . My GPU probably doesn't even have a half-precision pipeline.) I don't know if a method of constant switching between fp32 and fp16 mode is beneficial for performance when large textures are loaded and unloaded up to 200 times per second. (It's a different thing in games where many loaded textures are often used for several hours.)

[tetsuo55]

Is it within your capaibilities to make a patch to fix what you found?