Shader implementation of the NEDI algorithm

[Shiandow]

Yes the YCbCr shader should be after Chroma-NEDI. Glad it's working!

[FireFreak111]

Small thing, your YCbCr to RGB shader is bugged for SD, its only outputting 16:235 (example being a fully black 1st frame is grey). On HD, Blacks is fully functional.

For this SD content, the primaries are apparently SMPTE C (not BT.709, as usual). The shader isn't handling this type.

[Shiandow]

I added a different RGBtoYCbCr shader to the first post that should also work for SD content, but again likely not all. You could also use the RGBtoYCbCr shader I added instead of using the YCbCr hack, this should at least keep the colours the same.

[FireFreak111]

Absolutely perfect, working completely without the hack (which was messing with my gamma for some reason). Thankyou for this shader.

[Shiandow]

Minor update: the shaders chromaNEDI-I and II have been improved a bit by using a slightly larger (more disk shaped) window, the difference is most noticeable when not using chromaNEDI-III (which seems to remove some detail, including aliasing).

[FireFreak111]

Is there a way to view Chroma only, so I can compare different algorithms? Is there a shader I could use?

[Shiandow]

You can just modify the YCbCrtoRGB shader slightly so it will remove the luma channel:

sampler s0;
float2 c0;

float4 main(float2 tex : TEXCOORD0) : COLOR
{
	float3 s1 = tex2D(s0, tex).rgb;// original pixel
	s1.yz -= 0.5;
	s1.r = 0.3; // erase luma
	if(c0.x < 1120 && c0.y < 630) return float3(s1.x+1.402*s1.z, dot(s1, float3(1, -.202008/.587, -.419198/.587)), s1.x+1.772*s1.y).rgbb;// SD Y'CbCr to RGB output
	return float3(s1.x+1.5748*s1.z, dot(s1, float3(1, -.1674679/.894, -.4185031/.894)), s1.x+1.8556*s1.y).rgbb;// HD Y'CbCr to RGB output
}

[FireFreak111]

This works perfectly except when the Chroma-NEDI shader is in the middle, then everything shows as normal, so I cant compare your shader to others. (Luma is included when using your shader, with just RGB-YCbCr-RGB I see chroma.

Also, alot of grey in this chroma, no blacks, is that normal? First frame of a movie for example is grey, it's black when the new value you added is at 0.0

[Shiandow]

What I usually do when I want to compare is to add a RGBtoYCbCr shader followed by the chroma only shader post-resize, then I can toggle chromaNEDI on/off by enabling disabling the pre-resize shaders. You could also use it instead of the YCbCrtoRGB shader but that makes it harder to compare.

You see a lot of grey because it fixes the luma to 0.25, if you set it to 0 then it's usually too dark to see anything.

[FireFreak111]

Using Frozen for chroma investigation, due to the movie's good chroma compression and heavy use of it, especially in the pink cloak constantly used.

NEDI seems to fare well against NNEDI3, except for some blur on distinct edges (pillars, characters lower cloak, reflection, shoes). Jinc3 (not shown) has the same definition here as NNEDI3, so its distinctly NEDI's problem, not an advantage of NNEDI3.

NEDI



NNEDI3-16



To be fair, NEDI is 2-3x faster then NNEDI3, entire chain taking only 11.7ms on 1080p24 content (at 1080p48 with no SM).

[Shiandow]

One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp. Without the luma channel it's hard to tell if NEDI is blurring an edge that it shouldn't or NNEDI3 is sharpening an edge that it shouldn't. Both are very well possible. I also found it interesting that NEDI seems to be better at bringing out detail on the pillars itself.

[leeperry]

Originally Posted by Shiandow

A slight adaptation of chromaNEDI-II should do it; it currently shifts things half a pixel vertically, but it could just as well shift them horizontally. Although I would like it better if the shifting was done simultaneously with scaling. Anyway the following code should shift the chroma channels half a pixel to the right. It expects the input to be in YCbCr, and should be used pre-resize. It tries to convert the result to RGB but I can't guarantee that that part will always work; it might be wrong about the colour space.

Sweet, thanks! All MPEG1 movies are using BT.601 but fair enough, if there can't a fail-safe way to work it out afterwards I'll just feed mVR with RGB32 for MPEG1. Would need to run test patterns in order to ensure that ffdshow does it properly duh ^^

[FireFreak111]

Chroma.rar

Here is some screenshots with Luma shots for NEDI and NNEDI3, Chroma for NEDI, NNEDI3, Jinc3 and Bicubic, compressed into a RAR. This is after disabling the no DXVA copyback option in madVR, which was bluring the Chroma channel (still a blur difference between NEDI and NNEDI3, causing some minor artifacts like the bottom of the character's cloak in the Luma shot)

On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?

[Shiandow]

Originally Posted by FireFreak111

On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?

The second shader needs the results of the first one so it's not really easy to do it in just one shader. It would be possible to split them in a slightly different way but I think it will obfuscate the code and it will only be faster because I haven't found a way to make the shader skip processing for a particular pixel.

[Shiandow]

With the current discussion around chroma channel alignment I thought it would be useful to add support for different chroma patterns. I've therefore updated the chromaNEDI shaders.

[cyberbeing]

Originally Posted by Shiandow

One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp.

Is this why the NEDI chroma shaders don't remove distinct 2x2 block aliasing steps like the others do? The source video in question was a 640x360 x264 1200Kbps stream.

__________


Nearest Neighbor Chroma (400% zoom)
__________


NEDI Chroma (400% zoom)
__________


Catmull-Rom Chroma (400% zoom)
__________


NNEDI3 64 Chroma (400% zoom)
__________

madVR's YCbCr hack also seems to result in an elevated purplish black level on SD video with the provided YCbCrtoRGB shader as the last step pre-resize. [Edit: This was caused by using a 3DLUT with the YCbCr hack, which I guess must not be supported]. The above images used madVR in normal mode with both RGBtoYCbCr (first step) and YCbCrtoRGB (last step) shaders to workaround this.

[madshi]

@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.

[cyberbeing]

Originally Posted by madshi

@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.

Okay, I've just sent a PM with a sample to you and Shiandow.

[Shiandow]

There seems to be a mismatch between what the shader thinks the chroma channels are and what they actually are. As a result some part of the chroma channel isn't scaled. Using the RGBtoYCbCr shader will ensure that the colours remain accurate even if it guesses the colour space incorrectly, but instead it will use the shaders on the wrong chroma channels. If you use the YCbCr hack you should get a smooth(er) result but the colours might be inaccurate.

If you use the chroma only shader I posted a few posts back, you'll see that the "chroma" part looks perfectly smooth. Unfortunately it is the wrong chroma channel.

Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.

[cyberbeing]

Originally Posted by Shiandow

Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.

If you think those shaders are incorrect, you could try adapting NVIDIA's sample vertex/pixel shaders for RGB <-> YCbCr conversions which I'd assume should be accurate.

NVIDIA RGBA to YUVA sample | NVIDIA YUVA to RGBA sample

The actual pixel shader portion of these samples seems to be the following, but would need to be made MPC-HC compatible:

#define QUAD_REAL float
#define QUAD_REAL2 float2
#define QUAD_REAL3 float3
#define QUAD_REAL4 float4

QUAD_REAL4 ToYUV(QuadVertexOutput IN,
		    uniform sampler2D SceneSampler) : COLOR
{   
    QUAD_REAL4 rgba = tex2D(SceneSampler, IN.UV);
    QUAD_REAL3 ctr = QUAD_REAL3(0,.5,.5);
    return QUAD_REAL4(rgb_to_yuv(rgba.xyz)+ctr,rgba.w); // don't lose alpha
}

QUAD_REAL3 rgb_to_yuv(QUAD_REAL3 RGB)
{
    QUAD_REAL y = dot(RGB,QUAD_REAL3(0.299,0.587,0.114));
    QUAD_REAL u = (RGB.z - y) * 0.565;
    QUAD_REAL v = (RGB.x - y) * 0.713;
    return QUAD_REAL3(y,u,v);
}

QUAD_REAL4 FromYUV(QuadVertexOutput IN,
		    uniform sampler2D SceneSampler) : COLOR
{   
    QUAD_REAL4 yuva = tex2D(SceneSampler, IN.UV) - QUAD_REAL4(0,.5,.5,0);
    return QUAD_REAL4(yuv_to_rgb(yuva.xyz),yuva.w); // don't lose alpha
}

QUAD_REAL3 yuv_to_rgb(QUAD_REAL3 YUV)
{
   QUAD_REAL u = YUV.y;
   QUAD_REAL v = YUV.z;
   QUAD_REAL r = YUV.x + 1.403*v;
   QUAD_REAL g = YUV.x - 0.344*u - 1.403*v;
   QUAD_REAL b = YUV.x + 1.770*u;
   return QUAD_REAL3(r,g,b);
}