Shader implementation of the NEDI algorithm

[Shiandow]

This post was written after I succeeded in implementing the NEDI algorithm (the one by Xin Li et al. not the nnedi3 algorithm by tritical) using shaders. I've since tried to write upscaling algorithms using variations on NEDI and other methods. The results of which can be found below.

NEDI:
For a short explanation of the NEDI algorithm see: http://chiranjivi.tripod.com/EDITut.html

NEDI's picture quality tends to be better than that of linear scaling algorithms (like Lanczos, Jinc, etc.) and in some cases beats nnedi3. It's especially good at scaling the image without aliasing. Development of this method has more or less finished, or otherwise superseded by SuperRes which uses NEDI as part of it's process.

Here is a quick comparison (these results are no longer up to date):
Nearest (http://imgur.com/BCEfaHn.jpg)
Jinc3 (http://i.imgur.com/JP3R1eZ.jpg)
nnedi3 (32 neurons) (http://i.imgur.com/gnG1WQN.jpg)
NEDI (http://i.imgur.com/NtovKwF.jpg)

The easiest way to try NEDI is to use MPDN (http://forum.doom9.org/showthread.php?t=171120)'s render script capabilities. For details on how to use renderscripts see here (http://forum.doom9.org/showthread.php?t=171120).

To use the NEDI shaders for 2x upscaling with MadVR you should set MadVR to output YCbCr, by adding an empty file called "YCbCr" in the MadVR folder (or use the RGBtoYCbCr shader), and set MadVR to use a Nearest filter for luma upscaling (using NNEDI3 will also work but is obviously slower, using any other algorithm will give incorrect results). Also make sure that you're resizing the video exactly 2x, if this doesn't fit your screen you can usually force the video player to scale 2x anyway. You should then use the NEDI-I and NEDI-II shaders (in that order) post resize, and then you need to convert the result back to RGB. For Rec. 709 media this can be done by using the YCbCrtoRGB shader, unfortunately it probably won't work for all video types.

ChromaNEDI:
ChromaNEDI is a way of using NEDI to upscale chroma using information from the luma channels.

This project has largely been abandoned after I found out that this method causes a lot of chroma bleeding. I have been able to solve this, partially, by performing the scaling in linear light but this makes the NEDI artefacts (too) visible.

Anway, the chromaNEDI shaders can be used for chroma upscaling, currently this only works for 4:2:0 subsampled video but that seems to be 99.9% of all video. The way to use these shaders with MadVR is similar to how you use the NEDI shaders but should be used pre resize instead of post resize and you should set chroma upscaling to Nearest instead of luma upscaling. It consists of three different shaders (chromaNEDI-I up to chromaNEDI-III), which must all be used in order. The first two shaders work similarly to the two passes of the NEDI algorithm, the third one is necessary to align the chroma channels with the luma channel.

The chromaNEDI shaders also have support for several different chroma patterns, you can switch between these by changing the line "#define pattern x" where x should be 1,2,3 depending on which pattern you want. Make sure that you pick the same pattern for all shaders.

Short explanation:
pattern 1 (https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/Yuvformats420samplingMPEG-2.svg/500px-Yuvformats420samplingMPEG-2.svg.png): this is most common for modern codecs.
pattern 2 (https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/Yuvformats420sampling.svg/500px-Yuvformats420sampling.svg.png): used by mpeg-2, seems to be common for older codecs.
pattern 3 (https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/Yuvformats422sampling.svg/500px-Yuvformats422sampling.svg.png): not used much but is useful for chroma-doubling (shaders should be used post-resize in that case and luma upscaling should be put to nearest. Only works for 2x resizing). You can skip the third shader when using this pattern.

SuperRes:
The SuperRes shaders use a different scaling method which can be used in combination with NEDI (or any other scaling algorithm). This method is explained in detail here (http://forum.doom9.org/showthread.php?p=1685124#post1685124). This method seems to give better results than just using NEDI, and rival those of NNEDI3. These are now also available as an MPDN renderscript.

SuperChromaRes:
With techniques similar to those of SuperRes it's also possible to do chroma scaling. One major advantage is that this makes it possible to do chroma scaling in linear light, which would normally be impossible. This can improve image quality greatly for images consisting of saturated colours (especially red) on a white background. This is also available as an MPDN renderscript, but I've also decided to make the original experimental shaders available to make it possible to try it out with other renderers. Be warned that support for these experimental shaders will be minimal, I will not be backporting all the improvements made in the renderscript, nor will I explain all the options, they also have some of the same issues as ChromaNEDI but will generally work well for HD sources.

Downloads:

SuperRes shader pack (includes NEDI) (http://www.mediafire.com/download/22o6ahnchkbzhef/Shaders.rar)
ChromaNEDI shader pack (Includes RGB <-> YCbCr conversion shaders) (http://www.mediafire.com/download/86bo6bl66cnwv2j/chromaNEDI.rar)
Experimental SuperChromaRes shaders (Includes a short manual) (http://www.mediafire.com/download/1fnutv48bb3k71k/SuperChromaRes.rar)


More information on using MPDN and renderscripts:
http://forum.doom9.org/showthread.php?t=171120

Code of the NEDI shaders:
NEDI-I:

sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define offset 0.5
#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy)))//-float4(0,0.5,0.5,0))
#define Get(xy) (Value(xy)[0]+offset)
#define Get4(xy) (float2(Get(xy+2*dir[0])+Get(xy+2*dir[1]),Get(xy+2*dir[2])+Get(xy+2*dir[3])))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual
float2 solve(float2x2 A,float2 b) {
float2 x = 1/4.0;
float2 r = b - mul(A,x);
float2 p = r;
float2 Ar = mul(A,r);
float2 Ap = Ar;
for (int k = 0;k < 2; k++){
float a = min(100,dot(r,Ar)/dot(Ap,Ap));
x = x + a*p;
float2 rk = r; float2 Ark = Ar;
r = r - a*Ap;
Ar = mul(A,r);
float b = dot(r,Ar)/dot(rk,Ark);
p = r + b*p;
Ap = Ar + b*Ap;
}
return x;
}

//Cramer's method
float2 solvex(float2x2 A,float2 b) { return float2(determinant(float2x2(b,A[1])),determinant(float2x2(A[0],b)))/determinant(A); }

float4 main(float2 tex : TEXCOORD0) : COLOR {
float4 c0 = tex2D(s0,tex);

//Skip pixels on wrong grid
if ((frac(tex.x*width/2.0)<0.5)||(frac(tex.y*height/2.0)<0.5)) return c0;

//Define window and directions
float2 dir[4] = {{-1,-1},{1,1},{-1,1},{1,-1}};
float4x2 wind[4] = {{{-1,-1},{-1,1},{1,-1},{1,1}},{{-3,-1},{-1,3},{1,-3},{3,1}},{{-1,-3},{-3,1},{3,-1},{1,3}},{{-3,-3},{-3,3},{3,-3},{3,3}}};

//Initialization
float2x2 R = 0;
float2 r = 0;
float4 d = 0;

//Define weights
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

//Calculate (local) autocorrelation coefficients
for (int k = 0; k<4; k+= 1){
float4 y = float4(Get(wind[k][0]),Get(wind[k][1]),Get(wind[k][2]),Get(wind[k][3]));
float4x2 C = float4x2(Get4(wind[k][0]),Get4(wind[k][1]),Get4(wind[k][2]),Get4(wind[k][3]));
R += w[k]*mul(transpose(C),C);
r += w[k]*mul(y,C);
d += lancz[k]*(Value(wind[k][0])+Value(wind[k][1])+Value(wind[k][2])+Value(wind[k][3]));
}

//Normalize
float n = 24;
R /= n; r /= n;

//Calculate a = R^-1 . r
float e = 0.005;
float2 a = solve(R+e*e*I,r+e*e/2.0);

//Nomalize 'a' (prevents overshoot)
a = .25 + float2(.5,-.5)*clamp(a[0]-a[1],-1,1);

//Calculate result
float2x4 x = float2x4(Value(dir[0])+Value(dir[1]),Value(dir[2])+Value(dir[3]));
float4 c = mul(float1x2(a),x);

//Fallback to lanczos
float t = saturate(1-500*sqr(x[0]-x[1]));
c += t*(d-mul(float1x2(1,1)/4.0,x));

return c;//+float4(0,0.5,0.5,0);
}


NEDI-II:

sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define offset 0.5
#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy)))//-float4(0,0.5,0.5,0))
#define Get(xy) (Value(xy)[0]+offset)
#define Get4(xy) (float2(Get(xy+2*dir[0])+Get(xy+2*dir[1]),Get(xy+2*dir[2])+Get(xy+2*dir[3])))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual
float2 solve(float2x2 A,float2 b) {
float2 x = 1/4.0;
float2 r = b - mul(A,x);
float2 p = r;
float2 Ar = mul(A,r);
float2 Ap = Ar;
for (int k = 0;k < 2; k++){
float a = min(100,dot(r,Ar)/dot(Ap,Ap));
x = x + a*p;
float2 rk = r; float2 Ark = Ar;
r = r - a*Ap;
Ar = mul(A,r);
float b = dot(r,Ar)/dot(rk,Ark);
p = r + b*p;
Ap = Ar + b*Ap;
}
return x;
}

//Cramer's method
float2 solvex(float2x2 A,float2 b) { return float2(determinant(float2x2(b,A[1])),determinant(float2x2(A[0],b)))/determinant(A); }

float4 main(float2 tex : TEXCOORD0) : COLOR {
float4 c0 = tex2D(s0,tex);

//Skip pixels on wrong grid
if ((frac(tex.x*width/2.0)<0.5)&&(frac(tex.y*height/2.0)<0.5)) return c0;
if ((frac(tex.x*width/2.0)>0.5)&&(frac(tex.y*height/2.0)>0.5)) return c0;

//Define window and directions
float2 dir[4] = {{-1,0},{1,0},{0,1},{0,-1}};
float4x2 wind[4] = {{{-1,0},{1,0},{0,1},{0,-1}},{{-1,2},{1,-2},{2,1},{-2,-1}},{{-1,-2},{1,2},{-2,1},{2,-1}},{{-3,0},{3,0},{0,3},{0,-3}}};

//Initialization
float2x2 R = 0;
float2 r = 0;
float4 d = 0;

//Define weights
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

//Calculate (local) autocorrelation coefficients
for (int k = 0; k<4; k+= 1){
float4 y = float4(Get(wind[k][0]),Get(wind[k][1]),Get(wind[k][2]),Get(wind[k][3]));
float4x2 C = float4x2(Get4(wind[k][0]),Get4(wind[k][1]),Get4(wind[k][2]),Get4(wind[k][3]));
R += w[k]*mul(transpose(C),C);
r += w[k]*mul(y,C);
d += lancz[k]*(Value(wind[k][0])+Value(wind[k][1])+Value(wind[k][2])+Value(wind[k][3]));
}

//Normalize
float n = 24;
R /= n; r /= n;

//Calculate a = R^-1 . r
float e = 0.005;
float2 a = solve(R+e*e*I,r+e*e/2.0);

//Nomalize 'a' (prevents overshoot)
a = .25 + float2(.5,-.5)*clamp(a[0]-a[1],-1,1);

//Calculate result
float2x4 x = float2x4(Value(dir[0])+Value(dir[1]),Value(dir[2])+Value(dir[3]));
float4 c = mul(float1x2(a),x);

//Fallback to lanczos
float t = saturate(1-500*sqr(x[0]-x[1]));
c += t*(d-mul(float1x2(1,1)/4.0,x));

return c;//+float4(0,0.5,0.5,0);
}

[TheElix]

Funny you should say that. I've just succeeded in implementing the NEDI algorithm (the one by Xin Li et al. not the nnedi3 algorithm MadVR uses) in a shader. This made it run at a speed similar to, if not faster than, Jinc3. Without too large a loss in quality compared to nnedi3.

Here it is compared to some other algorithms:
Nearest (http://imgur.com/BCEfaHn.jpg)
Jinc3 (http://i.imgur.com/JP3R1eZ.jpg)
nnedi3 (32 neurons) (http://i.imgur.com/gnG1WQN.jpg)
NEDI (http://i.imgur.com/NtovKwF.jpg)

I was hoping that this might be useful as a faster alternative to nnedi3.This is interesting, I'd like to give it a try. Can you share a link to your thread?

[ryrynz]

I was hoping that this might be useful as a faster alternative to nnedi3.

Wow.

NEDI destroys Jinc giving a much more natural presentation. Both Jinc and nnedi3 are very artificial looking in comparison and it's the same speed as Jinc? Could it possibly be even faster if it was coded within MadVR?

I actually prefer the look of NEDI here to MadVR's nnedi3 too, It's just too strong (I would like to make better comparisons using a full screen images though)

[Motenai Yoda]

I still prefer nnedi3

[Nevilne]

Looks quite beneficial for madvr.
Was there ever a nedi plugin for avisynth by the way?

[Shiandow]

EDIUpsizer and FastEDIUpsizer by tritical are apparently based on the NEDI algorithm. This later developed into nnedi3 but for some reason the NEDI algorithm was abandoned at some point.

[ryrynz]

Faster than Jinc? I enabled all four of these in the right order and it's delivering a few frames per second, not sure why as both CPU and GPU usage are fairly low. i5 3570K, HD 4600, windows 7.

Is it possible this is only any good on Nvidia or AMD hardware?

[Shiandow]

Well, it does need a lot of texture calls, then again so does Jinc. What program did you use?

Edit: You can also try making the window smaller by changing the line:
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

to
float4 lancz = {0.328511,-0.0365013,0,0};
lancz /= dot(lancz,4);
float4 w = {1,1,0,0};

or even
float4 lancz = {1,0,0,0};
lancz /= dot(lancz,4);
float4 w = {1,0,0,0};

[leeperry]

Oh wow, very impressive! Sharp, yet not quite artificial looking like NNEDI3 and this will also make the AMD interop lag a non-issue. :thanks: for sharing, can't wait for madshi to add it to mVR :)

[pie1394]

Just looking forward to see if madshi has time to integrate your implementation into madVR!

About the resolution enhancement (i.e. sharpness) of your image samples, my eyes feel :

Nearest >> NNEDI3_32 > NEDI > Jinc3 (+AR or not?)

Of course it definitely causes serious jaggy / unstable edges on motion objects with Nearest scaling mode. :p

In fact sometimes I still notice that with Luma NNEDI3_32 mode. Luma NNEDI3_64 mode solves that to give more stable motion like Jinc3AR and even more sharper image, but the cost is somewhat too high for HD contents. It will be great if NEDI allows to preserve better original resolution + same motion edge stability than Jinc3AR while it maintains the similar cost. :D

ps: My TV is Sony 65" HX920 with reality creation option (i.e. super-resolution function) at 20%. But it is still able to tell some difference among different scaling options in madVR. There is no need to say, madVR's Lanczos4_AR, Jinc3_AR, NNEDI3 options have better performance than the ability of TV set's scaler.

[StinDaWg]

How do you use this in MPC-HC?

[burfadel]

Yeah it would be good if it were incorporated into MadVR, NNEDI3 just doesn't seem practical for everyday use. It's not that the output of NNEDI3 is bad (although maybe a little overdone as some people have pointed out), it's the efficiency of the actual processing. NNEDI3 requires a fair bit of processing power, which in turn requires a fair bit of electricity. If this NNEDI shader can use 'as little' amount of electricity as Jinc, with the quality close to NNEDI3, then that is a very good thing! This is even if the processing of a pixel twice can't be resolved. If it can be worked out, it would require even less processing power...

[madshi]

I have implemented a fair number of algorithms in Delphi/Pascal, just to try them out. The original NEDI is one of them, also the Zhao-Xin Li improvement of the original NEDI algorithm and the iNEDI algorithm. I've also tried MEDI, ICBI and several others. In some images NEDI looks great. But there are also a lot of images where it looks totally unusable, IMHO. Which is why I never even considered using it in madVR.

But maybe my NEDI implementation was buggy, I don't know. If you guys want to make sure, try all the test images from the following thread:

http://forum.doom9.org/showthread.php?t=145358&page=4

I think after trying all of them you'll probably agree with me that it's not a suitable algorithm for madVR due to heavy artifacts. If you do still like it, please post all the 4x upscaled NEDI images of all those test images from the linked thread here, and maybe I'll reconsider. In any case, thanks to Shiandow for your efforts!

P.S: Here's the iNEDI paper which showcases some of the NEDI problems:

http://www.tecnick.com/pagefiles/papers/85_Asuni_Giachetti_iNEDI_VISAPP2008.pdf

Please note that the iNEDI authors "replaced" iNEDI later with the ICBI algorithm. And ICBI is worse than Jinc AR, IMHO. Basically the NEDI algorithm and all its improvements produce directional artifacts and a "fractal like" look. On the positive side, they can be quite sharp. All the newer algorithms which are trying to reduce the NEDI artifacts, are much much slower and generally make the image look less like NEDI and get nearer to Jinc/NNEDI3. Please also note that NNEDI3 while having a similar name is a *completely* different algorithm compared to NEDI. IMHO, NNEDI3 is so much better than NEDI etc that it's not even funny. At least when comparing a lot of different images. The key reason is that NNEDI3 only has very few artifacts. NEDI is *FULL* of artifacts.

[madshi]

P.S: Also look here:

http://www.general-cathexis.com/manual2/#anchor8

Scroll down a bit until you get the table with "Artifact Avoidance, Relative Times, and Visual Comparisons". In that table "Xin Li" is the original NEDI algorithm. I hope you'll agree with me that with this test image, NEDI produces an extremely artificial looking image. Looks like watercolor to me, or some other sort of "art" distortion.

[burfadel]

Madshi, are you talking about NEDI or NNEDI? They're not the same thing, apparently!

[vivan]

Of course he knows what he is talking about...

[Shiandow]

P.S: Also look here:

http://www.general-cathexis.com/manual2/#anchor8

Scroll down a bit until you get the table with "Artifact Avoidance, Relative Times, and Visual Comparisons". In that table "Xin Li" is the original NEDI algorithm. I hope you'll agree with me that with this test image, NEDI produces an extremely artificial looking image. Looks like watercolor to me, or some other sort of "art" distortion.

Indeed NEDI does look horrible on that image. However the shader I created seems to avoid quite a lot of the artefacts (but not all). To make the shader more stable I've had to add a slight preference for 'simpler' solutions which also prevents it from overfitting.

Anyway I've tested it on the images from this page (http://forum.doom9.org/showthread.php?t=170661&highlight=nnedi3), I think it performs reasonably well, although it might be better to fall back to lanczos somewhat more aggressively to avoid some of the artefacts.

Car show. (http://i.imgur.com/2R6Hj2W.jpg)

Castle. (http://i.imgur.com/sKd4IcU.jpg)

Cat. (http://i.imgur.com/3cQWFYO.png)

Clown. (http://i.imgur.com/alZSwlQ.jpg)

Flowers. (http://i.imgur.com/LSOXDaJ.jpg)

Lighthouse. (http://i.imgur.com/jefxJCK.jpg)

Meter. (http://i.imgur.com/QhNPKFr.jpg)

"Wc". (http://i.imgur.com/Z4Br7aW.jpg)

Pixel art. (http://i.imgur.com/qMdhB2p.jpg)

[Procrastinating]

Some of those tests still feel a little "oil painty" for me. If you can deliver any consistent results over jinc/lanczos however, I don't see any reason why it shouldn't be incorporated into madVR. Many people seem to have hardware that is only just, or just under the required performance for NNEDI3 16. It seems like it could at least make NNEDI3 16 redundant.

[madshi]

Ok, it seems Shiandow's modifications do reduce the artifacts quite nicely, compared to the original NEDI algorithm, but I still don't consider it good enough for my taste yet. E.g. compare the Castle image:

NEDI (http://i.imgur.com/sKd4IcU.jpg)
NNEDI3 (http://madshi.net/castleNNEDI3.png)

It's not even in the same class. NNEDI3 is like 10 times better with this image. NNEDI3 is sharper, more focused, has less ringing and much less directional/weird artifacts. I think I would also very much prefer Jinc3 AR over NEDI - with this image at least.

The problem is this: If you just want to upscale one specific photograph/image, you can play with different algorithms and pick one which looks best for just that image. But madVR is about real time video playback with all kinds of different videos/scenes/images. So I need algorithms which always looks at least decent. I can't use algorithms which look great on some specific images but look horrible on some other images. I don't think NEDI is good enough for general purpose madVR use due to the heavy directional artifacts in many images. Of course that's only my personal opinion. And it only applies to the current version of Shiandow's shader...

[Shiandow]

Well, there's not much I can do about the getting a sharper result, the way NEDI works just forces some lines to get blurred (since there's no way to tell on which side of the edge you are, at least not by looking at a small part of the image). I might be able to prevent some of the artefacts, but likely not all. And it's probably possible to prevent some ringing by falling back to bicubic instead of lanczos.

I would like to mention that most artefacts only become problematic when you use NEDI twice, when you only use NEDI once they aren't that much of a problem.

[turbojet]

Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

[Shiandow]

Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

They should be post resize, it basically throws away 3/4 of the pixels and then tries to interpolate them. To simulate actually upscaling an image you need to tell MadVr to either use nearest or nnedi3, in both cases it will keep the pixels of the original image and try to interpolate the rest.

You could probably avoid using the "pre" and "post" shaders. You can get a similar result by changing:

#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy))[0])

to

#define Value(xy) (dot(tex2D(s0,tex+float2(px,py)*(xy)).rgb,1/3.0))

and

return float4(c,c0.gba);

to

return c0-Value(0)+c;

[turbojet]

Thanks and both 1 and 2 should be enabled at the same time? They look pretty similar, are these passes?

[Shiandow]

Yes, it will basically fill in the pixels in the following pattern:

o . o . o
. x . x .
o . o . o
. x . x .
o . o . o
Where the "o" are known pixels, the "x" are calculated from the surrounding "o" by NEDI-1 and then NEDI-2 will fill in the remaining "."s.

[madshi]

One thing this might be good for is chroma upsampling. Instead of using the neighbor pixels to create the interpolation weights, the algorithm could use the luma channel (or both the luma and chroma channels). This way the luma channel would guide the chroma upsampling. This makes a lot of sense because in most cases both luma and chroma change at the same time, and the luma channel already has 4 times the resolution. Of course the algorithm would have to do some safety checks to make sure that nothing bad happens if luma and chroma channels happen to be not related in some rare situations. But I think this could be a *really* good algorithm. I've had this idea for a long time, but never actually got around trying/implementing it yet. @Shiandow, maybe you'd have fun trying your luck with that? FWIW, one big problem with my idea is that sometimes the chroma channel was created with incorrect filtering (e.g. nearest neighbor) or sometimes the chroma channel is offset slightly. One additional thing a good chroma upsampling algorithm could do is to downscale the luma channel to chroma resolution and then check whether the chroma channel likely has a wrong offset or not.

(I had asked tritical at one point whether he'd consider making a special NNEDI3 version for chroma upsampling which takes the luma channel into account, but he was busy with other things, sadly.)

[leeperry]

please post all the 4x upscaled NEDI images of all those test images from the linked thread here, and maybe I'll reconsider.
I would like to mention that most artefacts only become problematic when you use NEDI twice, when you only use NEDI once they aren't that much of a problem.
IMHO, put your resources into luma doubling (maybe a little into chroma upscaling).
Guys, I might be missing something here but the kangaroo test pattern looks fantastic with NEDI and as madshi doesn't really advise using quad from what I understand, then when why comparing 4x upscales at all? We could use NEDI for luma doubling and either NNEDI3/J3AR or that new NEDI idea from madshi for chroma?

Most of the time, I personally find NNEDI3 too sharp for chroma and if we could get off the hiccupy OpenCL train then far more GPU's could be useful to mVR than just the GCN and Maxwell architectures.

[madshi]

Maybe the artifacts are worse with 4x upscales, but they're still there with 2x upscaling, too. IMHO NEDI in its current form is not suitable for a general purpose video upscaling algorithm, as I said before. Of course that's only my personal opinion, but unless I see evidence that suggests that I'm wrong, I won't add NEDI image/luma upscaling to madVR.

[Shiandow]

One thing this might be good for is chroma upsampling.

Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling. (http://i.imgur.com/7GZif44.png)
Jinc3AR chroma doubling. (http://i.imgur.com/V7YQkF8.png)

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.

Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.

[pie1394]

Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling. (http://i.imgur.com/7GZif44.png)
Jinc3AR chroma doubling. (http://i.imgur.com/V7YQkF8.png)

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.


To me the above sample's NEDI version has better edge smoothness, but at the cost of losing some higher freqeuency details (i.e. too soft) than the Jinc3AR version. It can be even noticed on the text body and the tree's texture at a normal viewing distance with the so so Dell U2412M. I guess the difference will be even higher on a TV with super-resolution engine's high-frequency signal restoration enhancement.

[Shiandow]

Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

[pie1394]

Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

The version's sharpness on the tree's texture indeed improves, but it is still somwhat softer than Jinc3AR verison. Yet it creates more noticeable ring effects on the leaf's edge and "T" and "P" characters. :p

Here is my subjective visual experience opinions:

[High-frequency signal preservation]
NEDI v2 > Jinc3AR >> NEDI v1

[Low-frequency signal preservation]
Jinc3AR > NEDI v2 >> NEDI v1

[Angled Object edge smoothness]
NEDI v1 >= NEDI v2 > Jinc3AR

[Ring-effect artifact issue]
NEDI v2 > Jinc3 AR >> NEDI v1

[Shaded? Text clarity]
Jinc3AR > NEDI v1 > NEDI v2

[madshi]

Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.
Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

FWIW, here are two samples which are good candidates for chroma tests:

red fonts (http://madshi.net/redfonts.mkv)
chroma test (http://madshi.net/chromatest.mkv)

Btw, if you create an empty file named "YCbCr" in the madVR folder, madVR will output YCbCr instead of RGB. This might allow you to test better because color conversion to RGB is simply skipped completely.

[Bloax]

madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?

[Shiandow]

Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma? Otherwise I would have to disagree.

Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR (http://i.imgur.com/TfkNr0h.jpg)
chromaNEDI (http://i.imgur.com/zvRy3il.jpg)

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI (http://i.imgur.com/OpLoF3F.jpg)

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.

[nevcairiel]

Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma?

Thats what "chroma upscaling" always refers to in the madVR context, so thats safe to assume.

[Shiandow]

Ok thanks, I wasn't really sure since chroma upscaling is done by doubling the resolution.

[madshi]

Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR (http://i.imgur.com/TfkNr0h.jpg)
chromaNEDI (http://i.imgur.com/zvRy3il.jpg)
Looks nice!

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI (http://i.imgur.com/OpLoF3F.jpg)
The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.
I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.

May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

[madshi]

madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?
I don't see that on my PC. If I type in "http://madshi.net" the browser seems to show that without any redirection. At least no redirection that I can see?

[Bloax]

Apparently it's just my AV pulling dirty, dirty tricks because it thinks your site is suspicious - carry on.

[Shiandow]

The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?


I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.


I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.


What I find somewhat frustrating is that, so far, NEDI has looked good on all video content I've tried. But I'll admit that it has some quirks, I'll try to see if I can iron these out.


May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

Feel free to use them. I'd appreciate it if you'd keep me updated on possible improvements you've made.

Here are some possible improvements that I couldn't implement because I was restricted to using a simple shader:

You should be able to increase the speed by about 1.7 (1.75/3) times just by preventing it from unnecessarily calculating pixels that weren't even supposed to change.
You could calculate some of the values upfront, preventing some texture calls. This will probably work best for the values of "Get4".
As I mentioned before you could probably detect some unwanted behaviour by comparing the original chroma values with a downscaled version of the upscaled chroma.

[pie1394]

I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.


To me the softer result by your NEDI looks like a female with cosmetics on the face -- if compared to BiCubic75AR version. :D

Yet I don't know if the original contents should look exactly like this, or other sharper algorithms add more artifical enhancements to make it look more pop / stereo...

[Shiandow]

I've discovered that the ChromaNEDI-III shader didn't actually use any edge information so it has been replaced by one that does.

Edit: Quick before (http://i.imgur.com/zvRy3il.jpg)/after (http://i.imgur.com/e6FZaI6.jpg).

[madshi]

I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.
FWIW, if you downscale a sharp image 50% and then upscale it again with NNEDI3, the result is soft compared to the original image.

What I find somewhat frustrating is that, so far, NEDI has looked good on all video content I've tried. But I'll admit that it has some quirks, I'll try to see if I can iron these out.
Ok, let me know when you're finished with your optimizations, then I'll give it a try with some videos.

I've discovered that the ChromaNEDI-III shader didn't actually use any edge information so it has been replaced by one that does.
Does that improve image quality visibly?

[Shiandow]

Does that improve image quality visibly?

I've added a quick example to my previous post. It seems to have removed some ringing and aliasing.

The previous shader wasn't capable of detecting the direction of an edge so it was basically the same as a box filter except it multiplied the result by some value which depended on whether there was an edge or not. This did actually prevent some 'bleeding' but it was unstable and caused some artefacts. The current shader can detect the direction of an edge and tries to interpolate along that direction.

[Shiandow]

On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr (http://i.imgur.com/e6FZaI6.jpg) / resized from RGB (http://i.imgur.com/xMztLRj.jpg).

[madshi]

You mean when luma doubling is enabled and chroma doubling is disabled?

[Shiandow]

Yes, I've configured it to use NNEDI3 for luma and Jinc3AR for chroma, except it seems treat the YCbCr image as if it were RGB which causes it to apply NNEDI3 on some other part of the colour space.

[madshi]

madVR first converts everything to RGB. Then if you enable luma doubling but disable chroma doubling, the image is converted to YCbCr just before scaling is done (always using BT.709 for that, IIRC), then NNEDI3 is performed on the Y channel, only, then chroma is upscaled with a different algorithm, then everything is converted back to RGB right after scaling. madVR runs custom pixel shaders always in RGB.

[huhn]

madVR first converts everything to RGB. Then if you enable luma doubling but disable chroma doubling, the image is converted to YCbCr just before scaling is done (always using BT.709 for that, IIRC), then NNEDI3 is performed on the Y channel, only, then chroma is upscaled with a different algorithm, then everything is converted back to RGB right after scaling. madVR runs custom pixel shaders always in RGB.

have you think about using YCoCg for this not YCbCr?
it can be used lossless for rgb and it's super fast too. it needs more bits than the source has and i don't have a clue how to use it with 16 bit. the Y channel doesn't change the bit deep in the 10 bit rgb example:

http://wiki.multimedia.cx/index.php?title=YCoCg

[nevcairiel]

have you think about using YCoCg for this not YCbCr?
it can be used lossless for rgb and it's super fast too. it needs more bits than the source has and i don't have a clue how to use it with 16 bit. the Y channel doesn't change the bit deep in the 10 bit rgb example:

http://wiki.multimedia.cx/index.php?title=YCoCg

If you're doing this in floating point back and forth, there is no loss in doing this in YCbCr. Its just math.

[huhn]

If you're doing this in floating point back and forth, there is no loss in doing this in YCbCr. Its just math.

but the result is "only" stored in 16 bit not float point so this should be faster and more precision.

[nevcairiel]

but the result is "only" stored in 16 bit not float point so this should be faster and more precision.

Its 16-bit float to my knowledge.

[madshi]

Lossless conversion between RGB <-> YCbCr is useless for me. I'm calculating in 32bit+ float and storing results in 16bit+ int/float, and final output is only 8bit. So there's gigantic headroom, no need for lossless conversion. It's *MUCH* more important to use a conversion that matches human perception. So BT.709 is a much better choice than YCgCo in this specific situation.

[huhn]

i only ask this to understand this better.
It's *MUCH* more important to use a conversion that matches human perception
if a RGB is lossless transform to YCgCo doesn't that mean it doesn't matter what colorspace RGB was before? it's the same colorspace again after the YCbCr -> RGB as long as the MATH done with it doesn't change the colorspace?
i don't understand this. i'm just missing the knowledge.

[nevcairiel]

His point is that he does processing on the luma plane after the color decorrelation, and for that to be really useful, this luma plane should match what we humans perceive as luma the most closely.
YCgCo was designed with compression efficiency and ease of computation in mind, not to match the human perception.

[huhn]

i never thought there is a different between luma in YCbCr and YCgCo. so luma information aren't simply luma informations good to know.
enough off topic from me i'm sorry.

[madshi]

His point is that he does processing on the luma plane after the color decorrelation, and for that to be really useful, this luma plane should match what we humans perceive as luma the most closely.
YCgCo was designed with compression efficiency and ease of computation in mind, not to match the human perception.
Exactly.

I could also scale RGB directly with NNEDI3. I'm not converting to YCbCr because I *have* to. I do that only because I get a better quality/performance ratio if I apply NNEDI3 only on the information which is most important to the human eye (= luma information). Scaling directly in RGB is 3x slower than scaling only the Luma channel.

[Shiandow]

Minor change to chromaNEDI-III: the line "//a /= dot(a,1)" shouldn't be commented out after all, doing so causes too many artefacts. The same line in chromaNEDI-I and -II doesn't seem to cause problems.

[Asmodian]

I could also scale RGB directly with NNEDI3. I'm not converting to YCbCr because I *have* to. I do that only because I get a better quality/performance ratio if I apply NNEDI3 only on the information which is most important to the human eye (= luma information). Scaling directly in RGB is 3x slower than scaling only the Luma channel.

To make sure I remember correctly; if one wants NNEDI3 to scale RGB directly, without a RGB->YCbCr->RGB conversion, one simply needs to set both madVR's luma and chroma doubling to the same neuron value? Maybe this would be a better way to compare the algorithms? Though it does make NNEDI3 even slower in comparison.

[madshi]

I always do the RGB->YCbCr->RGB conversion because you usually want to use more neurons for luma than for chroma, even if you enable both luma+chroma doubling.

[Asmodian]

I always do the RGB->YCbCr->RGB conversion because you usually want to use more neurons for luma than for chroma, even if you enable both luma+chroma doubling.

Wow, I had badly misread your explanation (http://forum.doom9.org/showthread.php?p=1664703#post1664703). :o

Of course separate luma and chroma doubling (only doubling luma) has always been better in my tests but if it is low effort maybe a special case for the future when we can all run NNEDI3 256 Luma + Chroma doubling? Or does the math workout the same either way?

On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr (http://i.imgur.com/e6FZaI6.jpg) / resized from RGB (http://i.imgur.com/xMztLRj.jpg).

From my limited understanding there would be a difference between NNEDI3 or NEDI on R, G, and B vs NNEDI3 or NEDI on Y, U, V converted to RGB. The HVS important luma information is more connected to R, G, and B than it is to U or V and both NNEDI3 and NEDI might be sort of human like as far as edge recognition goes? Also the source usually starts with half resolution U and V. Combining the chroma with the higher resolution luma information (via YUV -> RGB) might result in sharper easier to recognize edges for NNEDI3 or NEDI? Of course there is no longer a luma channel so all channels would have low resolution chroma mixed into them but your example seems to say RGB is better. :p

[madshi]

if it is low effort maybe a special case for the future when we can all run NNEDI3 256 Luma + Chroma doubling? Or does the math workout the same either way?
I think the math would either work out the same, or if it does not, YCbCr might be better than RGB. Just a guess, though.

On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr (http://i.imgur.com/e6FZaI6.jpg) / resized from RGB (http://i.imgur.com/xMztLRj.jpg).
Hmmmm... I'm not sure what you mean exactly with this. The pre-resize shader should already run in RGB. All of madVR's custom pixel shaders (both pre-resize and post-resize) are always fed with RGB data. So why would you convert to RGB, if the data is already RGB? Can you list the exact processing chain for both of these images?

[Shiandow]

I think the math would either work out the same, or if it does not, YCbCr might be better than RGB. Just a guess, though.


Since NNEDI3 is nonlinear the math doesn't exactly work out to be the same. I think all other algorithms in MadVR (except spline) are linear so in those cases it doesn't really matter which colour space you use (unless you change the gamma).


Hmmmm... I'm not sure what you mean exactly with this. The pre-resize shader should already run in RGB. All of madVR's custom pixel shaders (both pre-resize and post-resize) are always fed with RGB data. So why would you convert to RGB, if the data is already RGB? Can you list the exact processing chain for both of these images?

I fear I've been slightly unclear. This is happened when I put an empty file called YCbCr in the MadVR folder, and used NNEDI3 for luma doubling and Jinc3AR for image upscaling. It seems that MadVR skips the YCbCr -> RGB conversion but doesn't skip the RGB -> YCbCr conversion after the pre-resize pixel shaders. Which means that it's performing NNEDI3 on the wrong colour space which lowers the picture quality.

It seems that this is also the reason I thought NEDI looked superior, and it may be the reason that improving chroma upscaling seemed to slightly improve the luma upscaling in some of those images.

[madshi]

Since NNEDI3 is nonlinear the math doesn't exactly work out to be the same.
Yeah, it won't be identical, but probably in the end it will look similar.

I fear I've been slightly unclear. This is happened when I put an empty file called YCbCr in the MadVR folder, and used NNEDI3 for luma doubling and Jinc3AR for image upscaling. It seems that MadVR skips the YCbCr -> RGB conversion but doesn't skip the RGB -> YCbCr conversion after the pre-resize pixel shaders. Which means that it's performing NNEDI3 on the wrong colour space which lowers the picture quality.
Oooh, I think you're right! Sorry about that. I had implemented the "YCbCr" empty file hack just a long time ago for testing purposes and when implementing the RGB->YCbCr->RGB conversions for NNEDI3 I didn't think of that the "YCbCr" would need special treatment. So it seems NNEDI3 image doubling doesn't work well with the "YCbCr" empty file hack atm. Will put that on my to do list to fix...

[Shiandow]

It seems that the current versions of the ChromaNEDI shaders only use the directional information very weakly, apparently this is still enough to outperform most other algorithms but it causes a loss of detail. I think I've been able to fix this using a few of the same tricks I used in the NEDI shaders (and then some). I'll post these improved versions and a couple of comparisons when I have time, they still require some fine-tuning.

Edit: Later I'll post a more detailed explanation of what I changed but first some comparisons between the new and old method:
(Chroma only)
New method (http://i.imgur.com/IVtA8oc.jpg)
Old method (http://i.imgur.com/9BAfkrE.jpg)
Bicubic (http://i.imgur.com/jOoK1WN.jpg)

With luma (http://i.imgur.com/xS2qGfw.jpg)

One thing I find worrying is that Bicubic seems to be shifted slightly to the left, I've been unable to confirm if this is correct. For my implementation I assumed that the chroma channel corresponded to the average of each 2x2 block.

[pie1394]

Edit: Later I'll post a more detailed explanation of what I changed but first some comparisons between the new and old method:
(Chroma only)
New method (http://i.imgur.com/IVtA8oc.jpg)
Old method (http://i.imgur.com/9BAfkrE.jpg)
Bicubic (http://i.imgur.com/jOoK1WN.jpg)

With luma (http://i.imgur.com/xS2qGfw.jpg)


Personal opinions ...

[High-frequency details]
New method > Bicubic >> Old method

[Low-frequency details]
Bicubic > New method > Old method



This time I use the Sony Fit 11A's 11.6" 1920x1080 to view these samples. This monitor's backlit is QD-LED --- so richer colors than most LCD monitors, and more similar to a good TV set. But it has ridiculous high gamma value by factory default setting. The object / skin-tone colors also look weird (too vivid) if its RGB LUT settings are not adjusted in the display driver.

Anyway it looks impressive with your new method's sample on this monitor. The little dinosaur's head looks more pop/stereo. The girl's hair colors are also distinguishable one by one even on such small-sized FHD panel.

[madshi]

One thing I find worrying is that Bicubic seems to be shifted slightly to the left, I've been unable to confirm if this is correct. For my implementation I assumed that the chroma channel corresponded to the average of each 2x2 block.
All modern compression algorithms (MPEG2, VC1, h264, h265) have Chroma moved half a pixel to the left for 4:2:0 content:

chroma placement image (http://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/Yuvformats420samplingMPEG-2.svg/504px-Yuvformats420samplingMPEG-2.svg.png)

[Shiandow]

All modern compression algorithms (MPEG2, VC1, h264, h265) have Chroma moved half a pixel to the left for 4:2:0 content:

Then it seems that I have made the current version for the wrong "type" of 4:2:0. Although, on the bright side, that means that the image quality can be improved even further.

Edit: Well luckily that was somewhat easy to fix, and the resulting image (http://i.imgur.com/4v2Lr70.jpg) is definitely better.

[leeperry]

Then it seems that I have made the current version for the wrong "type" of 4:2:0
While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win :)

[Shiandow]

Allright, I think the improved chroma shaders are now more or less ready. The new shaders only use a 2nd order interpolation instead of a 4th order, the linear equation that you needed to solve for a 4th order interpolation has the annoying property that it becomes unstable if you get close to an edge, which means that you either have a lot of artefacts or you can only use the directional information very weakly. The new shaders also use a more stable method for solving these equations which also reduces artefacts. I've also added a smarter way of limiting the resulting coefficients, which forces the chroma intensity to stay the same and prevents it from interpolating too strongly in one direction.

I'll add the new shaders to the first post.

By the way, the only purpose of the third shader is to shift the chroma channel half a pixel upwards; it would probably be better to do this simultaneously with scaling.

[huhn]

While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win :)

a h264 stream with: chromaloc 1 should provide all infos needed. and mpeg1 input should work too.

[Shiandow]

While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win :)

A slight adaptation of chromaNEDI-II should do it; it currently shifts things half a pixel vertically, but it could just as well shift them horizontally. Although I would like it better if the shifting was done simultaneously with scaling. Anyway the following code should shift the chroma channels half a pixel to the right. It expects the input to be in YCbCr, and should be used pre-resize. It tries to convert the result to RGB but I can't guarantee that that part will always work; it might be wrong about the colour space.


// $MinimumShaderProfile: ps_3_0
sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual
float2 solve(float2x2 A,float2 b) {
float2 x = 1/2.0;
float2 r = b - mul(A,x);
float2 p = r;
float2 Ar = mul(A,r);
float2 Ap = Ar;
for (int k = 0;k < 3; k++){
float a = min(100,dot(r,Ar)/dot(Ap,Ap));
x = x + a*p;
float2 rk = r; float2 Ark = Ar;
r = r - a*Ap;
Ar = mul(A,r);
float b = dot(r,Ar)/dot(rk,Ark);
p = r + b*p;
Ap = Ar + b*Ap;
}
return x;
}

float4 toRGB(float4 s1) {
s1.yz -= 0.5;
if(width < 1120 && height < 630) return float3(s1.x+1.402*s1.z, dot(s1, float3(1, -.202008/.587, -.419198/.587)), s1.x+1.772*s1.y).rgbb;// SD Y'CbCr to RGB output
return float3(s1.x+1.5748*s1.z, dot(s1, float3(1, -.1674679/.894, -.4185031/.894)), s1.x+1.8556*s1.y).rgbb;// HD Y'CbCr to RGB output
}

#define Col(xy) (tex2D(s0,tex+float2(px,py)*(xy)).yz-0.5)
#define Get(xy) (tex2D(s0,tex+float2(px,py)*(xy)).x+0.25)
#define Get4(xy) (float2(Get(xy+dir[0]),Get(xy+dir[1])))

float4 main(float2 tex : TEXCOORD0) : COLOR {
float4 c0 = tex2D(s0,tex);

float2 dir[2] = {{-1,0},{1,0}};
float2 wind[4] = {{-1,0},{1,0},{0,1},{0,-1}};
float2 pos[2] = {{0,0},{1,0}};

float w = 2;
float2x2 R = w*mul(float2x1(Get4(0)),float1x2(Get4(0)));
float2 r = w*Get(0)*Get4(0);
float4x2 C = {Get4(wind[0]),Get4(wind[1]),Get4(wind[2]),Get4(wind[3])};
float4 y = {Get(wind[0]),Get(wind[1]),Get(wind[2]),Get(wind[3])};
R += mul(transpose(C),C);
r += mul(y,C);

//Normalize coefficients
float n = 16;
R/= n; r /= n;

//Solve equations
float2 a = solve(R+0.000001*I,r);

//Limit the coefiicents of a
float2 b = float2(a[0]+a[1]-1,a[0]-a[1]);
b[0] = clamp(b[0],-.02,.02); //Clamp intensity
b[1] = clamp(b[1],-1,1); //Clamp "directedness"
a = .5 + float2(.5,.5)*b[0] + float2(.5,-.5)*b[1];

c0.a = dot(c0.gb,1);
c0.gb = 0.5+mul(a,float2x2(Col(pos[0]),Col(pos[1])));

return toRGB(c0);
}

[FireFreak111]

Running the chroma shaders in MPC-HC in pre-resize in order, the first two blur the image, and with the third one the image looks blown out and grainy. With the YbCbCr file, basically the same problem. Chroma set to NN.

Clearly not working here.

GTX 660, newest Madvr, nightly MPC-HC, Windows 8.1.1.

From the samples shown however, this looks like a great middle-ground between Chroma Jinc and NNEDI3, which for DVD and lower quality content I just cant combine with Luma NNEDI3 16N with both double and quad enabled on this 660 and i7 870.

Wish NNEDI3 ran on DirectCompute.

[Shiandow]

Could you post a few screenshots of what happens?

[FireFreak111]

Nearest Neighbour
http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot176_zpse8df30f7.png~original

Chrome-NEDI
http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot177_zps105b58a0.png~original

[madshi]

Thanks for your efforts so far, Shiandow, I'll definitely play with your shaders when I find some time!

[Shiandow]

Oh it turns out that I copied the wrong shader for chromaNEDI-III, it should work with the current one. Thanks FireFreak111 for noticing that.

[FireFreak111]

Sorry for the full size images, photobuckets linking is messed when trying to use a full size image.

There is a distinct blur when using these shaders, it doesn't appear to be working correctly.

Nearest Neighbour:
http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot183_zpsbef10c1d.png~original

Chroma-NEDI
http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot182_zpse2f23ff7.png~original

[Shiandow]

Are you sure that the input is in YCbCr? Because it looks like the kind of images you get when the input is RGB.

[FireFreak111]

Thankyou, I had named the YCbCr file wrong. The YCbCr to RGB should be post Chroma-NEDI right? It seems to be working correctly now. :)

[Shiandow]

Yes the YCbCr shader should be after Chroma-NEDI. Glad it's working!

[FireFreak111]

Small thing, your YCbCr to RGB shader is bugged for SD, its only outputting 16:235 (example being a fully black 1st frame is grey). On HD, Blacks is fully functional.

For this SD content, the primaries are apparently SMPTE C (not BT.709, as usual). The shader isn't handling this type.

[Shiandow]

I added a different RGBtoYCbCr shader to the first post that should also work for SD content, but again likely not all. You could also use the RGBtoYCbCr shader I added instead of using the YCbCr hack, this should at least keep the colours the same.

[FireFreak111]

Absolutely perfect, working completely without the hack (which was messing with my gamma for some reason). Thankyou for this shader.

[Shiandow]

Minor update: the shaders chromaNEDI-I and II have been improved a bit by using a slightly larger (more disk shaped) window, the difference is most noticeable when not using chromaNEDI-III (which seems to remove some detail, including aliasing).

[FireFreak111]

Is there a way to view Chroma only, so I can compare different algorithms? Is there a shader I could use?

[Shiandow]

You can just modify the YCbCrtoRGB shader slightly so it will remove the luma channel:


sampler s0;
float2 c0;

float4 main(float2 tex : TEXCOORD0) : COLOR
{
float3 s1 = tex2D(s0, tex).rgb;// original pixel
s1.yz -= 0.5;
s1.r = 0.3; // erase luma
if(c0.x < 1120 && c0.y < 630) return float3(s1.x+1.402*s1.z, dot(s1, float3(1, -.202008/.587, -.419198/.587)), s1.x+1.772*s1.y).rgbb;// SD Y'CbCr to RGB output
return float3(s1.x+1.5748*s1.z, dot(s1, float3(1, -.1674679/.894, -.4185031/.894)), s1.x+1.8556*s1.y).rgbb;// HD Y'CbCr to RGB output
}

[FireFreak111]

This works perfectly except when the Chroma-NEDI shader is in the middle, then everything shows as normal, so I cant compare your shader to others. (Luma is included when using your shader, with just RGB-YCbCr-RGB I see chroma.

Also, alot of grey in this chroma, no blacks, is that normal? First frame of a movie for example is grey, it's black when the new value you added is at 0.0

http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot188_zps371ed53f.png

[Shiandow]

What I usually do when I want to compare is to add a RGBtoYCbCr shader followed by the chroma only shader post-resize, then I can toggle chromaNEDI on/off by enabling disabling the pre-resize shaders. You could also use it instead of the YCbCrtoRGB shader but that makes it harder to compare.

You see a lot of grey because it fixes the luma to 0.25, if you set it to 0 then it's usually too dark to see anything.

[FireFreak111]

Using Frozen for chroma investigation, due to the movie's good chroma compression and heavy use of it, especially in the pink cloak constantly used.

NEDI seems to fare well against NNEDI3, except for some blur on distinct edges (pillars, characters lower cloak, reflection, shoes). Jinc3 (not shown) has the same definition here as NNEDI3, so its distinctly NEDI's problem, not an advantage of NNEDI3.

NEDI

http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot198_zpsaa40bef4.png~original

NNEDI3-16

http://i1292.photobucket.com/albums/b568/FireFreak111/Screenshot199_zps13293d9a.png~original

To be fair, NEDI is 2-3x faster then NNEDI3, entire chain taking only 11.7ms on 1080p24 content (at 1080p48 with no SM).

[Shiandow]

One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp. Without the luma channel it's hard to tell if NEDI is blurring an edge that it shouldn't or NNEDI3 is sharpening an edge that it shouldn't. Both are very well possible. I also found it interesting that NEDI seems to be better at bringing out detail on the pillars itself.

[leeperry]

A slight adaptation of chromaNEDI-II should do it; it currently shifts things half a pixel vertically, but it could just as well shift them horizontally. Although I would like it better if the shifting was done simultaneously with scaling. Anyway the following code should shift the chroma channels half a pixel to the right. It expects the input to be in YCbCr, and should be used pre-resize. It tries to convert the result to RGB but I can't guarantee that that part will always work; it might be wrong about the colour space.
Sweet, thanks! All MPEG1 movies are using BT.601 but fair enough, if there can't a fail-safe way to work it out afterwards I'll just feed mVR with RGB32 for MPEG1. Would need to run test patterns in order to ensure that ffdshow does it properly duh ^^

[FireFreak111]

Chroma.rar (https://mega.co.nz/#!t9MkXAaS!akDh2MHzzdq4bsmHqVpQyjZvIZEM8OFS8fG31q4-ZR4)

Here is some screenshots with Luma shots for NEDI and NNEDI3, Chroma for NEDI, NNEDI3, Jinc3 and Bicubic, compressed into a RAR. This is after disabling the no DXVA copyback option in madVR, which was bluring the Chroma channel (still a blur difference between NEDI and NNEDI3, causing some minor artifacts like the bottom of the character's cloak in the Luma shot)

On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?

[Shiandow]

On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?

The second shader needs the results of the first one so it's not really easy to do it in just one shader. It would be possible to split them in a slightly different way but I think it will obfuscate the code and it will only be faster because I haven't found a way to make the shader skip processing for a particular pixel.

[Shiandow]

With the current discussion around chroma channel alignment I thought it would be useful to add support for different chroma patterns. I've therefore updated the chromaNEDI shaders.

[cyberbeing]

One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp.

Is this why the NEDI chroma shaders don't remove distinct 2x2 block aliasing steps like the others do? The source video in question was a 640x360 x264 1200Kbps stream.

__________

http://i.imgbox.com/RCePQhlS.png
Nearest Neighbor Chroma (400% zoom) (http://i.imgbox.com/RCePQhlS.png)
__________

http://i.imgbox.com/OQgsmgOJ.png
NEDI Chroma (400% zoom) (http://i.imgbox.com/OQgsmgOJ.png)
__________

http://i.imgbox.com/6mazhDjy.png
Catmull-Rom Chroma (400% zoom) (http://i.imgbox.com/6mazhDjy.png)
__________

http://i.imgbox.com/xwmMcj59.png
NNEDI3 64 Chroma (400% zoom) (http://i.imgbox.com/xwmMcj59.png)
__________

madVR's YCbCr hack also seems to result in an elevated purplish black level on SD video with the provided YCbCrtoRGB shader as the last step pre-resize. [Edit: This was caused by using a 3DLUT with the YCbCr hack, which I guess must not be supported]. The above images used madVR in normal mode with both RGBtoYCbCr (first step) and YCbCrtoRGB (last step) shaders to workaround this.

[madshi]

@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.

[cyberbeing]

@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.
Okay, I've just sent a PM with a sample to you and Shiandow.

[Shiandow]

There seems to be a mismatch between what the shader thinks the chroma channels are and what they actually are. As a result some part of the chroma channel isn't scaled. Using the RGBtoYCbCr shader will ensure that the colours remain accurate even if it guesses the colour space incorrectly, but instead it will use the shaders on the wrong chroma channels. If you use the YCbCr hack you should get a smooth(er) result but the colours might be inaccurate.

If you use the chroma only shader I posted a few posts back, you'll see that the "chroma" part looks perfectly smooth. Unfortunately it is the wrong chroma channel.

Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.

[cyberbeing]

Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.

If you think those shaders are incorrect, you could try adapting NVIDIA's sample vertex/pixel shaders for RGB <-> YCbCr conversions which I'd assume should be accurate.

NVIDIA RGBA to YUVA sample (http://developer.download.nvidia.com/shaderlibrary/packages/post_RGB_to_YUV.fx.zip) | NVIDIA YUVA to RGBA sample (http://developer.download.nvidia.com/shaderlibrary/packages/post_RGB_from_YUV.fx.zip)

The actual pixel shader portion of these samples seems to be the following, but would need to be made MPC-HC compatible:
#define QUAD_REAL float
#define QUAD_REAL2 float2
#define QUAD_REAL3 float3
#define QUAD_REAL4 float4

QUAD_REAL4 ToYUV(QuadVertexOutput IN,
uniform sampler2D SceneSampler) : COLOR
{
QUAD_REAL4 rgba = tex2D(SceneSampler, IN.UV);
QUAD_REAL3 ctr = QUAD_REAL3(0,.5,.5);
return QUAD_REAL4(rgb_to_yuv(rgba.xyz)+ctr,rgba.w); // don't lose alpha
}

QUAD_REAL3 rgb_to_yuv(QUAD_REAL3 RGB)
{
QUAD_REAL y = dot(RGB,QUAD_REAL3(0.299,0.587,0.114));
QUAD_REAL u = (RGB.z - y) * 0.565;
QUAD_REAL v = (RGB.x - y) * 0.713;
return QUAD_REAL3(y,u,v);
}

QUAD_REAL4 FromYUV(QuadVertexOutput IN,
uniform sampler2D SceneSampler) : COLOR
{
QUAD_REAL4 yuva = tex2D(SceneSampler, IN.UV) - QUAD_REAL4(0,.5,.5,0);
return QUAD_REAL4(yuv_to_rgb(yuva.xyz),yuva.w); // don't lose alpha
}

QUAD_REAL3 yuv_to_rgb(QUAD_REAL3 YUV)
{
QUAD_REAL u = YUV.y;
QUAD_REAL v = YUV.z;
QUAD_REAL r = YUV.x + 1.403*v;
QUAD_REAL g = YUV.x - 0.344*u - 1.403*v;
QUAD_REAL b = YUV.x + 1.770*u;
return QUAD_REAL3(r,g,b);
}

[Shiandow]

Oddly enough those seem to contain a typo. In the line "QUAD_REAL g = YUV.x - 0.344*u - 1.403*v" the number 1.403 seems to have been copied from the previous line, it should be somewhere around 0.714. Also the coefficients have been rounded to 3 decimal places which is less accurate than what I was using.

As far as I can tell I'm just using the wrong colour space somehow, I suspect it has something to do with the video using different primaries, which MadVR corrects but the YCbCrtoRGB shader does not. I'll try to see if I can find a quick fix, but I'd rather not spend too much time trying to copy the entire MadVR colour processing chain.

[Shiandow]

Well, it doesn't seem to be the primaries, but I can't figure out what does cause the problems. There seems to be something weird with the source levels; the "Y" channel isn't 0 when the source is black. Presumably MadVR does something to fix that but I can't figure out what.

[cyberbeing]

Unsure exactly what actually you're troubleshooting right now, but are you taking into consideration how madVR has a TV-range workflow which preserves BTB & WTW information during conversions, rather than clipping it off?

[Shiandow]

I was trying to see if there was a quick fix such that I could at least make the YCbCrtoRGB shader display the correct colours when you used it with the YCbCr hack on your clip. Since just using the Bt.601 matrices didn't seem to work I figure I might need to convert the primaries. I though I more or less understood how to go from one set of primaries to another, but something goes wrong. One of the things I can't figure out is why changing the primaries in MadVR also changes the black level when you use the YCbCr hack. This may have something to do with the TV-Range workflow you mentioned.

Edit: I did find a quick workaround for the sample you sent. If you set the primaries to EBU/PAL in MadVR then at least the chroma upsampling will work, even if you do not use the YCbCr hack, unfortunately the colours will be slightly incorrect.

[Shiandow]

It took some time but I think I've finally found a way to get improve NEDI enough to make it competitive with NNEDI3. It took a while since all algorithms related to NEDI (including aQua, SAI, and adaptations thereof) seem to suffer from the same flaw: there seems to be no way to get them simultaneously fast, numerically stable, and sharp (especially on vertical/horizontal edges).

So I needed to find a different approach. And I found one, in the article "Image Interpolation by Super-Resolution" by Alexey Lukin, Andrey S. Krylov, and Andrey Nasonov. The approach they took was to treat upscaling as a sort of inverse downscaling. They also show that this works nicely together with NEDI. From the kind of images I was able to create, using this approach, I'm also reasonably sure this was part of the inspiration behind the SmartEdge algorithm that Alexey Lukin showcases on his website (http://audio.rightmark.org/lukin/graphics/resampling.htm).

Anyway here is an example of the images I got using NEDI combined with the "SuperRes" method:

Castle (NEDI + SuperRes) (http://i.imgur.com/2rep1gA.png)
Castle (NNEDI3, 16 neurons) (http://i.imgur.com/mR3H3fi.png)
Castle (NEDI) (http://i.imgur.com/jWxTvsG.png)
Castle (Jinc3AR) (http://i.imgur.com/4qI2MNj.png)

Lighthouse (NEDI+SuperRes) (http://i.imgur.com/1AKVb1Q.png)
Lighthouse (NNEDI3, 16 neurons) (http://i.imgur.com/X1oCQs9.png)
Lighthouse (NEDI) (http://i.imgur.com/midSYiK.png)
Lighthouse (Jinc3AR) (http://i.imgur.com/f5rD8Bl.png)

By the way, I think there's still room for improvement; the super-resolution method is incredibly flexible. Although, I think it will be difficult to find the right parameters by hand.

[madshi]

Could you add a comparison image with your original NEDI algorithm (without SuperRes), so we can easily see the improvement from NEDI to NEDI+SuperRes? And maybe Jinc3 AR as another point of comparison? Thanks!! :)

[Shiandow]

Done. The version of NEDI I used might be marginally better than the one in the first post, I didn't bother to update it since the improvement was only marginal.

The main drawback to using SuperRes seems to be that it introduces some aliasing, this could probably be improved (I think I know what causes it).

[ryrynz]

At a quick glimpse SuperRes is doing much better on the windows on the castle.. If you can take care of that aliasing I think we're on to a winner. I already want to use this for all my upscaling, incredible find there Shiandow.

[foxyshadis]

How is the speed with the new algorithm? Still comparable to plain NEDI, or closer to NNEDI?

[Shiandow]

The speed should be close to NEDI. It still needs to use NEDI so it will be slower, but since SuperRes "refines" the image (removing artefacts etc.) it's possible to take some shortcuts in the NEDI algorithm.

[madshi]

This looks like a quite big improvement over NEDI to me. Less artifacts in the castle image, and it has a generally more "in focus" look. And on a quick check it might show less "fractal like" artifacts in image areas like grass/trees compared to NNEDI3. But, as you say, your current SuperRes algorithm adds quite a bit of aliasing compared to the original NEDI algorithm. This is quite noticeable in the circular hand rail at the top of the lighthouse. If you can fix that we probably have a winner!! :)

Btw, when I tested the original SmartEdge test application, I found that the SuperRes post processing made everything look identical, regardless of whether you started with NEDI or Bicubic. So once you've fixed (if possible) the aliasing problem, it might be worth a try to test Bicubic or Lanczos + SuperRes, just to see how it compares. At least Bicubic/Lanczos + SuperRes might be another option with a good speed/performance ratio, if SuperRes can improve on Bicubic/Lanczos.

[Shiandow]

I think I've lessened the aliasing a bit. It's still not quite as good as NEDI but then again I suspect that NEDI (and NNEDI3) deform the rail in order to improve aliasing. I also seems to have improved sharpness, without actually intending to do so.

Using Lanczos does result in a very similar image but using NEDI seems to improve aliasing a bit. But since Lanczos is faster you can have more iterations of the SuperRes algorithm. I think that in most cases more iterations will be better than less aliasing, but this doesn't seem to apply to the lighthouse image.

New Results: NEDI + SuperRes (4 iterations) (v0.2) (http://i.imgur.com/Dm67pUR.png), Lanczos + SuperRes (6 iterations) (v0.2) (http://i.imgur.com/6cadtZC.png)

[ryrynz]

Using Lanczos does result in a very similar image but using NEDI seems to improve aliasing a bit. But since Lanczos is faster you can have more iterations of the SuperRes algorithm. I think that in most cases more iterations will be better than less aliasing, but this doesn't seem to apply to the lighthouse image.


As you say it's very close.. upon closer inspection you can see NEDI doing some of it's magic on the bottom rail along the top of the lighthouse.
So it looks like a new set of resizing possibilities have opened up for us to compare.. fun.

[madshi]

The lighthouse image is a bit "mean" because that rail already has some ringing around it in the original image which makes it really hard to handle for scaling algorithms. For some reasons NNEDI3 handles this situation especially well. But some other parts of the image actually look better in your latest NEDI + SuperRes image compared to NNEDI3, e.g. the fence. I wonder what happens if you run NNEDI3 + SuperRes? :) Maybe it could be an option to use NNEDI3 with 16 neurons + SuperRes instead of using NNEDI3 with more neurons.

From what I remember, though, the original SmartEdge 2 algorithm didn't have *any* aliasing at image edges. So maybe there's still room for improvement? But the original SmartEdge 2 algo was also slow as hell...

[Shiandow]

At the moment using SuperRes with NNEDI3 gives an identical result to using it with NEDI, it's currently just too aggressive too leave any difference between them intact. But I have noticed that less aggressive versions of SuperRes tend to have less aliasing, so NNEDI3 with a less aggressive version of SuperRes could look quite nice. Anyway I think it's best if I write a short explanation of the algorithm and clean up the code a bit so you can try and see for yourselves what options and trade-offs there are.

[madshi]

Sounds great to me. Just wish I had a bit more free time atm...

[Shiandow]

Oh, whoops, while trying to explain the algorithm I discovered a mistake. I was trying to be clever by combining a few of the steps of the algorithm, but it turns out that this wasn't possible after all. This (http://i.imgur.com/wsIUU7r.png) is what I got after fixing that (and some other changes). Apparently that was (part of) the cause behind the aliasing.

It'll probably take some time before I get the explanation ready, I'd rather not rush it. Especially since there are apparently still parts of the algorithm that aren't entirely correct/clear.

[madshi]

I've often found that when I tried to explain something complicated, that can help seeing things clearer for myself. FWIW, I don't see that much difference between the latest result and the previous "NEDI + SuperRes (4 iterations) (v0.2)" result. Maybe a touch less aliasing. But maybe also a tiny bit less sharpness. NNEDI3 still reproduces the rail a bit better. Anyway, please keep going. I'm really looking forward to try to understand the algorithm once you get around explaining it.

Btw, another great test image is the clown image. Those 3 (castle, lighthouse, clown) are my favorites for testing resampling algorithms. Well, that, and the park meter image to a lesser degree.

[foxyshadis]

The lighthouse image is a bit "mean" because that rail already has some ringing around it in the original image which makes it really hard to handle for scaling algorithms.

Definitely, and it's in enough real-life images and videos that my workflow was generally dehalo->limitedsharpen->NNEDI->sharpen->line thinning. (More aliased input usually meant slightly better output.) Without dehaloing, they'd inevitably look positively glowing at 4x.

Oh, whoops, while trying to explain the algorithm I discovered a mistake. I was trying to be clever by combining a few of the steps of the algorithm, but it turns out that this wasn't possible after all. This (http://i.imgur.com/wsIUU7r.png) is what I got after fixing that (and some other changes). Apparently that was (part of) the cause behind the aliasing.

It'll probably take some time before I get the explanation ready, I'd rather not rush it. Especially since there are apparently still parts of the algorithm that aren't entirely correct/clear.

I find the "broken" version was slightly more pleasant thanks to the extra sharpness. The aliasing had already been reduced enough that further reduction in the fixed didn't quite balance the sharpness loss, but either way, it's a very subtle difference. It might be a worthwhile trade if it speeds things up. Would be interesting to make the sharpness/aliasing tunable, if that's possible.

I can't wait to play with the shader.

[Shiandow]

Okay, so I'll now try to explain how the super resolution method works and how I've implemented it. If you only want to know how to configure it just skip to the end.

The general idea behind the super resolution method that Alexey Lukin et al. explained in their paper is to treat upscaling as inverse downscaling. So the aim is to find a high resolution image which, after downscaling, is equal to the low resolution image.

The problem is that this is usually not well defined. For instance let's take the simplest possible example i.e. an image consisting of just 1 pixel, with value X. And say we want to find an image consisting of 2 pixels with values which we'll denote A and B. Now we need to decide on a downsampling algorithm from the image of 2 pixels to the image of 1 pixels. The obvious choice is to just average the two pixels, i.e. X = (A+B)/2. So if we had the values of A and B then we could find X but we're working backwards so we know the value of X and we want to find the values of A and B. However for any value of A there is a value of B such that (A+B)/2 = X, so there's no way to decide upon a solution. This can be solved by requiring the values of A and B to be close to each other, in which case the unique(!) solution is to make A and B equal X.

The SuperRes algorithm works similar to this except with more pixels, it requires the image to be "regular" (pixel close to each other should have "similar" values) and instead of requiring the result to be exactly equal to the original image after downscaling we simply require it to be "faithful". This method has an enormous amount of flexibility since we can choose which downscaling method to use, how to measure "regularity", and how to measure "faithfulness".

My implementation uses a very simple downscaling method which just averages the pixel values over a disk shaped region (with radius sqrt(2)). For measuring "faithfulness" I just square the difference between the downscaled result and the original. The method of measuring "regularity" is somewhat more complicated, since you only want the pixel values to be close when there is no edge, but not when there is. But it basically consists of looking at all pixels that are close and try to minimize some "distance" between the pixel values, I'll explain how I chose this distance later since it's related to how the algorithm works.

To find an image which is both "regular" and is "faithful" to the original image it is simplest to use the gradient descent method, which basically means that we look if it's better to lower or raise a pixel value and change it accordingly. This is similar to viewing the "regularity" and "faithfulness" as some kind of energy and calculate the resulting forces that act on the pixel values, where "regularity" pulls them closer together and "faithfulness" pulls them closer to the original values.

This brings us to how I've chosen to measure regularity, since instead of defining "regularity" directly and try to calculate the forces I just directly define the forces. The force I've chosen looks like this (http://www.wolframalpha.com/input/?i=plot+2+x+%28+%281+-+b%29%2F%281+%2B+%28a+x%29%5E2%29%5E2+%2B+b+%2F%281+%2B+Abs%5Ba+x%5D%29%29+with+a+%3D+7.5+and+b+%3D+0.25) in this plot "x" is the difference between two adjacent pixel values. From the plot you can see that the force increases rapidly when values that were close move away from each other, but remains fairly constant when the difference was large (since that likely means that there's an edge). The corresponding distance is a bit more complicated but looks like this (http://www.wolframalpha.com/input/?i=plot+-%28%28%28-1+%2B+b%29+x%5E2%29%2F%281+%2B+a%5E2+x%5E2%29%29+%2B+%282+b+%28Sqrt%5Ba+x%5E2%5D+-+Log%5B1+%2B+Sqrt%5Ba+x%5E2%5D%5D%29%29%2Fa+where+a+%3D+7.5%2C+b+%3D+0.25). This distance behaves like the square of the difference close to 0 but behaves more like an absolute value for large differences.

Now that we've defined all of required parts the algorithm consists of the following steps:

Calculate an initial guess
Downscale and calculate differences with original image.
Calculate forces, resulting from "regularity" and "faithfulness".
Apply forces.
Repeat steps 2-4 several times.


The mistake I discovered earlier was that I was trying to combine steps 2 and 3 together, but you can't calculate the forces if you haven't calculated the differences yet. In the end I had to split step 2 in (yet another) shader. This made the algorithm somewhat slower but I think this could be avoided by first doing step 2 and 3 for 1/4 of the pixels and then do step 3 for the other 3/4 of the pixels, this should at least prevent any unnecessary texture calls. Splitting step 2 and 3 also meant that you now need even more shaders to get it all working, which doesn't seem to improve the stability of MadVR (it crashes sometimes when it's using a lot of shaders, usually during start up).

Now there are still some small parts of the algorithm left that I haven't mentioned:

The first is the way I normalise the forces acting on a pixel. I did this by reinterpreting part of the forces as the "weight" and reinterpreting "2x" as the actual force and then use the weighted average instead of just adding them. The idea behind this is that the weight is going to be small when it is close to an edge in which case you want the values to pull together faster to prevent ringing. To make this even effect even more pronounced I actually divide by the square of the total weight. I'm still not 100% sure that this part of the algorithm is actually that beneficial, but changing it would meant that I have to recalibrate it again so I'll just leave it in for now.

The second is the way I store the original values and the differences of the downscaled result with the original. I store these in the alpha channel of the pixels, where for every 2x2 block, the difference is stored in the top-left and the original value is stored in the bottom-right. This results in the lowest possible number of texture calls.

That concludes the description of the algorithm. Which leaves us with the way to use the shaders. Firstly here is a list of parameters and what they do:

strength: total strength of the force. If it's larger the algorithm will converge faster but it might go too far, resulting in artefacts.
softness: relative strength of the regularizing force. Larger values make the resulting image smoother.
radius: effective radius of regularizing force (it uses a Gaussian with that radius as weights). A larger radius should make the image smoother, but only slightly.
acuity: controls the threshold for what is considered an edge. If the difference between two pixel values is larger than 1/acuity then it will assume that there's an edge between them.
baseline: the baseline of the regularizing force, ensures that even pixels across edges don't get too far from each other. Makes edges softer (putting it to 0 makes edges amazingly crisp but also very rough).

If you want to you can also control the downscaling weights (called "weights"), just make sure that you pick the same weights for each of the 3 shaders (SuperRes, SuperRes-pre, SuperRes-inf).

Secondly, you need to put the shaders in the right order. The chain of shaders needed is getting a bit complicated, but in general it looks like this:

NEDI-pre -> Upscale-I -> Upscale-II -> SuperRes-pre -> SuperRes -> { SuperRes-inf -> SuperRes } -> NEDI-pst

The part between brackets can be repeated as many times as you want. I'd recommend using 3 iterations of the SupeRes shader, but perhaps you can use less if you raise the "strength" parameter. You can replace "Upscale" by "fNEDI", "NEDI" or "Lanczos" (I'd recommend using fNEDI) or just skip those shaders and use NNEDI3 for image upscaling. It you're not using NNEDI3 then you should set image upscaling to nearest. You should also make sure that you're resizing 2x (in both directions).

Finally you can download the needed shaders here (https://www.mediafire.com/?22o6ahnchkbzhef). If you want to use them for MPC-BE then just put them in the shader folder, for MPC-HC you need to change the extension to ".hlsl" or add them manually using the shader editor, depending on which version of MPC-HC you have (they recently removed the shader editor, no idea why). For PotPlayer you should change the extension to ".txt" and put them in the shader folder. I'll also change the NEDI shaders in the first post, but I think it no longer makes sense to add all of the code to the first post.

[Shiandow]

I find the "broken" version was slightly more pleasant thanks to the extra sharpness. The aliasing had already been reduced enough that further reduction in the fixed didn't quite balance the sharpness loss, but either way, it's a very subtle difference. It might be a worthwhile trade if it speeds things up. Would be interesting to make the sharpness/aliasing tunable, if that's possible.

Using the "broken" method is only faster because I have very little control over the output size, I think the "correct" method could be made almost as fast.

Ironically the difference in sharpness was caused because I was trying to make the sharpness easier to configure. You can tune it by using the "softness" parameter, but more sharpness does tend to lead to more ringing. It's somewhat harder to tune aliasing, as far as I can tell you can avoid it by either making the edges softer (by raising the "baseline" parameter), or you need to lower the "strength" parameter, which means that you may need more iterations to get a good result.

[madshi]

Sounds like a cool algorithm! Is there some scientific paper about the original algorithm available somewhere? Or where did you get the ideas from?

[Shiandow]

Sounds like a cool algorithm! Is there some scientific paper about the original algorithm available somewhere? Or where did you get the ideas from?

I though I had mentioned it in one of my previous posts but I got the main ideas from the publicly available paper Image interpolation by Super-Resolution (http://www.graphicon.ru/2006/proceedings/papers/we08_94_LukinKrylovNasonov.pdf) by Alexey Lukin, Andrey S. Krylov, Andrey Nasonov. To devise the regularization force and refine the definition of faithfulness I took some ideas from an other paper (http://imaging.cmc.msu.ru/pub/2009.ICIP.Krylov_Lukin_Nasonov.IntEdgePres.en.pdf) (also cowritten by Alexey Lukin) on page two they list the result from using various different regularization methods and definitions of faithfulness. My method is somewhere in between their p=1,n=2,m=1 and p=1,n=2,m=2.

I also took some inspiration from a different paper Bilateral back projection for single image super-resolution (http://www.ece.northwestern.edu/~yingwu/papers/conference/2007/ICME07_Dai_final.pdf), although what I ended up with is quite different from what they described, the main resemblance is the use of bilateral filtering (use interpolation weights depending on the difference in color space, not just in position), but I use it to define "regularity", they use it to define "faithfulness".

[madshi]

Hmmm... Thanks, I'll have to give those papers a read. I've just tested the SmartEdge 2 demo tool. It seems that they've found a way to tame NEDI even more. Their NEDI first pass (before running the Super-Res passes) looks very clean, almost without any directional/fractal artifacts. Not sure how they did that. It does have pretty strong ringing, though (which is later removed/reduced by the post-processing). That's weird because NEDI normally doesn't ring. Makes me wonder...

[Shiandow]

On this page (http://audio.rightmark.org/lukin/graphics/resampling.htm) they mention "12 taps" NEDI, they also mention that they implemented some improvements by "G. d. Haan" which makes me think that they are using the version described in this paper (http://www.ics.ele.tue.nl/~dehaan/pdf/88_SPIE_IVCP2003.pdf). Unfortunately most of their changes are just too expensive to be implemented on a shader. They increase the window size and also increase the interpolation order which means that you have to invert an 8x8 matrix. Given that it is nearly impossible to even multiply by an 8x8 matrix I don't think that this is feasible on a simple shader, maybe it could be implemented in OpenCL but I doubt that it will be faster than just using NNEDI3.

Edit: on an unrelated note, I'm investigating a way of shuffling the pixels around which should make it possible to skip some calculations (in a way that actually improves performance). I'll report if this had any success later.

Followup: It didn't work, for some reason you can't branch when one side of the if statement contains texture calls. Or at least I couldn't find a way to do so.

[XRyche]

unneeded

[Hyllian]

I've successfully managed to port your NEDI shader to Retroarch shader specs. (Here (https://github.com/libretro/common-shaders/tree/master/nedi))

Congrats for this great filter. It works well with retro games. Some screenshots (http://imgur.com/a/YqL8X).

[XRyche]

I've successfully managed to port your NEDI shader to Retroarch shader specs. (Here (https://github.com/libretro/common-shaders/tree/master/nedi))

Congrats for this great filter. It works well with retro games. Some screenshots (http://imgur.com/a/YqL8X).

Have you thought of collaboration with maybe........guest.r of the Epsxe forums or maybe Asmodean of the PCSX2 forums to make a multi-psuedo pass shader for Pete's OpenGL2 gpu plugin for PSEmu based emulators. Asmodean did a whole psuedo multi-pass shader suite for the PCSX2 which he ported to use with Pete's OpenGL2 plugin.