Log in

View Full Version : Shader implementation of the NEDI algorithm


Pages : [1] 2 3

Shiandow
4th June 2014, 16:13
This post was written after I succeeded in implementing the NEDI algorithm (the one by Xin Li et al. not the nnedi3 algorithm by tritical) using shaders. I've since tried to write upscaling algorithms using variations on NEDI and other methods. The results of which can be found below.

NEDI:
For a short explanation of the NEDI algorithm see: http://chiranjivi.tripod.com/EDITut.html

NEDI's picture quality tends to be better than that of linear scaling algorithms (like Lanczos, Jinc, etc.) and in some cases beats nnedi3. It's especially good at scaling the image without aliasing. Development of this method has more or less finished, or otherwise superseded by SuperRes which uses NEDI as part of it's process.

Here is a quick comparison (these results are no longer up to date):
Nearest (http://imgur.com/BCEfaHn.jpg)
Jinc3 (http://i.imgur.com/JP3R1eZ.jpg)
nnedi3 (32 neurons) (http://i.imgur.com/gnG1WQN.jpg)
NEDI (http://i.imgur.com/NtovKwF.jpg)

The easiest way to try NEDI is to use MPDN (http://forum.doom9.org/showthread.php?t=171120)'s render script capabilities. For details on how to use renderscripts see here (http://forum.doom9.org/showthread.php?t=171120).

To use the NEDI shaders for 2x upscaling with MadVR you should set MadVR to output YCbCr, by adding an empty file called "YCbCr" in the MadVR folder (or use the RGBtoYCbCr shader), and set MadVR to use a Nearest filter for luma upscaling (using NNEDI3 will also work but is obviously slower, using any other algorithm will give incorrect results). Also make sure that you're resizing the video exactly 2x, if this doesn't fit your screen you can usually force the video player to scale 2x anyway. You should then use the NEDI-I and NEDI-II shaders (in that order) post resize, and then you need to convert the result back to RGB. For Rec. 709 media this can be done by using the YCbCrtoRGB shader, unfortunately it probably won't work for all video types.

ChromaNEDI:
ChromaNEDI is a way of using NEDI to upscale chroma using information from the luma channels.

This project has largely been abandoned after I found out that this method causes a lot of chroma bleeding. I have been able to solve this, partially, by performing the scaling in linear light but this makes the NEDI artefacts (too) visible.

Anway, the chromaNEDI shaders can be used for chroma upscaling, currently this only works for 4:2:0 subsampled video but that seems to be 99.9% of all video. The way to use these shaders with MadVR is similar to how you use the NEDI shaders but should be used pre resize instead of post resize and you should set chroma upscaling to Nearest instead of luma upscaling. It consists of three different shaders (chromaNEDI-I up to chromaNEDI-III), which must all be used in order. The first two shaders work similarly to the two passes of the NEDI algorithm, the third one is necessary to align the chroma channels with the luma channel.

The chromaNEDI shaders also have support for several different chroma patterns, you can switch between these by changing the line "#define pattern x" where x should be 1,2,3 depending on which pattern you want. Make sure that you pick the same pattern for all shaders.

Short explanation:
pattern 1 (https://upload.wikimedia.org/wikipedia/commons/thumb/5/5d/Yuvformats420samplingMPEG-2.svg/500px-Yuvformats420samplingMPEG-2.svg.png): this is most common for modern codecs.
pattern 2 (https://upload.wikimedia.org/wikipedia/commons/thumb/1/11/Yuvformats420sampling.svg/500px-Yuvformats420sampling.svg.png): used by mpeg-2, seems to be common for older codecs.
pattern 3 (https://upload.wikimedia.org/wikipedia/commons/thumb/3/3e/Yuvformats422sampling.svg/500px-Yuvformats422sampling.svg.png): not used much but is useful for chroma-doubling (shaders should be used post-resize in that case and luma upscaling should be put to nearest. Only works for 2x resizing). You can skip the third shader when using this pattern.

SuperRes:
The SuperRes shaders use a different scaling method which can be used in combination with NEDI (or any other scaling algorithm). This method is explained in detail here (http://forum.doom9.org/showthread.php?p=1685124#post1685124). This method seems to give better results than just using NEDI, and rival those of NNEDI3. These are now also available as an MPDN renderscript.

SuperChromaRes:
With techniques similar to those of SuperRes it's also possible to do chroma scaling. One major advantage is that this makes it possible to do chroma scaling in linear light, which would normally be impossible. This can improve image quality greatly for images consisting of saturated colours (especially red) on a white background. This is also available as an MPDN renderscript, but I've also decided to make the original experimental shaders available to make it possible to try it out with other renderers. Be warned that support for these experimental shaders will be minimal, I will not be backporting all the improvements made in the renderscript, nor will I explain all the options, they also have some of the same issues as ChromaNEDI but will generally work well for HD sources.

Downloads:

SuperRes shader pack (includes NEDI) (http://www.mediafire.com/download/22o6ahnchkbzhef/Shaders.rar)
ChromaNEDI shader pack (Includes RGB <-> YCbCr conversion shaders) (http://www.mediafire.com/download/86bo6bl66cnwv2j/chromaNEDI.rar)
Experimental SuperChromaRes shaders (Includes a short manual) (http://www.mediafire.com/download/1fnutv48bb3k71k/SuperChromaRes.rar)


More information on using MPDN and renderscripts:
http://forum.doom9.org/showthread.php?t=171120

Code of the NEDI shaders:
NEDI-I:

sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define offset 0.5
#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy)))//-float4(0,0.5,0.5,0))
#define Get(xy) (Value(xy)[0]+offset)
#define Get4(xy) (float2(Get(xy+2*dir[0])+Get(xy+2*dir[1]),Get(xy+2*dir[2])+Get(xy+2*dir[3])))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual
float2 solve(float2x2 A,float2 b) {
float2 x = 1/4.0;
float2 r = b - mul(A,x);
float2 p = r;
float2 Ar = mul(A,r);
float2 Ap = Ar;
for (int k = 0;k < 2; k++){
float a = min(100,dot(r,Ar)/dot(Ap,Ap));
x = x + a*p;
float2 rk = r; float2 Ark = Ar;
r = r - a*Ap;
Ar = mul(A,r);
float b = dot(r,Ar)/dot(rk,Ark);
p = r + b*p;
Ap = Ar + b*Ap;
}
return x;
}

//Cramer's method
float2 solvex(float2x2 A,float2 b) { return float2(determinant(float2x2(b,A[1])),determinant(float2x2(A[0],b)))/determinant(A); }

float4 main(float2 tex : TEXCOORD0) : COLOR {
float4 c0 = tex2D(s0,tex);

//Skip pixels on wrong grid
if ((frac(tex.x*width/2.0)<0.5)||(frac(tex.y*height/2.0)<0.5)) return c0;

//Define window and directions
float2 dir[4] = {{-1,-1},{1,1},{-1,1},{1,-1}};
float4x2 wind[4] = {{{-1,-1},{-1,1},{1,-1},{1,1}},{{-3,-1},{-1,3},{1,-3},{3,1}},{{-1,-3},{-3,1},{3,-1},{1,3}},{{-3,-3},{-3,3},{3,-3},{3,3}}};

//Initialization
float2x2 R = 0;
float2 r = 0;
float4 d = 0;

//Define weights
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

//Calculate (local) autocorrelation coefficients
for (int k = 0; k<4; k+= 1){
float4 y = float4(Get(wind[k][0]),Get(wind[k][1]),Get(wind[k][2]),Get(wind[k][3]));
float4x2 C = float4x2(Get4(wind[k][0]),Get4(wind[k][1]),Get4(wind[k][2]),Get4(wind[k][3]));
R += w[k]*mul(transpose(C),C);
r += w[k]*mul(y,C);
d += lancz[k]*(Value(wind[k][0])+Value(wind[k][1])+Value(wind[k][2])+Value(wind[k][3]));
}

//Normalize
float n = 24;
R /= n; r /= n;

//Calculate a = R^-1 . r
float e = 0.005;
float2 a = solve(R+e*e*I,r+e*e/2.0);

//Nomalize 'a' (prevents overshoot)
a = .25 + float2(.5,-.5)*clamp(a[0]-a[1],-1,1);

//Calculate result
float2x4 x = float2x4(Value(dir[0])+Value(dir[1]),Value(dir[2])+Value(dir[3]));
float4 c = mul(float1x2(a),x);

//Fallback to lanczos
float t = saturate(1-500*sqr(x[0]-x[1]));
c += t*(d-mul(float1x2(1,1)/4.0,x));

return c;//+float4(0,0.5,0.5,0);
}


NEDI-II:

sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define offset 0.5
#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy)))//-float4(0,0.5,0.5,0))
#define Get(xy) (Value(xy)[0]+offset)
#define Get4(xy) (float2(Get(xy+2*dir[0])+Get(xy+2*dir[1]),Get(xy+2*dir[2])+Get(xy+2*dir[3])))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual
float2 solve(float2x2 A,float2 b) {
float2 x = 1/4.0;
float2 r = b - mul(A,x);
float2 p = r;
float2 Ar = mul(A,r);
float2 Ap = Ar;
for (int k = 0;k < 2; k++){
float a = min(100,dot(r,Ar)/dot(Ap,Ap));
x = x + a*p;
float2 rk = r; float2 Ark = Ar;
r = r - a*Ap;
Ar = mul(A,r);
float b = dot(r,Ar)/dot(rk,Ark);
p = r + b*p;
Ap = Ar + b*Ap;
}
return x;
}

//Cramer's method
float2 solvex(float2x2 A,float2 b) { return float2(determinant(float2x2(b,A[1])),determinant(float2x2(A[0],b)))/determinant(A); }

float4 main(float2 tex : TEXCOORD0) : COLOR {
float4 c0 = tex2D(s0,tex);

//Skip pixels on wrong grid
if ((frac(tex.x*width/2.0)<0.5)&&(frac(tex.y*height/2.0)<0.5)) return c0;
if ((frac(tex.x*width/2.0)>0.5)&&(frac(tex.y*height/2.0)>0.5)) return c0;

//Define window and directions
float2 dir[4] = {{-1,0},{1,0},{0,1},{0,-1}};
float4x2 wind[4] = {{{-1,0},{1,0},{0,1},{0,-1}},{{-1,2},{1,-2},{2,1},{-2,-1}},{{-1,-2},{1,2},{-2,1},{2,-1}},{{-3,0},{3,0},{0,3},{0,-3}}};

//Initialization
float2x2 R = 0;
float2 r = 0;
float4 d = 0;

//Define weights
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

//Calculate (local) autocorrelation coefficients
for (int k = 0; k<4; k+= 1){
float4 y = float4(Get(wind[k][0]),Get(wind[k][1]),Get(wind[k][2]),Get(wind[k][3]));
float4x2 C = float4x2(Get4(wind[k][0]),Get4(wind[k][1]),Get4(wind[k][2]),Get4(wind[k][3]));
R += w[k]*mul(transpose(C),C);
r += w[k]*mul(y,C);
d += lancz[k]*(Value(wind[k][0])+Value(wind[k][1])+Value(wind[k][2])+Value(wind[k][3]));
}

//Normalize
float n = 24;
R /= n; r /= n;

//Calculate a = R^-1 . r
float e = 0.005;
float2 a = solve(R+e*e*I,r+e*e/2.0);

//Nomalize 'a' (prevents overshoot)
a = .25 + float2(.5,-.5)*clamp(a[0]-a[1],-1,1);

//Calculate result
float2x4 x = float2x4(Value(dir[0])+Value(dir[1]),Value(dir[2])+Value(dir[3]));
float4 c = mul(float1x2(a),x);

//Fallback to lanczos
float t = saturate(1-500*sqr(x[0]-x[1]));
c += t*(d-mul(float1x2(1,1)/4.0,x));

return c;//+float4(0,0.5,0.5,0);
}

TheElix
4th June 2014, 21:05
Funny you should say that. I've just succeeded in implementing the NEDI algorithm (the one by Xin Li et al. not the nnedi3 algorithm MadVR uses) in a shader. This made it run at a speed similar to, if not faster than, Jinc3. Without too large a loss in quality compared to nnedi3.

Here it is compared to some other algorithms:
Nearest (http://imgur.com/BCEfaHn.jpg)
Jinc3 (http://i.imgur.com/JP3R1eZ.jpg)
nnedi3 (32 neurons) (http://i.imgur.com/gnG1WQN.jpg)
NEDI (http://i.imgur.com/NtovKwF.jpg)

I was hoping that this might be useful as a faster alternative to nnedi3.This is interesting, I'd like to give it a try. Can you share a link to your thread?

ryrynz
4th June 2014, 23:21
I was hoping that this might be useful as a faster alternative to nnedi3.

Wow.

NEDI destroys Jinc giving a much more natural presentation. Both Jinc and nnedi3 are very artificial looking in comparison and it's the same speed as Jinc? Could it possibly be even faster if it was coded within MadVR?

I actually prefer the look of NEDI here to MadVR's nnedi3 too, It's just too strong (I would like to make better comparisons using a full screen images though)

Motenai Yoda
4th June 2014, 23:58
I still prefer nnedi3

Nevilne
5th June 2014, 00:27
Looks quite beneficial for madvr.
Was there ever a nedi plugin for avisynth by the way?

Shiandow
5th June 2014, 00:38
EDIUpsizer and FastEDIUpsizer by tritical are apparently based on the NEDI algorithm. This later developed into nnedi3 but for some reason the NEDI algorithm was abandoned at some point.

ryrynz
5th June 2014, 00:43
Faster than Jinc? I enabled all four of these in the right order and it's delivering a few frames per second, not sure why as both CPU and GPU usage are fairly low. i5 3570K, HD 4600, windows 7.

Is it possible this is only any good on Nvidia or AMD hardware?

Shiandow
5th June 2014, 00:50
Well, it does need a lot of texture calls, then again so does Jinc. What program did you use?

Edit: You can also try making the window smaller by changing the line:
float4 lancz = {0.328511,-0.0365013,-0.0365013,0.0040557};
lancz /= dot(lancz,4);
float4 w = {1,1,1,0};

to
float4 lancz = {0.328511,-0.0365013,0,0};
lancz /= dot(lancz,4);
float4 w = {1,1,0,0};

or even
float4 lancz = {1,0,0,0};
lancz /= dot(lancz,4);
float4 w = {1,0,0,0};

leeperry
5th June 2014, 01:48
Oh wow, very impressive! Sharp, yet not quite artificial looking like NNEDI3 and this will also make the AMD interop lag a non-issue. :thanks: for sharing, can't wait for madshi to add it to mVR :)

pie1394
5th June 2014, 04:43
Just looking forward to see if madshi has time to integrate your implementation into madVR!

About the resolution enhancement (i.e. sharpness) of your image samples, my eyes feel :

Nearest >> NNEDI3_32 > NEDI > Jinc3 (+AR or not?)

Of course it definitely causes serious jaggy / unstable edges on motion objects with Nearest scaling mode. :p

In fact sometimes I still notice that with Luma NNEDI3_32 mode. Luma NNEDI3_64 mode solves that to give more stable motion like Jinc3AR and even more sharper image, but the cost is somewhat too high for HD contents. It will be great if NEDI allows to preserve better original resolution + same motion edge stability than Jinc3AR while it maintains the similar cost. :D

ps: My TV is Sony 65" HX920 with reality creation option (i.e. super-resolution function) at 20%. But it is still able to tell some difference among different scaling options in madVR. There is no need to say, madVR's Lanczos4_AR, Jinc3_AR, NNEDI3 options have better performance than the ability of TV set's scaler.

StinDaWg
5th June 2014, 05:29
How do you use this in MPC-HC?

burfadel
5th June 2014, 05:31
Yeah it would be good if it were incorporated into MadVR, NNEDI3 just doesn't seem practical for everyday use. It's not that the output of NNEDI3 is bad (although maybe a little overdone as some people have pointed out), it's the efficiency of the actual processing. NNEDI3 requires a fair bit of processing power, which in turn requires a fair bit of electricity. If this NNEDI shader can use 'as little' amount of electricity as Jinc, with the quality close to NNEDI3, then that is a very good thing! This is even if the processing of a pixel twice can't be resolved. If it can be worked out, it would require even less processing power...

madshi
5th June 2014, 08:03
I have implemented a fair number of algorithms in Delphi/Pascal, just to try them out. The original NEDI is one of them, also the Zhao-Xin Li improvement of the original NEDI algorithm and the iNEDI algorithm. I've also tried MEDI, ICBI and several others. In some images NEDI looks great. But there are also a lot of images where it looks totally unusable, IMHO. Which is why I never even considered using it in madVR.

But maybe my NEDI implementation was buggy, I don't know. If you guys want to make sure, try all the test images from the following thread:

http://forum.doom9.org/showthread.php?t=145358&page=4

I think after trying all of them you'll probably agree with me that it's not a suitable algorithm for madVR due to heavy artifacts. If you do still like it, please post all the 4x upscaled NEDI images of all those test images from the linked thread here, and maybe I'll reconsider. In any case, thanks to Shiandow for your efforts!

P.S: Here's the iNEDI paper which showcases some of the NEDI problems:

http://www.tecnick.com/pagefiles/papers/85_Asuni_Giachetti_iNEDI_VISAPP2008.pdf

Please note that the iNEDI authors "replaced" iNEDI later with the ICBI algorithm. And ICBI is worse than Jinc AR, IMHO. Basically the NEDI algorithm and all its improvements produce directional artifacts and a "fractal like" look. On the positive side, they can be quite sharp. All the newer algorithms which are trying to reduce the NEDI artifacts, are much much slower and generally make the image look less like NEDI and get nearer to Jinc/NNEDI3. Please also note that NNEDI3 while having a similar name is a *completely* different algorithm compared to NEDI. IMHO, NNEDI3 is so much better than NEDI etc that it's not even funny. At least when comparing a lot of different images. The key reason is that NNEDI3 only has very few artifacts. NEDI is *FULL* of artifacts.

madshi
5th June 2014, 08:20
P.S: Also look here:

http://www.general-cathexis.com/manual2/#anchor8

Scroll down a bit until you get the table with "Artifact Avoidance, Relative Times, and Visual Comparisons". In that table "Xin Li" is the original NEDI algorithm. I hope you'll agree with me that with this test image, NEDI produces an extremely artificial looking image. Looks like watercolor to me, or some other sort of "art" distortion.

burfadel
5th June 2014, 11:42
Madshi, are you talking about NEDI or NNEDI? They're not the same thing, apparently!

vivan
5th June 2014, 11:53
Of course he knows what he is talking about...

Shiandow
5th June 2014, 13:22
P.S: Also look here:

http://www.general-cathexis.com/manual2/#anchor8

Scroll down a bit until you get the table with "Artifact Avoidance, Relative Times, and Visual Comparisons". In that table "Xin Li" is the original NEDI algorithm. I hope you'll agree with me that with this test image, NEDI produces an extremely artificial looking image. Looks like watercolor to me, or some other sort of "art" distortion.

Indeed NEDI does look horrible on that image. However the shader I created seems to avoid quite a lot of the artefacts (but not all). To make the shader more stable I've had to add a slight preference for 'simpler' solutions which also prevents it from overfitting.

Anyway I've tested it on the images from this page (http://forum.doom9.org/showthread.php?t=170661&highlight=nnedi3), I think it performs reasonably well, although it might be better to fall back to lanczos somewhat more aggressively to avoid some of the artefacts.

Car show. (http://i.imgur.com/2R6Hj2W.jpg)

Castle. (http://i.imgur.com/sKd4IcU.jpg)

Cat. (http://i.imgur.com/3cQWFYO.png)

Clown. (http://i.imgur.com/alZSwlQ.jpg)

Flowers. (http://i.imgur.com/LSOXDaJ.jpg)

Lighthouse. (http://i.imgur.com/jefxJCK.jpg)

Meter. (http://i.imgur.com/QhNPKFr.jpg)

"Wc". (http://i.imgur.com/Z4Br7aW.jpg)

Pixel art. (http://i.imgur.com/qMdhB2p.jpg)

Procrastinating
5th June 2014, 13:43
Some of those tests still feel a little "oil painty" for me. If you can deliver any consistent results over jinc/lanczos however, I don't see any reason why it shouldn't be incorporated into madVR. Many people seem to have hardware that is only just, or just under the required performance for NNEDI3 16. It seems like it could at least make NNEDI3 16 redundant.

madshi
5th June 2014, 14:49
Ok, it seems Shiandow's modifications do reduce the artifacts quite nicely, compared to the original NEDI algorithm, but I still don't consider it good enough for my taste yet. E.g. compare the Castle image:

NEDI (http://i.imgur.com/sKd4IcU.jpg)
NNEDI3 (http://madshi.net/castleNNEDI3.png)

It's not even in the same class. NNEDI3 is like 10 times better with this image. NNEDI3 is sharper, more focused, has less ringing and much less directional/weird artifacts. I think I would also very much prefer Jinc3 AR over NEDI - with this image at least.

The problem is this: If you just want to upscale one specific photograph/image, you can play with different algorithms and pick one which looks best for just that image. But madVR is about real time video playback with all kinds of different videos/scenes/images. So I need algorithms which always looks at least decent. I can't use algorithms which look great on some specific images but look horrible on some other images. I don't think NEDI is good enough for general purpose madVR use due to the heavy directional artifacts in many images. Of course that's only my personal opinion. And it only applies to the current version of Shiandow's shader...

Shiandow
5th June 2014, 17:04
Well, there's not much I can do about the getting a sharper result, the way NEDI works just forces some lines to get blurred (since there's no way to tell on which side of the edge you are, at least not by looking at a small part of the image). I might be able to prevent some of the artefacts, but likely not all. And it's probably possible to prevent some ringing by falling back to bicubic instead of lanczos.

I would like to mention that most artefacts only become problematic when you use NEDI twice, when you only use NEDI once they aren't that much of a problem.

turbojet
6th June 2014, 00:27
Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

Shiandow
6th June 2014, 01:47
Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

They should be post resize, it basically throws away 3/4 of the pixels and then tries to interpolate them. To simulate actually upscaling an image you need to tell MadVr to either use nearest or nnedi3, in both cases it will keep the pixels of the original image and try to interpolate the rest.

You could probably avoid using the "pre" and "post" shaders. You can get a similar result by changing:

#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy))[0])

to

#define Value(xy) (dot(tex2D(s0,tex+float2(px,py)*(xy)).rgb,1/3.0))

and

return float4(c,c0.gba);

to

return c0-Value(0)+c;

turbojet
6th June 2014, 06:01
Thanks and both 1 and 2 should be enabled at the same time? They look pretty similar, are these passes?

Shiandow
6th June 2014, 09:10
Yes, it will basically fill in the pixels in the following pattern:

o . o . o
. x . x .
o . o . o
. x . x .
o . o . o
Where the "o" are known pixels, the "x" are calculated from the surrounding "o" by NEDI-1 and then NEDI-2 will fill in the remaining "."s.

madshi
6th June 2014, 13:30
One thing this might be good for is chroma upsampling. Instead of using the neighbor pixels to create the interpolation weights, the algorithm could use the luma channel (or both the luma and chroma channels). This way the luma channel would guide the chroma upsampling. This makes a lot of sense because in most cases both luma and chroma change at the same time, and the luma channel already has 4 times the resolution. Of course the algorithm would have to do some safety checks to make sure that nothing bad happens if luma and chroma channels happen to be not related in some rare situations. But I think this could be a *really* good algorithm. I've had this idea for a long time, but never actually got around trying/implementing it yet. @Shiandow, maybe you'd have fun trying your luck with that? FWIW, one big problem with my idea is that sometimes the chroma channel was created with incorrect filtering (e.g. nearest neighbor) or sometimes the chroma channel is offset slightly. One additional thing a good chroma upsampling algorithm could do is to downscale the luma channel to chroma resolution and then check whether the chroma channel likely has a wrong offset or not.

(I had asked tritical at one point whether he'd consider making a special NNEDI3 version for chroma upsampling which takes the luma channel into account, but he was busy with other things, sadly.)

leeperry
6th June 2014, 20:21
please post all the 4x upscaled NEDI images of all those test images from the linked thread here, and maybe I'll reconsider.
I would like to mention that most artefacts only become problematic when you use NEDI twice, when you only use NEDI once they aren't that much of a problem.
IMHO, put your resources into luma doubling (maybe a little into chroma upscaling).
Guys, I might be missing something here but the kangaroo test pattern looks fantastic with NEDI and as madshi doesn't really advise using quad from what I understand, then when why comparing 4x upscales at all? We could use NEDI for luma doubling and either NNEDI3/J3AR or that new NEDI idea from madshi for chroma?

Most of the time, I personally find NNEDI3 too sharp for chroma and if we could get off the hiccupy OpenCL train then far more GPU's could be useful to mVR than just the GCN and Maxwell architectures.

madshi
6th June 2014, 20:27
Maybe the artifacts are worse with 4x upscales, but they're still there with 2x upscaling, too. IMHO NEDI in its current form is not suitable for a general purpose video upscaling algorithm, as I said before. Of course that's only my personal opinion, but unless I see evidence that suggests that I'm wrong, I won't add NEDI image/luma upscaling to madVR.

Shiandow
6th June 2014, 22:37
One thing this might be good for is chroma upsampling.

Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling. (http://i.imgur.com/7GZif44.png)
Jinc3AR chroma doubling. (http://i.imgur.com/V7YQkF8.png)

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.

Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.

pie1394
7th June 2014, 01:30
Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling. (http://i.imgur.com/7GZif44.png)
Jinc3AR chroma doubling. (http://i.imgur.com/V7YQkF8.png)

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.


To me the above sample's NEDI version has better edge smoothness, but at the cost of losing some higher freqeuency details (i.e. too soft) than the Jinc3AR version. It can be even noticed on the text body and the tree's texture at a normal viewing distance with the so so Dell U2412M. I guess the difference will be even higher on a TV with super-resolution engine's high-frequency signal restoration enhancement.

Shiandow
7th June 2014, 01:44
Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

pie1394
7th June 2014, 03:19
Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

The version's sharpness on the tree's texture indeed improves, but it is still somwhat softer than Jinc3AR verison. Yet it creates more noticeable ring effects on the leaf's edge and "T" and "P" characters. :p

Here is my subjective visual experience opinions:

[High-frequency signal preservation]
NEDI v2 > Jinc3AR >> NEDI v1

[Low-frequency signal preservation]
Jinc3AR > NEDI v2 >> NEDI v1

[Angled Object edge smoothness]
NEDI v1 >= NEDI v2 > Jinc3AR

[Ring-effect artifact issue]
NEDI v2 > Jinc3 AR >> NEDI v1

[Shaded? Text clarity]
Jinc3AR > NEDI v1 > NEDI v2

madshi
7th June 2014, 07:56
Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.
Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

FWIW, here are two samples which are good candidates for chroma tests:

red fonts (http://madshi.net/redfonts.mkv)
chroma test (http://madshi.net/chromatest.mkv)

Btw, if you create an empty file named "YCbCr" in the madVR folder, madVR will output YCbCr instead of RGB. This might allow you to test better because color conversion to RGB is simply skipped completely.

Bloax
7th June 2014, 21:35
madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?

Shiandow
7th June 2014, 21:47
Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma? Otherwise I would have to disagree.

Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR (http://i.imgur.com/TfkNr0h.jpg)
chromaNEDI (http://i.imgur.com/zvRy3il.jpg)

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI (http://i.imgur.com/OpLoF3F.jpg)

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.

nevcairiel
7th June 2014, 22:54
Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma?

Thats what "chroma upscaling" always refers to in the madVR context, so thats safe to assume.

Shiandow
7th June 2014, 23:03
Ok thanks, I wasn't really sure since chroma upscaling is done by doubling the resolution.

madshi
7th June 2014, 23:20
Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR (http://i.imgur.com/TfkNr0h.jpg)
chromaNEDI (http://i.imgur.com/zvRy3il.jpg)
Looks nice!

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI (http://i.imgur.com/OpLoF3F.jpg)
The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.
I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.

May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

madshi
7th June 2014, 23:21
madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?
I don't see that on my PC. If I type in "http://madshi.net" the browser seems to show that without any redirection. At least no redirection that I can see?

Bloax
7th June 2014, 23:50
Apparently it's just my AV pulling dirty, dirty tricks because it thinks your site is suspicious - carry on.

Shiandow
8th June 2014, 00:45
The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?


I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.


I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.


What I find somewhat frustrating is that, so far, NEDI has looked good on all video content I've tried. But I'll admit that it has some quirks, I'll try to see if I can iron these out.


May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

Feel free to use them. I'd appreciate it if you'd keep me updated on possible improvements you've made.

Here are some possible improvements that I couldn't implement because I was restricted to using a simple shader:

You should be able to increase the speed by about 1.7 (1.75/3) times just by preventing it from unnecessarily calculating pixels that weren't even supposed to change.
You could calculate some of the values upfront, preventing some texture calls. This will probably work best for the values of "Get4".
As I mentioned before you could probably detect some unwanted behaviour by comparing the original chroma values with a downscaled version of the upscaled chroma.

pie1394
8th June 2014, 04:54
I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.


To me the softer result by your NEDI looks like a female with cosmetics on the face -- if compared to BiCubic75AR version. :D

Yet I don't know if the original contents should look exactly like this, or other sharper algorithms add more artifical enhancements to make it look more pop / stereo...

Shiandow
8th June 2014, 08:29
I've discovered that the ChromaNEDI-III shader didn't actually use any edge information so it has been replaced by one that does.

Edit: Quick before (http://i.imgur.com/zvRy3il.jpg)/after (http://i.imgur.com/e6FZaI6.jpg).

madshi
8th June 2014, 08:46
I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.
FWIW, if you downscale a sharp image 50% and then upscale it again with NNEDI3, the result is soft compared to the original image.

What I find somewhat frustrating is that, so far, NEDI has looked good on all video content I've tried. But I'll admit that it has some quirks, I'll try to see if I can iron these out.
Ok, let me know when you're finished with your optimizations, then I'll give it a try with some videos.

I've discovered that the ChromaNEDI-III shader didn't actually use any edge information so it has been replaced by one that does.
Does that improve image quality visibly?

Shiandow
8th June 2014, 09:02
Does that improve image quality visibly?

I've added a quick example to my previous post. It seems to have removed some ringing and aliasing.

The previous shader wasn't capable of detecting the direction of an edge so it was basically the same as a box filter except it multiplied the result by some value which depended on whether there was an edge or not. This did actually prevent some 'bleeding' but it was unstable and caused some artefacts. The current shader can detect the direction of an edge and tries to interpolate along that direction.

Shiandow
8th June 2014, 11:16
On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr (http://i.imgur.com/e6FZaI6.jpg) / resized from RGB (http://i.imgur.com/xMztLRj.jpg).

madshi
8th June 2014, 12:23
You mean when luma doubling is enabled and chroma doubling is disabled?

Shiandow
8th June 2014, 12:40
Yes, I've configured it to use NNEDI3 for luma and Jinc3AR for chroma, except it seems treat the YCbCr image as if it were RGB which causes it to apply NNEDI3 on some other part of the colour space.

madshi
8th June 2014, 14:12
madVR first converts everything to RGB. Then if you enable luma doubling but disable chroma doubling, the image is converted to YCbCr just before scaling is done (always using BT.709 for that, IIRC), then NNEDI3 is performed on the Y channel, only, then chroma is upscaled with a different algorithm, then everything is converted back to RGB right after scaling. madVR runs custom pixel shaders always in RGB.

huhn
8th June 2014, 15:42
madVR first converts everything to RGB. Then if you enable luma doubling but disable chroma doubling, the image is converted to YCbCr just before scaling is done (always using BT.709 for that, IIRC), then NNEDI3 is performed on the Y channel, only, then chroma is upscaled with a different algorithm, then everything is converted back to RGB right after scaling. madVR runs custom pixel shaders always in RGB.

have you think about using YCoCg for this not YCbCr?
it can be used lossless for rgb and it's super fast too. it needs more bits than the source has and i don't have a clue how to use it with 16 bit. the Y channel doesn't change the bit deep in the 10 bit rgb example:

http://wiki.multimedia.cx/index.php?title=YCoCg

nevcairiel
8th June 2014, 15:46
have you think about using YCoCg for this not YCbCr?
it can be used lossless for rgb and it's super fast too. it needs more bits than the source has and i don't have a clue how to use it with 16 bit. the Y channel doesn't change the bit deep in the 10 bit rgb example:

http://wiki.multimedia.cx/index.php?title=YCoCg

If you're doing this in floating point back and forth, there is no loss in doing this in YCbCr. Its just math.