Shader implementation of the NEDI algorithm

[turbojet]

Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

[Shiandow]

Originally Posted by turbojet

Are these shaders recommend as all pre-resize? I'm getting some pretty nasty artifacts at times if that's the case.

Can these be consolidated into less shaders?

They should be post resize, it basically throws away 3/4 of the pixels and then tries to interpolate them. To simulate actually upscaling an image you need to tell MadVr to either use nearest or nnedi3, in both cases it will keep the pixels of the original image and try to interpolate the rest.

You could probably avoid using the "pre" and "post" shaders. You can get a similar result by changing:

#define Value(xy) (tex2D(s0,tex+float2(px,py)*(xy))[0])

to

#define Value(xy) (dot(tex2D(s0,tex+float2(px,py)*(xy)).rgb,1/3.0))

and

return float4(c,c0.gba);

to

return c0-Value(0)+c;

[turbojet]

Thanks and both 1 and 2 should be enabled at the same time? They look pretty similar, are these passes?

[Shiandow]

Yes, it will basically fill in the pixels in the following pattern:

o . o . o
. x . x .
o . o . o
. x . x .
o . o . o

Where the "o" are known pixels, the "x" are calculated from the surrounding "o" by NEDI-1 and then NEDI-2 will fill in the remaining "."s.

[madshi]

One thing this might be good for is chroma upsampling. Instead of using the neighbor pixels to create the interpolation weights, the algorithm could use the luma channel (or both the luma and chroma channels). This way the luma channel would guide the chroma upsampling. This makes a lot of sense because in most cases both luma and chroma change at the same time, and the luma channel already has 4 times the resolution. Of course the algorithm would have to do some safety checks to make sure that nothing bad happens if luma and chroma channels happen to be not related in some rare situations. But I think this could be a *really* good algorithm. I've had this idea for a long time, but never actually got around trying/implementing it yet. @Shiandow, maybe you'd have fun trying your luck with that? FWIW, one big problem with my idea is that sometimes the chroma channel was created with incorrect filtering (e.g. nearest neighbor) or sometimes the chroma channel is offset slightly. One additional thing a good chroma upsampling algorithm could do is to downscale the luma channel to chroma resolution and then check whether the chroma channel likely has a wrong offset or not.

(I had asked tritical at one point whether he'd consider making a special NNEDI3 version for chroma upsampling which takes the luma channel into account, but he was busy with other things, sadly.)

[leeperry]

Originally Posted by madshi

please post all the 4x upscaled NEDI images of all those test images from the linked thread here, and maybe I'll reconsider.

Originally Posted by Shiandow

I would like to mention that most artefacts only become problematic when you use NEDI twice, when you only use NEDI once they aren't that much of a problem.

Originally Posted by madshi

IMHO, put your resources into luma doubling (maybe a little into chroma upscaling).

Guys, I might be missing something here but the kangaroo test pattern looks fantastic with NEDI and as madshi doesn't really advise using quad from what I understand, then when why comparing 4x upscales at all? We could use NEDI for luma doubling and either NNEDI3/J3AR or that new NEDI idea from madshi for chroma?

Most of the time, I personally find NNEDI3 too sharp for chroma and if we could get off the hiccupy OpenCL train then far more GPU's could be useful to mVR than just the GCN and Maxwell architectures.

[madshi]

Maybe the artifacts are worse with 4x upscales, but they're still there with 2x upscaling, too. IMHO NEDI in its current form is not suitable for a general purpose video upscaling algorithm, as I said before. Of course that's only my personal opinion, but unless I see evidence that suggests that I'm wrong, I won't add NEDI image/luma upscaling to madVR.

[Shiandow]

Originally Posted by madshi

One thing this might be good for is chroma upsampling.

Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling.
Jinc3AR chroma doubling.

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.

Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.

[pie1394]

Originally Posted by Shiandow

Well, it seems that using NEDI for chroma doubling indeed does work quite well. I did have some trouble with aligning the luma and chroma grid, but I sort of succeeded with the following result:

NEDI chroma doubling.
Jinc3AR chroma doubling.

The NEDI method could probably be made a bit sharper since I'm currently using a simple box filter to shift it 1/2 a pixel, which could be replaced by something better (in fact you could just use NEDI again). I'm also completely ignoring the chroma channel which may backfire on some images, but does seem to produce the best results. Even better, doing so seems to hide some source artifacts when these don't occur in the luma channel.

To me the above sample's NEDI version has better edge smoothness, but at the cost of losing some higher freqeuency details (i.e. too soft) than the Jinc3AR version. It can be even noticed on the text body and the tree's texture at a normal viewing distance with the so so Dell U2412M. I guess the difference will be even higher on a TV with super-resolution engine's high-frequency signal restoration enhancement.

[Shiandow]

Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

[pie1394]

Originally Posted by Shiandow

Using NEDI to shift the image, instead of a box filter, improves it quite a bit: http://i.imgur.com/DhbOS34.png.

The version's sharpness on the tree's texture indeed improves, but it is still somwhat softer than Jinc3AR verison. Yet it creates more noticeable ring effects on the leaf's edge and "T" and "P" characters.

Here is my subjective visual experience opinions:

[High-frequency signal preservation]
NEDI v2 > Jinc3AR >> NEDI v1

[Low-frequency signal preservation]
Jinc3AR > NEDI v2 >> NEDI v1

[Angled Object edge smoothness]
NEDI v1 >= NEDI v2 > Jinc3AR

[Ring-effect artifact issue]
NEDI v2 > Jinc3 AR >> NEDI v1

[Shaded? Text clarity]
Jinc3AR > NEDI v1 > NEDI v2

[madshi]

Originally Posted by Shiandow

Edit: I've just confirmed that only processing the chroma channel indeed does backfire sometimes, I've simultaneously confirmed that using NEDI to interpolate the chroma channel is capable of actually reconstructing the original values, which I find quite impressive, unfortunately it does get it wrong sometimes. Hopefully it's possible to prevent this by checking if the chroma and luma channels actually correlate.

Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

FWIW, here are two samples which are good candidates for chroma tests:

red fonts
chroma test

Btw, if you create an empty file named "YCbCr" in the madVR folder, madVR will output YCbCr instead of RGB. This might allow you to test better because color conversion to RGB is simply skipped completely.

[Bloax]

madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?

[Shiandow]

Originally Posted by madshi

Fine tuning this could be difficult. But I think there's some potential there. FWIW, I think this would be more beneficial for chroma upscaling than chroma doubling.

Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma? Otherwise I would have to disagree.

Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR
chromaNEDI

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.

[nevcairiel]

Originally Posted by Shiandow

Just to be sure, by "chroma upscaling" you mean scaling the chroma to the same resolution as luma?

Thats what "chroma upscaling" always refers to in the madVR context, so thats safe to assume.

[Shiandow]

Ok thanks, I wasn't really sure since chroma upscaling is done by doubling the resolution.

[madshi]

Originally Posted by Shiandow

Anyway, I've more or less finished with implementing the NEDI (4:2:0) chroma upscaling algorithm. For some images it noticeably improves the picture quality. There are some cases where it behaves differently from bicubic, but it's usually hard to tell whether the result is worse or better. In any case these situation could probably be avoided by downscaling the chroma again and comparing it to the original.

Here is an example, taken from the Sintel film, where it is not only different but clearly better:

Bicubic75AR
chromaNEDI

Looks nice!

Originally Posted by Shiandow

In both cases it was followed by a single NNEDI3 (16 neurons) pass, to make the differences more obvious. Incidentally this is also one of the cases where NEDI image doubling looks better than NNEDI3:

chroma+luma NEDI

The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?

Originally Posted by Shiandow

In fact I think you might want to try watching Sintel using NEDI, if that doesn't convince you that using NEDI is worth the amount of artefacts it causes then I don't know what will. I'll update the first post to include both the chroma NEDI code and a version of NEDI which is suitable for video playback.

I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.

May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

[madshi]

Originally Posted by Bloax

madshi: Any ideas as for why your domain redirects to 92.242.144.160/http://madshi.net(..)?

I don't see that on my PC. If I type in "http://madshi.net" the browser seems to show that without any redirection. At least no redirection that I can see?

[Bloax]

Apparently it's just my AV pulling dirty, dirty tricks because it thinks your site is suspicious - carry on.

[Shiandow]

Originally Posted by madshi

The NEDI+NEDI image looks better than NEDI+NNEDI3 in some image areas, but worse in others. Some edges look cleaner (less aliased) with NEDI, but NNEDI3 is overall sharper and more detailed in some areas. Which neuron count did you use for NNEDI3? Maybe only 16?

I used 16 neurons although using 128 neurons doesn't result in a noticeable difference. In this particular image I might actually prefer the way NEDI is a bit softer, it looks somewhat more natural. The extra sharpness of NNEDI3 looks nice at first glance but when fine tuning the NEDI algorithm I've encountered (admittedly rare) cases when what I thought was an artefact of NEDI turned out to be some detail that NNEDI3 decided to erase.

Originally Posted by madshi

I just don't think it's an algorithm a user could "set and forget", one which works for any kind of video content. All the other algorithms qualify for that, but NEDI doesn't, IMHO.

What I find somewhat frustrating is that, so far, NEDI has looked good on all video content I've tried. But I'll admit that it has some quirks, I'll try to see if I can iron these out.

Originally Posted by madshi

May I use your chroma upscaling shaders for madVR? Maybe I'll find a way to improve them further. If so, I'd post my changes here.

Feel free to use them. I'd appreciate it if you'd keep me updated on possible improvements you've made.

Here are some possible improvements that I couldn't implement because I was restricted to using a simple shader:

• You should be able to increase the speed by about 1.7 (1.75/3) times just by preventing it from unnecessarily calculating pixels that weren't even supposed to change.
• You could calculate some of the values upfront, preventing some texture calls. This will probably work best for the values of "Get4".
• As I mentioned before you could probably detect some unwanted behaviour by comparing the original chroma values with a downscaled version of the upscaled chroma.