nnedi3 - OpenCL rewrite

[madshi]

@SEt,

tried for 2 days to find a faster implementation than yours, but I have to admit that I failed. Tried lots of different approaches, but none of them were faster than yours. So congrats, you seem to have done a very good job!

Have been working with AMD's CodeXL. Surprisingly, your kernels only run at 20% occupancy on GCN, due to using too many registers. But after those 2 days of trying for myself, I have to say that AMD's OpenCL compiler pretty much sucks. It wastes registers like crazy, often without any sense. For example, I tried modifying your "float8" logic to "float4" in order to save a few registers, in the hope that this might improve occupancy and performance. But after my changes AMD's OpenCL compiler actually spent *MORE* registers on the code than before. Which makes absolutely no sense. Argh... I suppose in a few months/years, when hopefully the OpenCL compiler has matured a bit, maybe there's hope in improving the kernels further to decrease occupancy and improve performance. But for now I've given up on finding a faster/different approach.

So I've played with your kernel a bit and found a small performance improvement. You're doing:

#if defined(__GPU__) && defined(__AMD__) && __OPENCL_VERSION__ >= 110
float8 t = (float8)((*(__local float3*)&in[j][0]).s0012, *(__local float4*)&in[j][3]);
#else
float8 t = (float8)(0, in[j][0], in[j][1], in[j][2], in[j][3], in[j][4], in[j][5], in[j][6]);
#endif
#pragma unroll
for (uint i = 0; i < xdia; i++)
{
t = (float8)(t.s1234, t.s567, in[j][i+7]);
sum1 += t*w[i];
sum2 += t*w[i+8];
}
By replacing that code with the following code I got a ~ 5% performance improvement on my GCN card:

float8 t = *((__local float8*) &in[j][0]);
#pragma unroll
for (uint i = 0; i < xdia - 1; i++)
{
sum1 += t*w[i];
sum2 += t*w[i + 8];
t = (float8) (t.s1234, t.s567, in[j][i + 8]);
}
sum1 += t*w[xdia - 1];
sum2 += t*w[xdia - 1 + 8];
I haven't actually tested if my code is correct, though, so you might want to double check. Just been benchmarking so far...

One more thing: In all my image upscaling tests I've always preferred 8x4 over 8x6. I've found that 8x4 produces less weird artifacts. And my preference for 8x4 had nothing to do with performance, it was based purely on overall image quality. So I would suggest switching to 8x4 (or to offer it as an option). At least for image upscaling. Don't know if the situation is maybe different for deinterlacing. Switching to 8x4 of course also gives another nice performance boost. When only testing the "y" kernel on only one color channel, I got from 95fps to about 130fps by doing the small code change posted above, and by going from 8x6 to 8x4.

[SEt]

Occupancy doesn't matter if you know what you are doing. The code was optimized for Cayman architecture, but not to the point to hurt others much. So +5% for GCN from that change is reasonable but likely hurts something else if I didn't use that obvious implementation, don't remember exact cases though. Also, your code won't run on non-AMD cards.
8x4 - maybe, it was designed to be easy subcase.

[madshi]

A while ago when testing resampling algorithms with various test images I tried all the options NNEDI3 offered and to my eyes 8x4 produced the overall best results. Anyway, just my 2 cents. Feel free to ignore... :)

Personally, I think optimizing for the latest generation of GPUs makes more sense than optimizing for older generations. Of course that's only true if the gain on newer generations is not lower than the cost on older generations. So it's a balance act, of course. FWIW, I can't see a reason why the original code should be faster on any GPU than the modified code I suggested. My code simply does less work. But after my experience with the AMD OpenCL compiler I've lost trust in what appears logical, so I can't be sure.

Having only tested Y resampling before, I've now switched over to X testing. FYI, I've found that simply "reusing" the Y kernel for X (meaning: I copied the X kernel and just swapped the X/Y coordinates when reading/writing pixels) produces faster results on my GCN GPU compared to the special X kernel. However, I'm not using OpenCL buffers. Instead I'm using OpenCL image objects (D3D9 interop is based on image objects). Maybe using image objects reduces the stride memory access pattern problem? I don't know. I think I read somewhere that GPUs can optimize the cache for stride access when using image objects, so using image objects can work better than buffers for 2D data. Not sure if that is of any use to you, but wanted to mention it, just in case you want to try. If you do try, you may want to use a single channel image format ("CL_R"). That gives me similar performance to buffers (maybe a tiny bit slower), while CL_RGBA is (naturally) slower than CL_R.

[SEt]

Optimizing makes most sense for what hardware you have. Your code does 1 more load or the same work depending on compiler. Data shuffles are free.
The code isn't memory bound – load/store shouldn't matter much, though x kernel does have more work.

[madshi]

Why would my code use one more load? I don't see that. As far as I can see, my code should have the same number of loads, or one less, depending on compiler. And it should have one less shuffle. Please note that the loop is 1 shorter in my ocde ompared to yours.

[SEt]

Ah, indeed 1 shorter – then it's the same and we are looking at compiler difference. There are no shuffles: compiler just renames registers. I guess I'll retest such sequence when resume my work on the code: there is at least one more feature I want to implement.

Tested: indeed, it's a bit slower on Cayman. Roughly the same on NV. Also, NV bluescreened during testing – and you say AMD OpenCL implementation is bad? :rolleyes:

[madshi]

Yeah, I've now tested my changes on all Intel, AMD and NVidia. And I have to say, NVidia's OpenCL implementation is by far the worst. Not only is it limited to OpenCL 1.1, but it also has weird effects. E.g. I wondered why you stored 68 weight floats per nnst, instead of 66 (2 empty). Now I know why: When using 66 floats, NVidia produces a corrupted image (AMD and Intel don't). Furthermore, using write_imagef() with out of range coordinates makes NVidia go bonkers. So I now basically had to add a couple "if"s just to make NVidia play nice. In comparison AMD's and Intel's OpenCL implementation worked flawless right from the start. And the latest AMD driver now also supports D3D9 interop (finally!). Still not happy with AMD's OpenCL compiler optimizations, though.

Just in case you're interested (if you want to benchmark using the y kernel for x when using image objects), here's the code I've currently ended up with. It contains several changes, though, so it might not be useful to you:

http://madshi.net/nnedi3ocl.zip

JFYI:

> your code won't run on non-AMD cards

It did run just fine on NVidia on Intel GPUs, too.

[SEt]

AMD interops are buggy (at least with OpenGL) and not so great speed-wise. NV works... while you have only 1 card; add second and it becomes extremely slow.

Ok if it works, but its memory accesses are not-compliant. AMD hardware allows you to break many restrictions, but NV and Intel are less forgiving as you've seen.

It looks like you are using clamp-to-edge padding instead of original mirroring. Is there much difference? I've faithfully implemented original behavior without experiments in this area.

Your code has 4 or 8 times more memory accesses then mine when reading and writing image data. I believe the kernels are not memory-bound in mine implementation, so it would be interesting to see how much speed impact it has.

[madshi]

I guess I could use CLK_ADDRESS_MIRRORED_REPEAT instead of CLK_ADDRESS_CLAMP. Haven't tried that yet. Will double check if that produces better edges and whether it affects performance. Thanks for the hint.

Yeah, I have to issue 8 separate image_read/writef instructions for what you're doing with one float8 assignment. However, my code doesn't seem to execute slower than yours on my GCN card. I'm not sure why. Maybe it's simply because the kernel is not memory-bound. I do notice a (small) slowdown, though, when using 16bit images instead of 8bit.

[SEt]

Would be interesting to see if CLK_ADDRESS_MIRRORED_REPEAT works: first of all you will have to use normalized coordinates (the question is accuracy of course) and even still repeat won't be the same type as in original nnedi3.

[tObber166]

When uisng nnedi3x_rpow2() , why do I get "there is no function named "lsY8" error?

[madshi]

Would be interesting to see if CLK_ADDRESS_MIRRORED_REPEAT works: first of all you will have to use normalized coordinates (the question is accuracy of course) and even still repeat won't be the same type as in original nnedi3.
I've double checked the mirroring, and you were right. The bottom and right most lines were black with my kernel, while they were valid image content with yours. Thanks again for bringing this to my attention. So now I added some code to my kernels to internally mirror the image_readf() calls. I liked that better than to switch to normalized coords. Seems to work fine. Image output now looks more or less identical compared to your kernel.

http://madshi.net/nnedi3ocl.zip

Do you happen to remember from the top of your head whether with a 480 height image, the first "invalid" line number 480 should it map to 479 (so that lines 479 and 480 are identical), or should it map to 478? I wasn't sure about that.

FWIW, I've done a small benchmark comparison between your and my kernels. But let me start by saying that the comparison is not fair, because my kernel only does 8x4, while yours does 8x6. So your kernel does about 50% more work. So your kernel is expected to run slower. Anyway, with my AMD 7770 GPU, my kernel runs with 92.7fps while yours runs with 64.0fps. With my Intel HD4000, mine produces 8.546fps, while yours produces 6.056fps. Would be interesting to compare the kernels with the same workload.

[madshi]

P.S: Tried to modify your kernel to run 8x4, too. Not sure if I did it correctly, though. At least I changed ydia to 4 and decreased the y kernel reads. Wasn't sure how to change the x kernel, though, on a quick check. So probably my edit was not complete (too many reads for x kernel). Anyway, with the changes I did I got 7.720fps with HD4000 and 85.0fps with the AMD 7770. So it seems to me that my modification of your kernels might not be slower than your original kernels, maybe even a tiny bit faster (maybe due to 2D texture cache). I'm not 100% sure, though. But it might be worth trying on your side. My code also has the following advantages:

(1) No ugly buffer stride/offset calculations.
(2) No extra padding kernels.
(3) x and y kernels are almost identical. And both kernels use the same "process_local" function.
(4) Input/output pixel bitdepth can be changed via runtime APIs, using the same kernels.

[SEt]

It should be -478-479-478-.
I don't have time now for comparisons, but I had plans for trying textures so definitely will test this way later.

tObber166, nnedi3x_rpow2() is AviSynth26 function and you are likely running it on 2.5. Only base plugin call nnedi3ocl() is 2.5 compatible.

[madshi]

It should be -478-479-478-.
Thanks.

[madshi]

Sorry for flooding this thread with posts. I think this will be the last one for now. At least I think I'm now done with polishing my nnedi code. The latest version now only uses one kernel for everything (padding/mirroring, x and y resampling). Which cuts down OpenCL initialization time:

http://madshi.net/nnedi3ocl.zip

I've also now run benchmarks with my kernel patched to 8x6, so I can finally deliver fair results:

AMD 7770:
x only, buffer vs image: 161.8 fps vs 199.3 fps
y only, buffer vs image: 103.8 fps vs 109.3 fps
x + y, buffer vs image: 64.0 fps vs 71.5 fps

Intel HD4000:
x only, buffer vs image: 23.22 fps vs 19.07 fps
y only, buffer vs image: 8.10 fps vs 10.36 fps
x + y, buffer vs image: 6.05 fps vs 6.77 fps
I find it interesting that your special x kernel runs noticeably faster with the HD4000, but noticeably slower with the 7770, compared to my solution which reuses the y kernel for x upscaling.

[SEt]

Initialization time is reduced by totally different means: dropping the compilation step. nnedi3ocl actually has code for it but it's currently disabled to allow free and intuitive modification of OpenCL part.

Also, why are you putting LGPLv2.1 license in the archive when the code is under LGPLv3?

[madshi]

Sorry, didn't notice the version difference. Your download didn't contain a license file, so I copied the first LGPL file I found. Will replace it with LGPLv3 right way.

[DrZine]

I don't have MT installed. Using Avs+ So I did the fps/tflops calculation from the none mt mode. Close to a stock 7970 :)

32.85, no MT, 11.66, Radeon HD7870, 1100, 2, FX8350, 2013.8.12

Frames processed: 1000 (0 - 999)
FPS (min | max | average): 31.78 | 33.17 | 32.85
CPU usage (average): 0%
Thread count: 3
Physical Memory usage (peak): 62 MB
Virtual Memory usage (peak): 66 MB
Time (elapsed): 000:00:30.444

Kinda funny seeing a script run with 0% cpu usage.

[SEt]

Really nice result for Radeon HD7870. Though I would like to also have MTMode result to add it to the table (because without MTMode you likely don't see 100% GPU load so the fps is somewhat arbitrary lower).

0% CPU usage is exactly what you should see: all the work is on the GPU side and nothing heavy should occupy the CPU.

[tObber166]

i7 4770K @ Stock, GTX 780 SLI


http://i44.tinypic.com/2irucxx.jpg

[SEt]

tObber166, just so you know SLI or other multi-gpu configurations are not used so far, so your result is by single GTX 780.

[tObber166]

Yes, I am fully aware of that :)
It's just a odd habit to post full spec sometimes =)

[DrZine]

Really nice result for Radeon HD7870. Though I would like to also have MTMode result to add it to the table (because without MTMode you likely don't see 100% GPU load so the fps is somewhat arbitrary lower).

Ok Set, I reinstalled Alpha5 and swapped in your newest build of MT and reran the script. It should be noted that the stock clock speed on the card is 1000 mhz. I am running a small overclock. 2.56 is the stock TFLOPS for the card so that gets adjusted up to 2.816 with the overclock.

32.89, 37.53, 13.327, Radeon HD7870, 1100, 2, FX8350, 2013.12.8

[SEt]

So, Pitcairn takes the efficiency crown now.

You don't need to reinstall anything to switch versions of Avisynth or Avisynth+: replacing avisynth.dll is enough.

[kasper93]

Results on my HD5870

https://gist.github.com/kasper93/ef7b3a0b2165d56b5b0a

https://gist.github.com/kasper93/de93cad4d410a9bcb575

I find it quite strange that with SetMTMode(2,4) I get unstable fps. Is that measurements error or my GPU doesn't like MT mode? HD5870 is not newest one but well it's still quite powerful.

[pie1394]

My code also has the following advantages:

(1) No ugly buffer stride/offset calculations.
(2) No extra padding kernels.
(3) x and y kernels are almost identical. And both kernels use the same "process_local" function.
(4) Input/output pixel bitdepth can be changed via runtime APIs, using the same kernels.

I guess you have known about Tahiti (HD97x0) architecture, which has 12 memory channels. It is interleaved in 256 bytes with a specially designed mapping. The same channel mapping mechanism is repeated every 16KB address -- 1/3 channel conflict rate every 2KB jumping, 1/6 rate every 256bytes. Yet Pitcairn(HD78x0) has 8 channels, and Verde(HD77x0) has 4 channels --- cycled every 2KB.

After a little bit study about AMD GCN architecture's memory controller design, I think non-linear-strided frame buffer design could utilize more memory bandwidth between L2 cache and GDDR5 memory and reduce latency on fetching particular 4x4 or 8x8 2D pixel data from VRAM...

How do you think about 32-bit-pixel component with 4x4 or 8x8 tile temporary frame-buffer design? With the 8-tap filtering in both X and Y-direction, 8x8-tile seems a better choice on 256-byte-per-bank design w/o copying pixel data into CU's local memory and out again.

[Groucho2004]

I find it quite strange that with SetMTMode(2,4) I get unstable fps. Is that measurements error or my GPU doesn't like MT mode? HD5870 is not newest one but well it's still quite powerful.
These wild fluctuations can be observed in many multi-threaded scenarios involving SEt's Avisynth MT and I think it has to do with Avisynth's caching algorithm. It's not a measurement error and your GPU is not responsible for it.

[kasper93]

I see my results are in first post. To fill remaining informations, it was on i7 920@4.0, PCI-E 2.0 and GPU on stock 850 clock.

[jmac698]

Just to note that I got this running on a lowly GT620, GPU load was 83%, cpu went up 20% (core duo 2.66Ghz), GPU core 700MHz, GPU mem 300MHz, function 2x scale nnedi3ocl(dw=1)

Looks like I need to get avsmeter. Will update later.

[BeNooL]

Results running version 2013.12.08-beta on a GTX 660 (1137MHz core clock, PCIe 3.0 @ x8) with an i7 3770 @ 4.0 GHz


FPS with MTMode(2,4):

Frames processed: 1000 (0 - 999)
FPS (min | max | average): 3.689 | 474957 | 14.96
CPU usage (average): 12%
Thread count: 16
Physical Memory usage (peak): 563 MB
Virtual Memory usage (peak): 550 MB
Time (elapsed): 000:01:06.858


FPS with no MTMode:

Frames processed: 1000 (0 - 999)
FPS (min | max | average): 10.59 | 14.93 | 14.68
CPU usage (average): 12%
Thread count: 12
Physical Memory usage (peak): 276 MB
Virtual Memory usage (peak): 272 MB
Time (elapsed): 000:01:08.134

[dajaja]

Hi everybody. I am am using nnedi3 with the ffdshow Avisynth plugin to deinterlace live TV (SD). The setup works pretty well, but performance is limiting nns to 0 or 1 and I would like to speed it up by switching to nnedi3ocl. My regular script is the following:


temp=nnedi3(field=3,nsize=6,nns=1,qual=1,etype=0,threads=16)
yadifmod(order=1, mode=1, field=0, edeint=temp)


I modified this to use nnedi3ocl as follows:


temp=nnedi3ocl(field=3,nsize=0,nns=1,qual=1,etype=0)
yadifmod(order=1, mode=1, field=0, edeint=temp)


Now, all I get is an interlaced image with strong ghosting. Screenshots are attached. I played with many settings, but nothing eliminates the ghosting. Avisynth is version 2.6 MT, video card is a Radeon 7790 on Catalyst 14.4.

What could be the problem?

[BeNooL]

At the risk of being off topic, have you given QTGMC a try ? see http://forum.doom9.org/showthread.php?t=156028

Given it is SD content, running it live might work with moderate quality preset.

[dajaja]

At the risk of being off topic, have you given QTGMC a try ? see http://forum.doom9.org/showthread.php?t=156028

Given it is SD content, running it live might work with moderate quality preset.

Yes, thanks. I have tried it before and it works with the "Very Fast" preset - but only until it crashes :D Seems to be an MT problem and I was not able to find a stable setting, yet. If I did, nnedi3ocl could be used in QTGMC, too to speed it up.

[huhn]

i see about 5 times blended?? frame in the one frame you showed can you post a 5 sec sample?

[dajaja]

i see about 5 times blended?? frame in the one frame you showed can you post a 5 sec sample?

I think I'd have to install extra software to capture it. But I don't see the advantage of looking at the video. The still shows the problem perfectly. The only thing missing is the info that if there is no movement in the video, there are of course no artifacts visible :)

[Seedmanc]

Is there any chance of this plugin going x64 bit? I'm using it in a chain of graphedit-ffdshow-avisynth as a custom upscaler for ps2 emulator and having both graphedit and ffdshow available in x64, this is the only thing keeping me in 32bit.

Also noticed a weird thing, there is no nnedi3ocl_rpow2, so I have to use that nnedi3x_rpow2 from .avsi, but at the same time AVSPmod suggests that there exist a "nnedi3ocl_nnedi3ocl" function and avisynth accepts it, so it indeed exist (does the same as nnedi3ocl).

I tried all 3 custom .cl files for NVidia posted here earlier, first two made no difference in speed for me, while 3rd (posted by madshi) gave error "Unable to allocate OpenCL resources". Is there no hope for GTX 560?

[SEt]

As there is no usable x64 Avisynth – no x64 version for now. This plugin wouldn't benefit from it anyway.

nnedi3ocl is basic functionality implemented as plugin. nnedi3x and nnedi3x_rpow2 – scripts that extended beyond that. Basically, what is reasonable to do in script – should be done in script.

GTX 560 should work just fine. Slower than similar Radeons or CC2.0 cards, but still ok.

[huhn]

As there is no usable x64 Avisynth – no x64 version for now. This plugin wouldn't benefit from it anyway.

so this doesn't work with avisynth+ ?

[SEt]

It should work without problems, but Avisynth+ has no stable multithreading yet. Speed-wise now I would recommend AviSynth 2.6 with MTMode (in general, not for this plugin in particular).

[huhn]

my question was more about 64 bit avisynth+.

or is it still not usable and I'm just lucky the hole time?

[Groucho2004]

my question was more about 64 bit avisynth+.
32 Bit plugins don't work with 64 Bit Avisynth and vice versa.

or is it still not usable and I'm just lucky the hole time?
:confused:

[huhn]

32 Bit plugins don't work with 64 Bit Avisynth and vice versa.

i know that's why I ask.

As there is no usable x64 Avisynth – no x64 version for now. This plugin wouldn't benefit from it anyway.
my question was more about 64 bit avisynth+.

or is it still not usable and I'm just lucky the hole time?

or in short. avisynth+ 64 works fine for me but there is no 64 bit nnedi3 openCL.

and the reason is this:
As there is no usable x64 Avisynth – no x64 version for now. This plugin wouldn't benefit from it anyway.

[SEt]

For me Avisynth is not usable without threading.

[SamKook]

There is a version of avisynth+ with MT that apparently works(haven't tried it myself), there's just no official release for it.
I haven't really seen any more big reports of MT failing in the months since it appeared (in terms of its actual stability), save for the bounds checking issue with the audio cache that I reported in here and on Github. If I'm correct about that assessment, perhaps it's time to just bite the bullet and merge it to master and bump the version number to 0.2. The psychological effect of just going ahead and releasing it might help too.
P.S. It was a subtitle filter(SupTitle) that was failing in this case.

[SEt]

Last version of Avisynth+ I've tried had speed issues with threading rather than stability. My usual post-processing script run on it significantly slower (realtime -> not realtime). Maybe it's not hard to tweak the existing implementation for similar or better speeds, but regrettably I have absolutely no time now to dedicate there.

[yup]

Hi All!

how I can use nnedi3ocl for conversion 25 Hz interlaced source to 50 Hz progressive. When using nnedi3 it is simple
nnedi3(field=-2)
When I use
nneedi3ocl(field=-2)
I see doubling framerate and frame quantity, frame from 1 to N see fine, but from N+1 to 2*N it is not changed, freeze frame N.
N number frame in interlaced source.


yup.

[SEt]

yup, see pm.

Note: it looks like new AMD driver 14.12 broke some part of used OpenCL on HD6xxx hardware, so check other driver versions if you get corrupted results.

[TheProfileth]

After updating my driver on my 7970 it also no longer works and instead produces weird results where only a fraction of the screen is actually intelligible.

[Asmodian]

Note: it looks like new AMD driver 14.12 broke some part of used OpenCL on HD6xxx hardware, so check other driver versions if you get corrupted results.

Sounds like they broke it on the 7970 as well. :(