Deathray - OpenCL GPU accelerated spatial/temporal non-local means de-noising

[Jawed]

No news, dead due to lack of interest (mostly my lack of interest, it seems).

My lack of interest is due to the fact that I prefer the results from MDegrain.

My website is available. There are 3 files:

Deathray_1_00.zip - see post 1
Deathray_1_00_DLL.zip - see post 1
DeathrayNV110120004.zip - see post 36

Please note that the source code I've provided in post number 1 does not match the source code for post 36.

Please read the entire thread to learn how best to use this.

For the best quality I recommend DeathrayNV110120004.zip. This contains a tweaked algorithm (hence the improved quality) as well as a fix to ensure it works on NVidia. This DLL works with AMD, too. The downside with this version is that it creates a deathray.log file in the same folder as the Avisynth script.

I have no idea whether Deathray works with recent AMD or NVidia drivers. In theory the newest drivers will reject my code because they are more strict.

Obviously, anyone is welcome to play with the code.

[xiulet]

No news, dead due to lack of interest (mostly my lack of interest, it seems).

My lack of interest is due to the fact that I prefer the results from MDegrain.

My website is available. There are 3 files:

Deathray_1_00.zip - see post 1
Deathray_1_00_DLL.zip - see post 1
DeathrayNV110120004.zip - see post 36

Please note that the source code I've provided in post number 1 does not match the source code for post 36.

Please read the entire thread to learn how best to use this.

For the best quality I recommend DeathrayNV110120004.zip. This contains a tweaked algorithm (hence the improved quality) as well as a fix to ensure it works on NVidia. This DLL works with AMD, too. The downside with this version is that it creates a deathray.log file in the same folder as the Avisynth script.

I have no idea whether Deathray works with recent AMD or NVidia drivers. In theory the newest drivers will reject my code because they are more strict.

Obviously, anyone is welcome to play with the code.

ok thanks you for the info my friend :)

you mean MVTools+MDegrain...¿?

[Keiyakusha]

My lack of interest is due to the fact that I prefer the results from MDegrain.

So how about porting Mdegrain to gpu then? Porting motion search proved to be possible and if i'm not mistaken its opensource (since mvtools is opensource). But they had different purpose and ported only mflowfps(and renderer part of that is closed source). However they claim their motion search is better than original, which I so far can't see, unfortunately. So maybe it requires some tweaks.

[Jawed]

So how about porting Mdegrain to gpu then?
One of those things I just didn't get round to, and now behind other things.

I logged on to this forum today for the first time in quite a while and discovered VapourSynth.

Previous time I logged in, I discovered people are doing 10-bit encodes.

I'm out of touch.

Porting motion search proved to be possible and if i'm not mistaken its opensource (since mvtools is opensource). But they had different purpose and ported only mflowfps(and renderer part of that is closed source). However they claim their motion search is better than original, which I so far can't see, unfortunately. So maybe it requires some tweaks.
Even just downloading the code and setting up a development project is more effort than I want to get involved in, right now. Sorry.

[Jawed]

I've just realised I could make a git version of this:

Deathray on github.com (https://github.com/JawedAshraf/Deathray)

This is v1.00 without any of the bug fixes or improvements that I created during the debugging work done for NVidia users.

It's also for an ancient SDK.

It will definitely require work to get it working with latest drivers and/or SDKs.

---

And for those of you who understand Git branches, the branch called FindFixes is my attempt to identify the code that makes the DeathrayNV110120004 version.

[lansing]

i have a question.

I just brought a new nvidia graphic for encoding purposes, and when I did test on both deathray and nlmeanscl with avsmeter, the default nlmeanscl setting gave me 252fps with 70% gpu loaded, while deathray in default setting gave me 47fps and over 90% gpu loaded. It makes me wonder what's the difference between the two to have a huge different in speed like this?

[Jawed]

i have a question.

I just brought a new nvidia graphic for encoding purposes, and when I did test on both deathray and nlmeanscl with avsmeter, the default nlmeanscl setting gave me 252fps with 70% gpu loaded, while deathray in default setting gave me 47fps and over 90% gpu loaded.

Wow, it still works? I was expecting newer drivers to stop it working.

It makes me wonder what's the difference between the two to have a huge different in speed like this?
Looking at my notes, I see for 1920x798 video I had about 32fps on HD5870. This is "compute-limited", i.e. memory accesses have no impact on performance. For standard definition I had 122fps theoretical, but memory accesses meant that this was only about 25fps in practice, so I needed to watch 5 videos in parallel (in Media Player Classic) to get 100% GPU usage.

Alternatively a temporal radius of 2 would produce 100% GPU usage (tY=2, tUV=2) on standard definition clips.

Because I coded for AMD, it uses on-chip memory in a way that's potentially "unfriendly" towards NVidia. In other words I took advantage of the very large register capacity on AMD and I wouldn't be surprised if NVidia cards ran out of registers. The newest NVidia cards (Kepler) have much better register capacity, so in theory even if this was a problem a few years ago, it shouldn't be a problem for the latest NVidia cards.

Due to lack of interest, no performance comparisons were made between AMD and Nvidia, so I don't know if this is really true or not.

I don't know the size of the window in nlmeanscl. Deathray uses a 7x7 size, and the larger it is the more effort involved.

Also I don't know the search area in nlmeanscl. Deathray does not search the entire frame to get other 7x7 windows, it only searches near the pixel being filtered (it looks at 48 other 7x7 windows). Again, searching more of the frame is more effort. In fact searching the entire frame is counter-productive, it is better to search a limited area near the pixel being filtered.

I don't think there's much point in the nlmeans algorithm. It is only mildly useful when used with temporal settings and pretty much a waste of time without temporal settings. I prefer MDegrain, which is why I don't use Deathray.

Do you like the results of either Deathray or nlmeanscl?

Do you prefer it when you use temporal settings?

Have you used MDegrain?

[lansing]

mdegrain didn't fit my taste in which it manipulated the image too much. It's going to be either a hit or miss. On moving scene, sometime it gives a good result, and sometime it smooths out the original shape.

I'd did some test with this sample, i uploaded here:
sample (http://www.mediafire.com/download/nk5tbg5fo5stkcf/anime_low_light_thin_line_sample.264)

Both deathray and nlmeanscl pretty much gave the same result, except in some low light area with thin lines, deathray at default desaturated the color by a little.

This sample also gives the reason why i don't use mdegrain, notice the panning scene from 2-6 second, there're some spider webs across the screen. When i used smdegrain, it pretty much wiped out half of them. While deathray and nlmeanscl still preserved them.

well on the temporal side, smdegrain did a better job at calming the grains.

[Jawed]

I trimmed off the first frame, which is broken:

http://www.mediafire.com/download/0wn3l95nmiynrpf/anime_low_light_thin_line_sample_JA.mp4

I have to say I'm horrified that apparently recent anime can be such a disaster zone.

[Jawed]

Updated the first post with information on version 1.01. Please note that www.cupidity.f9.co.uk is not being maintained with source or DLL. All mirrors in this thread should be considered defunct.

[lansing]

Jawed, I replaced the 1.01 version and there's an error message saying that there's no such function.

[Jawed]

Make sure you are not running an encode or previewing an avisynth script in an editor such as AvsP at the time you delete the old Deathray.dll. After you've deleted the DLL refresh the explorer window to make sure that the Deathray.dll has really gone.

Then download the 1.01 DLL from the github page and copy to the Avisynth plugins folder.

I've just done those exact steps here and it works fine.

You might like to try a re-boot of your PC.

I can't think of anything else that could be causing your problem. Maybe someone else can report their experience?

[lansing]

i'm using the avisynth mt version in here (http://forum.doom9.org/showthread.php?t=148782)

version 1.00 works fine

[Jawed]

That's the version of avisynth I'm using :confused:

I have the avisynth DLL in two places on my system:

SysWOW64 - used by AvsP
MeGUI\tools\avs - used by MeGUI

---

When you say 1.0 works do you mean the NVidia NV110120004 version?

[lansing]

i don't know, the date of that deathray dll is 1/17/2011.

When i use the old one, avspmod recognized it, but when i replaced it with the new one, it said "there's no function name deathray"

[Jawed]

A friend helped me debug this and discovered that he is missing msvcp110.dll. And then msvcr110.dll.

In Virtualdub this is clearly shown as a sequence of two error messages. The first says that msvc*110.dll is missing. The second message says that Deathray is not a function.

AvsPmod on his system says "Deathray is not a function".

Sorry about that. I think I can recompile with an older runtime (90), but that will be a couple of hours at the earliest. I'm not 100% sure of that though.

Now I wonder if I can eliminate the runtime?

[Morte66]

A friend helped me debug this and discovered that he is missing msvcp110.dll. And then msvcr110.dll.

Those two DLLs are bundled with Lord Mulder's Simple x264 launcher, by the way. At least, that's where I found them.

[Jawed]

Please try:

Non-runtime DLL (https://github.com/JawedAshraf/Deathray/tree/v1_1_no_runtime)

and tell me if it works.

[lansing]

yes the new one is working now

[Jawed]

I have made version 1.1 compile to a non-runtime-required DLL. I have updated github (master and 1_1 branches) with the new DLL and I have removed the test branch 1_1_no_runtime.

[Jawed]

This new release is primarily aimed at de-noising more evenly across the tonal range.

Two new parameters, z and b control this.

z is more important and I would recommend it is set to true. It works best with temporal mode (tY > 0 and tUV > 0).

l is probably a waste of time, but feel free to play (it increases de-noising in shadows and reduces de-noising in highlights).

c defaults to true, which reproduces the behaviour of 1.01. I recommend using false when z and/or b are set to true.

A good set of parameters to start your experiments with:

hy=1,huv=0,tY=2,tuv=0,s=0.7,x=3,l=false,c=false,z=true,b=true

It's a bit slow though.

[lansing]

i played around with the new parameter, and the z parameter is giving me issues.

For my anime source, when i set z=true and hy less than 2, all the small white dots in the video were filled with black. When i set hy >= 2, everything is back to normal.

[Jawed]

Hi there, can you try the 1.3 branch that I have just released?

I discovered what I think is the same problem (though I had to use hY < 0.5 with z=true).

Are you happy to switch to the v1_3 branch on Github and download the DLL?

[lansing]

yes it's fixed, thanks.

I've also uploaded a sample video here for testing. It's an anime scene of flashing light with very heavy grain on top. So far I have no success with any denoiser except temporaldegrain.

http://www.mediafire.com/download/616tj9u8rd6av6t/flashing.mkv


EDIT: which one is the parameter to control the temporal strength, it seems like that the temporal denoise is doing very little effect to my source.

[Jawed]

yes it's fixed, thanks.I'm glad to hear that works.

EDIT: which one is the parameter to control the temporal strength, it seems like that the temporal denoise is doing very little effect to my source.
The tY and tUV parameters control the count of frames. If you set tY to 3 then you get 3 frames before the current frame, 3 frames after it + the current frame. So that's 7 frames.

You can control colour temporal with tUV.

The hY setting works together with tY and hUV works with tUV.

As for your flashing clip, I doubt I can help much. The near-black nastiness is pretty horrible.

You might have more chance of help if you make a thread about it, rather than posting in this thread. Most people ignore this sub-forum. Presumably there are people on Doom9 who have some expertise in cleaning up this kind of noise on anime.

You might want to turn on L (linear), which will make denoising much stronger near black - this sounds contradictory to setting z=true and b=true - but I think the combination can work.

Try s = 0.7. And try x=2 or x=3. Try tY=1 or 2.

I don't think you have much choice but to spend a long time experimenting.

I'm still experimenting with new code for Deathray, so perhaps I'll have something more useful.

[Jawed]

lansing, I would like your opinion on these encodes of the videos you uploaded a while ago:

https://dl.dropboxusercontent.com/u/81074020/Vicious%20Deathray%20Alpha.zip

Encoding them is very slow, about 0.4fps, but I think it's worth it :)

[lansing]

they look pretty nice to me, the grain pattern after the denoise were distributed evenly. in comparison to before where there're grain of different sizes here and there.

[Selur]

Any one got a working binary?
Using the one from: https://github.com/JawedAshraf/Deathray/commit/17a6f0ba4499c411a31ea0bc3bfbb97fc7ed154d
and
LoadCPlugin("G:\Hybrid\avisynthPlugins\ffms2.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\Deathray.dll")
# loading source: F:\TestClips&Co\test.avi
FFVideoSource("F:\TESTCL~1\test.avi",cachefile="H:\Temp\avi_0197468a3716844ade54f3f30f60eeda_491.ffindex",fpsnum=25)
# denoising
Deathray()
return last
Ii just get:
Error reading source frame 0: Avisynth read error: Deathray: Copy Y to Device status = 7 and OpenCL status = -36'
in Virtual Dub. Am I missing something? (I'm on Windows 10 and an NVIDIA Geforce GTX 980 Ti)

Cu Selur

[Jawed]

-36 is Invalid command queue, an OpenCL error.

7 is FILTER_COPYING_TO_PLANE_FAILED

I don't know what your test clip looks like, but please make sure that the video frame size is at least 8x8 in size.

NVidia support for OpenCL has not been very good. You may have to try a variety of drivers to get one that works. I think NVidia has been making changes to OpenCL support in its drivers recently, in theory to make it better.

Alternatively, well there's a bug in my code. This binary works on my current AMD system with up-to-date drivers. But sometimes the way AMD and NVidia compile code can show that there are subtle errors in C++ or OpenCL code. Which would be my fault :confused:

Move to a later frame and try a preview again.

Also, instead of trying to preview the video in Virtual Dub, encode a short clip. Use TRIM(1000,2000) to make a 1001 frame clip on a line just before the return last. Then send the script to your encoder.

Also try the parameter:

colorspace="YV12"

in FFVideoSource. I actually use FFmpegSource2, so I'm not sure if that parameter works for FFVideoSource. The video you supply to Deathray must be in a planar format, such as YV12.

[Selur]

Trim, doesn't help, encoding (with x264/x264/vpxenc) doesn't work (that is why I switched to Virtual Dub to check if something is wrong ;)).
Input is a avi with 640x352 (Xvid, Yv12). -> Video format should be planar Yv12. (you can get the test.avi from my google drive (https://drive.google.com/folderview?id=0B_WxUS1XGCPASUZibG5XZkRfeTg&usp=sharing), but the problem happens with any clip I throw at Deathray,... tested multiple SD, HD and 4k sources to make sure it's not an resolution issue)
Adding 'ConvertToYv12' before Deathray() and/or adding 'colorspace="YV12" ' to the FFVideoSource call doesn't help either.
-> Some OpenCL incompatibility is probably the best guess.
Will try older drivers later (currently using latest version: 361.43)

Cu Selur

[Groucho2004]

This works for me:
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio()
Deathray()


WinXPSP3
GTX750
Drivers: 350.12
OpenCL DLL: Khronos OpenCL ICD 1.2.11.0 (supplied with the driver)

[Jawed]

I've thought of another reason why this might not work with NVidia: Deathray requires OpenCL 1.1 including features that were marked for deletion in a future version.

NVidia's latest driver may have removed some or all of those 1.1 features that were marked for deletion (deprecated).

If that's true, then I would need to compile a newer version without those 1.1 features :(

[Selur]

"This works for me" -> doesn't work here, gives me the same error as any other source.

including features that were marked for deletion in a future version.
.. using depreciated option is always a bad idea ;)

[Jawed]

The deprecation occurred when OpenCL 1.2 was announced. After I'd written this software.

If both the driver and ICD versions reported by Groucho2004 fail to work on your 980Ti, then I'm afraid to say it prolly isn't deprecation that's caused the problem.

But it's impossible for me to rule that out, since NVidia might have decided to write software for Maxwell version 2 GPUs that deprecates 1.1, but Maxwell version 1 GPUs are using some other software.

If I fix deprecation it'll be when I have evidence of that being the case. And the fix will probably be in Deathray2 (not officially published but for those of you that can make Deathray work, google is your friend).

[Selur]

How can I check the ICD version that is installed with NVIDIA drivers, or should I install something from https://www.khronos.org/registry/cl/ manually?

[Jawed]

I'm afraid I don't know if it's safe for you to try to install another ICD. Hopefully Groucho2004 will be able to show you how to check your system configuration.

Or perhaps someone else who owns a GTX9xx card will be able to help, if their system works.

A couple more thoughts: do you have multiple GPUs in your computer? Do you have an Intel OpenCL driver installed?

[Selur]

Got an i7 4770k which comes with an HD 4600 and I didn't install any special OpenCL drivers,...
Installing latest drivers: https://downloadcenter.intel.com/download/25489/Intel-Graphics-Driver-for-Windows-10-15-40-4th-Gen- didn't help with Deathray.

[Groucho2004]

How can I check the ICD version that is installed with NVIDIA drivers
"%windir%\system32\opencl.dll" (or "%windir%\syswow64\opencl.dll") right-click on "Properties" -> "Version".

FYI - I had to manually copy the OpenCL.dll that comes with the driver to the system directory. It was the only way to get KNLMeansCL working. I don't know why the driver installer didn't overwrite the old version.

[Jawed]

Got an i7 4770k which comes with an HD 4600 and I didn't install any special OpenCL drivers,...
Installing latest drivers: https://downloadcenter.intel.com/download/25489/Intel-Graphics-Driver-for-Windows-10-15-40-4th-Gen- didn't help with Deathray.
This Intel driver, or the driver that this replaced, might be causing a problem.

I don't know if it is possible or safe to de-install the Intel driver. If the OpenCL component of the Intel driver is installed then it seems very likely that it is interfering (or that Deathray won't work it). Perhaps it's possible to install the Intel driver but not the OpenCL component?

Deathray asks the computer for the first GPU it can find, but only from the first OpenCL platform. The computer you are using almost certainly has both the Intel and NVidia OpenCL platforms available.

Is there a reason you have the Intel graphics driver installed?

(My CPU is AMD, so I only have a single OpenCL platform on my computer.)

[Jawed]

I have made a new test build of Deathray DLL:

https://github.com/JawedAshraf/Deathray/tree/Platform-Exclude-Intel

Try to download the new DLL and see what happens.

It looks at the vendor of the OpenCL platform. If it sees the word "Intel" in there, it skips past that platform.

The code is written so that it will crash if there are no platforms without the word "Intel" inside. Hopefully the NVidia platform will be found, and it won't crash, and Deathray will work :)

[Selur]

Nice! That seems to work!

Cu Selur

[Jawed]

Well, that's awesome. Glad to hear it worked.

[Selur]

May be adding and option to set a preference which platform to use (Intel,AMD, NVIDIA) might be a nice thing.

[Jawed]

It's already in my to-do list for Deathray2.

[kalehrl]

How does it compare to knlmeanscl?
I suppose they are quite similar.

[Khanattila]

How does it compare to knlmeanscl?
I suppose they are quite similar.


In fact I never checked, but I think there are differences.

[mastrboy]

Where can I find binary releases? The release page on github is empty: https://github.com/JawedAshraf/Deathray/releases

[Reel.Deel]

Where can I find binary releases? The release page on github is empty: https://github.com/JawedAshraf/Deathray/releases

Look for Deathray.dll in the list here: https://github.com/JawedAshraf/Deathray

[mastrboy]

Thank you, an oversight on my part being to used to github users using the release page.

[Magik Mark]

Do you have a 64bit version?