CUDASynth Filters

[eac3to_mod]

Thread split from KNLMeansCL: OpenCL NLMeans de-noising algorithm [2018-01-29]:-https://forum.doom9.org/showthread.php?p=1997163#post1997163

DGDenoise from DGToolsNV is actually a NLMeans denoiser.
That's true but a major update is coming shortly with much better performance in both speed and quality, so I would wait for that.

[kedautinh12]

That's true but a major update is coming shortly with much better performance in both speed and quality, so I would wait for that.

Nice, thanks

[ChaosKing]

DGDenoise from DGToolsNV is actually a NLMeans denoiser.

It has no temporal denoising only spatial mode.

[tormento]

It has no temporal denoising only spatial mode.


The author has expressed the intention to add temporal mode too. For more info come to DGTools support forum.

[ChaosKing]

The author has expressed the intention to add temporal mode too. For more info come to DGTools support forum.

I asked about this like 5 years ago. Don't think it will happen soon.

[eac3to_mod]

Don't bet the farm on that. A lot happens in 5 years, or one day.

[ChaosKing]

Maybe 2024 will be the year, fingers crossed :)

[eac3to_mod]

Is there a temporal denoiser whose performance impresses you? We're deciding on an appropriate algorithm. We will do it within DGSource() and not require an extra filter as BM3D does.

[ChaosKing]

I would say I had the best results with:
1. MVTools in combination with a prefilter (prefilter can also be mvtools)
2. knlmeans with ref parameter
3. bm3d is bit harder to use, but can give impressive results then used in combination with mvtools

[eac3to_mod]

Please, what is "with ref parameter"? I don't see such an option for KNLMeansCL.

[ChaosKing]

Sorry it's called rclip aka reference clip https://github.com/Khanattila/KNLMeansCL/wiki/Filter-description#advanced

[eac3to_mod]

Thank you. And what would the reference clip be?

[ChaosKing]

A filtered clip. Can also be overfiltered.

Depends on how good/bad the spource is. wref is also interessting:

float wref [default: 1.0]
Amount of original pixel to contribute to the filter output, relative to the weight of the most similar pixel found.

clip rclip [default: not set]
Extra reference clip option to do weighting calculation.

[LigH]

:thanks:

pinterf and eac3to_mod and everyone keeping free software competitive.

[LigH]

Sounds like if(a = b) instead of if(a == b) ... happened so many times to me :rolleyes:

[tormento]

pinterf and eac3to_mod and everyone keeping free software competitive.
There are so many to thank.

[ChaosKing]

It crashes in vapoursynth / vsedit and avisynth also does not work for me - illegal instruction. I have a Ryzen 5600G + 3070TI

[StainlessS]

Blankclip
Info

shows avisynth determined capabilites
https://i.postimg.cc/QKDcrBQh/Flags.jpg (https://postimg.cc/QKDcrBQh)

[eac3to_mod]

Good idea. I will compare to Info() source code.

[StainlessS]

Prev image was for i7-8700, is similar for i7-12700T,
BUT, for
i5-11400T
https://i.postimg.cc/18zwPndr/flags-i5-11400-T.jpg (https://postimg.cc/18zwPndr)

EDIT: There is supposed to be some kind of hack available to switch on AVX512 for 12 gen i7/i9 CPU, may need to switch off E cores in BIOS firmware.
Switched off E cores in my firmware, no dice, flags same as for i7-8700 again.
EDIT: The AVX512 hack is permanently disabled for 12 gen i7/i9, by intel.

[ChaosKing]

I re-downloaded it. Same error in Vdub2:
Avisynth open failure:
System exception - Illegal Instruction

Even if I simply use dgsource() without any path.

[tormento]

There is supposed to be some kind of hack available to switch on AVX512 for 12 gen i7/i9 CPU, may need to switch off E cores in BIOS firmware
Just for the initial batches, later they laser cut the capability.

[guest]

Just a thought...

Would it be easier to create one for AVX/AVX2, and a separate one for AVX512 ??

There's others out there do this.

[ChaosKing]

Why not check how other plugins solved it? Like here https://github.com/sekrit-twc/zimg/blob/71431815950664f1e11b9ee4e5d4ba23d6d997f1/src/testapp/cpuinfoapp.cpp#L32 (I just randomly searched for avx2)
Or a simpler one here which uses some parts from zimg https://github.com/Irrational-Encoding-Wizardry/descale/blob/8c53f5d1297dee286e5a854ae5731103614a0583/src/x86/cpuinfo_x86.h#L26

[tormento]

To be fair, AVX512 is supported on latest AMD and on Xeon processors only, making it a niche instruction set.

I love Intel compiler but it has a steeper curve than MSVC. I have the best performance results with builds from it.

I would be sad to see AVX2 only, as older processors are the ones that benefit most from GPU offloading.

If you can't cope the challenge, I will get over it.

Unfortunately that will prevent a lot of people not only from using the newer CUDA filtering (as it was before) but also from updating even the decode part of DGSource and that would be pure evil decision.

[guest]

Sure, I could make separate DLLs but it's just not something I want to do.

My reasoning is that to get the best out of DGDecNV you're going to have a good modern GPU together with a good modern CPU. AVX2 has been around since 2013 and I don't feel like I need to help out luddites that stick to things like Win XP. The luddites can stick with DGDecNV build 251.

I do plan to move this intrinsics code to the GPU at some point but for now if y'all want anything it's going to require AVX2. I'll just wait for any more people to weigh in before I give an AVX2 test build.

EDIT: Ah I see you were more wanting to get the AVX512 support. I thought about that but haven't decided. Once I go down the route of multiple DLLs, I might as well include no-AVX/AVX2. But I prefer not to go there. I'd rather switch to the Intel compiler and do things right.

I realise that AVX/AVX2 has been around for ages, but to put it into a bit more perspective...

I have several older Ryzen's (3950X, 5900X, 5950X) that only support AVX2 at best, and I also have a gen 13 13900KF CPU which is only AVX2 (changed after Gen 12), however, I have a Ryzen 7950X, which does support AVX512.

So for me, it would be a no brainer to have 2 variants.

Just out of curiosity, what is your preferred OS ??

I actually had to Google "luddite's", I like it.

I need to mention, that whatever you do, I probably won't be able to use it with my preferred encoding app (tried it in the past, was too much work, and there were other issues that are out of my hands, it's just the way the app works), and I didn't use KNLMeansCL all that much either.

[eac3to_mod]

Win 10.

What is the encoding app and why is it too much work? What issues were out of your hands?

[tormento]

Win 10.
Could you please reup test1? It was working just fine.

[tormento]

I don't have a build for that as it has evolved into test 2, and I didn't keep the zip file.
Test2 isn't working :D

[tormento]

Use test 1 if you just want the 33% speed-up.


That is the reason I was asking you.

@all somebody kept that?

[guest]

Win 10.

What is the encoding app and why is it too much work? What issues were out of your hands?

Good to know, I'm on W11 23H2, I personally just don't understand why so many ppl are still on W7....

Read my "sig"...

It's a L-O-N-G story, but basically, you had to import a new job, then do a fair bit of manually editing, adding & redoing things to get it to encode, but then the best function of the app did not work, so it's just a big waste of time :(

[eac3to_mod]

It's a L-O-N-G story, but basically, you had to import a new job, then do a fair bit of manually editing, adding & redoing things to get it to encode, but then the best function of the app did not work, so it's just a big waste of time Well, golly gee, I'd be happy to work with the author to make things integrate more smoothly. It's just a source filter like all the rest, after all. Do you have any links describing the issues? What is the "best function" that did not work?

[guest]

Well, golly gee, I'd be happy to work with the author to make things integrate more smoothly. It's just a source filter like all the rest, after all. Do you have any links describing the issues? What is the "best function" that did not work?

Well, Curly, the author is, IMO, a "piece of work", he is very set in his ways, and is VERY reluctant to add options that are asked about, repeatedly.

I was involved in a customised build of the app, that added many more filtering options, and keeping it right up to date, but the main behind the scenes "gut's" of the app are untouchable, it's written on a pretty old platform (can't recall atm).

So if I recall, you had to import a job, then also run that same job thru DGIndex (or whatever you use to get the .dgi file), then manually edit some commands, and hope for the best.

But, the best function of the app is Distributed Encoding, and I'm pretty sure it would not initiate that, so it was just a single PC encode, instead of multiple PC's helping.

I do recall someone, saying that the nature of DGDecNV didn't lend itself to the way that DE worked, it might have been someone from your neck of the woods, DG, that is.

So you'd have to play around with the app for yourself, I'd say.

Like I said, it's a LONG sad story.

[eac3to_mod]

Thank you for the explanation. With the latest DGDecNV the index is automatically generated so at least that hassle should be avoidable. But if the author is unwilling to do anything to move into the modern era we'll just let it go.

[tebasuna51]

test3 in CPU with only AVX2 crash with:

cuMemAlloc() failed

with:
C:\Portable\Avs\AVSMeter64 test.avs -gpu

test.avs only:

dgsource("test.dgi")

than work without problems with other DGDecodeNV.dll versions

[guest]

Thank you for the explanation. With the latest DGDecNV the index is automatically generated so at least that hassle should be avoidable. But if the author is unwilling to do anything to move into the modern era we'll just let it go.

Sad, but he's the epitome of a "luddite", anything other than W7, is no good....

[tormento]

Test3 gives me

System exception - Illegal instruction

again.

[eac3to_mod]

@tebasuna51

What is your GPU and how much memory does it have?
What is the resolution of the video?

Also your post is not clear. Does it fail with AVSMeter64 without the -gpu option? Does it fail with the -gpu option? Does it fail in VirtualDub?

Hey, we're making progress. It's not crashing with illegal instructions, except for luddites.

BTW, I was able to make a build configuration for AVX512, so management of multiple DLLs will be possible without too much pain. We already do it for 64 versus 32 bit.

[tebasuna51]

Only a cheap NVIDIA GeForce GT 1030 with 2 Gb GDDR5
Video resolution 1920x1040

[StainlessS]

Quote from recent post in another D9 forum, a little related and maybe of interest.


Now for Intel it's more drastic. Their little cores for Alder Lake were upgraded to support avx2 and new instructions were added because they couldn't do avx512 anymore on the whole system. They're large p cores could, but they disabled them.
People wondered when they would return, well Intel wants to rework avx for hybrid processors, but it won't support avx512 for a long time and probably won't support current cpus

I've got a Lenovo ThinkCenter M70Q gen 2 Tiny, with i5-11400T (35W TDP) that claims AVX512 (at least partial).
Also, a Dell Optiplex 7000 Micro, with i7-12700T (35W TDP) that professes not to support AVX512 [8 P cores {+ 8 hyperthreaded}, 4 E cores].

I find below a little interesting but have not as yet investigated it.

Software support

Alder Lake requires special support from the operating system due to its relatively unusual-for-x86 hybrid nature. For software unable to be upgraded, a UEFI-provided compatibility mode may be used to disable the E cores; it is enabled by the user turning on scroll lock.[29]

This problem has been fixed in a microcode update. The P and E cores now return the same CPUID when both are enabled. A different CPUID is reported when E cores are disabled and only P cores are enabled. The AVX-512 instruction set extension is implemented in the P cores but disabled due to incompatibility with the E cores.[32] Hackers have shown that it is possible to enable the AVX-512 instructions on the P cores when the E cores are disabled and an old microcode version is used.[33]
https://en.wikipedia.org/wiki/Alder_Lake#Dies

When using MeGUI x64 to encode Avisynth script, I'm finding that the i7-12700T machine is basically using only the 4 E cores to encode
(well a few % of the 8 P cores [+ 8 hyperthreaded logical cores] is also used), thats got me a little perplexed.
I'll havta look into it a little more next time I use the i7-12700T machine [I can turn off any number of E cores or P cores in BIOS UEFI Firmware].

EDIT: I'm using Win10 on all machines [I really dont want W11].
EDIT: Long ago, I use to used a program called something like "Prio" to set default Processor Affinity & Priority for certain applications [Addon TAB for TaskManager].
EDIT: This seems to be it, not tested in W10. [EDIT: Supposed to be W7+ compatible].:- https://www.prnwatch.com/prio/
EDIT: This seems to be a similar app to prio, Process Lasso {There is a limited free version}:- https://bitsum.com/
EDIT: 6 Tools to Permanently Set Process Priority in Windows:- https://whatsoftware.com/permanently-set-process-priority-in-windows-task-manager-with-prio/

EDIT: Flags determined by Avisynth for i7-12700T with E cores disabled in BIOS(same as for i7-8700) : No change to flags whether E cores/Scroll-Lock enabled or disabled.

Blankclip
info
[clickMe]
https://i.postimg.cc/5HmN7n44/Flags-i7-12700-T.jpg (https://postimg.cc/5HmN7n44)

And for i5-11400T [Has Avx512 flags]
https://i.postimg.cc/18zwPndr/flags-i5-11400-T.jpg (https://postimg.cc/18zwPndr)

EDIT: i5-12500T does not have AVX512 either and as it does not have E cores, then does not seem to have any reason to not support AVX512 (unless its to avoid users buying i5 rather than i7/i9 to get AVX512 without the P/E cores hybrid incompatability problem).

[guest]

Poor old Curly... he's bending over backwards to get this to work, which it must be for him, on his system, then whenever others use it, it errors due to different OS's, snd dodgy cheap GPU's, etc.

[eac3to_mod]

@teba

That GPU should be fine. Please grab the download again as I rebuilt it completely just to be sure.

Fingers crossed!

[tormento]

I clearly said it requires AVX2. I may support AVX later.
Somewhere I read AVX2 and below and it stuck in my mind.
I found test1 on backup and re-uploaded it.

:thanks:

[eac3to_mod]

Poor old Curly... he's bending over backwards to get this to work, which it must be for him, on his system, then whenever others use it, it errors due to different OS's, snd dodgy cheap GPU's, etc. That's the thing, it works great for me, especially the AVX512 build. i7-11700K

I wouldn't call the 1030 dodgy but I get your point.

[StainlessS]

I did try on my Lenovo Tiny (i5-11400T) but got platform code 126 (missing requirement),
then realized that none of my 35W TDP machines have discrete GPU, duh.
Works in i7-8700 without probs.

[eac3to_mod]

Ooh, finally it works for someone else. Happy Day! :p
You scared the poop out of me with your first sentence. Did you do that on purpose? ;)

And thank you for the info you posted previously.

OK, I'll go ahead and make the per-CPU builds and make them available.

[ChaosKing]

Test3 works for me too. With dn_enable=2 I get a message box: avx512

[StainlessS]

Did you do that on purpose?
Nope, did not think of it.

How many people will hate me if I require AVX2?
If X people hate you, and Y people will hate you for AVX2 requirement, then would be X + Y people. :)
{EDIT: Assumes that X and Y are distinct sets}
Its your plug, do whatever you feel happiest with, improve as time permits.
The multitude of different instruction sets are a nightmare, dont forget MMX and same era instruction sets, horrible!

EDIT: The "platform code 126 (missing requirement)" thingy might be something else,
I'm not sure that the AVS setup on that machine is fully functional, not used it in a while and changed things about.
It though does not have GPU and so would fail anyways.
The 35W TDP machines are only babies, ~ 7" x 7" x 1.5", are low power, quiet, and with a bunch of them, can just let them go encode till they're done, and main i7-8700 machine free to do other stuff.

EDIT:
Test3 works for me too. With dn_enable=2 I get a message box: avx512
Snap, me gets dat too for i7-8700 {with GTX1070+8GB} without AVX512.

[eac3to_mod]

@ChaosKing

dn_enable=3, dn_quality="best" works pretty good with the sample you gave me back in early 2021, methinks. I'm a pack rat, I keep everything.

[eac3to_mod]

@teba

For the avx512, there are lots of different subflags, and different CPUs have different combos of them. I'll have to check specifically for the ones I need. Give me a while. For now grab the AVX2 DLL.

Your benchmarks look fishy. I'll try here. Also, bench with dn=0 just to see raw frame serving.

EDIT: Hey, where'd the benchmarks go?

[tebasuna51]

Work now for me with test3 DGDecodeNV_AVX2.dll,
but is very slow:

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r4062, master, x86_64) (3.7.3.0)

Number of frames: 507
Length (hh:mm:ss.ms): 00:00:21.146
Frame width: 1920
Frame height: 1040
Framerate: 23.976 (24000/1001)
Colorspace: YUV420P16

1) Old DGDecodeNV.dll dgsource("test.dgi").DGDenoise()

2) Test3 DGDecodeNV_AVX2.dll dgsource("test.dgi",dn_enable=1)

3) Test3 DGDecodeNV_AVX2.dll dgsource("test.dgi",dn_enable=3)

1) 2) 3)
----------------- ----------------- -----------------
Frames processed: 507 (0 - 506) 507 (0 - 506) 507 (0 - 506)
FPS (min | max | average): 12.05|23.99|23.49 3.810|4.724|4.656 1.512|9.359|4.579
Process memory usage (max): 393 MiB 598 MiB 627 MiB
Thread count: 22 19 19
CPU usage (average): 8.0% 8.2% 8.3%

GPU usage (average): 89% 98% 97%
VPU usage (average): 3% 1% 1%
GPU memory usage: 849 MiB 856 MiB 834 MiB
GPU Power Consumption (average): 24.3 W 29.9 W 29.7 W

Time (elapsed): 00:00:21.585 00:01:48.902 00:01:50.716

[tebasuna51]

Previous version was fast with dn_enable=1 (not work with 2 or 3)

Frames processed: 507 (0 - 506)
FPS (min | max | average): 6.357 | 9.447 | 9.260
Process memory usage (max): 524 MiB
Thread count: 17
CPU usage (average): 8.3%

GPU usage (average): 97%
VPU usage (average): 2%
GPU memory usage: 825 MiB
GPU Power Consumption (average): 29.7 W

Time (elapsed): 00:00:54.754

[eac3to_mod]

OK, everything is fine. :rolleyes:

Here's the deal.

We had searchw 5/7/9.
Now we have good/better/best.

The good/better/best are scaled higher than searchw (this was discussed at DG forum). That means "good" is already equivalent to searchw = 9. So for accurate comparison to old searchw = 9 you have to choose "good".

And you have to turn off temporal to compare to old.

I hope it is clear. I'll give my numbers shortly.

[eac3to_mod]

For SSE4.1 no AVX2 or AVX512:

Old with searchw = 9: 475 fps (spatial only)
New with dn=1, quality good: 522 fps (spatial only)(good = searchw 9)
New with dn=3, quality good: 485 fps (spatial and temporal)(good = searchw 9)

For spatial there is a good speed increase. You get temporal denoising for free while still being a little faster. And you have the higher qualities of better and best available that you didn't have before. And don't forget that raw frame serving is 33% faster too in my tests.

Will give AVX numbers shortly.

[eac3to_mod]

AVX2:

Old with searchw = 9: 475 fps (spatial only)
New with dn=1, quality good: 524 fps (spatial only)(good = searchw 9)
New with dn=3, quality good: 505 fps (spatial and temporal)(good = searchw 9)

AVX512:

Old with searchw = 9: 475 fps (spatial only)
New with dn=1, quality good: 524 fps (spatial only)(good = searchw 9)
New with dn=3, quality good: 508 fps (spatial and temporal)(good = searchw 9)

So AVX is a big step up. AVX512 is very minor over AVX2.

AVX is used only for the temporal denoising. When I move it to the GPU it will get even better. I did it this way to leverage Avisynth+ frame caching and secondarily because I could do it quickly. I'll have to cache myself to move it to the GPU.

Another thing not to forget is that the more filters in the chain, the more relative speed improvement there will be, as more and more expensive PCIe transfers are avoided. This is why I intend to add more filters, even those supplied by third parties. If not too difficult I'll even port them to CUDA. This case with one filter doesn't showcase the CUDASynth philosophy, but it already shows signifcant gains. The filter chain with HDR to SDR and denoising starts to get quite impressive, for example.

[tormento]

Only a cheap NVIDIA GeForce GT 1030 with 2 Gb GDDR5 Video resolution 1920x1040
Doesn't this important progress in CUDA filters deserve its own thread?

Perhaps much better not to confuse people.

[tormento]

Here's the deal.
Eagerly waiting my queue to finish to try it.

Have dn_strength and dn_cstrengh linear or non linear effects? In the second case, how is the "curve" shaped?

When you will have time, please consider an arbitrary number for searchw.

[eac3to_mod]

@teba

Can you please post a screen shot of Info() for your CPU? It will help me to address your issue with AVX512, though AVX512 is not a big improvement over AVX2 in this application. Thank you.

[tormento]

I could consider another bump to ultra but, honestly, when I tried that I could not perceive any improvement.
You tried with really noisy material?
The technical reason is that CUDA loop unrolling can only be done when the loops have constant limits.
Can I pretend to have understood? :p

About dn_quality, dn_strength and dn_cstrengh: what if I want to use different "values" for spatial and temporal denoising?

Do you plan to release a standalone cuda filter, detached from the DGSource video decoding?

Some videos, such as AVC 10 bit or with insane bframes/ref numbers can't be decoded in hardware.

[tormento]

Decoding only tested and working on i7-2600k, i.e. AVX only.

I will let it run for a whole night on different material to see if there are hiccups.

[ChaosKing]

Was is "temporal_sample.mkv"? Dropbox says it was 5 years ago... time flies

[eac3to_mod]

Yeah, that's the one. ;)

[tormento]

The DGDenoise() in the new DLL already uses the new stuff.
My two cents for the future developement of the separate function.

If possible, now that we have 2 filters (and more I hope will come) that works in GPU memory, let's have a single function that can apply the different filters to the same frame timespan internally (x number of frames in a GPU buffer?).

To be clear: let's say we have to convert HDR to SDR and denoise it.

It would be a pity to have to call 2 different functions that moves the frame back and forth.

I hope to have explained my idea.

[tormento]

Filters that are able to run in place do so.
Thanks for your explanation.

:thanks:

[DTL]

Another option is to move to the Intel compiler, which supports runtime dispatching of code for different architectures. However, it is still possibly a bridge too far for me.

Typically enabling AVX512 in the C-compiler makes only very few peerformance benefit. So you can simply make several builds for SSE2/AVX2/AVX512 executable and mark for users. So each user can select the executable file for current machine to run.

If you really make some parts of program for any SIMD - you do not need to enable its architecture support in the compiler. It is auto-selected by compiler from program text. And all selection of used program parts performed manually using AVS returned flags (so user can lower CPUcaps flags for debug if requied using script function).

So you build single executable for SSE2 and it will be compatible with most current chips and can have AVX512 parts where required. The performance penalty typically very few. Most benefit typically from using very advanced compilers like LLVM (but also not as great as manual programing for SIMD).



So AVX is a big step up. AVX512 is very minor over AVX2.



If your application is not hardly limited by host RAM performance the AVX512 can provide 2..4x+ performance gain if used properly. I see no sources of the current AVX512 implementation of this software. Typically for AVX512 you need to grab workunit about 4x larger in size over AVX2 (32x512 vs 16x256 bit datawords) into register file of AVX512 SIMD co-processor and organize compute with attempt to use new shorter instructions and combine several operations for superscalar processing by several dispatch ports in parallel. Some examples are in the Asd-g sources of AVX512 for vsTTempSmooth - https://github.com/Asd-g/AviSynth-vsTTempSmooth/blob/master/src/vsTTempSmooth_AVX512.cpp

Unfortunately current C-compilers have too few AI inside to do such job internally (auto loop-unroll with auto-size of work unit size for each SIMD arch and auto-combine instructions for super-scalarity) so programmers need to write large blocks of repeating intrinsics and adjust workunit size for each loop pass manually for best performance at target SIMD arch.


The temporal denoising has no parameters to tweak. It is a modified temporal median filter rolled quickly because I had to make ChaosKing happy.

There is also some sort of temporal median implementation in AVX2 and AVX512 - https://github.com/Asd-g/AviSynth-vsTTempSmooth/blob/master/src/vsTTempSmooth_pmode1_AVX512.cpp
https://github.com/Asd-g/AviSynth-vsTTempSmooth/blob/master/src/vsTTempSmooth_pmode1_AVX2.cpp

With temporal radius > 2 (1?) it runs several times faster over old implementations of temporal median (in MedianBlurTemporal()) and uses hardware gathering instructions of AVX2/AVX512 to keep all computing in SIMD domain without fallback to scalar gathering loop (greatly destroying total idea of SIMD parallelism).

But anyway it is only temporal median computing implementaton - for real moving pictures denoise it must use motion-compensated frame sequence as input (or the performance/quality will be very poor using simple 'thresholding skip' method). So if CUDA will provide ME data from hardware ASIC and you can use it to make motion compensated frames you can use temporal median as final output stage (or simple weighted averaging as in MDegrain). As I read NVIDIA also provide some API for CUDA-programmers to get ME data from MPEG encoder ASIC where available. Other implementation is for DX12.
See https://docs.nvidia.com/video-technologies/video-codec-sdk/11.1/nvenc-video-encoder-api-prog-guide/index.html
Motion Estimation Only Mode

NVENC can be used as a hardware accelerator to perform motion search and generate motion vectors and mode information. The resulting motion vectors or mode decisions can be used, for example, in motion compensated filtering

Also some temporal noise reduction in openCV ? Hardware accelerated by NVIDIA ? Or any OpenCV device ? https://docs.nvidia.com/vpi/sample_tnr.html

[tebasuna51]

@teba

Can you please post a screen shot of Info() for your CPU? It will help me to address your issue with AVX512, though AVX512 is not a big improvement over AVX2 in this application. Thank you.
Here is, with 8 GB DDR4:

[tebasuna51]

Can you please change the title to simply "CUDASynth Filters", as there will be more filters than you listed...
Done.
BTW you are the thread owner if you want change it one more time.

Your CPU does not support AVX512, so you'll have to run the AVX2 DLL.
Of course.

[tormento]

>8 hours of encoding and no video decoding hiccups at all. :cool:

@eac3to_mod

I suggest to stick to AVX/AVX2 now, as AVX architecture is going to change in 6 months with Arrow Lake and its AVX10 instruction set.

I doubt, anyway, that Intel will reintroduce AVX-512 on consumer products, until it will solve its power consumption problems.

Perhaps the topic title could be a bit misleading. Cudasynth was a 2013 (i.e. dead) project about VST synthesizer that offloaded processing to Cuda-enabled devices.

Plus, if I am not wrong, it was the title of the DG branch of AVS and its not the case here, as we are still using standard AVS+.

Perhaps 'DG CUDA filters'? :)

[tormento]

Thank you for your interest and contributions.
You are the most welcome and thank you for your hard work in facilitating our hobby (for someone) and job (for others).

[Julek]

For AVX selection the VLC2 lib (https://github.com/vectorclass/version2) can help, you can use instrset_detect() as in this example (https://github.com/HomeOfVapourSynthEvolution/VapourSynth-TCanny/blob/14ac2ceeb59afc7089974d0ae233fe8d0ea183c8/TCanny/TCanny.cpp#L585).

[eac3to_mod]

@Julek

Thank you for the links.

[tormento]

1) What is the bitdepth of internal processing? Does it use 8 bits for 8 bit source and 16 bits for >8 bit source?

2) If the first, could we force to use 16 bits internally for more precision and output 8 or 16 bits?

3) If we internally resize the video, the denoise/hdrtosdr will apply to the resized or the full size?

There is an error in Notes.txt:

dn_cstrengh (default 0.0)

instead of

n_cstrenght (default 0.0)

Beside that, the denoising quality is great! I will try it with some more footage.

I will test DGHDRtoSDR now.

Please consider to add HLG output to DGHDRtoSDR. I know it will be trivial for you.

[tormento]

Just playing around, I have found something strange.

Look here (https://slow.pics/c/Zhk7nyKj).

First script (crop, resize and SDR conversion)

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("M:\In\- 1_58 Il labirinto del fauno 4k\fauno4k.dgi", ct=48,cb=48,cl=0,cr=0, rw=1920,rh=1032, h2s_enable=1,h2s_white=2000)

Second script (first+denoise 0.05)

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("M:\In\- 1_58 Il labirinto del fauno 4k\fauno4k.dgi", ct=48,cb=48,cl=0,cr=0, rw=1920,rh=1032, dn_enable=3,dn_strength=0.05,dn_cstrength=0.05,dn_quality="best", h2s_enable=1,h2s_white=2000)

Third script (first+denoise 0.10)

LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("M:\In\- 1_58 Il labirinto del fauno 4k\fauno4k.dgi", ct=48,cb=48,cl=0,cr=0, rw=1920,rh=1032, dn_enable=3,dn_strength=0.10,dn_cstrength=0.10,dn_quality="best", h2s_enable=1,h2s_white=2000)

I can see 2 things:

1) color artifacts on scene change
2) sort of "duplicate" image superimposing, a bit hard to see on screenshots but really evident in motion
3) when denoised, video brightness is different
4) posterization (look at the face of the girl) but perhaps 0.05 denoise is too high for that scene

dn_enable=1 (spatial) works perfectly
dn_enable=2 (temporal) gives both color artifacts and smearing
dn_enable=3 (spatial+temporal) gives both color artifacts and smearing, exacerbating color artifacts
dn_cstrength=0.0 (no chroma denoising) solves the color artifacts but not smearing
dn_strength=0.00,dn_cstrength=0.05 solves nothing :)

Here (https://pixeldrain.com/u/eeN317Qo) you can find the test clip (on the left there is the download button).

Frames are 83-85.

[tormento]

If I recall well, some time ago DGSource had the capability to force input to 16 bitdepth. Could you introduce it again so we can apply all the internal filters with 16 bit precision?

The resizing applies to the decoded frame from NVDec and before any further filtering.
That is a blessing (speed) and a damnation. Noise, especially the digital/postproduction one, can be single pixel grain that would "smudge" the image when resized. It would be nice (as I said on other board) to be able to apply the resizing when we need. And what about HDRtoSDR, is it applied before or after denoising? I think that sorting out a way to order filters would be nice, the more you add, as it can lead to different results. Take, as example, all the various deinterlacing/decimating ones.
there is DGPQtoHLG, which could be considered for integration
I completely forgot that filter. If integrated it would be a great thing.
Thank you for the clip, which will be useful for my testing.
As always you are the most welcome.

[anton_foy]

Great to see this coming up!!!
I dont know if this is of any help but:

https://github.com/AN3223/dotfiles/blob/master/.config/mpv/shaders/nlmeans_temporal.glsl

Uses some kind of motion estimation for the temporal denoising.

[eac3to_mod]

Thank you for the link. I don't grok shaders and it claims to be buggy with no releases, so it won't be of much help.

[anton_foy]

Yeah I figured as much. Looking forward to the next version with upped temporal denoising quality. The temporal part I think is the most important as it is motion we are dealing with.

Edit: when it comes to spatiotemporal denoisers I think FFT3D renders pretty good results but the temporal part should have been more developed and one side effect of the denoising is banding/blotches issue which for example vaguedenoiser does not suffer from (my favourite spatial denoiser btw.).

[DTL]

Great to see this coming up!!!
I dont know if this is of any help but:

https://github.com/AN3223/dotfiles/blob/master/.config/mpv/shaders/nlmeans_temporal.glsl

Uses some kind of motion estimation for the temporal denoising.

Estimation is only begining of the process. Before temporal denoise frames must be backward motion compensated (so any before and after frames motion compensated to 'current' frame while keeping all noise/distortions from that frames) and after compensation it is finally passed to some denoise engine (median statistical based or simple arithmetic mean weighted blending).

So practical temporal denoise pipeline is of 3 stages:
Motion Estimation -> Motion Compensation -> Denoise Stage.

For better results stages maybe recursed (multi-generation motion estimation refining where previous generation denoised frame returned as current or reference frame to motion estimation analyzer). The ME stage is mostly compute-loaded and MC and DS are simple enough but requires large memory subsystem performance.

So with full hardware accelerated temporal denoiser it is expected ME data from MPEG encoder hardware ASIC (NVenc in the NVIDIA case) and MC and DN stages are shader-based dispatched at the general purpose GPU cores connected with high-bandwidth GPU RAM.

It is possible to design ME stage (simple MAnalyse shader-based) at the general purpose GPU compute cores and it may have better quality (same as onCPU MAnalyse) but performance maybe lower in compare with MPEG hardware ASIC. Also as I remember there is some already working ME of simple MAnalyse on CUDA - it is copied in pinterf repository with CUDA filters - in KTGMC part https://github.com/pinterf/AviSynthCUDAFilters/tree/master/KTGMC .

Where https://github.com/pinterf/AviSynthCUDAFilters/blob/master/KTGMC/MV.cpp looks like simplified MSuper+MAnalyse implementation on CUDA with real MVs search. So for complete temporal denoise only simple MC+DN parts need to be designed for CUDA too (like copy of old simple MDegrainN).


Those of you that were around in the early days of desktop image processing starting with VirtualDub filters may remember a filter called TemporalCleaner by Jim Casaburi. The concept was simple:

---
For each pixel in the current frame, if the difference between the previous frame's corresponding pixel and the current pixel is below a threshold then replace the current pixel by the average of the previous and current pixels, otherwise keep the current pixel.
---



It looks the most advanced implementation of these ideas currently in vsTTempSmooth(). But without motion compensation it still very poor in quality or can only remove very slight visible noise (though it still visible help to MPEG encoder in some use cases).

[DTL]

vsTTempSmooth is sort of advanced by usage of some weighting from close temporal radius (higher weight) to far samples (optional lower weight). So it attempts to lower possible motion blur without using motion compensation.

If you simply try any method of motion detection and skip denoise in these areas it will cause significantly uneven denoise over the total frame. Only mostly static areas will be denoised. The purpose of motion compensation not only saves from motion blur but also makes denoising of moving areas possible. So it is the key operation for denoise of moving pictures in comparison with static images denoising.

" based on the long-dead CUDA-enabled Avisynth."

Yes - it uses CUDA (GPU) memory management by (some branch of) AVS. So a programmer of a separate filter must rewrite it for its own GPU memory management. As for quality you can check classic mvtools old versions like 2.7.45 or even more old. I think the main algorithms of MVs search (see PlaneofBlocks class) have not changed significantly from the very old mvtools designs and at some point were ported into that KTGMC project.
The main difference is that before Win10 and the main effort by Microsoft to make ME API for hardware ASICS available for end users we only had the software-based ME engines (onCPU or some onGPU implementations like that KTGMC project). And after Win10 we finally have compatible ME API from hardware accelerators at end-users GPU chips. Maybe with CUDA SDK NVIDIA provides some UNIX-compatible ME API too for CUDA-software designers.
So currently we have 3 different open source and public freeware API for most compute-loaded ME processes:
1. onCPU mvtools
2. onGPU CUDA-based KTGMC
3. onGPU hardware ME ASIC accelerated from MPEG encoder via DX12 or CUDA-API.

All 3 methods return compatible MVs format with precision up to qpel. The only limitation of hardware ASIC typically only blocksize of 8x8 or 16x16 and colour format of NV12 (YV12). Software implementations can support any blocksize and bitdepth and colour format searches.

Method 3 allow to use most of possible at the enduser machine resources: hardware MPEG ASIC (for ME) + GPU general purpose cores (for MC+DN) + CPU general purpose cores for other processing so can provide max performance in theory.

[eac3to_mod]

Alice in Wonderland.

[LigH]

:confused:

I may be missing something; but reading such a reply which seems to lack respect for another person's efforts could discourage me from helping any further...

[tebasuna51]

I accidentally deleted the thread trying to edit my first post. You can either restore it or not I don't really care. It won't be so useful after I delete all my posts. If they want to take over my thread let them have it.

I will only be posting about eac3to from now on.

I restore the thread, for preserve users contribution. But I closed it because owner don't want continue posting here.