DG NV tools

[tormento]

Would you like to suggest any specific spatial denoising algorithm that I could implement?
I'd leave the speech to Didee, one of the avisynth man I was tought most. My dream is MVTools2 CUDA ;)

[Groucho2004]

* Fixed a race condition between CUDA init and filter deinstantiation that could cause
a crash when DGDecodeNV is instantiated and then deinstantiated without a call
to GetFrame(). Some third-party applications do that to get the video clip properties
returned by an Avisynth script.

http://neuron2.net/dgdecnv/dgdecnv.html

Nice. I just encoded a MPEG-2 sample with Procoder 2 for the first time using DGDecNV. :)

[Guest]

Good news! Thanks for the test result and your valuable contribution to fixing this nasty little bug.

[LigH]

Now I wonder if DGAVCDecDI might have a similar issue. Trying the batch encode with ivfenc on an AviSynth script which uses DGAVCDecDI instead of DGDecNV (I have only a 6800 GS at home), I get pseudo-crashes too (as described in the former report (http://forum.doom9.org/showthread.php?p=1433633#post1433633) - just analoguely).

ED_0360_700_ivf.batE:\Programme\ivfenc\ivfenc.exe --codec=vp8 --passes=2 --pass=1 --fpf=ED_0360.stats --best --threads=1 --token-parts=1 --end-usage=0 --target-bitrate=700 ED_0360_DI.avs ED_0360_700.ivf
E:\Programme\ivfenc\ivfenc.exe --codec=vp8 --passes=2 --pass=2 --fpf=ED_0360.stats --best --threads=1 --token-parts=1 --end-usage=0 --target-bitrate=700 ED_0360_DI.avs ED_0360_700.ivf
E:\Programme\MKVtoolnix\mkvmerge.exe -o ED_0360_700.webm -w ED_0360_700.ivf ED.ogg

ED_0360_DI.avsLoadPlugin("E:\Programme\DGAVCDecDI\DGAVCDecodeDI.dll")
DGSource("ED_1080_CRF06_DI.dgi")
Spline36Resize(640, 360)

But that should rather be discussed in the DGAVCDecDI thread (http://forum.doom9.org/showthread.php?t=156238), I believe.

[Guest]

But that should rather be discussed in the DGAVCDecDI thread (http://forum.doom9.org/showthread.php?t=156238), I believe. Yes, please. I'm aware of some crashing issues on random access that I am working on.

[jpsdr]

This version fixes the virtualdub crash problem jobs i've reported. Thanks.
As soons as i can, i'll check if it fixes the crash with SetMTMode(5) under MT avisynth.

Edit :
SetMTMode(5) apparently now works.

[Guest]

Fantastic. Thank you for your test results!

[jpsdr]

Yes.... but... my test crash at around 50% of a 100 frames video with a kernell32 error... on QTGMC with SetMTMode(5,6) (and SetMTMode(2) just after DGDecodeNV and before QTGMC) (with avisynthx64).
Nevertheless, maybe 3GB of memory is not enough for doing 6 thread on a 1080i video with QTGMC...

[Guest]

If you can get a DrWatson dump of the crash it may help us to find the cause.

[jpsdr]

... I don't know how to do that.
But when i take a look at the task manager afterward, the pic memory usage was at 3GB...

[Didée]

Quick report on SetMTmode(5) ...

Yes, with 2026 it generally works. That's a big and important improvement! The only drawback seems to be that it somehow uses more memory. It's not the "private bytes" and not the "working set" in Taskmanager, those values are about the same. It's the "virtual memory" which seems to be ~200 MB bigger, compared to using mpeg2source.
Now, 200MB is not quite trivial ... I assume that's the reason why some scripts with SetMemoryMax(bigvalue) still crash, though they work with mpeg2source: private bytes and working set are sufficiently below 2GB, but the virtual memory could exceed. (Though, I never saw that value actually exceed ... the crashes came earlier, while the memory value was increasing quickly.)


Edit:

Some more toying-around reveals it is related to the number of instances of DG[Multi]Source. Each single instance is using a good bunch of memory. (The above numbers happened to be with 3 source calls present.)

With a script like this:

SetMTMode(5,6)
SetMemoryMax(1024)

v1 = DGsource(...) # or mpeg2source, respectively
v2 = ...
v3 = ...
...etc...

v1 + v2 + v3 + ...

SetMTMode(2)
TGMC_b2(1,1,1)

return( last )


Trying with 1, 2, 4, 6, 8, 10 source calls, TaskManager shows the following numbers for "virtual memory" of VirtualDub 1.9.9 :

source mpeg2source DGSourcce
calls

1 1'186'116 1'250'288

2 1'209'072 1'320'428

4 1'252'808 1'454'332

6 1'297'636 1'584'588

8 1'342'396 1'716'780

10 1'385'896 1'845'544

Seeing that, it becomes understandable that DGDecodeNV runs out of memory much sooner than mpeg2source.


It's very good news that SetMTMode(5) finally works.
Now, if you could find a way to minimize allocation of virtual memory, it would be even better. :)

[crasus]

Dear neuron2,

It so happens that sometimes after demuxing the .h264 from a Bluray with eac3to (latest) I see a certain number of frames in the log (say 128540) and after indexing with dgdecnv 2024 (and other versions in the past) I see another number in its log (say 128526).

May I ask what's the source of this and I am doing something wrong somewhere? I do apologize if it's already been answered or asked.

All the very best and warmest for your excellent software (which saved me time ... more than once!)

Dear neuron2,

An addition to my earlier comment. I tryied to index the said file (along with another one from a very different Bluray but with the same situation under 2026) with older DGIndexNV versions.

I tested with all the versions on your website at the present : from 2020 to 2026. From 2021 to 2026, the DGIndexNV log showed a different frame count than eac3to. BUT! 2020 popped up the correct frame number. So, I searched around the web for more and came across DGDecNV 2012 and tried that as well. Yet again, the correct number of frames. It would seem that the problem started with 2021.

Do let me know, what I might do to help pinpoint this possible issue.

Warmly,
Crasus

P.S. : For reference, Win7 x64 with GT 240 (258.96 driver) and the latest eac3to.

[mini-moose]

I noticed the recent versions of dgavcnv don't come packed with the cuvid server. I was always just overwriting the old files so all my old ones were kept so I didn't notice up till now. Is it not required anymore ?

if it was discussed before apologies in advance.

[Sharktooth]

cuvid is not needed anymore with recent DGNVTools.

[mini-moose]

awesome. thanks Sharktooth. does this means no more crashes if I use rdp or screen saver ?

[royia]

Sure they are capable of it but these things are not currently exposed in the CUVID API.

Since I now know how to write postprocessing functions that run on CUDA (I use it for the NV12->RGB conversion in DGIndexNV), I could contemplate writing some filters. At first, spatial only. Would you like to suggest any specific spatial denoising algorithm that I could implement?

Any decent Edge Preserving Noise Reduction would be great.
Something like hq3dnd (Or whatever the NR Plug IN is called) would be a killer feature.
I would buy it instantly.

[LigH]

ivfenc 1.5 (Nic's AVS mod) encodes without any pseudo crash dialogs in 2 passes using DGDecNV 2026.

[Sharktooth]

awesome. thanks Sharktooth. does this means no more crashes if I use rdp or screen saver ?
dont know. never had those problems since i use no screen savers and i dont use rdp for that purpouse.

@neuron2: sorry for the delay. im really busy and we're also in the process to release a new megui stable build, so ill upload the clips as soon as i find 10 minutes to do that...

[tormento]

Little OT (please Neuron2 don't ban me, simply delete this post if needed, it comes from the hearth =P)

It's a pleasure to see here great programmers like you, Didee and Sharktooth "work" for us, completely free.

Thanks, guys.

[hydra3333]

Just wondering ... how about a deblocking filter. It's sort of denoising, in a way :)

[XadoX]

I have a Nvidia Quadro NVS 290. While recoding a h264 Video with DGdecNV and x264 and watching a Video on YouTube or in MPC-HC my PC freezes.

[Hagbard23]

I have a NVIDIA 9500GS.

I can get DGTools2026 working with MPEG2Files, only when i use "Use_D3D=1" (in the *.ini file). Both, DgIndexNV and DGSource() are working quite well.

But when i load *.x264 files into DGINDEXNV, it crashes with the following error message:

"Failed to start GPU Decoder" Error#100

Maybe it is the Graphic Card, which is garbage... ;)

BTW: What is the D3D-Mode all about - what exactly is the difference and disadvantages to native driver support? I know that there should be disadvantages, but i am just curious what the technical background is...maybe someone can answer this question...Thanks..

[gahz]

just wanted to thank you. whatever you did fixed my problem with having to open the preview window every time i opened a .avs in megui!

quick question, though i dont know if it relates to dgtools or not. sometimes i'll open up one of the various tools in megui while i have an encode going and it will crash out megui but if you look at task manager the encode is still running. it eventually finishes up at the time it's supposed to and the file is intact. any clues as to what's going on?

[Zathor]

quick question, though i dont know if it relates to dgtools or not. sometimes i'll open up one of the various tools in megui while i have an encode going and it will crash out megui but if you look at task manager the encode is still running. it eventually finishes up at the time it's supposed to and the file is intact. any clues as to what's going on?

I think it is more related to MeGUI. Nevertheless what kind of action do you start when it crashs (does it involve opening an avs file with DGDecodeNV.dll?) and how much graphics memory do you have? Have you tried to monitor the usage with GPU-Z? And what kind of crash is it? Do you have any crash report?

@neuron: move the post to the GUI forum if necessary.

[Hagbard23]

Neuron2 - can you add the "FieldOrder" in the Stats-Window again? Or is it technically impossible?

[Guest]

It's possible. I'll add it to my to-do list. I also need to have DGecodeNV set it properly for Avisynth.

Thank you for the suggestion.

[tormento]

Neuron2, if any spare time, please make DGIndexNV remember in and out directory separately. Sometimes the ripping source drive is one, the output project one is another.

And the filling status in in the Windows7 bar (a-la-MeGui to tell one), would be very useful if audio is off (hint: office) and the demux is long...

Thanks ;)

[gahz]

I think it is more related to MeGUI. Nevertheless what kind of action do you start when it crashs (does it involve opening an avs file with DGDecodeNV.dll?) and how much graphics memory do you have? Have you tried to monitor the usage with GPU-Z? And what kind of crash is it? Do you have any crash report?

@neuron: move the post to the GUI forum if necessary.

hiya zathor, i can tell you the easiest way to reproduce this (well at least for me):

1. load 5 jobs into megui
2. close megui, then open megui
3. hit start

at this point megui should gray out. you then have no option but to close megui in which case you get a crash window with this error:

Problem signature:
Problem Event Name: APPCRASH
Application Name: MeGUI.exe
Application Version: 0.3.5.12
Application Timestamp: 4c8d3471
Fault Module Name: nvcuda.dll_unloaded
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 4c379085
Exception Code: c0000005
Exception Offset: 000000000945564f
OS Version: 6.1.7600.2.0.0.256.48
Locale ID: 1033
Additional Information 1: 2264
Additional Information 2: 2264db07e74365624c50317d7b856ae9
Additional Information 3: dbaf
Additional Information 4: dbaf93d46593b450aa6261886a14bd19

after it closes down you will notice x264 is still running in taskmanager and cpu usage confirms that it is indeed encoding

script i use if it mattters:

# Set DAR in encoder to 16 : 9. The following line is for automatic signalling
global MeGUI_darx = 16
global MeGUI_dary = 9
LoadPlugin("C:\Program Files\Megui_x64\tools\dgindexnv\DGDecodeNV.dll")
DGSource("D:\8\00067.dgi",fieldop=0)

this happens in both 32 and 64 bit megui. sorry if this doesnt belong in this thread but i didnt know how else to reply to zathor's post directly.

*edit* if you do not close megui and then open it back up as per step #2, all will be fine. it will not gray out and crash.
*edit #2 forgot to answer zathor's questions* graphics memory is 512meg. usage on gpu-z is normal, around 8%. i explained the crash type and the report is posted.
*edit #3* after crashing this way around 4 or so times megui gets messed up and i have to reinstall otherwise it crashes anytime i even attempt to enque a job.

[krosswindz]

Neuron2, if any spare time, please make DGIndexNV remember in and out directory separately. Sometimes the ripping source drive is one, the output project one is another.


+1

This is something really very useful.

[asarian]

"1. Activation of a screensaver or standby mode may interfere with proper operation of the NV tools. They should be disabled during use of the NV tools."

Hmnm, dumb question, no doubt, but does that also mean you can't game while DGSource is running?

[Sharktooth]

@neuron2: as promised here's an MKV with vorbis audio (dgindexnv cant demux it). it's 2 channels, if you need a MKV with vorbis 5.1 channels audio just ask.
http://www.mediafire.com/?p50eerrupiuyciq

[Underground78]

Hello,

I have warnings "CuvidDecodePicture: 1" when seeking in DGIndexNV with some VOB files loaded, can I just ignore them safely as there are just warnings or do I need to provide you a sample so that you can check ? :)

Underground78

[asarian]

Sadly, I seem to have a problem with DGDecNV, after all. :( All seemed to go so well. I was trying to encode the following job:

LoadPlugin("C:\Program Files (x86)\dgdecnv2026\DGDecodeNV.dll")

DGSource("f:\jobs\push.dgi").ConvertToYV12()

Crop(0, 132, 0, -132)

MCTemporalDenoise(settings="low")
GradFun2DBmod(thr=1.4,thrC=1.0,mode=0,str=1.2,strC=0.0,temp=50,adapt=64,mask=false,show=false)

SupTitle("f:\jobs\push_exp.sup", forcedOnly=false, swapCbCr=false, relocate=true, relocOffset="0,0,0,0")

Decoding speed was never spectacular; it started off at around 1.46fps (which is pretty normal for a job using MCTemporalDenoise and 'placebo' settings for x264). But gradually performance started to degrade, getting ever slower over time. I aborted the process after a day when the frame processing rate had dropped to about 0.89fps (and CPU usage to an ultimate low 2%).

Also, DGSource seems to use a wicked amount of memory. At ca. ~5 GB total physical memory, it's using 1.5-2GB over what I'm used to. Not that I mind, per se; but I actually expected lower memory usage (as some would be offloaded to the GPU memory?).

So, I'm suspecting something is wrong, really. I use Windows 7, 64-bit, with a GTX285 and the latest 258.96 drivers.

[Groucho2004]

Decoding speed was never spectacular; it started off at around 1.46fps (which is pretty normal for a job using MCTemporalDenoise and 'placebo' settings for x264). But gradually performance started to degrade, getting ever slower over time. I aborted the process after a day when the frame processing rate had dropped to about 0.89fps (and CPU usage to an ultimate low 2%).

Also, DGSource seems to use a wicked amount of memory. At ca. ~5 GB total physical memory, it's using 1.5-2GB over what I'm used to. Not that I mind, per se; but I actually expected lower memory usage (as some would be offloaded to the GPU memory?).

What makes you so sure that the decoder is the culprit? Have you tried without the filters? Are you using a modified version of Avisynth? Have you tried the "official" Avisynth 2.58? Have you tried a different decoder (FFMS2, Directshow, etc)?

[asarian]

What makes you so sure that the decoder is the culprit? Have you tried without the filters? Are you using a modified version of Avisynth? Have you tried the "official" Avisynth 2.58? Have you tried a different decoder (FFMS2, Directshow, etc)?

I've tried FFVideoSource and DSS2. Both perform as expected (ca 1.5fps). Only after using DGSource does performance gradually degrade. I'm re-encoding now with FFVideoSource, and there's no longer a decline in perfomance (and uses ca. 2G less memory).

Memory usage also steadily increases with DGSource; not dramatically, but still noticeable; from 4.6 to 5.1 GB over time (at which point I aborted the encode, since, at 2% CPU usage for x264, the decoder hardly seemed to feed any frames any more).

And yes, I use the official 2.58 AviSynth release.

[tormento]

Am I the only one who noticed bigger "desktop lagging" under Win7 with 260.x drivers while using DGDecodeNV.dll?

Even encoding speed seems slower..

[Sharktooth]

those drivers are in beta stage for a reason...

[Groucho2004]

Memory usage also steadily increases with DGSource; not dramatically, but still noticeable; from 4.6 to 5.1 GB over time (at which point I aborted the encode, since, at 2% CPU usage for x264, the decoder hardly seemed to feed any frames any more).

And yes, I use the official 2.58 AviSynth release.

The official Avisynth is 32 Bit so it's restricted to 2 GB of memory usage.
Which process uses how much memory? Did you check memory usage with Process Explorer?

[levi]

"1. Activation of a screensaver or standby mode may interfere with proper operation of the NV tools. They should be disabled during use of the NV tools."

Hmnm, dumb question, no doubt, but does that also mean you can't game while DGSource is running?

I guess it depends on the game. WOW seems fine.

[tormento]

those drivers are in beta stage for a reason...
We can count WHQL drivers release on two hands. And we know that WHQL is a way for Microsoft to get money too, not a proof of quality.

I need feedback, not moaning ;)

[hajj_3]

Found this, it was posted in late July on a forum:

http://3.bp.blogspot.com/_I4-UBBtkAT4/TC8q1Bhp0aI/AAAAAAAAAG4/5h6f3gV3CTs/s1600/roadmap.jpg

thought i'd post it for Neuron as Q4 2010 was the only info of when ATI Stream 2.3 SDK would be out, now it appears a more accurate month of December is targetted.

For those that don't already know, this will allow Neuron to create a version of DG NV tools for ATI gpu's that support ATI Stream technology.

[Sharktooth]

We can count WHQL drivers release on two hands. And we know that WHQL is a way for Microsoft to get money too, not a proof of quality.

I need feedback, not moaning ;)

WHQL and BETA are 2 different things. Your drivers are BETA. So do not expect they work as a final product.
So, here's your feedback: go to www.nvidia.com and download the latest OFFICIAL and STABLE drivers.

[crasus]

Good evening, neuron2. Did you happen to get the time to look into the issue I mentioned? Thanks.

[Hagbard23]

Maybe you post it in the brandnew SupportForum on N2.Net: http://neuron2.net/board/index.php

[Mixer73]

I'd love to register at the new support forum but the captcha there is so secure it seems impossible for a human to actually complete it...

[kypec]

I'd love to register at the new support forum but the captcha there is so secure it seems impossible for a human to actually complete it...
Just ignore the little dots, commas and hyphens. Other than that, this CAPTCHA seemed rather crystal-clear to me.:)

[Hagbard23]

NEWZ:

I mailed TRITICAL, about the problem, that TIVTC expects "D2V" Files, which could not be achieved with the *new* DGNVTOOLS, since we get "DGI" files out of it. I don't exactly know if the D2V brings some better perfomance of TIVTC, but i assume it. So i asked Tritical if it would be possible to rewrite TIVTC, so that it gets along with DGI-Files.

ATM he needs to know, wether the same type of information is stored in the DGI's,or not - if yes, he would rewrite the TIVTC Code. I will send a DGI Sample to him and hope the best.

THX Tritical for all your work...

[Groucho2004]

NEWZ:

I mailed TRITICAL, about the problem, that TIVTC expects "D2V" Files, which could not be achieved with the *new* DGNVTOOLS, since we get "DGI" files out of it. I don't exactly know if the D2V brings some better perfomance of TIVTC, but i assume it. So i asked Tritical if it would be possible to rewrite TIVTC, so that it gets along with DGI-Files.

ATM he needs to know, wether the same type of information is stored in the DGI's,or not - if yes, he would rewrite the TIVTC Code. I will send a DGI Sample to him and hope the best.

THX Tritical for all your work...

The DGI format is completely different, just look at the files and the documentation.

[Hagbard23]

The DGI format is completely different, just look at the files and the documentation.

Ask yourself, why i am posting it here...thanks for your "smart" answer, but unless you are able to tell, what exactly the differences are, your comment is worthless, because we all know, that the formats are different. We want to figure out, how to connect those two tools anyway..

If you can help achieving this: Thank You.
If not: Good Bye.

[Groucho2004]

Ask yourself, why i am posting it here...thanks for your "smart" answer, but unless you are able to tell, what exactly the differences are, your comment is worthless, because we all know, that the formats are different. We want to figure out, how to connect those two tools anyway..

If you can help achieving this: Thank You.
If not: Good Bye.

Mine was indeed a smart answer because I pointed out that all there is to know about the two file formats is in the documentation which is available to everyone and a much better approach to figure out the differences than by just looking at the DGI file.

Again, just so you also understand - All details about the two file formats are in the documentation.