DGAVCDecNV 1.0.13: GPU decoding on Nvidia

[Guest]

Out-of-process COM servers are factor 10 to 16 times slower than inproc. Where will be the interface level? I suspect that if you plan to transport decoded images over COM, you will lose all speed advantages of hardware decoding. I said quite clearly in my post above that I plan to transfer the frame in shared memory.

I am still not sure what the problem with MeGUI is. I can run multiple instances of vdub GPU AVS scripts. What is so special with MeGUI? Again, as I said above, it is a managed application, and for some reason that I have not yet discovered, managed applications crash on the second D3D instantiation.

[meatwad]

I'm experiencing a bug. X264 hangs at 99.9% with VirtualDubMod and ASXGui 1.9.4. I'm not sure why it's not closing? The two programs use different builds of X264 and it's happening in both XP and Vista.

[LoRd_MuldeR]

I'm experiencing a bug. X264 hangs at 99.9% with VirtualDubMod and ASXGui 1.9.4. I'm not sure why it's not closing? The two programs use different builds of X264 and it's happening in both XP and Vista.

Does it happen also when running x264 via CLI? Any other filter in addition to DGAVCDecNV in your Avisynth script?

BTW: x264 + VDubMod means you use x264 VfW, which probably is not the best idea...

[meatwad]

I just use the Nvidia Pure Deinterlace option:

loadplugin("Z:\dgavcdecnv102\DGAVCDecodeNV.dll")
avcsource("__vid__", deinterlace=true)

VFW for virtualdub is correct, but ASXGui 1.9.4 uses CLI. I'll use a different CLI of X264 with it and see what happens. I just thought it was weird that it hangs with both programs in two different operating systems. I'm sure the problem is on my end because no one else seems to be experiencing it.

[meatwad]

Testing CLI now. hopefully it doesn't hang.

[meatwad]

With CLI it hangs at 81.9% of the encode job. Something's not right with my system.

[meatwad]

I tried one more time with CLI and it hung at 99.9%. If I use a directshow source it doesn't hang. It only hangs if I use DGAVCDecodeNV. I've tested it with 3 different versions of X264. I'll keep messing with it.

[Guest]

Can you cut the end of the stream where it hangs and post a link to it?

Also, try it with DGAVCDec (not NV) and let me know what happens.

[meatwad]

http://www.zshare.net/download/200136624c2bfc26/

I'll get back to you in a few minutes on DGAVCDec.

[meatwad]

All went well with DGAVCDec. Except of course the nasty PAFF blocks.

[meatwad]

This isn't looking good for me. It behaves exactly the same in XP, Vista 32 and Vista 64. It will get to around 99% and not close. It's more than likely something about my system:

Q6600
Gigabyte p35-ds3r
EVGA 9800gt 512mb
4gb ram
1.5tb raid 0 array.

It's done this from day one, but I assumed it was virtualdub. I'd never tried to do a complete encoding with another program until the other day. That's when I realized it wasn't virtualdub.

The memory and video card are brand new so I suppose I should run a memory check. It's not overclocked either.

[Guest]

Does it happen with all streams?

[meatwad]

Yes it does. I'm using the Cuda Driver and Cuda SDK. I've tried avisynth 2.57 and the beta.

[meatwad]

My memory test is at 50% with no errors and I'm suspecting my GPU. I can transplant the GPU into another PC and install the necessary software, but you'd have to send me a new key for it or a test version with expiration.

GPU: EVGA 512-P3-N975-AR 9800GT 512M

I can also do a bare install of XP with the necessary software to test the decoder in case it's something I have installed in all three OS that's causing the problem.

[ReferenceDivx]

It seems to be working fine for me. I never installed the cuda sdk though. Maybe your x264 switches aren't correct since they changed some of the options a couple of days ago.

[meatwad]

It still hangs even with a clean minimal install. The only codec I installed was YV 12.

@ReferenceDivx I've tested several builds and the results are the same. I don't have the sdk or the toolkit installed this time. Just the Cuda Driver. I'm going to put the card in a different machine and see if it still hangs. I hate to ask DG for another key.

[LoRd_MuldeR]

Maybe your x264 switches aren't correct since they changed some of the options a couple of days ago.

When using invalid options x264 will simply complain and abort right at the beginning.
But it certainly doesn't start to encode and then decides to stall at 99.9% :p

[meatwad]

BTW, I've tried to use different compressors like XVID and the encoder will still hang at the end.

[Zwitterion]

meatwad, what do you mean by cuda driver? This one?
http://www.nvidia.com/object/winxp_178.13_whql.html

[LoRd_MuldeR]

BTW, I've tried to use different compressors like XVID and the encoder will still hang at the end.

What if you open the AVS script in VirtualDub and seek all the way to the end?

[meatwad]

Yes. I was just about to post that I believe it completes the encode, just the encoder doesn't close. In VirtualDub I can see the completed encode but I have to stop the encoder. If I stop X264 CLI when it hangs I can't view the video. Naturally I can't do any batch encoding running like this.

@Zwitterion: I've been using this one from the CUDA page NVIDIADisplayWinVista(177_84)Int.exe, I believe I tried the regular driver as well since others have said they used it without any issues. I'll try it again before I move the card to another system.

I really appreciate everyone's help with this. Sorry I'm such a pain DG.

Update: I just found that if I seek to the end and then go back it locks up VirtualDub.

[meatwad]

I just tested with 178.13_geforce_winvista_32bit_english_whql.exe installed and I'm still getting the same results.

[meatwad]

I found what was wrong, it doesn't like my avc .ts streams. It will work with other avc sources but not my transport streams (even if they're demuxed). Neuron2, could you please add support for them in a future release? I'll send you all the samples you need.

They're all I really work with. I would forever be in your debt if you could also get it to decode the same type of avc streams in .vdr (video disk recorder) format.

Also, were you able to get the sample to encode with vdub or x264 cli?

[Guest]

Also, were you able to get the sample to encode with vdub or x264 cli?
I am able to duplicate your issue with the file that you provided. So I'll try to fix it today.

EDIT: OK, I know what is happening. When you hit EOF the decode thread is waiting for a signal from the decoder to say that a requested frame is available, but the EOF processing doesn't send that signal to allow the decode thread to terminate. I don't know why it was happening only on these streams, but it's a design issue, so I'll release an update ASAP.

[meatwad]

You are far and above the cut. Thank you so much for addressing the issue. I've included a link to a sample avc .vdr file so you can see the similarities. They are a pain to work with, unlike the mpg version (just rename the files to .mpg and they'll work with most anything). Linux gurus probably eat them for breakfast, but it takes me several steps to get a finished .mkv file. The streams inside the vdr file should be the same make as the transport stream sample.

http://www.zshare.net/download/200884986b8c802a/

If you don't have the time to look at the sample don't worry about it. Just fixing the transport stream issue is more than enough.

Once again, thank you for making such a wonderful product.

[Casimir666]

@neuron2
Marc99 post a message on the mpc-hc thread that puzzle me :
http://forum.doom9.org/showpost.php?p=1191981&postcount=4468

The weird things is that VSFilter is not supposed to be compatible with DXVA for 2 reasons :
- IMediaSample::GetPointer is not supposed to return buffer (because it's DX Surface)
- Video decoder cannot initiate the DXVA negociation with EVR (because of intermediate filters)

But marc99 screenshot show that mpc detect H264 bitstream mode. Mpc-hc show this only if :
- Someone ask interface IDirectXVideoDecoderService on the EVR
- Someone successfully call IDirectXVideoDecoderService::CreateVideoDecoder


So I have a small question for you : are you using the IDirectXVideoDecoderService interface one way or another in your AVISynth filter (asking to EVR filter or using Media foundation)?

[Guest]

So I have a small question for you : are you using the IDirectXVideoDecoderService interface one way or another in your AVISynth filter (asking to EVR filter or using Media foundation)? No. If you follow the Nvidia development dialog at my website, you'll know exactly what I am using.

http://neuron2.net/dgavcdecnv/cuda/cuda.html

[Casimir666]

No. If you follow the Nvidia development dialog at my website, you'll know exactly what I am using.

http://neuron2.net/dgavcdecnv/cuda/cuda.html


Ok thank you for you answer, it's a really interesting dialog.

[Guest]

@meatwad

Your issue has made it to the Nvidia development dialog, as it appears to expose a regression in nvcuvid.dll. Scroll to the bottom of the dialog and you'll see the analysis.

http://neuron2.net/dgavcdecnv/cuda/cuda.html

[meatwad]

I was hoping it was going to be something minor, but I have faith they'll come through. You were right about Nvidia being extremely helpful. It's in their best interest after all as it will help them sell more GPUs (and I'm proof of that).

[Pookie]

@meatwad - sounds like the same problem I was experiencing -

http://forum.doom9.org/showthread.php?p=1186758#post1186758

[Guest]

Oh, I'm glad you reminded me about that. I have meatwad's stream working and just now tested with yours and they're not yet fixed. Investigating...

[Guest]

OK, Pookie, I've figured out the problem with your streams.

The stream is sending a lump of 8 PPSs every so often. When I seek, though, I was just injecting the previous 5 known PPSs, so the decoder was silently discarding the frames with orphaned PPSs, and when I reached the end of the stream, my app thought there were more frames to come but the decoder couldn't provide any more. I have to rethink my heuristics for SPS/PPS injection on a seek. For now I've bumped it to inject the last 15 prior to the seek point. I should be able to think of something a little more intelligent, however, as I do have the stream indexed at that point and know about all the PPSs.

I spent two days being confused because I thought the problem was caused by end of stream handling when all along it was caused by start of stream handling!

Also, Nvidia has delivered a new nvcuvid.dll with better performance and some bug fixes. So I'll make a release tonight with that, the temporary PPS fix, and the fix for meatwad. That should keep us going until I redesign the SPS/PPS heuristics properly.

[Guest]

1. Fixed an uninitialized variable in the CUDA parser creation that could cause random suboptimal decoding performance.

2. Changed handling of NALUs with forbidden_zero_bit set. They are now silently discarded.

3. Fixed a problem that could cause a stream to not be properly flushed on end of stream, which in turn could cause DGAVCDecNV to hang.

4. Fixed D3D device object leak for XP.

5. Improved the heuristic for SPS/PPS injection when seeking. This fixes some other cases of hanging at the end of the stream.

6. Added a safety timeout for end of stream hangs (although I think they are all solved now).

7. Updated the Nvidia Development dialog at my web site.

http://neuron2.net/dgavcdecnv/dgavcdecnv.html

I'm still working on compatibility with managed applications, i.e., the COM server solution.

[Guest]

I've included a link to a sample avc .vdr file so you can see the similarities. They are a pain to work with, unlike the mpg version (just rename the files to .mpg and they'll work with most anything). Linux gurus probably eat them for breakfast, but it takes me several steps to get a finished .mkv file. The streams inside the vdr file should be the same make as the transport stream sample. Do you have any information on what these VDR streams are inside, i.e., specs? What do you use to demux them?

[meatwad]

I've been using elecard xmuxer. I'll post a link with a sample already demuxed.

[Guest]

I don't need the demuxed sample!

Testing Elecard now, standby...

[Guest]

Hmm, Elecard works, but I can't recognize the file type in my hex editor.

[meatwad]

Unfortunately I don't have a clue either. I've asked in some Linux forums but no one responded. I'd probably just put this on the back burner as you have much bigger fish to fry with all your other projects. Besides, I'm looking forward to your avc cutter and you've already done more than enough for me in the last few days. Thanks again for the fix and I'll test it out later tonight.

[Guest]

Ah, OK, it's PES. I can handle that. :) I'll put it on the container to-do list with MKV and MP4.

[meatwad]

I do believe another donation will be heading your way soon. Thanks again.

[meatwad]

I just gave it a test drive and it works GREAT! Hopefully Pookie will have the same results.

[Quark.Fusion]

Is it ever possible to generate index on the fly, if target request frames in linear pattern? This can greatly reduce time for encoding stream.

[Guest]

Is it ever possible to generate index on the fly, if target request frames in linear pattern? This can greatly reduce time for encoding stream. If you aren't using random access, there's no need for an index at all.

What is this great time reduction you are talking about? The time for indexing is small compared to encoding.

[Quark.Fusion]

Indexing time isn't so small if you encode BD to some small device at realtime speed. And before encoding I'm usually set cropping and denoise filters — which I can't before index completes.

So how to open file without indexing, will «AVCSource("00001.m2ts")» work? (Your documentation says that it need dga file)
Tried with DGAVCDecode — it says:Avisynth open failure:
AVCSource: The input file is not a DGA project file.

[Sagekilla]

@Quark.Fusion: Here's a little trick: Try opening the m2ts file in DSS (Directshowsource("file.grf")) and then start indexing the file in DGAVCIndex(NV). Apply filters as necessary on your video in AvsP w/ the DSS instance ;)

I do this kind of multitasking all the time and it works great.

[Quark.Fusion]

Yeah, great idea, but it crashes…

[Sagekilla]

Eh? Which crashes? Cause I've never had an issue doing this before.

[Audionut]

Hi neuron2,

Getting this error on a file.

PicParset 0 references an invalid (uninitialized) Sequence Parameter Set with ID 0, expect the unexpected.


A couple of samples.


33 is the first 500 or so frames
55 is a random selection from the middle


No problems on this file with the software version.

[Guest]

@Quark.Fusion

Sorry, no, I don't intend to support anything like that. Just make yourself a cup of coffee, or balance the checkbook, while your stream is indexing. Or make a small project using the range feature first. Then start your full project. You can tweak using the small project while the full one is cranking.

@Audionut

I'll look at it this evening.

@all

I ditched the COM idea because it's a PITA, with all the GUIDs, registry stuff, etc. I have to make shared memory anyway to pass the frame, so I'm using it also to implement an RPC mechanism. So that together with some events and a mutex gives me all I need without the COM insanity. The way it will look to the user is that you first open a CUDA decoder server application. You can minimize it but as long as it stays alive the CUDA decoder is available to clients, i.e., AVCSource() instances. The clients access it using the named shared memory, events, and mutex. I'll publish the interface, the server source code, and a basic client so that it can be re-used. My DG client applications will remain closed.