LAV CUVID Decoder - High Quality Hardware decoding for NVIDIA

[SamuriHL]

Nev,

You were right. I took a clip (http://www.mediafire.com/?ubgs42nnihjhl41) that makes it happen and converted it back to m2ts and then opened it. Plays perfect. So, play around with that clip and see if you can repro it. Then use tsMuxer to convert it to m2ts if you want and it plays fine. Thanks!!!

[nevcairiel]

It looks like the problem is simple. The SPS/PPS Header stored in the MKV is not valid for that part of the file, so you get corrupted playback until the bitstream contains a new one, but then it corrupts again because i actually inject the header before every IDR frame..

I'll have to figure out how to deal with that. Maybe i can analyse the stream and see if it contains its own SPS/PPS NALUs, or only feed the container header to the decoder once at the beginning.
If anyone has some insight when the decoders expect to receive the SPS/PPS, please do tell. =)

[madshi]

FWIW, eac3to muxes MKV files practically unchanged. All headers remain in the bitstream. SPS/PPS headers are stored "in front of" the video frames which follow them. Works great for all MPEG2, VC-1 and h264 this way.

Not sure how to handle playback if you seek and the bitstream doesn't contain SPS/PPS headers. How are you (as a splitter) supposed to know whether the SPS/PPS headers in the container are still valid for the seek point? How are you supposed to know whether the stream itself contains valid headers or not? I guess the best way to handle this would be to keep current headers in a cache and update them whenever they occur in the bitstream. After a seek you should probably check whether the seeked to frame has headers in front of them. If so, use them. If not, well then you're screwed... In that case I'd probably add those headers from the container.

Hmmm... But then: If the headers don't ever change, then why injecting them into the stream at all? Maybe you should simply not inject headers *AT ALL*?

[nevcairiel]

My current plan would be to just inject them at the beginning and after a seek.
DXVA probably requires this as well, i should check with MPC-HCs code how they deal with that.

[madshi]

Originally Posted by nevcairiel

My current plan would be to just inject them at the beginning and after a seek.

What if the bitstream itself contains headers at every seek point? If you inject headers you might overwrite the correct headers with wrong ones.

[nevcairiel]

If i send my headers to the decoder before any other data, shouldn't a new header that was actually contained in the data overwrite mine?
I'll have to think about this.

Too bad just decoding a buffer that only contains SPS and PPS to init it doesn't work, for some reason it doesn't like that.

What the DXVA MPC-HC codec does is basically decode the headers of every frame, updating its context as it goes forward. But i do not keep my own context, its all in the CUVID driver. I'll have to figure out when it actually "forgets" about the context - does a full flush (as it happens on seeks) already cause it to forget? If not, sending the container SPS/PPS on start would be good enough, i figure.

[nevcairiel]

Ok, some testing revealed, a Flush does actually drop its state, but feeding it the SPS/PPS after the flush does seem to work just fine. The Sample from SamuriHL plays fine now, and files that were directly encoded into MKV (so, no SPS/PPS NALUs in the bitstream), are still working fine as well.

I'll do some more tests, and finish working on some configuration that i had lined up.

[SamuriHL]

That's awesome, Nev and Madshi! Thank you for looking into that!

[ranpha]

After testing more Xvid videos, it seems that Xvid videos (a lot of them I say) in AVI containers stutters a lot. Remuxing the very same videos into .mp4/.mkv containers and they started playing smoothly! Anyone else seen the same issue?

[nevcairiel]

Originally Posted by ranpha

After testing more Xvid videos, it seems that Xvid videos (a lot of them I say) in AVI containers stutters a lot. Remuxing the very same videos into .mp4/.mkv containers and they started playing smoothly! Anyone else seen the same issue?

I have seen this, its mostly because AVI really doesn't have much of a timestamp. If using MPC-HC and EVR-CP, turn on frame-time correction. madVR should also work fine. At least thats what i've seen so far.

[ranpha]

Originally Posted by nevcairiel

I have seen this, its mostly because AVI really doesn't have much of a timestamp. If using MPC-HC and EVR-CP, turn on frame-time correction. madVR should also work fine. At least thats what i've seen so far.

I use madVR latest version (I will redo my tests with EVR custiom presenter later), and the stuttering is still there.

edit: Just tested some of the problematic MPEG4-ASP AVI files with EVR custom presenter, and the problem disappear.

[CruNcher]

Nev your decoder also fails @ 1080p 60 fps H.264 on a VMR9 Renderless surface in MPC-HC (demuxer and ffdshow audio render) same as almost every ISV Decoder existing it drops frames like there is no tomorrow this happens either in Cuda or DXVA mode (always on VMR9 Renderless) there are only 2 Decoder yet who survive this test on Nvidia VP2 and that's CoreAVC CUDA and Cyberlinks DXVA H.264 Decoder neither Mainconcept DXVA does playback it without framedrops nor does Arcsofts Decoder manages this.
Every of the Decoder plays fine on VMR9 and VMR7 Renderless Renderer but only CoreAVC and Cyberlink survive VMR9 Renderless @ 1080p 60 fps flawless yet

Yours the same as the Chinese Nvcuvid Decoder and others stay stable for 12 seconds then start to drop frames like crazy with that Sony 60 fps Stream on VMR9 Renderless

[nevcairiel]

Its not optimized at all yet. If by fails you mean it drops frames or is too slow, there is still alot of room for improvement.

Btw, any convenient place to find a 1080p/60 test stream?

[CruNcher]

This is the one i use for testing http://e.dl.playstation.net/e/wipeou...D_EN_1080p.zip

[nevcairiel]

Thanks, i'll see what i can do to get some more speed out of this.

[CruNcher]

With a Mpeg-2 .ts Test stream it fallsback to Video Renderer doesn't connect to VMR9 Renderless @ all (Cyberlink works flawless)

[BatKnight]

Originally Posted by CruNcher

This is the one i use for testing http://e.dl.playstation.net/e/wipeou...D_EN_1080p.zip

@CruNcher
I can play this file flawless at VMR9 Renderless with CUVID decoder. No drop frames. But I own a GF104, which has VP4, and nevcairieal's graphics card is also a VP4.
I've tried different combinations and this Wipehout video never breaks, delays or get unwatchable.

@nevcairiel
I've manage to sort all those problems I've refered before on previous posts by reducing the EVR Buffers to 5 or 10 (I had them at 25).
But I've found a reproduceable bug: When I play a MKV with h264/AVC and FLAC track with LAV Audio then the audio isn't sync with the video. But if I force madflac instead, then the audio is sync with video.
This doesn't happen with AC3 or DTS tracks, only FLAC.

EDIT: Looks like that CUVID isn't the problem, ffdshow also does this. The only constant is the FLAC track. Sorry to have posted this here.

To reproduce: MKV with h264/AVC + FLAC track + LAV Audio

Nuno

PS: All these videos with FLAC tracks were at 1920x1080

[jmone]

Well I'll be ... there could be a mainstream nVidia GPU in my future! I managed to get the following working on my MBA 11" (Late 2010) series (Win7 Only build - no OSX). The MBA is my only nVidia GPU equipped PC *and took some "encouragement" to get V260+ drivers on) and it only has a 320M GPU, 2GB total System RAM (shared with GPU), and a C2D.

- J.River Media Center as the Player using: LAV Splitter, LAV CUVID, FFDSHOW Subtitle Filter, LAV Audio Decoder, madVR (bottom settings), and Directsound --> Output is 1080p &5.1 LPCM over HDMI (using mini display port to HDMI adapter)

It all worked including:
- x264, VC-1, MPEG-2
- Subtitles
- VC-1 60i being deinterlaced nicely

The only issue I saw with my test files was x264 1080/50p (Camcorder) footage that the MBA clearly does not have the horse power to play smoothly BUT, the Audio drifted out of sync very quickly (I've seen this with other filters previously). A short clip is here http://forum.cyberlink.com/forum/pos...tach/3802.page but let me know if you need a longer sample.

[jmone]

Originally Posted by CruNcher

Nev your decoder also fails @ 1080p 60 fps H.264 on a VMR9 Renderless surface in MPC-HC (demuxer and ffdshow audio render) same as almost every ISV Decoder existing it drops frames like there is no tomorrow this happens either in Cuda or DXVA mode (always on VMR9 Renderless) there are only 2 Decoder yet who survive this test on Nvidia VP2 and that's CoreAVC CUDA and Cyberlinks DXVA H.264 Decoder neither Mainconcept DXVA does playback it without framedrops nor does Arcsofts Decoder manages this.
Every of the Decoder plays fine on VMR9 and VMR7 Renderless Renderer but only CoreAVC and Cyberlink survive VMR9 Renderless @ 1080p 60 fps flawless yet

Yours the same as the Chinese Nvcuvid Decoder and others stay stable for 12 seconds then start to drop frames like crazy with that Sony 60 fps Stream on VMR9 Renderless

FYI, from what I've seen on playing back 264/AVC 1080/50p content, ffdshow ffmpeg-mt plays back smoothly with a range of renderers (my current setup)

[nevcairiel]

Originally Posted by CruNcher

With a Mpeg-2 .ts Test stream it fallsback to Video Renderer doesn't connect to VMR9 Renderless @ all (Cyberlink works flawless)

The only reason it wouldn't connect is when VMR9 does either not accept NV12, or does not understand a VIDEOINFOHEADER2 format.
In any case, the output media type is not dependent on the input type (except for width/height and AR), so it doesn't make any sense that the mpeg2 would not work, but others would.