LAV Filters - DirectShow Media Splitter and Decoders

[Plutotype]

Hi Samuri,
Nice setup. Why do you prefer to use ArcSoft and Cyberlink video decoders and additionally pass them over to ffdshow raw?
Im just curious if Arcsoft decoders doing better job than ffdshow video decoders. Im using ffdshow for H.264/AVC+VC1+MPEG2 decoding and outputting as YV12 to madVR. Is there any PQ improvement or cpu/gpu saving by using your setup?
Thanks
Pluto

[SamuriHL]

I can honestly say that with the Cyberlink decoder I'm getting great CPU utilization and incredible PQ with the way I have it set up. I'm using ffdshow for resizing when necessary. :D (I.E. DVD's I've remade into MKV's). It's quite outstanding. If what you have works, don't mess with it. I mean, it's simple really...just add the video decoder, set it to prefer, and uncheck ffdshow and compare. If it's not an improvement, go back to your original setup. That's what I've been doing. I wanted to see how the ArcSoft decoder compared. It's not bad, but, I think the Cyberlink decoder works better in this environment. For my bedroom machine I use CoreAVC because the CPU is too slow for video decoding. :( Nonetheless, this setup works PERFECTLY for all videos I've tried it on.

[SamuriHL]

To get DXVA in Cyberlink to stick, I have to do this:

Open MPC-HC > go to external filters and double-click Cyberlink Decoder > change decoding to DXVA. Then, without closing MPC-HC, drag a VC-1 encoded file into it.

You may also need to add GUID {31435657-0000-0010-8000-00AA00389B71} as a media subtype to MEDIATYPE_Video of CyberLink Video Decoder in MPC-HC's external filter page.

The bad thing about Cyberlink's decoder though: it doesn't seem to work well for VC-1 inside an mkv. For that, Arcsoft's decoder seems best when dealing with interlaced VC-1 (ironically, at least on my system, I get stuttering on progressive VC-1 inside mkv with Arcsoft :confused:).

Ok, I'm now configuring my laptop for DXVA in MPC-HC and I did indeed have to add that mediatype to get DXVA to work. So thanks for that! It wouldn't connect in DXVA without it. Now it works great. I have it setup so I can switch between ArcSoft and Cyberlink decoders on all my machines now. On my main HTPC, I'm still determining what looks best with madVR, but, so far I think Cyberlink has the edge. Testing a movie with ArcSoft decoder now. With my bedroom machine and laptop I've yet to get the Cyberlink decoder to work with DXVA until now, so, I haven't done a good comparison. On the plus side, my bedroom machine is able to use the ArcSoft decoder to play in 24Hz a VC-1 MKV which is damn hard to do. So, I think I'm really getting there with all these machines and my MKV playback. Nev, keep up the most excellent work. I look forward to Blu-ray support. :)

[robpdotcom]

Glad to hear I could help (that's a rare occurrence for me :rolleyes:).

On my system, it takes much longer to open a VC-1 file with Cyberlink, and interlaced VC-1 is also not as smooth as it is Arcsoft. I'm interested to hear if you have similar results.

[SamuriHL]

I'll give it a try maybe tomorrow with a VC-1 MKV I have. (Golden Compass blu-ray...created to torture ArcSoft cause they can't play it in an MKV container. Interestingly it can from MPC-HC now. :D) I don't have any interlaced VC-1 content, though. But yea, you definitely helped so thanks. :)

[nevcairiel]

Classic DXVA disables madVR, which is a no-no, but there are other hardware acceleration solutions.. :)

<teaser>If you're an NVIDIA user, i'll have a treat for you soon.</teaser>

[madshi]

<teaser>If you're an NVIDIA user, i'll have a treat for you soon.</teaser>
CUDA Decode? :)

[nevcairiel]

Technically its not using CUDA to decode, just to access the hardware decoder.
I like to call it what NVIDIA calls that interface, a CUVID decoder.

[madshi]

Yeah, it's looking quite promising, I have the sample on my harddisk, too... :) There's one disadvantage: You'll have to copy the decoded frame back from GPU -> system RAM, but from what I've measured (madNV12Test) that should be very fast (~ 500fps with 1080p content on newer cards), so no real problem.

[nevcairiel]

Yeah, NVIDIA put real focus on getting data from the GPU back to the CPU, its used alot in CUDA. I didn't actually measure performance when using the CUDA interface to copy the frame back instead of using the D3D functions to read a surface, but i figure it'll not be slower, possibly even faster. A quick test in GraphStudios decoder performance gives me about 70fps in a 1080p H264 file (don't remember the complexity of it right now, though), about the same as CoreAVC in CUDA mode, so i must be doing something right.

I'll post some more measured stats when i release it.

(Oh, the main difference to CoreAVC? In addition to H264, mine supports VC-1, MPEG-2 and MPEG-4 ASP as well - given hardware support, of course.)

[madshi]

Looking forward to it! I think those 70fps must be limited by the actual decoding performance, not by the GPU -> System RAM copy.

What I found clever is the way NVidia handles the NV12 surfaces: Those allocated for DXVA decoding are optimized for reading. That makes sense because applications will never write to those surfaces, if they access them at all, they will read from them. But NV12 surfaces not allocated for DXVA are optimized for writing instead. Again that makes a lot of sense. Only NVidia has optimized things this way. ATI and Intel have not.

[nevcairiel]

Looking forward to it! I think those 70fps must be limited by the actual decoding performance, not by the GPU -> System RAM copy.

Indeed.
I learned that the hardware decoder runs the same speed on all cards of the same generation, and is not influenced by the actual clock of the GPU, that makes benchmarking it alot easier, and the values more meaningful.

I'm not sure if the deinterlacing is done in the fixed function hardware or handed over to the shaders, though.

In any case, hardware deinterlaced cheese slice test on madVR = awesome.

Its mostly done, as well.
I only need to work out some kinks with video renderers that request a certain stride, somehow my handling of this is rather wonky right now =)

[madshi]

Indeed.
I learned that the hardware decoder runs the same speed on all cards of the same generation, and is not influenced by the actual clock of the GPU, that makes benchmarking it alot easier, and the values more meaningful.

I'm not sure if the deinterlacing is done in the fixed function hardware or handed over to the shaders, though.

In any case, hardware deinterlaced cheese slice test on madVR = awesome.
So you plan to do deinterlacing, too? That'd be quite cool, of course! :D

[nevcairiel]

There is a flag you can set, its values are "weave", "bob" or "adaptive", and then CUVID deinterlaces for you. Nothing you really have to do. Thats why i love CUVID. :)

[madshi]

Cool. I'm still wondering how IVTC works, though. Do you really get 24fps out of a 60i stream? Who controls the output framerate? Is that your job or does the deinterlacing hardware do that? I think it's your job? So you'd have to offer a switch video (60p/50p output) vs. film (24p/25p output) mode for optimal results?

[nevcairiel]

I'm quite sure the decoder just outputs 24 frames per second when IVTC'ing NTSC 30fps content (i don't think i have 60i content, i'll check), but i'll run some more tests on that.
The question is if it tells me that its reducing it to 24fps, so i can set the frame rate in the media type appropriately, as i'm not sure if renderers rely on that value, because from where i stand, the frame rate is simply determined by the distance between two frames, not the frame rate in the media type.

[madshi]

Well, madVR doesn't really care right now what the media type says. However, automatic refresh rate changing might depend on that value being set correctly, so it would be quite useful having it set correctly. If you find out how the decoder behaves exactly with IVTC, I'd love to know. (NTSC 30fps = 60i, it's one and the same)

[hoborg]

@nevcairiel:

Do you plan to support subtitles rendering too (something like FFDshow DxVA decoder do)?

[nevcairiel]

Why?

You can use ffdshow in raw mode, or MPC-HCs internal renderer, there are no limitations.

[hoborg]

Why?

You can use ffdshow in raw mode, or MPC-HCs internal renderer, there are no limitations.

Becouse there are some players that doesn't support subtitles, like nPVR/GB-PVR, WMP too (i think) or have limitations (MediaPortal cannot render PGS), etc...
FFDshow RAW is a solution, but not very "clean".

[nevcairiel]

Subtitle rendering is annoying, and i probably won't touch it. I use a player that can render them properly scaled on the final image, i don't need to render them on the image before scaling, like ffdshow does.

For people that don't use such players, there are several solutions, ffdshow raw and DirectVobSub should work in most if not all players.

[madshi]

Agreed, spending time to render subtitles on the decoded image pre-scaling simply makes no sense. Subtitle rendering must be done after scaling for proper quality. Which means that subtitle rendering can not be done in the video decoder.

[hoborg]

Yes, i understand that.

This is the reason why i use FFDshow video decoder => resize => subtitles rendering => video renderer for SD and
FFDshow DxVA => subtitles overlay => video renderer for HD.

Possibly i can use LAVF video decoder => FFDshow RAW (resize + subtitles rendering) => video renderer, but i am afraid some players won't add FFDshow RAW between LAVF video decoder and renderer.

Looks like i will need to stay using FFDshow after all.

[nevcairiel]

My video decoder will for the time being only be for NVIDIA, going by your signature, you don't fit that profile.

A general purpose software and DXVA video decoder is in the planning/alpha stage, but don't hold your breath, it might not happen at all, or take a year.

[hoborg]

My video decoder will for the time being only be for NVIDIA, going by your signature, you don't fit that profile.

A general purpose software and DXVA video decoder is in the planning/alpha stage, but don't hold your breath, it might not happen at all, or take a year.

OK, no problem on my side, i just want to be informed :)

[ranpha]

Sample is from here (https://github.com/Nevcairiel/LAVFSplitter/issues#issue/3).

The LAV Audio Decoder supposed to support Vorbis tracks right? With the latest version of the decoder, it wasn't used when playing the Vorbis track in the said sample.

[nevcairiel]

It just supports vorbis when using LAV Splitter as well, with other splitters it will most likely not work due to not supporting the vorbis-specific media format descriptor.

[SamuriHL]

Classic DXVA disables madVR, which is a no-no, but there are other hardware acceleration solutions.. :)

<teaser>If you're an NVIDIA user, i'll have a treat for you soon.</teaser>

Right, I know. On my bedroom machine and laptop I can't use madVR. They just aren't powerful enough to do software decoding. CoreAVC allows me to do CUDA decoding on the bedroom machine, but, I don't have a solution for VC-1 decoding. So I'm stuck with DXVA on that machine. My main HTPC is a quad core monster and has plenty of power for decoding. Using that with the Cyberlink decoder gives me the best results. And yes, my bedroom machine has an nVidia 450. Am I going to be happy? :)

[SamuriHL]

Yeah, NVIDIA put real focus on getting data from the GPU back to the CPU, its used alot in CUDA. I didn't actually measure performance when using the CUDA interface to copy the frame back instead of using the D3D functions to read a surface, but i figure it'll not be slower, possibly even faster. A quick test in GraphStudios decoder performance gives me about 70fps in a 1080p H264 file (don't remember the complexity of it right now, though), about the same as CoreAVC in CUDA mode, so i must be doing something right.

I'll post some more measured stats when i release it.

(Oh, the main difference to CoreAVC? In addition to H264, mine supports VC-1, MPEG-2 and MPEG-4 ASP as well - given hardware support, of course.)

OMG! DROOL DROOL DROOL!!!! Oh HELL YEA!!! That's EXACTLY what that machine needs!!!!!!!! You guys ROCK!!!!

[clsid]

Wow, a nice surprise.

Will it handle incompatible videos properly by falling back to whatever software decoder is installed? Maybe there are CoreAVC users here that have some samples for you that don't work with CUDA.

[nevcairiel]

That depends. If it can be detected from the header that its incompatible, it'll refuse connection. If some problem occurs during playback, there isn't anything one can do. There isn't really a sane way to let some other filter take over once playback has started.

[SamuriHL]

Well, I for one am waiting anxiously to get to try this out. It sounds like the perfect solution for my bedroom machine to finally be able to use madVR. We'll see how that thing handles 24Hz playback of my MKV's. You've already given me an almost nearly flawless solution for that anyway, but, this would make it even more impressive. No chance of STREAM being used in the future for those AMD people? :D I'm kidding...no one uses STREAM. LOL!

[Mercury_22]

I'm about to upgrade my Video card (I was waiting for HD6670 (http://www.amd.com/us/products/desktop/graphics/amd-radeon-hd-6000/hd-6670/Pages/amd-radeon-hd-6670-overview.aspx) :) ) so I'm really interested to know if your decoder it will "do" VC-1 i too ?

[Kotik]

Well great news indeed.

When/if LAV Video decoder is out, it is going to be the 1st Open Source Video Decoder to support VC-1i + DXVA!

Sounds really promising. My ION is ready for some testing Nev :)

[nevcairiel]

Its not DXVA, and i haven't decided yet if i should open-source it.

And yes, it supports VC-1 interlaced.

[SamuriHL]

If you get bitstreaming added to the audio decoder, I won't need ffdshow in the chain anymore. That's pretty scary! :)

[Kotik]

Nev, i know it is too early to ask but i will ask anyway:)

Are you planning at some point to do the same by using ATI Stream Universal Computing?

Would like your thoughts on this one.

[nevcairiel]

Are you planning at some point to do the same by using ATI Stream Universal Computing?


No.

First off, you have to realize that its not actually using CUDA to decode. Its just using CUDA to access the hardware decoder, this is done through an extension called CUVID.
That may bring you to the realization that STREAM might not even have such an API to access the video decoder (I actually have no idea if they do).

In any case, i have never owned a AMD/ATI GPU, and i'm not starting now. Also, writing this all over again for another GPU would be like starting over, as the code is very NVIDIA specific, there is barely any code that could be used for another decoder in there - all the heavy lifting is done by the NVIDIA driver and hardware.

[SamuriHL]

Hey! I already asked that! :P And he ignored me. Probably cause I mentioned that almost NO ONE uses STREAM. It would be nice to have, but, the architectural differences in how they go about video decoding in their respective APIs will mean that it's almost a completely different decoder. So, the question to ask is...are you going to write a decoder that uses STREAM? :)

EDIT: Nev Ninja'd me. :)

[ranpha]

The upcoming LAV Video decoder, will it support MVC videos too?

[nevcairiel]

The upcoming LAV Video decoder, will it support MVC videos too?

No.
The current CUVID interface does not support MVC decoding yet.

[SamuriHL]

I think what Nev is trying to say is that pretty much anything the nVidia cards support he'll be able to do with this driver. It's basically sending the video to the card for hardware decoding and getting the frames shoved back to the CPU. Truly awesome.

EDIT: Ninja'd again. :) Nev, they're not exposing all the functionality of the hardware decoder?? That seems odd.

[nevcairiel]

Maybe the upcoming CUDA SDK 4.0 supports it, i don't know - i cannot download the RC without registering for their CUDA developers program.

Personally, i'm not that invested in 3D anyhow.

[SamuriHL]

Fascinating. And no, I'm not doing 3D either even though my bedroom setup (minus the TV) is ready for it. If I got a new TV I could do it but I don't care. I've been struggling to get basic 24Hz MKV playback working since I got the 450. With your latest LAVF, latest MPC-HC, and ArcSoft/Cyberlink decoders, I'm able to do DXVA for 24Hz. That's pretty darn awesome. Your decoder should throw madVR into the mix and that's just frosting on top as far as I'm concerned. :)

[madshi]

CUDA SDK 4.0 does not support MVC decoding, either.

[noee]

Not that I'm arguing for it, but isn't this kind of the same thing (limited right now to h.264 and obviously a bit behind CUVID) on the AMD side:

OVD API (pdf) (http://developer.amd.com/gpu/AMDAPPSDK/assets/OpenVideo_Decode_API.PDF)

"The OpenDecode API is the part of OpenVideo that allows applications to use the GPU’s UVD
engine. OpenVideo supports full bitstream decoding acceleration. The decoded output then can
be used for either displaying directly using the GPU, or for other post-processing operations
through Open CL kernels run on the GPU shaders (post-process filtering or transcoding
operations). The OpenVideo API is fully interoperable with the OpenCL API: it allows for shared
surfaces between the two domains."

[nevcairiel]

Sounds like its similar, maybe someone else feels like doing that then. :p

[Kotik]

hehe :)

[SamuriHL]

Not it! :D

[madshi]

Forget about it with AMD. You can use it, but it's quite similar to DXVA. The NVidia solution allows very fast transfering of the decoded frames back to System RAM, which means that you can create a DirectShow decoder which works similar to a software decoder with no funny limitations. With ATI, there's no fast method to transfer the decoded frames back to System RAM, which means the ATI solution only makes sense if you have a renderer which can directly work with the decoded frames in GPU RAM. That's exactly the same limitation that DXVA has, so the ATI OVD API has no real benefit over DXVA (other than that it supports MVC decoding).