Xvid and (Windows) DXVA acceleration?

[pcdvdguy]

Has anyone investigated the possibility of adding DXVA-acceleration to the XVid codec? For applications involving HDTV resolutions (1280x720, 1920x1080), I think hardware-acceleration is the way to go. At least until CPUs reach 5GHz or so :)

I've download the DirectX VA API specification (v1.01), skimmed, re-skimmed, and re-re-skimmed it multiple times. As far as I can tell, the document has no specific 'profile' for MPEG-4 simple profile (let alone 'advanced simple profile'.) But of the profiles DXVA outlines for H.263 (starting at 'H263_D'), I think a few of them might be able to handle MPEG-4. So far, I've only found 'Quarter-pel' to be definitely NOT supportable (no DXVA equivalent function for it.) B-frames look ok, and I need to read more about GMC.

[Nic]

The only person to even begin to look at was trbarry. I think at that time no video cards were supporting the extensions in any useful way...

Good luck in your search, MS documentation has never been great...

Cheers,
-Nic

ps
Your old website taught me all I needed to know about iDCT when I looked sometime ago, so thanks for that ;)

[trbarry]

I'm not a dshow expert so mostly I just posted a few nagging question here and there and didn't discover anything useful other than that it should be possible.

But this ATI & Divx agreement (http://forum.doom9.org/showthread.php?s=&threadid=42561) is promising, and at least shows they think it's doable.

At the very least it means the Divx player should be able to play Xvid HDTV clips soon. Hopefully some dshow/dxva guru can chime in here with some better info on whether this Divx support could also go into Xvid.

Anybody understand what all this is about?

- Tom

[temporance]

If you have a high-end PC, you can already play 1280x720p24 DivX - I've seen it done. Possibly even 1920x1080p24. OK, so you might not be able to do GMC / qpel but you'll be waiting a very long time for graphics cards that support these modes.

Perhaps the xvid decoder is not quite as fast.

[Cyberace]

Is it ok to bump this (or this other one (http://forum.doom9.org/showthread.php?t=22408)) or should I start a new thread?, (thought I revive the discussion since DivX is now working on it too (http://forums.divx.com/eve/ubb.x/a/tpc/f/776104432/m/255101732)):

Microsoft DirectX Video Acceleration (DirectX VA) support

This feature/function request is both for hardware acceleration (motion compensation) and post-processing via Microsofts DirectX Video Acceleration (DXVA) API (http://msdn.microsoft.com/archive/default.asp?url=/archive/en-us/dx81_c/directx_cpp/htm/directxvideoacceleration.asp), (be nice if the options should be added in to XviD codec config settings (or FFdshow (http://ffdshow.sourceforge.net)) when auto-detect if the video cards supported it and to what extent );

Hope XviD (http://www.xvid.org) or FFdshow (http://ffdshow.sourceforge.net) developer(s) can look into if this could be fully or partially used for most DXVA supported graphic/video controllers?

Summery: Microsoft DirectX Video Acceleration (DXVA) allows DirectShow based software decoders to accelerate video playback directly on the graphics processors (GPU). If your graphics processor supports DXVA and has built-in technology to accelerate MPEG-4 ASP file playback, then DXVA can provide (GPU) hardware acceleration. DVXA is an application programming interface (API) and a corresponding motion compensation (http://msdn.microsoft.com/library/en-us/graphics/hh/graphics/ddraw_2ab799f5-77bc-497c-9c64-68903bb940fb.xml.asp) device driver interface (DDI) for acceleration of digital video decoding. DDIs are also provided as part of DXVA; a deinterlacing DDI (http://msdn.microsoft.com/library/en-us/graphics/hh/graphics/dxvaref_d72180f0-84bc-4acf-b263-6f6c7e9f7b22.xml.asp) for deinterlacing and frame-rate conversion of video content, and a to support ProcAmp DDI (http://msdn.microsoft.com/library/en-us/graphics/hh/graphics/dxvaref_557ca488-5ca9-4595-994f-18eafa171f77.xml.asp) control and postprocessing of video content. DXVA provides an interface definition that is foremost focused on support of MPEG-2 "main profile" video (formally ITU-T H.262 | ISO/IEC 13818-2), but is also intended to support other key video codecs, like example; H.261, and MPEG-1, H.263 (MPEG-4 ASP), and H.264 (MPEG-4 AVC).

A good exampel of the benifits of using DVXA for video acceleration is that you could decode HDTV (720p/1080i) MPEG-4 with a PIII 1400Mhz CPU (possible even less), a task that would normaly require a P4 2,400Mhz CPU if the card does not support DVXA HWMC. Or if you have a faster CPU then you can record something at the same time as you are playing back something else on the same Windows PC, (as with DXVA most of the playback effort gets loaded onto the GPU so the CPU can be doing something more/else).

Other post-processing featutes achivable via DVXA (if the video adapter hardware support it):
- Hardware iDCT motion compensation and subpicture decoding
- Hardware based IDCT (Inverse Discrete Cosine Transform)
- IQ (Inverse quantization)
- Fullframe video playback of HDTV content on slow CPU's (1000Mhz+ ?)
- Independent hardware color controls for video overlay
- Hardware colorspace conversion (YUV 4:2:2 and 4:2:0)
- 5tap horizontal by 3tap vertical filtering
- 8:1 up/down scaling
- Perpixel color keying
- Multiple video windows supported for CSC and filtering
- Subpicture alphablended compositing
- Alpha-Blending Surface

Developer links:
- Microsoft Developer information on DirectX Video Acceleration (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/graphics/hh/graphics/dxvaguide_30c21a63-691d-4ada-b469-08790b3b6c29.xml.asp)
- How Decoders Use IAMVideoAccelerator (http://msdn.microsoft.com/archive/default.asp?url=/archive/en-us/directx9_c/directx/htm/howdecodersuseiamvideoaccelerator.asp)
- Mapping DirectX Video Acceleration to IAMVideoAccelerator (http://msdn.microsoft.com/archive/default.asp?url=/archive/en-us/directx9_c/directx/htm/52mappingdirectxvatoiamvideoaccelerator.asp)
- DVXA API/DDI Specification (Rev 1.0) (DirectX 8.1 C++ Archive) (http://msdn.microsoft.com/archive/en-us/dx81_c/directx_cpp/htm/directxvideoacceleration.asp)
- DirectShow DirectX Video Acceleration Video Subtypes (http://msdn.microsoft.com/library/en-us/directshow/htm/directxvideoaccelerationdxvavideosubtypes.asp)
- Enabling DirectX Video Acceleration in a custom player (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wmform95/htm/enablingdirectxvideoacceleration.asp)
- Windows Media Format 9.5 SDK Enabling DirectX Video Acceleration (http://msdn.microsoft.com/library/en-us/wmform95/htm/enablingdirectxvideoacceleration.asp)
- DirectX Video Acceleration Video Subtypes (DirectX 9.0 C++ Archive) (http://msdn.microsoft.com/archive/en-us/directx9_c/directx/htm/directxvideoaccelerationdxvavideosubtypes.asp)
- IAMVideoAccelerator Operational Specification (DirectX 9.0 C++ Archive) (http://msdn.microsoft.com/archive/en-us/directx9_c/directx/htm/522directxvaiamvideoacceleratoroperationalspecification.asp)
- DirectX Video Acceleration Motion Compensation Callbacks (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/graphics/hh/graphics/ddraw_28178ef1-ce67-476d-bdf0-f854fc342b4c.xml.asp)
- Calling the Deinterlace DDI from a User-Mode Component (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/graphics/hh/graphics/dxvaguide_6691221e-f352-47f9-8345-b38d0955fd2a.xml.asp)
- Per-Pixel Alpha Blending (DirectX 9.0 C++ Archive) (http://msdn.microsoft.com/archive/en-us/directx9_c/directx/htm/perpixelalphablending.asp)
- Video Content for Deinterlace and Frame-Rate Conversion using DXVA (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/graphics/hh/graphics/dxvaguide_a72c9f34-2b11-48d2-9d4d-5be85933cdbd.xml.asp)
....AND MUCH MORE available on a Microsoft Development Network search (http://search.microsoft.com/search/results.aspx?qu=dxva&View=msdn&st=b&c=0&s=1&swc=0)

PS! I know most GPUs don't support DVXA HWMC for MPEG-4 ASP yet but more will sure to come soon, at least the latest ATI Radeoon-Series, nVIDIA GeForce 7800 series (PureVideo™ (http://www.nvidia.com/page/purevideo.html)), also VIA's Embedded Platforms/Mainboards (EPIA) (http://www.viaembedded.com), GPU/IGP (GammaChrome/CN400) (http://www.via.com.tw/en/products/graphics/) and processor (Luke) (http://www.hexus.net/content/reviews/review.php?dXJsX3Jldmlld19JRD0xMjIz) do for starters ;)), todays programmable vertex/pixel/ARB shaders (2.0 and 3.0) are really more than capable to blow your mind on this :D ...not to mention future GPU technologies and the next DirectX version which will give programmers even more control

[bill_baroud]

If you look on the Xvid forums, there was a guy who started something with DXVA, but there was some problems, iirc :
- the dxva doc is old, and not updated anymore. It's seems that Microsoft doesn't want other people to use it freely anymore (read : w/o paying big bucks)
- the gpu he was trying to use didn't respond correctly to his dxva request... like said OK to any configuration... which was obviously not the case.
- Nvidia video support was buggy on NV3X and even NV40, or they want you to think that. They much prefer sell their own mpeg2 decoder rather than allow everybody to do the same for free.
- the last update of dxva was for WMP10 _only_

Well, that's my totally wilds thoughts on the question, but i think that every actor in this particular sheme (MS and the GPU manufactors) doesn't want independent individuals to develop some tools using those functions. So we're screwed.

[el divx]

If you look on the Xvid forums, there was a guy who started something with DXVA, but there was some problems, iirc :
- the dxva doc is old, and not updated anymore. It's seems that Microsoft doesn't want other people to use it freely anymore (read : w/o paying big bucks)
- the gpu he was trying to use didn't respond correctly to his dxva request... like said OK to any configuration... which was obviously not the case.
- Nvidia video support was buggy on NV3X and even NV40, or they want you to think that. They much prefer sell their own mpeg2 decoder rather than allow everybody to do the same for free.
- the last update of dxva was for WMP10 _only_

Well, that's my totally wilds thoughts on the question, but i think that every actor in this particular sheme (MS and the GPU manufactors) doesn't want independent individuals to develop some tools using those functions. So we're screwed.

Someone could then reverse engineer some of the existing players/filters/products that use DXVA, having alongside the documentation provided by MS and/or GPU makers, see how it's done and do something similar and maybe better, no? It would be at least a starting point.

[Koepi]

lol.

Sure. The only tiny problem is to find someone willing to do that. There's nobody, else you'd already see a result.

[el divx]

It was just a thought. No need to become sarcastic.

Anyway, the postprocessing could be done through shaders by sending the video down the 3D pipeline. Gabest has i.e. included a shader in mpc for doing deinterlace. Something similar could be done in xvid's ds filter and in opengl for the linux and mac os decoder, although I may be totally wrong about it.

[mimungr]

The DXVA docs are both current and free. Do a search for "DirectX Video Acceleration Interfaces" on MSDN.

It seems unlikely that there are any undocumented, WMP-only APIs. That would be such a flagrant violation of the court settlement that even Microsoft wouldn't be that reckless. If you know of any, send an e-mail to some of Microsoft's competitors :). I think the update that bill_baroud is referring to was to add DXVA support to WMP10, not to add secret WMP10-only APIs to the OS. This is a good thing. If WMP10 can take advantage of DXVA, perhaps Microsoft with its considerable clout will start pressuring GPU manufacturers to improve their support.

Please don't flame me. I'm not defending Microsoft. I just don't want devs to be unnecessarily discouraged from using DXVA.

[Cyberace]

Sounds to me like when the XviD devs last looked at DXVA (back in 2003) Microsoft docs where not accurate and/or not up-to-date, but it looks to me like the docs today (see links in my post above) are both up-to-date and much more detailed than before, (though I am not a dev myself), ...however Koepi is right, the problem now is finding a skilled enough developer who's willing to volenteer (and owns or can get a graphic-adapter that supports DXVA for MPEG-4), then there is two aspects of DXVA; video-acceleration and post-processing, (post-processing which in turn has several different features), and that dev might only be insteresting in one of those.

[bill_baroud]

well it's true that i didn't go read the new docs, my statement was just based on what i find on the subject when i investigated it, last year. All the dxva docs was dated from 2001, so i wasn't expecting something new after another year. Well i stand corrected and i'll write down 100 times "I must read the documents linked before replying to a thread"


and just a thought on gpu drivers reverse engineering : given the hard times Linux guys have with nvidia drivers (and all their tricks to hide the registers used), don't expect anything on something totally undocumented and unsupported by nvidia (strange, the ultra-w00t kickass PureVideo Engine 2.0 of my FX5900 totally disappeared from the commercial product pages...)

[LiFe]

Related Technology:
http://arstechnica.com/reviews/os/macosx-10.4.ars/16
http://www.apple.com/macosx/features/coreimage/
Apple's new Core technology builds filters dynamically using OpenGL, which is then uploaded to the GPU and processed in real time.

The list of built in Image units is impressive, all which can be used on video.

The way they've developed the Image units leads one to believe they have paved the way for Audio units as well (Core Audio exists, but they havn't implemented it in hardware yet).

[Cyberace]

@LiFe, thay may be indirectly related but it's not the same and thus off-topic, DXVA is a Microsoft standard for Windows only.

[bill_baroud]

Okay, i re-read the docs linked and they are :
- outdated (the same i read 1 year ago) and marked as such by microsoft
- useless (how to use the WMV9 DMO ...)
- irrevelent (how to wrote drivers callback to support DXVA).

If you didn't read how dxva works, basically, the video renderer expose some interface on his input pin that say it can be used for hardware acceleration, the decoder then send (after negociation) data in a known structure to the video renderer, which then call some driver callbacks to do the hardware stuff and render the video.
So now you see easily where we are screwed : if nvidia or ati doesn't include (or hide/mess/whatever) any callbacks in their drivers, you can't use dxva.

The only solution is to go the hard way and use the programmable pipelines for DCT and ME/MC...

A good start would be :
http://www.mee.tcd.ie/~sigmedia/publications/publis/spie2004/spie_interp_04.pdf
and
http://download.developer.nvidia.com/developer/SDK/Individual_Samples/samples.html
http://download.developer.nvidia.com/developer/SDK/Individual_Samples/DEMOS/OpenGL/gpgpu_dct.zip