Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[JanWillem32]

Originally Posted by egur

I managed to reproduce the drag & drop causing a freeze to MPC-HC. ffdshow.ax stay resident in memory but my decoder DLL is unloaded. Looks like some kind of race condition. Almost impossible to debug a this doesn't occur if I place breakpoints
I couldn't reproduce with ZoomPlayer, maybe MPC-HC is handling the loading differently. If I knew how, I could find a solution. Unless I'm wrong here, this is a low-medium priority bug. If an MPC-HC developer can give a hint that would help.

I'm quite familiar with the MPC-HC graph builder and related parts. There are several initialization problems. I'm currently evaluating two fixes for problems that are indeed causing race conditions (and a scaling bug for DVD menus). I'm also looking at possibilities for seamless playback. The default settings for debugging MPC-HC are okay for a quick checkup on a project, but not for core debugging of renderers, profile-guided optimization, assembly checkup, et cetera.

Originally Posted by egur

Bicubic is not good enough when scaling factors are high (>2), the image is a little blurry. Bilinear shouldn't be used for anything as it creates horrible scaling artifacts.
The SNB HW scaler has the capability to downgrade to Lanczos4, Lanczos3, Lanczos2 and all the bicubic variants. I'm not sure if the driver supports any of these modes though, but I'll check. It makes sense to utilize it in any case.

A few questions for everyone:
* Does the EVR have an interface to configure the scaler quality?
* Is the EVR-CP the same one supplied with Media SDK 4.0b4 (they have the same file name)?
* A tough one - does anyone know how to create a virtual adapter - so DXVA can enumerate a GPU not connected to a screen?

The basic VMR and EVR property pages can be called when the appropriate DLL files are registered for it (proppage.dll and evrprop.dll, separate versions exist for x86 and x64).
For reading the additional EVR-CP mixer settings, querying the mixer interface is required. The same rule applies for applying most settings. As for the scaling quality, I've simply assumed it's always bilinear coming out of the scaling by VMR and EVR judging by the type of square or rectangular scaling artifacts and blur.

As for the variants of custom renderers, MPC-HC has one. It's completely shared between the VMR-9 r., EVR-CP, RealMedia DX9 and Quicktime DX9 mixers. EVR Sync contains a nearly 1:1 copy of that code. It evolved from the DirectX 7, then 8 renderer that was there before it. I never liked it.

When I decided that I could come up with something better, I dumped the renderer core and started developing: http://forum.doom9.org/showthread.php?t=161047 .
In the beginning, the renderer wouldn't work properly at all, but I'm slowly making progress over time. Once I've finally figured out how to receive the raw bits from a DirectShow pin (or if anyone would help with that part), I'll also add a custom mixer in time to replace the "borrowed" mixers. I'm already trying to disable all possible built-in filters of the mixers and making implementations of useful filters in the custom renderer core. That includes all resizing options (currently doesn't always work for disabling chroma filtering, though). The handling of deinterlacing at the mixer level is an abomination, by the way.

A few virtual DirectX 9 COM pointer functions can be derived from a DirectX 10/11 device, but that's for expert-level DirectX programmers. The resource management is difficult.
For DirectX usage without a screen, look into DirectX 10.1 and 11 DirectCompute. Else, just ignore the implicit swap chain completely and set the window handle to invisible.

I can assist with debugging. I can upload a full source code of what I'm working on, explain the debug methods for some parts and help with other things in my field of expertise.

[CruNcher]

Something is strange im trying ffdshow-quicksync with MPC-HC test and their is some strange thing going on with the Internal OSD not the one of MPC-HC but the one of ffdshow and Colorspaces, if the OSD is off the output from ffdshow-quicksync is NV12 as it should be and if i enabled the OSD while playback it stays @ NV12 but if i leave the OSD on now and reopen the file the output from ffdshow-quicksync changes to YV12 ?????

Happens also with the normal MPC-HC so it seems to be a ffdshow thing.


Open the OSD while playback:



OSD still enabled on next file open:

[JanWillem32]

Both NV12 and YV12 are valid types for transporting progressive Y'CbCr 4:2:0 video, but it's indeed a bit odd that the type changes on re-opening. The mixer input format is a bigger problem: "YUY2" indicates format conversion to an incompatible type.
Can you take a look at the output pins of the video parts in the "Play", "Filters" menu? Neither EVR or VMR-9 should ever use a YUY2 input pin if NV12, YV12, IYUV or I420 is offered. A problem factor can be insertion of the color space converter filter.
You can get a full graphic overview of the DirectShow playback chain with GraphEdit. Simply use "Connect to Remote Graph...". You might be able to find out things easier when using that interface.
For an overview of all supported video decoder and processor types, with and without deinterlacing, see DXVAChecker. If there are no compatible processor types for Y'CbCr 4:2:0 video, we will have to take a look at the options for color space conversion to a more proper type than YUY2.

[CruNcher]

Hmm it remembers me that ffdshows NV12 implementation might be buggy anyways if your mixer cant recognize it there where some heavy issues with it's NV12 when put into a full Hardware decoding chain without any conversions (it crashed the whole rendering) of Nvidias Nvcuvid it's a little problematic to repeat that test with the Quicksync Decoder though as ffdshow is now a part of it

http://forum.doom9.org/showthread.ph...ow#post1501021


PS: I cant connect to the remote graph tried both with Elevated Admin rights (Graphstudio 64, MPC-HC Tester 64) but it doesn't connect, i see the Graph but it wont connect nothing happens if i push connect the Graph isn't loaded.
And with Graphedit 64 from the Windows SDK it doesn't even shows any Graph to connect to @ all




Here it is (this is a mockup i wish MPC-HC would be finally able to do this window multitasking without interrupting its workflow )

[nevcairiel]

ffdshows problem is with raw NV12 input, and the problem is quite obvious if you get it.

[egur]

ffdshow has issues with NV12 - libavcodec bug actually. It crashes when copying NV12 surfaces in some cases. ffdshow has an alternative method which works fine - that's what I used.
Regarding the connection issues with MPC-HC. It should be fixed for the next release. I'm almost done with the time stamp code fix so a release is very close. Probably last release before integration with ffdshow's official code base.

[egur]

Originally Posted by JanWillem32

A few virtual DirectX 9 COM pointer functions can be derived from a DirectX 10/11 device, but that's for expert-level DirectX programmers. The resource management is difficult.
For DirectX usage without a screen, look into DirectX 10.1 and 11 DirectCompute. Else, just ignore the implicit swap chain completely and set the window handle to invisible.

Thanks for the explanations, but I need a DirectX adapter to be enumurated for a screen less adapter. Otherwise the Media SDK will not initialize.
Do you know why the EVR CP drops frames? There's plenty of compute headroom and it falls to ~30fps, dropping about half the frames.

[CruNcher]

@jan

is it possible to keep this active by default for EVR-CP (or selectable via the EVR-CP config tab,or even better directly in the Menu Options bound to the stop graph call it needs) ? it works but it disables itself after a close and you have to enable it again (would be much easier to have access via the normal menu for on demand usage)



That seems to be Intels Hardware Deinterlacer

And this Intels IVTC




The Deinterlacer depending on the Interlaced frames config though fails sometimes in terms of motion results (i wonder if it does any frame analyzing @ all as it shows NumForwardRefSamples = 0 NumBackwardRefSamples = 0) but better then nothing if no flag is available combined with the flag call from the decoder, it seems a nice combo, and it doesn't really seem to hit the progressive frames

I wonder if its possible to get access to the Denoiser (NoiseFilterTechnology) and Sharpener (DetailFilterTechnology) also via this DXVA2 interface it suggests so

[JanWillem32]

Originally Posted by egur

Thanks for the explanations, but I need a DirectX adapter to be enumurated for a screen less adapter. Otherwise the Media SDK will not initialize.
Do you know why the EVR CP drops frames? There's plenty of compute headroom and it falls to ~30fps, dropping about half the frames.

If you're trying to use the internal VSync and/or flush functions, you'll see that the GPU will be doing nothing about half of the time. These functions are made to flush the command cue, hold the GPU and the paint thread (on the CPU) inactive until the estimated next VBlank period, and then call a present. After the present call, the paint sequence for the next frame is stalled for a while, until the scheduler is sure that the next frame won't be presented early. In that time the presenter thread and GPU are idle.
A normal renderer never flushes and rarely stalls, except for reset and world transition situations. It's pretty normal to allow up to about 3 fully rendered frames in a queue.
The trunk MPC-HC build renderer also can't properly queue even basic drawing sequence commands, let alone queue entire frames.
For the first question, are you looking for the IDirect3D9 adapter functions? http://msdn.microsoft.com/en-us/libr...=VS.85%29.aspx
When creating a DX9 or DX9Ex device, the default adapter is always given the number 0. Don't forget to give a private HWND input pointer if you intend to make the window invisible (don't use the one of the main window). An ignored swapchain will generally be 1×1 in size, and have a single back buffer, that is never used. Not creating an implicit swap chain is not allowed under DirectX 9.
I'm looking forward to your fixes, I'll keep an eye on this thread to see if I can help with anything.

@CruNcher: Video processor devices are registered by the display driver and are also handled by the display driver in the external EVR mixer phase, as a black box to the video application. There are some options the video application can set, but most items are are vendor-specific and many settings are simply ignored when initializing the external mixer. Graphics drivers should have a tab for video options in their control panel.
I was already not amused by setting up the color controls for EVR on the Miscellaneous page. These sort of controls won't ever work for RGB input types, are a duplicate of those in most recent control panels and most importantly, are not transparent to the user what software is responsible for executing this filter (not the video player itself in this case). That's why I'm against integrating these sorts of controls in MPC-HC.
In the pictures you see the debug window for the loaded EVR filter. It doesn't keep settings. It's the graphics driver's job to write out defaults and settings to the registry, and in the the case of interlacing types, even for several scenarios. Remember that the vanilla EVR also doesn't come with a regular settings panel for these sorts of things, too.

[egur]

Originally Posted by JanWillem32

If you're trying to use the internal VSync and/or flush functions, you'll see that the GPU will be doing nothing about half of the time. These functions are made to flush the command cue, hold the GPU and the paint thread (on the CPU) inactive until the estimated next VBlank period, and then call a present. After the present call, the paint sequence for the next frame is stalled for a while, until the scheduler is sure that the next frame won't be presented early. In that time the presenter thread and GPU are idle.
A normal renderer never flushes and rarely stalls, except for reset and world transition situations. It's pretty normal to allow up to about 3 fully rendered frames in a queue.
The trunk MPC-HC build renderer also can't properly queue even basic drawing sequence commands, let alone queue entire frames.
For the first question, are you looking for the IDirect3D9 adapter functions? http://msdn.microsoft.com/en-us/libr...=VS.85%29.aspx
When creating a DX9 or DX9Ex device, the default adapter is always given the number 0. Don't forget to give a private HWND input pointer if you intend to make the window invisible (don't use the one of the main window). An ignored swapchain will generally be 1×1 in size, and have a single back buffer, that is never used. Not creating an implicit swap chain is not allowed under DirectX 9.
I'm looking forward to your fixes, I'll keep an eye on this thread to see if I can help with anything.

Thanks for the explanations; they should come in handy in the future. Unfortunately, the MSDK initialization (educated guess) queries the IDirect3D9 interface for available adaptors via calls to GetAdapterCount and GetAdapterIdentifier and look for an Intel GPU. When the Intel GPU is not connected to a monitor, it will not enumerate in the above calls. My home setup has a Radeon connected to screen. If the monitor is connected to the Radeon, the Intel GPU will be hidden. It will show in Windows device manager, but not accessible from DirectX.
The big question is how do I force DirectX to enumerate it?
Is there a way to programmatically connect the disconnected GPU to a virtual monitor?

[JanWillem32]

A virtual monitor driver is new to me. I've seen functions to force enable an analog "tv" output, though. It installs a standard VGA monitor on an adapter and forces it to output.
You can try something else first. If the combination of GetAdapterCount and GetAdapterIdentifier doesn't return the adapter you are looking for, EnumAdapters1 probably will (although I've never used it before): http://msdn.microsoft.com/en-us/libr...=VS.85%29.aspx . I don't know if the DXVA helper function can work on a DXGI-derived device. I've never tried to derive one for functions like that.

[CruNcher]

hmm maybe this can be somehow helpfull http://channel9.msdn.com/Events/BUIL...D2011/SAC-217T http://channel9.msdn.com/Events/BUILD/BUILD2011/HW-220C ? though these talks are more targeted @ Windows 8 and it might be better to look @ the PDC 2008 talks and the introduction of 7 though only these talks are interesting multimedia wise http://channel9.msdn.com/Events/PDC/PDC08/PC04 http://channel9.msdn.com/Events/PDC/PDC08/PC05 http://channel9.msdn.com/Events/PDC/PDC08/PC07 though only the 2011 talks go deep into the WDDM and virtual display driver.

[egur]

New and improved version. Zip files contains installer and documentation, please read.

Download version 0.15 alpha:
32 bit http://www.multiupload.com/SW88AXIEAR
64 bit http://www.multiupload.com/3QH5R6N6CD
Source code http://www.multiupload.com/GQBEQ161DB

Revision highlights:
v1.15:
* Rewrote time stamp handling code. Decoder now calculates frame rate if missing, corrects for splitters reporting double frame rate for interlaced content. Handles PTS and DTS time stamps. Broken streams that alternate frequently between telecined and interlaced frames are not handles perfectly (yet!).
* Handled unsupported H264 formats by reverting to libavcodec silently within ffdshow. HW acceleration is limited to H264 simple, main and high profiles. Previous version would crash on unsupported formats.
* Added support for WMV3 (part of the VC1 HW decoder).
* Various bug fixes and better decoder error handling. As reported by various users for the 0.14 release.
* Cleaned up minor memory leaks.

[CruNcher]

Egur nice also some progress on my QuickSync Decode/Transcode Capture Framework (this is really the nicest piece of Hardware i ever used, Z510 and US15W was already impressive to work with but this kills everything, just thinking about ivy bridge and haswell and more performance @ lower watt geez)

http://www.mediafire.com/?fitumy3c9qf3p31

[egur]

I succeeded in running the QS decoder on the Intel GPU and EVR on a discrete GPU (Radeon HD6950).
Steps to reproduce:
* Connect discrete card to monitor.
* Connect IGP to a second input on the same monitor, wait for the driver to recognize it (might need to manually switch the monitor input on the monitor itself). Win7 extends the desktop the "new" monitor.
* Switch back to the main input.
* Play video.
* Test setup by messing with controls on the discrete GPU control panel (e.g. lower saturation to zero).

This survived a reboot so it's a one time setup.
Both control panels for the GPUs are now functional.
I'll test this with MadVR today and update my post.

v0.15 is not compatible with this setup (bug - couldn't test this ), but next version will support it.
If it's very important, I'll release it ASAP.

[nevcairiel]

There is one really annoying thing with that setup though. There is no longer a mouse boundary, as your desktop permanently expands to the second screen, even if its not selected as an active input.

[egur]

Originally Posted by nevcairiel

There is one really annoying thing with that setup though. There is no longer a mouse boundary, as your desktop permanently expands to the second screen, even if its not selected as an active input.

True - an ugly hack. Until a SW solution is found, this can be good for testing/evaluation.

[ajp_anton]

But what about falling back to another decoder (not just libavcodec within ffdshow) for unsupported streams?
ffdshow can't output some of them directly so it converts them to RGB. LAV video works better there, and is also faster.

[egur]

Originally Posted by ajp_anton

But what about falling back to another decoder (not just libavcodec within ffdshow) for unsupported streams?
ffdshow can't output some of them directly so it converts them to RGB. LAV video works better there, and is also faster.

My decoder only outputs NV12 and decodes several types of streams. If it fails to initialize for any reason ffdshow will choose the default internal decoder - usually libavcodec.
Output conversion (raw video) has nothing to do with my code, it's negotiated with the filter connected downstream. The downstream filter decides the raw format.
What do you mean by LAV is faster? What scenario?

[nevcairiel]

He wants to use LAV for 10bit H264 and other formats not compatible with hardware decoding, because ffdshow has some limitations decoding those
My short answer would be to wait until LAV supports Intel MSDK as well.....