Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[egur]

Originally Posted by pirlouy

Sorry if it has been asked.
Why QuickSync needs Intel GPU to be active ? From what I've read, I thought it was independant. Is there a possibility to use QuickSync to decode, and then send it to RAM for third party GPU for example ?
Is it an Intel limitation, or it's just something you don't have overlooked right now ?

It's part of the Intel GPU (physically) and operated by the Intel GPU driver.
My decoder copies the frames back to system memory so you can use a renderer on another GPU. You'll need an H67/Z68 chipset for this to work. See this post on how to enable multi GPU setup: http://forum.doom9.org/showthread.ph...86#post1532786

[hajj_3]

Egur, i don't suppose you know whether it is technically possible for sandy bridge's quicksync to hardware decode HEVC a.k.a h.265 when that is launched in 2013 or will the hardware on the gpu not be powerful enough to do it? It would be great if sandy/ivy bridge could hardware decode that with a software upgrade otherwise current pc's likely won't have the cpu power to decode it as it looks to be extremely cpu intensive.

[CruNcher]

@ Egur
i wont have the time to test both lav video and ffdshow-quicksync (i will though do certain cross compares on strange issues and cases) but primarily i will do all further tests with Lav Video i hope nev can help improve all the time stamping code in ffdshow-quicksync so that both will work perfectly in the end with Lav Splitter @ least for .ts now

if you cant reproduce it i guess we really need Jan here too i also get the same with Lav Video the actual decision when it recovers the lock seems non deterministic except when you seek

I'll make a Video of it (i hope i can capture it)

[nevcairiel]

Originally Posted by egur

update
Well the Media SDK doesn't report an aspect ratio change.
FFDShow with either limpeg2 or libavcodec do the same - no aspect ratio change. Tried with Haali splitter as well.
LAV video decoder + splitter (0.43) doesn't change AR as well (32 bit anyway).

Works for me in software decoding or CUVID decoding in LAV Video. The splitter plays no role here (or should not, its possible that Haali breaks it, because of its ugly stream AR overwriting)
Just confirmed that its indeed working just fine with LAV Splitter + LAV Video, both Software (avcodec) and CUVID. Seeking anywhere after the 0:27 mark switches AR instantly, seeking back to the start switches AR back. Playing the file from start to end switches at the proper time as well. (Make sure "Use Stream AR" option is on)

Also tested ffdshow (a rather old version i had around, r3967), and both avcodec and libmpeg2 switch the AR properly.

All tests in MPC-HC with vanilla EVR as well as EVR Custom.

[egur]

Originally Posted by nevcairiel

Works for me in software decoding or CUVID decoding in LAV Video. The splitter plays no role here (or should not, its possible that Haali breaks it, because of its ugly stream AR overwriting)
Just confirmed that its indeed working just fine with LAV Splitter + LAV Video, both Software (avcodec) and CUVID. Seeking anywhere after the 0:27 mark switches AR instantly, seeking back to the start switches AR back. Playing the file from start to end switches at the proper time as well. (Make sure "Use Stream AR" option is on)

Also tested ffdshow (a rather old version i had around, r3967), and both avcodec and libmpeg2 switch the AR properly.

All tests in MPC-HC with vanilla EVR as well as EVR Custom.

I tested with MPH-HC and saw what you described. ZoomPlayer have an issue with this clip as it doesn't change the renderer's AR.
I'll dig in deeper, but a quick fix is not probable.

[CruNcher]

I did some tests with this lock problem and tried this @ two refresh rates and indeed it locks faster @ 60 Hz then 75 Hz :P
@ 75 Hz it can be problematic as such that it takes longer or doesn't lock @ all not sure where this depends on Aero i guess, though im almost sure now this is more something on Jans Render side then a Splitter/Decoder issue Also the behavior on 60 hz Aero of both is pretty identical it deterministically never misses the lock and locks always after 20 second only the way to the lock differs a little

60 Hz (lock takes 20 seconds)

Fddshow Quicksync = http://www.multiupload.com/F3NDGFYDF7
Lav Video Quicksync = http://www.multiupload.com/NDDB6U5YU8

75 Hz (lock doesn't happen or takes very long, non deterministic, seeking forces it)

Fddshow Quicksync = http://www.multiupload.com/PMFRYQEWPI
Lav Video Quicksync = http://www.multiupload.com/VPG09HCV1H

PS: Btw trying to capture this with my GPU recording framework (Quicksync) failed same for the software based x264 counterpart this needs very low latency to be captured in Realtime Mirillis Low Latency I frame Codec was capable of achiving it without modifying the outcome result

[NikosD]

Originally Posted by NikosD

1) After the installation of latest driver Intel 15.22.52.2559 I found 3 MFT decoders by Intel at C:\Program Files\Common Files\Intel\Media SDK\s1\2.0\

The names are Intel Hardware H.264/MPEG-2/VC-1 Decoder MFT.

But during the enumeration of available codecs in DXVA Checker when I try to benchmark a video file, those decoders never show up.

Why?

Also in their properties they don't seem to have a DXVA option (enable/disable)

Originally Posted by egur

I don't know - I'm not part of the Media SDK dev team nor the graphics driver team. I'll forward your question.

Originally Posted by NikosD

4) Why Intel restricts such a POWERFUL DECODER like QS for 1920x1080 only?

I think that the driver's team should "open" the driver up to 4K x 2K that QS could handle with ease.

And of course your decoder and every other decoder using QS must be updated too, to include 4K x 2K.

Originally Posted by egur

See answer #1. My guess would be that it made the HW more expensive and not worth the cost. I'll forward the question.

Eric hi.

Any feedback from the Intel Media and drivers team for the above questions ?

Now that latest beta from PotPlayer v1.5.31323 supports DXVA H.264 4K x 2K resolutions with it's internal codec, it would be useful for us to check it out.

We only need Intel to update the drivers.

You could add those resolutions to your project too.

[Blight]

Thanks to an insight from nev, Zoom Player v8.1 final will support dynamic aspect ratio changes with EVR.
I just verified that it works against the test clip posted earlier with both LAV and Haali as the source filters

ETA to v8.1 release, ~4h

[CruNcher]

Originally Posted by NikosD

Eric hi.

Any feedback from the Intel Media and drivers team for the above questions ?

Now that latest beta from PotPlayer v1.5.31323 supports DXVA H.264 4K x 2K resolutions with it's internal codec, it would be useful for us to check it out.

We only need Intel to update the drivers.

You could add those resolutions to your project too.

4K works @ least non DXVA it seems



Though im not sure if it might have fallen back to Intels Software Decoding Core or Libav ? (ill try to check)

[hajj_3]

wouldn't 4k be 2160p (3840x2160)?

4096 x 2304 seems an odd size?

[CruNcher]

3840x2160 that's QFHD (2160p)
4096 x 2304 is 4K @ 16:9 AR
Full 4K 4096 × 3112

http://www.youtube.com/view_play_lis...F9E09ECEC8F88F

PS: I found out that Arcsofts Decoder on Intel is by default (when called from Directshow directly, not sure about TMT5) (@ least for VC-1) a "Copy Back Decoder" (and it has very poor performance the heavier the stream gets, comparable to what ffdshow-quicksync reached in the beginning without all the Copy improvements) :P

@Egur

So with this my testing @ least for playback stability of .ts (Lav Splitter) and with quicksync (ffdshow-quicksync,lav video quicksync) ended, i will finish the deinterlace test on lav video and then finally move on to .mp4/.mov/.mkv/.wmv/.mpg/.m2ts so only the 2 issues are left the Sync problems with those damaged streams that work fine @ Software Decoding (Libav) and this strange Decoding error and what i guess wont be fixed anymore on the Driver side the MC.ts VC-1 interlace issue
I hope especially someone is able to root cause the decoding error the Sync issue for damaged streams seems not so critical as it can be fixed manually by a short seek @ playback (though indeed i didn't tested if it becomes maybe unsync after some time again) and also not sure what happens @ Encoding i guess end result would be unsync though

Left Decoding stability issues ( Lav Splitter + FFdshow-quicksync/Lav Video Quicksync + Lav Audio Decoder):

Decoding Error = http://www.mediafire.com/?rld8gnlh52f03ud (not good) (Cyberlink DXVA = Fail, CoreAVC DXVA = OK, CoreAVC = OK, Arcsoft DXVA = OK, ffdshow-quicksync = Fail, Lav Video = OK, Lav Video Quicksync = Fail, Mainconcept DXVA = Fail, Mainconcept = Fail, Microsoft DTV = Fail, DivX = Fail, Mirillis = Fail, Potplayer DXVA = OK)
Sync issue = http://ibc.cdngc.net/Avidan/dongle_3.ts (Sync can be forced by seeking)

also Nev should look @ those last 2 issues from the Parser/Splitter part

[egur]

@NikosD
I had a crazy week and didn't get to do it.
Did you try posting on the driver or media sdk support forums?

I can't manage to download any of the 4k clips, can anyone help/share?

[CruNcher]

http://www.mediafire.com/?yn7pa6xe0cdp5cx

but they are rather easy to decode hence youtube and low bitrate harder are the QFHD samples with enormous bitrates 50 Mbps and up

[egur]

Originally Posted by CruNcher

PS: I found out that Arcsofts Decoder on Intel is by default (when called from Directshow directly, not sure about TMT5) (@ least for VC-1) a "Copy Back Decoder" (and it has very poor performance the heavier the stream gets, comparable to what ffdshow-quicksync reached in the beginning without all the Copy improvements) :P

This is very odd. How do you they use DXVA?
BTW, with frame copying the resolution and frame rate is what makes the difference, not the bitrate.

Originally Posted by CruNcher

So with this my testing @ least for playback stability of .ts (Lav Splitter) and with quicksync (ffdshow-quicksync,lav video quicksync) ended, i will finish the deinterlace test on lav video and then finally move on to .mp4/.mov/.mkv/.wmv/.mpg/.m2ts so only the 2 issues are left the Sync problems with those damaged streams that work fine @ Software Decoding (Libav) and this strange Decoding error and what i guess wont be fixed anymore on the Driver side the MC.ts VC-1 interlace issue
I hope especially someone is able to root cause the decoding error the Sync issue for damaged streams seems not so critical as it can be fixed manually by a short seek @ playback (though indeed i didn't tested if it becomes maybe unsync after some time again) and also not sure what happens @ Encoding i guess end result would be unsync though

Well, you know the last 1% is always the hardest...

On another matter, I'll release another update in a few days. There's a small bug related to multi threading. Afterwards, I want to really optimize the multithreaded path.
When this is done, it's time to put the video processing in.

[dukey]

Egur since you work for Intel. Can you get someone to fix the opengl drivers ? There's a list of serious problems with them .. In fact it's so bad it's enough to make developers want to abandon opengl.

eg
Intel -> http://i126.photobucket.com/albums/p...uff/image1.png
ATI/Nvidia -> http://i126.photobucket.com/albums/p...uff/Image2.png

[egur]

Originally Posted by dukey

Egur since you work for Intel. Can you get someone to fix the opengl drivers ? There's a list of serious problems with them .. In fact it's so bad it's enough to make developers want to abandon opengl.

eg
Intel -> http://i126.photobucket.com/albums/p...uff/image1.png
ATI/Nvidia -> http://i126.photobucket.com/albums/p...uff/Image2.png

Sorry, please post in driver support forum.

[NikosD]

Originally Posted by egur

@NikosD
I had a crazy week and didn't get to do it.
Did you try posting on the driver or media sdk support forums?

I can't manage to download any of the 4k clips, can anyone help/share?

I didn't post there because we have Intel here

Here you are for the samples:

http://xhmikosr.1f0.de/index.php?fol...xlcy8yMTYwcA==

[CruNcher]

Originally Posted by egur

This is very odd. How do you they use DXVA?
BTW, with frame copying the resolution and frame rate is what makes the difference, not the bitrate.



Well, you know the last 1% is always the hardest...

On another matter, I'll release another update in a few days. There's a small bug related to multi threading. Afterwards, I want to really optimize the multithreaded path.
When this is done, it's time to put the video processing in.

Yep that could maybe explain why it crashes very strangely @ a 720p 50 fps clip Microsofts 2011 Build Keynote and inside TMT5 it gets totally unsync :P

with higher res 1080p it works outside but i get 15 fps for a 30 fps clip and im pretty sure those are no parser issues

anyways here is a QFHD sample (Mainconcept) http://115.com/file/be83t4l4# it's heavier then the Life in the Garden 4K Bitrate wise

this really pressures Quicksync heavily and it gives it a run for its money also it seems Lav Video Quicksync Performed a tad better then ffdshow-quicksync but mostly performance is the same with audio brake ups and fps brakes, when changing to Software Decoding its flawless so im pretty sure Quicksync Decoded it (and IO was also no issue). Though i guess it doesn't even have todo with the Decoder but the Memory Copy i guess the Hardware would be capable of playing this flawless ?.
Trying some of the DXVA decoder on it

Nope DXVA fails as others already said or it falls back to Software Decoding

QFHD 50Mbps H 5.1

Lav Video



Lav Video Quicksync



So for now with this performance i would make the suggestion to fallback to Software (Libav) for these complexities though with the 4K youtube it has less problems so it wont be enough to just decide this based on resolution i guess

Lav Video Quicksync "Youtube 4K" H 5.1 Max 19.4 Mbps



So that runs rather ok but seeing those spikes i guess that's where the bitrate shoots higher and indeed the Quicksync Decoder gets problems to cope with that Bitrate Spikes + Resolution it seems. So if that starts here already it's no wonder it's dying with the QFHD sample though lets analyse the GPU MFX pressure (that's where it would be nice to have it directly in the OSD + Power consumption, craziest of course as another Realtime Graph )

So there are 2 possibilities

Either the Decoder cant cope with the bitrate (most probably) or the +~6 fps are enough pressure on the copy side to cause the playback to endup like this (doubtful)

And so in the first case no DXVA would help here and Sandy Bridge would only be able to play very restricted 4K @ all (even youtube looks slightly to much for it )

[egur]

Originally Posted by CruNcher

anyways here is a QFHD sample (Mainconcept) http://115.com/file/be83t4l4# it's heavier then the Life in the Garden 4K Bitrate wise

Can you share on multiupload, download breaks all the time.

[fano]

Hy egur

I'm in a lot of trouble:
I've downloaded last ffdshow compiled by you: quicksync is present during installation, but not after installation (on the normal ffdshow Video Codecs page): Mediaportal plays the file but the ffdshow icon says it's using libvacopec (cpu usage is high!)... in ffdshow video I can't configure QuickSync anymore

It's a bug or my system it's not supported?

Strange as my hardware is AsRock Core 100 HT:
http://www.asrock.com/nettop/overvie...Specifications

it shouldn't be supported?
If not why installer deludes me?
Official ffdshow have QuickSync selectable but in the they used libvacodec...

I've to install by hand (?) the famous QuickSync,dll? There's none in my system

for your support!