Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[vivan]

Some problems when choosing russian language in installer:
http://2.firepic.org/2/images/2011-1...uicafzgk79.png
http://2.firepic.org/2/images/2011-1...uvye7g62o1.png
http://2.firepic.org/2/images/2011-1...vo2z4dznm3.png
But with regular ffdshow everything is ok...

[jmone]

Well done egur, I have had good success testing this on my i7-2600K and it worked pretty well with little issues on dropping frames etc with madVR. I did not have a good result however with interlaced HD material when compared with LAV CUVID. LAV CUVID would output frame doubled material that looks great at both 50 and 59.94 where as the intel version is pushing out 25 and 29.97 frames respectively and the interlaced fields as easily visible. Also on one VC-1(i) clip (Eagles Farewell Tour muxed to M2TS) there were also bouts of video corruption that seemed to be related to scene changes.

Great start!
Thanks
Nathan

[nevcairiel]

It does not deinterlace at all yet, thats why you see the interlacing artifacts, and only get 25/29.97 fps.

[jmone]

That would explain it! I'll be very intersted to see if the IGP is fast enough to handled 50/60fps in madVR when/if deinterlacing works as the rendering times already look high(ish) compared to the 550Ti.

[ajp_anton]

Did some speed testing on a Blu-ray video (Pixar's Day&Night).
i7 2600K. CPU at 4000MHz, GPU at 1700MHz
I also looked at the CPU power draw readings, don't know how accurate they are, and they are fluctuating a bit...

decoder                 speed(fps)    power(W)
ffdshow (QuickSync)      460.4          33**
LAV video                326.1          90
ffdshow                  287.0          86
LAV video (1 thread)      67.9          48
ffdshow   (1 thread)      66.6          48

ffdshow (QuickSync)        24           12*
LAV video                  24           15*
ffdshow                    24           15*

** CPU at 1600MHz, GPU at 1700MHz
*  CPU at 1600MHz

[egur]

Originally Posted by ajp_anton

decoder                 speed(fps)    power(W)
ffdshow (QuickSync)      460.4          33**
LAV video                326.1          90
ffdshow                  287.0          86
LAV video (1 thread)      67.9          48
ffdshow   (1 thread)      66.6          48

ffdshow (QuickSync)        24           12*
LAV video                  24           15*
ffdshow                    24           15*

** CPU at 1600MHz, GPU at 1700MHz
*  CPU at 1600MHz

These are very good numbers. I didn't have the time to measure power yet. I think power would be better if the GPU wasn't overclocked so high, but 20% savings on 1080p@24 is a good start point.
Interlaced content should provide better results (than SW implementations) when using the EVR as it uses the HW deinterlacer.

Originally Posted by jmone

Well done egur, I have had good success testing this on my i7-2600K and it worked pretty well with little issues on dropping frames etc with madVR. I did not have a good result however with interlaced HD material when compared with LAV CUVID. LAV CUVID would output frame doubled material that looks great at both 50 and 59.94 where as the intel version is pushing out 25 and 29.97 frames respectively and the interlaced fields as easily visible. Also on one VC-1(i) clip (Eagles Farewell Tour muxed to M2TS) there were also bouts of video corruption that seemed to be related to scene changes.

Great start!
Thanks
Nathan

Thanks.
As nevcairiel pointed out, LAV Video decoder doesn't deinterlace the video. I also do not deinterlace, it's the job of the renderer. If I incorrectly flag a clip as progressive, please share it to help me fix the problem.
EVR uses HW deinterlacing and produces 50-60fps. That's the most stable renderer ATM.
MadVR compatibility is on my TODO list.
VC1 is a little problematic and I've seen corruption in some clips. I do not know the root cause of the problem (HW, driver, media SDK) but it's being dealt with by the Media SDK team. I hope a solution will come promptly. If you provide/share a short sample of the corrupted scene it would help.

I want to thank everyone for providing valuable feedback, this is the highway to a stable product and a good reference for other products.

[leomax]

Keep up the great job egur.
Would it be possible to use this in a notebook with switchable graphics (optimus)?

[egur]

Originally Posted by CruNcher

This issue might be not easy to find egur, currently ffdshow-quicksync has a tendency to silently selfdestruct after a lot of streams continuously loaded in MPC-HC it dies and MPC-HC falls @ the next stream then back to another Decoder ffdshow-quicksync wont load anymore after until MPC-HC is restarted it isn't a specific stream it seems already changing streams fast without unloading them can trigger this, like in MPC-HC drag & drop while another stream is still loaded.
Another problem though here is MPC-HC cant cleanly unload dshow filter when a file is closed the filter is still acquired not sure though if that has something todo with ffdshow-quicksync currently braking after a while. Though maybe it's also a memory leak i didn't looked into that yet.

I managed to reproduce the drag & drop causing a freeze to MPC-HC. ffdshow.ax stay resident in memory but my decoder DLL is unloaded. Looks like some kind of race condition. Almost impossible to debug a this doesn't occur if I place breakpoints
I couldn't reproduce with ZoomPlayer, maybe MPC-HC is handling the loading differently. If I knew how, I could find a solution. Unless I'm wrong here, this is a low-medium priority bug. If an MPC-HC developer can give a hint that would help.
I scanned for memory leaks and fixed them (for next release). They were minor and didn't affect anything.

Originally Posted by CruNcher

http://www.mediafire.com/download.php?ri4sdpjlafxxrry <- hangs (@ the cat) when the lion scene should follow (with both lav splitter and mpc-hc).

Btw the 4:2:2 Mpeg-2 Fallback already works perfectly in combination with Lav Video
Only High10, High422 and X264 losless fail

The crash/hang is actually a critical bug in my code - fixed.

Regarding the various H264 formats I now filter them within ffdshow and fallback to libavcodec silently.

[egur]

Originally Posted by leomax

Keep up the great job egur.
Would it be possible to use this in a notebook with switchable graphics (optimus)?

One report said it works. I don't have a system to check myself.
Please try and let me know.

[CruNcher]

Originally Posted by egur

I managed to reproduce the drag & drop causing a freeze to MPC-HC. ffdshow.ax stay resident in memory but my decoder DLL is unloaded. Looks like some kind of race condition. Almost impossible to debug a this doesn't occur if I place breakpoints
I couldn't reproduce with ZoomPlayer, maybe MPC-HC is handling the loading differently. If I knew how, I could find a solution. Unless I'm wrong here, this is a low-medium priority bug. If an MPC-HC developer can give a hint that would help.
I scanned for memory leaks and fixed them (for next release). They were minor and didn't affect anything.


The crash/hang is actually a critical bug in my code - fixed.

Regarding the various H264 formats I now filter them within ffdshow and fallback to libavcodec silently.

I asked Jan if he might have an idea, about the fallback so it's falling back to the Internal ffdshow libav not to the dshow chain right, could you maybe add a switch to let the user chose if he prefers internal/external in such a fallback case ?

[jmone]

Here is the first 1:20 of the VC(i) clip with showing the corruption that comes and goes --> http://www.megaupload.com/?d=40NY4V2C

Deinterlacing: For me the great benefit of LAV CUVID is not the decoding as much as that you get the best deinterlacing I've ever seen (for supported formats). It really is very very good and puts to shame anything that EVR or FFDSHOW/YADIF can do. Are you saying that access to GPU deinterlacing will not be part of the Intel GPU Video Decoder?

Integration with LAV Video: I know that nevcairiel has hinted that your project may be accessible from LAV Video (as an alternative to FFDSHOW) or am I reading this incorrectly? & if not any timeline?

Thanks again.

[Blight]

jmone:
I believe deinterlacing and other PP effects will be added once general-playback is considered stable.

[CruNcher]

jmone
Yes as egur stated already your issue is known and it's up to the Driver and Media SDK team @ Intel now to fix it and hopefully it is fixable @ all but also Nvidia fixed it so it seems to be doable without any ASIC replace, especially these Asics aren't so basic as they where years ago they are partly programmable nowdays, or like Nvidia did for a Mpeg-2 implementation bug just using the EUs to workaround the " hardware bug" in the hardest case (though Intels action space is smaller for this EU workarounds, being not so powerful).
But im very confident the Intel Engineers gonna find a way to fix this Problem one or another way

Also im sure Intel knows about this problem since some time now, as it hardly can be that none of the Top 4 ISVs (Mainconcept, Arcsoft, Cyberlink, Corel) gave that feedback already from their customers that experience this issue to Intel, just the pressure gets higher the more reports flowing in about this
Btw It took Nvidias Engineers 1 Driver cycle to fix this back then

[egur]

Originally Posted by Blight

jmone:
I believe deinterlacing and other PP effects will be added once general-playback is considered stable.

Correct. One thing at a time.

[pulbitz]

I'm sorry. I don't speak English very well.

sample files
http://www.mediafire.com/?sem1jx36pnnae2i or http://www.multiupload.com/VJPU41ELBI

file: slow video.20100518.직캠.서울시립대학교 축제.아이유(IU) - Boo.flv
file: slow video.20110516.S-OIL.TV-CM.즐거운 세상 만드는 좋은기름 1리터의 힘.20초.아이유(IU).RAiN.ts

quicksync is slow motion. libavcodec is OK.

file: can't decode.20101006.직캠.숭실대 얼Ssu!.6.아이유(IU) - 멘트.mkv

quicksync can't decode video(720x1280).
if HW can't decode it, then need fallback to libavcodec.

[egur]

Originally Posted by pulbitz

file: slow video.20100518.직캠.서울시립대학교 축제.아이유(IU) - Boo.flv
file: slow video.20110516.S-OIL.TV-CM.즐거운 세상 만드는 좋은기름 1리터의 힘.20초.아이유(IU).RAiN.ts

quicksync is slow motion. libavcodec is OK.

file: can't decode.20101006.직캠.숭실대 얼Ssu!.6.아이유(IU) - 멘트.mkv

quicksync can't decode video(720x1280).
if HW can't decode it, then need fallback to libavcodec.

Issue 1: Slow decode (Boo.flv). Root caused to several splitters not sending frame rate (defaulting to 24fps) - will be fixed in next release (I'm working hard on stabilizing the time stamps and identify frame rate ).
Unfortunately, I haven't locks on a good algorithm that takes care of all the corner cases. LAV decoder's solution for time stamp handling is working according to nevcairiel but I don't want to use it as it involves querying the filter graph and needs some meta data I don't have. I want to make a standalone algorithm independent from DirectShow - rely only on the time stamps given at input and outputted by the Media SDK decoder API. I may release a less than perfect implementation in order for other fixes to surface.

Issue 2: Slow decode (Oil TV.ts). The clip has inverse telecine flags at the beginning of the clip. Due to a bug, the QS decoder stays locked at 23.97. This fix will be solved with issue 1.

Issue 3: Can't decode. A duplicate bug reported by CruNcher. Already fixed in my code. Will be in next release - hopefully this week.


pulbitz - thanks a lot of the feedback!

[egur]

CruNcher:
The CPU utilization seems a little high.
During my testing on both desktop and laptop CPUs, during playback the CPU frequency would drop to a minimum (1600MHz for desktop and 800MHz for mobile) with single digit CPU utilization.
It's obvious that (unnecessary) surface conversion is going on not sure why. Did you use the standard EVR?

[CruNcher]

It cant drop lower because im using High Performance to keep overall system latency low as possible (but you right @ the beginning i used Balanced) and nope this is EVR-CP (Deinterlacing completely lost but different capabilities for Higher Quality output) and im not sure whats going on especially that YUY2 conversion seems to hit (its using in both cases billinear 2.0 ps for the scaling) still it does better then @ the beginning so that progress i wanted to visualize but still config stuff changes :P Though that the OSD for MPC-HC main gets differently rendered (blur) im not sure what causes this (could be either one of the Microsoft D2D or Intel Driver update)

Also my experience is SB is so efficient itself that clock changing doesn't impact power consumption much @ all and not very useful for a Desktop, we aren't in the early days of C&C and Speedstep anymore where you could save tons of watts, the whole Power Management from the lowest (cpu,chipset), middle (bios) up to the highest (OS) level is already efficient even running @ High Performance

PS: Also keep in mind i didn't compared yet EVR-CP (Bicubic PS 2.0) vs the Hardware Scaling used on EVR by Intels Driver yet (i got a quality idea of the Deinterlacing, IVTC and Sharpen PP so far but not how the Adaptive Scaling off different content works out compared to a hardcoded Bicubic PS approach running over the EUs).


Ok here is the current state 32 bit (Balanced)

ffdshow-quicksync 0.14 Alpha



Cyberlink DXVA (Worlds most efficient DXVA Decoder)



Please dont ask why the OSD is so blurred i have no idea what changed this (it's the same build and settings as the first day test of ffdshow-quicksync, just the system around changed Driver and Subsystems and also the Scaling is by default now Bicubic PS 2.0 im not sure anymore what i used on the first day test, but it is in now way responsible for the blur) (Microsoft Optional D2D Patches for fixing problems with IE9) that could cause this)
Feel free to tell if you have any idea or have this same issue with the OSD currently on Intel Graphics and the Main MPC-HC builds with EVR-CP

So this is how ffdshow-quicksync started its a big improvement

[egur]

Originally Posted by CruNcher

...
PS: Also keep in mind i didn't compared yet EVR-CP (Bicubic PS 2.0) vs the Hardware Scaling used on EVR by Intels Driver yet (i got a quality idea of the Deinterlacing, IVTC and Sharpen PP so far but not how the Adaptive Scaling off different content works out compared to a hardcoded Bicubic PS approach running over the EUs).

Bicubic is not good enough when scaling factors are high (>2), the image is a little blurry. Bilinear shouldn't be used for anything as it creates horrible scaling artifacts.
The SNB HW scaler has the capability to downgrade to Lanczos4, Lanczos3, Lanczos2 and all the bicubic variants. I'm not sure if the driver supports any of these modes though, but I'll check. It makes sense to utilize it in any case.

A few questions for everyone:
* Does the EVR have an interface to configure the scaler quality?
* Is the EVR-CP the same one supplied with Media SDK 4.0b4 (they have the same file name)?
* A tough one - does anyone know how to create a virtual adapter - so DXVA can enumerate a GPU not connected to a screen?

[CruNcher]

Btw i also lost Microsofts DTV Decoder for this file im also not sure yet why but it doesn't connect anymore, anyways for DXVA Cyberlink is the best choice so i added this to have a compare vs DXVA (it showed me that a lot of times in the past even in very picky situations where others DXVA implementations fail it keeps stable and performant throughout different DSPs, and just recently on a NT 6 test it showed again superior results when testing CoreCodecs new Implementation)

Egur all of these are very good questions
1. Somehow their must be else how should the driver have the possibility to manipulate EVR directly in terms of IVTC, Deinterlacing and Sharpen :P though i guess from the lower level it's not really documented @ all, so most probably only from the Driver Kernel Level ?
2. In theory Intel could have borrowed MPC-HCs version for their samples indeed
3. Yea there must be a way, though im not sure if this was done for security reasons too in terms of the Protected Media Path, hmm if Lucids Virtu is in dgpu mode though Quicksync can be still used also can't it ? so they must have found a way (or does encoding work only in igpu mode ?)