Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[nevcairiel]

Maybe a splitter issue, try using "LAV Splitter Source" in MPC-HC. If that fixes things, I'll try looking at the clips with "Async File Source" as source.

The splitter should make zero difference, at least when using LAV decoder.

[egur]

Version 0.32 beta is out with the following changes:
* Added HW deinterlacing - can't be disabled in this build. Works on content marked as interlaced. If this version give you problems, please report and revert to the previous version.
* Added support for HW Detail and Denoise filters. Disabled in this build. Need ffdshow GUI for these features.
* FFDShow rev4453

Downloads
* For the latest cutting edge FFDShow builds download my builds Intel QuickSync Decoder SourceForge home page (http://sourceforge.net/projects/qsdecoder/)
* FFDShow-tryout site (http://ffdshow-tryout.sourceforge.net/download.php)
* LAV Splitter builds (http://forum.doom9.org/showthread.php?t=156191)

[nevcairiel]

I assume with "can't be disabled" you mean in ffdshow, but with the API i can disable it?
I didn't look at the config API you provided yet, but how about forcing deinterlacing of all frames, is it possible, or if not yet, planned?

[egur]

I assume with "can't be disabled" you mean in ffdshow, but with the API i can disable it?
I didn't look at the config API you provided yet, but how about forcing deinterlacing of all frames, is it possible, or if not yet, planned?

Correct, can't be disabled in ffdshow (default settings of QS decoder), you can disable in LAV via config.
I didn't implement forced DI. Simple feature, I'll add it soon.
Current mode of operation for DI:
Interlaced segments - apply DI.
Telecined (3:2) segments - apply soft telecine. Time stamps are changed only if config.bTimeStampCorrection = true. Frame rate will dynamically change to 23.976.
A video clip can have several segments, some interlaced, some 3:2, the DI will work just on the purely interlaced sources.
Couldn't apply DI to 3:2 frames, maybe an MSDK limitation. WIP.
Some things like 2:2 film mode detection do not work as planned, deinterlacer works instead.
If all VPP features are disabled or unused the VPP pipeline is disabled and not used so progressive clips do not suffer any performance hit (in theory anyway :) ).

[cybersans]

dear guys.

my problem has been solved. intel hd graphics driver installer is not installing mediasdk because some string inside its Setup.if2
just remove some string and viola! ffdshow can decode h.264 using QuickSync \O/

[RainyDog]

dear guys.

my problem has been solved. intel hd graphics driver installer is not installing mediasdk because some string inside its Setup.if2
just remove some string and viola! ffdshow can decode h.264 using QuickSync \O/

Cybersans, glad you've sorted it.

Please can you confirm which string you have to remove in the Setup.if2 file?

Cheers.

[cybersans]

i am using windows server 2008 r2. according to their readme.txt the driver will install on server os. unfortunately there are some string inside setup.if2 that caused mediasdk won't install:

AND NOT IsWinN
AND NOT IsOS(WIN2008,WIN2008_MAXSP)
AND NOT IsOS(WIN2008_R2,WIN2008_R2_MAXSP)

just remove that string, or put ; in front of it. then mediasdk will be install in program files\common files.
problem solved. :sly:

[egur]

i am using windows server 2008 r2. according to their readme.txt the driver will install on server os. unfortunately there are some string inside setup.if2 that caused mediasdk won't install:

AND NOT IsWinN
AND NOT IsOS(WIN2008,WIN2008_MAXSP)
AND NOT IsOS(WIN2008_R2,WIN2008_R2_MAXSP)

just remove that string, or put ; in front of it. then mediasdk will be install in program files\common files.
problem solved. :sly:

You didn't report that it was Win 2008. I'm not sure everything is validated for this OS, hence the setup check.
Let me know if QS is working for you.
BTW, which driver and where did you get it?

[RainyDog]

Can confirm I've got Quick Sync working on my set-up now as well.

I didn't touch the setup.if2 file as cybersans method obviously wasn't relevant to my OS which is Windows 7 x64.

Basically, all I did was uninstall the windows update driver then install the same numbered driver but direct from Intel's site.

Quick Sync option was there as soon as I installed Eric's latest ffdshow.

So, yeah... I should've just listened to Eric all along about windows update drivers :)

[mecedo]

Maybe a splitter issue, try using "LAV Splitter Source" in MPC-HC. If that fixes things, I'll try looking at the clips with "Async File Source" as source.

I've tried different splitters but it didn't help.
But after installing rev4453_20120521 movies play successfully:)

[cybersans]

rainydog: right. never use windowsupdate driver installation. they just use basic driver without any custom settings and add-ons ;)

egur: i forgot about that.
QS works well for all mp4, mkv movie i just play, accept 1 mkv files seems like the video is fast than sound. when i check the movie fps @ ffdshow video vonfiguration/info & cpu, it says 59.99 :eek:
i think that might be a problem with that video and not decoder itself since others video just fine.
the funny things is, if i change to libavcodec for h.264, the video become slower/stutter than audio :confused: movie fps seems like around 29.99, drop to 12, and then 15, back to 29 and so on.

[pulbitz]

strange...

I test MPC-HC, KMPlayer and PotPlayer with LAV splitter 32bit.

Only ffdshow QuickSync like yadif output mode 25p/30p.
Other(LAV QuickSync, PotPlayer QuickSync, DXVA and libavcodec) like yadif output mode 50p/60p.


I tested new 0.32 beta.
movies fps is double frame rate.
but motion is 25p/30p. (not doubling frame rate)

moviefps.png:http://imageupload.org/thumb/thumb_226569.png (http://imageupload.org/en/file/226569/moviefps.png.html)

sample files
20090725.Music_Core.IU.You_know.MPEG2.ts
http://www.sendspace.com/file/qg251f
premiere-paff.AVC.ts
http://www.sendspace.com/file/pvw8xb

[egur]

...
QS works well for all mp4, mkv movie i just play, accept 1 mkv files seems like the video is fast than sound. when i check the movie fps @ ffdshow video vonfiguration/info & cpu, it says 59.99 :eek:
i think that might be a problem with that video and not decoder itself since others video just fine.
the funny things is, if i change to libavcodec for h.264, the video become slower/stutter than audio :confused: movie fps seems like around 29.99, drop to 12, and then 15, back to 29 and so on.
Please share your faulty clip or part of it.

I tested new 0.32 beta.
movies fps is double frame rate.
but motion is 25p/30p. (not doubling frame rate)

moviefps.png:http://imageupload.org/thumb/thumb_226569.png (http://imageupload.org/en/file/226569/moviefps.png.html)

sample files
20090725.Music_Core.IU.You_know.MPEG2.ts
http://www.sendspace.com/file/qg251f
premiere-paff.AVC.ts
http://www.sendspace.com/file/pvw8xb
I made sure that the output is truly 60p on true 60i content - each of the 60 frames is different. I'll check your clips and report back.

Update
The problem is my limited testing. The new frames do have time stamps. This is fine with EVR (on these clips too) but EVR-CP doesn't use those frames and displays 30p.
I'll add time stamps to the missing frames. I hope this doesn't complicate LAV's time stamp correction...
I'll also add a chicken bit to disable the time stamp interpolation.

[egur]

I'm wondering what's the "forced DI" behavior should be:
* On progressive source -> mark as tff/bff according to config and send to DI. Output normal/double rate according to config.
* On interlaced source - do not override tff/bff flags. Normal DI operation.
* On 3:2 content - perform soft telecine.Output 23.976p.

Anyone has suggestions/comments on this behavior?
I'm not using this feature myself so I'm not aware of the use cases...

Update:
By mistake, the detail filter is working at 50% in the last ffdshow build (only the 0.32 build).

[nevcairiel]

Soft Telecine should also be DI'ed when force is on, although only single rate.

[cybersans]

EDITED: the video has been uploaded at my latest posts.

[egur]

Soft Telecine should also be DI'ed when force is on, although only single rate.

Do you mean output 29.97fps for 3:2 content?
How is that possible when there are only ~24 frames per second?
Why not repeat behavior for double rate?
What's he use case? Where does this happen?

[cybersans]

guys,

here is the mkv video i told you about:
https://rapidshare.com/files/1911724036/Metallica_-_Rock.in.Rio.2011_00h18m34s-00h19m38s_.mkv

using latest ffdshow, decode using quicksync video become faster than sound.
decode using libavcodec its stutter.

[nevcairiel]

Do you mean output 29.97fps for 3:2 content?


Why would the number of frames increase just because i deinterlace them? Just send the 24 frames through the deinterlacer, so that a wrong encoded movie will get its interlacing artifacts removed.
The fps or any other attributes should be irrelevant. If you deinterlace, you either produce one frame per field (double rate), or one frame per two fields (single rate), either doubling the fps, or well, not. Which FPS that is, is irrelevant.

[egur]

guys,

here is the mkv video i told you about:
https://rapidshare.com/files/1911724036/Metallica_-_Rock.in.Rio.2011_00h18m34s-00h19m38s_.mkv

using latest ffdshow, decode using quicksync video become faster than sound.
decode using libavcodec its stutter.
I'll take a look.

Why would the number of frames increase just because i deinterlace them? Just send the 24 frames through the deinterlacer, so that a wrong encoded movie will get its interlacing artifacts removed.
The fps or any other attributes should be irrelevant. If you deinterlace, you either produce one frame per field (double rate), or one frame per two fields (single rate), either doubling the fps, or well, not. Which FPS that is, is irrelevant.

No problem, just wanted to get his right.

[egur]

@cybersans,
This clip uses an H264 feature which I didn't encounter yet - frame doubling.
ffdshow's telecine manager (with libavcodec as decoder) thinks it's 3:2 so it performs soft inverse telecine (to 23.976fps) wrongly.
My decoder improperly treats this flag causing the DI to fail and output 29.97fps. I didn't see any playback problem or sync issues when using EVR as render under ZoomPlayer.
The flags that come out out my decoder mark this clip as interlaced and left to the mercy of the renderer's deinterlacer.

The clip itself is marked as 59.94fps with real 29.97fps and every frame doubled.
I'll have a fix in a few days.

[cybersans]

egur: thank you for your response. i think like that too, because previous version ffdshow can play that clip flawlessly.
will wait for your latest release.

[nevcairiel]

No problem, just wanted to get his right.

Looks good so far.
One option i'm still missing right now is an option to override the field order for all content, not just when "force" is on. Sometimes the flags in the files are wrong, and an option to override that is a quick fix.

Oh, and for the forced mode i would prefer it to only overwrite the tff/bff flags if none was deteremined from the bitstream, so that i can activate force on interlaced material without the flags being overwritten.

Aaand another problem. It seems to crash when i try to play an interlaced file with DI (and all other VPP) turned off. Progressive seems fine.

[egur]

Looks good so far.
One option i'm still missing right now is an option to override the field order for all content, not just when "force" is on. Sometimes the flags in the files are wrong, and an option to override that is a quick fix.

Oh, and for the forced mode i would prefer it to only overwrite the tff/bff flags if none was deteremined from the bitstream, so that i can activate force on interlaced material without the flags being overwritten.

Aaand another problem. It seems to crash when i try to play an interlaced file with DI (and all other VPP) turned off. Progressive seems fine.
I can implement the features.
I can't reproduce a crash, please detail the setup used.

[nevcairiel]

I can't reproduce a crash, please detail the setup used.

Debug build of LAV here:
http://files.1f0.de/lavf/LAVFilters-QSDI-debug.zip

Options are tied into the QS settings, if i set deinterlacing to none/weave, it just crashes.

Sample used:
http://files.1f0.de/samples/Test_clip_avc.1080i59.94.ac3.5.1.mkv

It only crash if:
- bEnableVideoProcessing is 1
- bVppEnableDeinterlacing is 0
- Detail/Noise options are 0

If i set bEnableVideoProcessing to 0, or activate one of the processor options, it works.

[egur]

Debug build of LAV here:
http://files.1f0.de/lavf/LAVFilters-QSDI-debug.zip

Options are tied into the QS settings, if i set deinterlacing to none/weave, it just crashes.

Sample used:
http://files.1f0.de/samples/Test_clip_avc.1080i59.94.ac3.5.1.mkv

It only crash if:
- bEnableVideoProcessing is 1
- bVppEnableDeinterlacing is 0
- Detail/Noise options are 0

If i set bEnableVideoProcessing to 0, or activate one of the processor options, it works.

I'll take a look tonight.

Regarding overriding TFF/BFF flags, it's strange that you don't want to override them in forced mode if they exist - they may be wrong - isn't this the purpose of this feature?
I can do this either way (always override and override when missing), your call.

[nevcairiel]

Regarding overriding TFF/BFF flags, it's strange that you don't want to override them in forced mode if they exist - they may be wrong - isn't this the purpose of this feature?
I can do this either way (always override and override when missing), your call.

"Force" mode is mostly to send progressive frames through DI, in case someone encoded interlaced frames as progressive (it happens).
On the other hand i don't want to influence the deinterlacing of an interlaced file even when "Force" is active.

I just want those two options to be separate, so they can be used independently. Force DI, and Force a specific field order.
A user might activate "Force", but still leave field order at "Auto". He may then play a normal interlaced file, and he would not want the field order flags overriden. Especially in broadcasts, it also happens that half of the frames are marked progressive, the other half interlaced, yet all need to be deinterlaced. In this situation, i would want to obey the flags from the interlaced frames, and only set the fallback flag for the progressive frames that don't have any.

So to conclude, you really need 3 options:

- Force DI
- Field Order
- Force Field Order

Field Order would then only be used when its not available otherwise (ie. on progressive frames), or when force field order is true.

[egur]

"Force" mode is mostly to send progressive frames through DI, in case someone encoded interlaced frames as progressive (it happens).
On the other hand i don't want to influence the deinterlacing of an interlaced file even when "Force" is active.

I just want those two options to be separate, so they can be used independently. Force DI, and Force a specific field order.
A user might activate "Force", but still leave field order at "Auto". He may then play a normal interlaced file, and he would not want the field order flags overriden. Especially in broadcasts, it also happens that half of the frames are marked progressive, the other half interlaced, yet all need to be deinterlaced. In this situation, i would want to obey the flags from the interlaced frames, and only set the fallback flag for the progressive frames that don't have any.

So to conclude, you really need 3 options:

- Force DI
- Field Order
- Force Field Order

Field Order would then only be used when its not available otherwise (ie. on progressive frames), or when force field order is true.

OK.
BTW, aren't the DI options in LAV mutually exclusive: Force DI/Aggressive DI/force progressive?

[nevcairiel]

BTW, aren't the DI options in LAV mutually exclusive: Force DI/Aggressive DI/force progressive?

Yes they are, but that says nothing about field order :)

All these options need to be able to be on all the time (especially aggressive is active for quite some people), without breaking decoding of properly flagged interlaced material, which is why i want "Forced DI" to be separate from the "Force Field Order" flag.

[egur]

Committed r57 to SVN:
* Fixed several issues/bugs with deinterlacing - also checked with LAV debug posted earlier.
* Added support for H264 frame doubling/tripling flags.
* Added support for forcing a field order (bForceFieldOrder) - independent of actual deinterlacing. Options are auto (do nothing), bff, tff via new config parameter (eFieldOrder).
* Forced DI behavior - convert progressive and 3:2 frames to interlaced using either the first frame's interlaced flag (if not progressive) or the eFieldOrder config parameter. Setting eFieldOrder to auto will cause forced DI to use TFF.

Note - field order is only visible in true interlaced content (e.g. a frame consists of 2 field from different times) and the output is full rate (50p, 60p).

DI full rate operation is capped at 60i.
Forcing DI on 24p will result in 48p. There's no check for input frame rate other than it's equal or smaller than 30fps.

Performed limited testing, if all goes well I'll build a new test version (0.33).

[egur]

Version 0.33 beta is out with the following changes:
* Support for H264 frame doubling/tripling flags.
* Added forced DI mode - currently disabled.
* Added forced filter order - currently disabled.
* FFDShow rev4455

I'm working on adding GUI to ffdshow and then all new features will be enabled.

Downloads
* For the latest cutting edge FFDShow builds download my builds Intel QuickSync Decoder SourceForge home page (http://sourceforge.net/projects/qsdecoder/)
* FFDShow-tryout site (http://ffdshow-tryout.sourceforge.net/download.php)
* LAV Splitter builds (http://forum.doom9.org/showthread.php?t=156191)

[aufkrawall]

Could you please also compile an x32 build? :)

[egur]

Could you please also compile an x32 build? :)

Forgot to upload, it's there now

[cybersans]

egur: download and install ffdshow_rev4455_20120529_egur.exe
my mkv video (like the sample i just gave before) still playing fast than audio when using intel quicksync decoder

but it works, no delay like previous version when change decoder to libavcodec.

maybe intel mediasdk caused that? because last time when i was using sandy bridge hd 3000 with old graphic driver (v2476), i can watch that video flawlessly.
with ivy bridge and driver v2696, it happen like that :eek:

[egur]

egur: download and install ffdshow_rev4455_20120529_egur.exe
my mkv video (like the sample i just gave before) still playing fast than audio when using intel quicksync decoder

but it works, no delay like previous version when change decoder to libavcodec.

maybe intel mediasdk caused that? because last time when i was using sandy bridge hd 3000 with old graphic driver (v2476), i can watch that video flawlessly.
with ivy bridge and driver v2696, it happen like that :eek:

The Metallica clips plays fine on my SandyBridge system, I'll check on an IvyBridge in a few days, my test system is under installation. It could be a driver problem which was fixed since I use a newer driver.

[cybersans]

The Metallica clips plays fine on my SandyBridge system, I'll check on an IvyBridge in a few days, my test system is under installation. It could be a driver problem which was fixed since I use a newer driver.

like i said before, it was play fine in my previous sandy bridge too.
i'll try to downgrade my driver to 2618 or 2598 to see any changes.

[cybersans]

right. intel driver v2696 caused that problem.
downgraded to v2618. video play fine.
testing with v2598, also fine. so i stick with v2618.

[egur]

@cybersans
Can you summarize your system setup including rendere, player, OS, drivers, use of external GPU, Lucid Virtu so I can try to reproduce.
Please don't skip any details.
Also 2618 is quite old, try 2656.

[nevcairiel]

2696 seems to cause quite some regressions with timings.
My problem with timestamps that get modified for no reason also only started happening in 2696 (i want them untouched, not modified at all) (only when fps in the bitstream is zero, eric might remember the report that we worked around now)

[egur]

Version 0.33.1 beta is out with the following changes:
* Support for H264 frame doubling/tripling flags.
* Added forced DI mode.
* Added forced filter order
* FFDShow rev4457 with new QuickSync config!

This version has the same QS decoder with upgraded ffdshow.
Control most QS decoder options, including video post processing via a new ffdshow configuration dialog. Look for a new entry just below "decoder options".
Visit the configuration once for your new settings to stick.

Downloads
* For the latest cutting edge FFDShow builds download my builds Intel QuickSync Decoder SourceForge home page (http://sourceforge.net/projects/qsdecoder/)
* FFDShow-tryout site (http://ffdshow-tryout.sourceforge.net/download.php)
* LAV Splitter builds (http://forum.doom9.org/showthread.php?t=156191)

[aufkrawall]

Forgot to upload, it's there now
Thanks. I don't want to really test without madVR, who knows if it's not a renderer bug.

Sorry for being OT now that deinterlacing is the topic, but do you think there could be acceleration for 4:2:2 or even 4:4:4 in future?

[egur]

Thanks. I don't want to really test without madVR, who knows if it's not a renderer bug.

Sorry for being OT now that deinterlacing is the topic, but do you think there could be acceleration for 4:2:2 or even 4:4:4 in future?

MadVR uses the GPU and might be too much for it. Try EVR and see if the stutters improve. Disable QS DI for this clip.
I'm not aware of plans to support 4:2:2 or 4:4:4 - I just don't know. Luckily these are rare.

[aufkrawall]

MadVR uses the GPU and might be too much for it. Try EVR and see if the stutters improve. Disable QS DI for this clip.

I use dedicated GPU for madVR.


I'm not aware of plans to support 4:2:2 or 4:4:4 - I just don't know. Luckily these are rare.
I'd say unfortunately, not luckily. :(

[egur]

I'd say unfortunately, not luckily. :(
I mean luckily.
The human eye is less sensitive to chroma changes than it is to luma changes.
Given an arbitrary bitrate, it would be better spent on higher resolution and/or more bits compressing the 4:2:0 frames than compressing 2-4 times more chroma data.

4:2:2 is usually used as an intermediate almost lossless fashion for video editing using ultra high bitrates. Keeping this much video data is problematic and has very little benefits.
It might make sense for DVDs as their resolution is small, but for HD it makes little sense.
Better quality can be achieved (better) by:
* Higher resolution - up to a point
* More bits/pixel - look better on high brightness/contrast displays, less banding.
* Wide gamut (xvYCC) - more accurate colors, not suitable for Windows based systems as all pixels are converted to RGB and the extra colors are gone.
* No interlacing
* High frame rate (60p).

[aufkrawall]

I have to disagree, at least partially. ;)
4:4:4 can have huge image quality advantages over CSS, here I made a comparison:
http://forum.doom9.org/showpost.php?p=1560156&postcount=7
The picture with CSS looks partially totally unsharp, many details are killed.

Real world films may not be affected that much, especially since it seems to be modern to wash out the colors with stupid filters.
Who really likes this?
It's not like it was behind the Iron Curtain, where color was too expensive *lol*. :scared:
But especially red tones seem to be degraded by CSS.

Furthermore, with 10bit x264, 1080p60 I444 can be achieved with pretty common bitrates.
35mbit may be enough, I'll try it out. :)

[cybersans]

@cybersans
Can you summarize your system setup including rendere, player, OS, drivers, use of external GPU, Lucid Virtu so I can try to reproduce.
Please don't skip any details.
Also 2618 is quite old, try 2656.

1. intel core i7-3770k using IGP HD 4000 + v2618 driver.
fyi, 2656 caused "COM Surrogate has stopped working" when you viewing image (jpg/bmp/png etc) with windows image viewer. you can do a search using a keyword "com surrogate intel driver 2656" and many people experienced the same problem too :D
so v2618 is the previous version which i found stable with my system

2. windows media player 12
3. windows server 2008 r2 converted to "workstation look alike"
4. your latest ffdshow build.
5. just IGP as primary GPU, lucid virtu disabled and no pci-e GPU.

[egur]

Nev,
The time stamp problem doesn't appear in driver 2761 anymore.

Also found some small issues, so I'll release another version very soon.

[nevcairiel]

Nev,
The time stamp problem doesn't appear in driver 2761 anymore.


Great.
You wouldn't happen to know when that one will be available to the public? (or at least to me :D)

[hajj_3]

will there be any new drivers soon for core i5 1st gen laptop processor as the latest is v2622.

[egur]

I don't know driver release dates, I'm not part of the graphics group. Sorry.
I've recently asked internally to share the production drivers that are available to OEMs and several ISVs, I'll update if there's any news in front.