B-frames and Media Foundation

[filler56789]

The following post by zambelli (in the Graphstudionext thread):

Any plans to add Media Foundation support? Directshow might not be with us for much longer...

reminded me of an interesting article by RadScorpion:

http://blog.monogram.sk/janos/2008/06/08/b-frames-in-directshow/

which says:

Ever wondered about B-Frame support in DirectShow? … or do you think that DirectShow is perfect? Unluckily it’s not. Read the rest of the article for more info on this issue.

Therefore, now the question is:

after VfW and after DirectShow, does Media Foundation finally offer proper support for B-frames on Microsoft Windows ®?

[benwaggoner]

Yeah, MediaFoundation does great with B-frames. However, the link describes the problem with B-frames in .AVI, however. I don't know that DirectShow itself had any problems with B-frames. It certainly played MPEG-1 files with B-frames without issues. MediaFoundation was definitely an improvement over DirectShow for consumer media playback in a whole lot of ways. Most professional video programs still use DirectShow, however. One factor is probably that MediaFoundation wasn't close to feature complete until Windows 7.

I was there at the trade show where DirectShow was announced. It was originally called ActiveMovie. They promised us a Mac port by the end of 1998 IIRC :).

[filler56789]

Thanks for answering, at last. :)

the link describes the problem with B-frames in .AVI, however.

Not at all. Read it again. :sly: There is no better explanation for the fact that Haali's DSS2() can be frame-accurate at seeking, BUT is not accurate at frame-counting. ;)

[nevcairiel]

I don't know why people think DirectShow has no support for B-Frames. True is that it only has one timestamp per frame, so you have to decide if you want PTS or DTS, but since PTS makes more sense, thats usually a no-brainer. Timestamps and frames can be in any arbitrary order (in decoding order), there are no restrictions like in AVI, so i don't see the problems.
The only problems come from issues in some implementations that always use decoding timestamps (DTS), however in my experience, this is mostly only a problem with VC-1, and barely for H.264 (unless you have H.264 in AVI, but thats broken by design on the timestamp side)

It may be easier to handle if there were two timestamp fields, for DTS and PTS separately (and in MF you can much easier define your own fields if you wish to do so), however the real problems come from file formats not having proper support for B-Frames, and/or implementations only working with decoding timestamps instead of the better presentation timestamps, but its not a platform problem. B-Frames work just perfectly today.

[dukey]

AVI was part of video for windows api, something dating back to 1992. It actually pre-dates mpeg1. It's no wonder it doesn't work well for modern formats.

[filler56789]

^ Thanks for the contribution, even though it's rather off-topic :D

BTW, could someone say whether the following excerpt is or is not "nonsense" ???

So one could now argue AVI doesnt support b frames as it doesnt store PTS and would if the application needs to know PTS (simpler players dont need to know the PTS…) to calculate the PTS based upon frame type and DTS, but that argument against AVI+b frames has a critical flaw, MPEG-PS and MPEG-TS dont store PTS for every frame either but only require it to be stored every 0.5 seconds or so. Which means that the same complicated calculate the PTS from DTS + frame types code is needed for the official MPEG format too

{ from: http://guru.multimedia.cx/avi-and-b-frames/ }

EDIT: also,

http://forum.doom9.org/showthread.php?p=930964#post930964

and more importantly,

it seems ASF is not any better in terms of B-frame support then AVI (since decoder hacks are needed to get right association of timestamps).

[nevcairiel]

Most MPEG-TS files i have seen store a timestamp for every frame. But sure, they are not required to. If there is a timestamp every 0.5s, and otherwise no timestamp at all, you can just interpolate the missing ones and sync on the ones you have, and in practice you do this after frame re-ordering, so the presence of B-Frames or not makes no difference. The problem is when you have no PTS at all, then you actually need to try to use DTS, which is more complicated then just interpolating the missing timestamps (and since most stuff is constant frame rate, thats easy). The fact is that MPEG-TS/PS can actually store PTS and DTS, and usually also do have both, even if not on every frame (which is no problem as long as you have it on some frames to sync against)

ASF/WMV is equally stupid, it was initially designed for wmv1/wmv2 which just don't come with B-frames.

[foxyshadis]

ASF/WMV is equally stupid, it was initially designed for wmv1/wmv2 which just don't come with B-frames.

ASF frames have a mandatory Presentation Time field. My guess is that codecs, editors, or older versions of DirectShow itself played fast and loose with it or ignored it entirely and caused most of the problems.