DirectShow Media Types for Stereoscopic Content

[madshi]

I'd like to suggest a small addition. Something like this:

typedef struct tagOPENVIDEOINFOHEADER {
int versionNumber;
EXTRAVIDEOINFO extraVideoInfo; // All extra stuff not part of VIDEOINFOHEADER2
byte reservedSpace [1024];
VIDEOINFOHEADER2EX videoInfoHeader2; // Compatible with VIDEOINFOHEADER2
} OPENVIDEOINFOHEADER;
The new fields "versionNumber" and "reservedSpace" would give us the following advantages:

(1) ability to add further information fields in the future
(2) without breaking compatability to older software (older software would simply ignore the added fields)
(3) and with comfortable direct access to all structure elements.

Not sure if those 2 new fields should be outside or inside of EXTRAVIDEOINFO, that's not important for me. Anyway.

[nevcairiel]

Thats quite alot of reserved space. I know there are some reserved DWORD fields in other headers, do you really expect to use 1kb of data in the future?
On the version thing.. One version number might be good, for compatible changes. However, incompatible changes should be done with a new format GUID to avoid any problems whatsoever. Also doesn't require any custom code by implementing filters to reject the connection.

[madshi]

Thats quite alot of reserved space. I know there are some reserved DWORD fields in other headers, do you really expect to use 1kb of data in the future?
I've no idea. Maybe 1kb is overkill. But then: Isn't it better to reserve too much memory instead of later finding out that we were too cautious? Always remember how Bill Gates thought 640kb of RAM were enough.

On the version thing.. One version number might be good, for compatible changes. However, incompatible changes should be done with a new format GUID to avoid any problems whatsoever. Also doesn't require any custom code by implementing filters to reject the connection.
Well, yes, probably you're right. So let's use only one version number.

[nevcairiel]

Always remember how Bill Gates thought 640kb of RAM were enough.


Thats just a myth, he never said that =p

The amount of reserved memory doesn't really concern me anyway, 1kb isn't what i would call "wasted space", its not like we alloc this structure constantly all over again, its usually a one time deal.
It just seemed like alot, considering most headers barely scratch 150 bytes in size.

[madshi]

Thats just a myth, he never said that =p
He claims himself that he never said that. But I'm not convinced. But anyway, that's another topic... :D

[BetaBoy]

I am talking to Peter and Haali to getting them together to chat more about it.

[yesgrey]

I've just reviewed the spec posted on post #2 and took some notes about it. I don't know if this project is still alive, but here they are hoping they would still be helpful.

I'll setup a SourceForce project for it. It is certainly best that the spec is avaiable on a neutral place.

Before I put it on SourceForce, we should agree at least on a final name so that we don't have to rename the SourceForge project later on. Currently, the name is "Open Media Format".
I would stay with Open Media Format, its the best name yet.
If we keep the word "Open", we should at least rename it to Open Video Info. Open Media Format sounds like a container or codec. Video Info is more appropriate.
Did you setup the SourceForce project, or the name is still not clear?

My first note is exactly about the name. In the intro text (lines 1 to 42), sometimes you refer to it as "Open Video Info", and other times as "Open Media Format". Is that intended, or should it always be "Open Video Info"?

For the other notes I will indicate the line where the problem is, what the problem is, and then my suggestion.

Line 106:
// STEREOLAYOUT_RIGHT are >>>indetend<<< for content that contains only one view but
should be: intended.

Line 269:
// MFVideoChromaSubsampling >>>enumarations<<<.
should be: enumerations.

Line 312:
VideoPrimaries_BT709 = 0x02, // ITU-R BT.709, also used for >>sRGB and scRGB<<
should be: sRGB, scRGB and xvYCC

Lines 313 and 314:
VideoPrimaries_BT470_2_SysM = 0x03, // ITU-R >>BT.470-4<< System M (NTSC), original NTSC primaries
VideoPrimaries_BT470_2_SysBG = 0x04, // ITU-R >>BT.470-4<< System B,G (PAL)
should be: BT.470-6

Line 341:
VideoTransferFunction_22_709_Sym = 0x0B, // >>>Gamma 2.2 curve with a linear range in the low range (IEC 61966-2-4)<<
should be: Gamma 2.2 curve with a linear segment in the low range and origin symmetry (accepts negative values), used by xvYCC

Lines 345:
>>>VideoTransferFunction_22_BT1361 = 0x1E, // Gamma 2.2 curve with a linear range in the low range, used in BT1361 extended colour gamut system<<<
This transfer function is for BT1361 extended. The BT1361 is the same as BT.709, so I suggest a different name and a slightly different description:
VideoTransferFunction_22_1361_Neg = 0x1E, // Gamma 2.2 curve with a linear segment in the low range (accepts negative values), used in BT1361 extended colour gamut system
Note: if name this one as _Neg, maybe it would be preferable to also name the "VideoTransferFunction_22_709_Sym" as "_Neg" instead of "_Sym".

Also, since its value is 0x1E it should come before "VideoTransferFunction_26" whose value is 0x1F.

Lines 310 to 345:
Are used several different expressions: "used for", "used by" and "used in".
I think we should use always the same, the one which is more correct.

Lines 475 and 476:
VIDEOPROPS videoProps;
DWORD dwControlFlags;
In the VIDEOINFOHEADER2 struct dwControlFlags cames first, shouldn't it be the same here?

Lines 499:
typedef enum tagVIDEODEGRADATIONFLAGS {
I suggest another degradation flag:
VideoDegradation_TransferMatrixChanged = 0x00800000, // Image converted to use another transfer matrix

Lines 525:
} VIDEODEGRA>>>DEA<<<TIONFLAGS;
should be: DA

Lines 541:
UINT de>>>gre<<<dationFlags; // See VIDEODEGRA>>>DEA<<<TIONFLAGS enumeration for possible values
should be: "gra" and "DA"

Lines 555:
// The OPENVIDEOINFOHEADER is >>>similiar<<< to the VIDEOINFOHEADER2 structure but
should be: similar

[madshi]

Thank you, yesgrey - appreciate that!

@pwimmer and @BetaBoy, can we finalize this spec? Any more input from your side? @pwimmer, any comments on my suggestion from my post #51?

I'd love to have this spec available as v1.0, so that I could soon start to add support for it to my video renderer.

[pwimmer]

yesgreg, thank you very much for your excellent input!

Give me a few days and I'll update the files.

Fyi, the H.264 MVC decoder I've been working on during the last few weeks will simply use multiple pins for multiple views. I didn't implement the spec discussed on this thread yet.

[7000feet]

from the mvc programmers manual:

1 General Information
The JMVC (Joint Multiview Video Coding) software is
the reference software for the Multiview Video Coding
(MVC) project of the Joint Video Team (JVT) of the
ISO/IEC Moving Pictures Experts Group (MPEG) and the
ITU-T Video Coding Experts Group (VCEG). Since the MVC
project is still under development, the JMVC Software
as is also under development and changes frequently.
The JMVC software is written in C++ and is provided
as source code. Section 1.1 describes how the JMVC
software can be obtained via a CVS server. Information
about the structure of the CVS repository is presented
in section 1.2. Section 1.3 describes how the JMVC
software can be build on Win32 and Linux platforms, and
section 1.4 gives basic information about the
binaries that are contained in the JMVC software package.

1.1 Accessing the latest JMVC Software
In order to keep track of the changes in software
development and to always provide an up-to-date
version of the JMVC software, a CVS server for the
JMVC software has been set up at the Rheinisch-Westfälische
Technische Hochschule (RWTH) Aachen. The CVS server
can be accessed using WinCVS or any other CVS client.
The server is configured to allow read access only using
the parameters specified in Table 1. Write access to the
JMVC software server is restricted to the JMVC
software coordinators group.

Table 1: CVS access parameters

authentication: pserver
host address: garcon.ient.rwth-aachen.de
path: /cvs/jvt
user name: jvtuser
password: jvt.Amd.2
module name: jmvc

Example 1 shows how the JMVC software can be accessed
by using a command line CVS client.
Example 1: Accessing the JMVC software with a command
line CVS client.

cvs –d :pserver:jvtuser:jvt.Amd.2@garcon.ient.rwth-aachen.de:/cvs/jvt login
cvs –d :pserver:jvtuser@garcon.ient.rwth-aachen.de:/cvs/jvt checkout jmvc

[madshi]

@pwimmer,

any chance to get this finalized soon? I'd like to start implementing this in my video renderer soon.

Also please check out my suggestion in comment #51. I think it would be a useful addition.

[mariner]

Greetings pwimmer and madshi.

Came across this JVC 3D mvc (http://av.watch.impress.co.jp/video/avw/docs/424/208/ez0027.mp4) clip over here (http://av.watch.impress.co.jp/docs/series/zooma/20110202_424208.htm).

Will the new decoder/splitter be able to handle it?

Best regards.

[pwimmer]

I've updated the spec.

Thanks again yesgrey for your great and detailed suggestions, I've adopted all of them.

The spec also includes some "reserved" fields for future use. I didn't add the version field, because I believe it's more useful and error-proof that future fields should be defined in a way that a value of zero means "not used", while a value other than zero should be interpreted by a filter. The spec requests that the currently unused fields should be set to zero when writing.

[madshi]

Thanks, pwimmer. I'm pretty happy with the header now.

Could you please add me to the copyright? (Mathias Rauen, madshi (at) gmail (dot) com). Also it would be great if you could create a sourceforge project sooner or later, but both of these things are not urgent at all.

[yesgrey]

I've updated the spec.

Thanks again yesgrey for your great and detailed suggestions
You're welcome. :)

I've adopted all of them.
Not quite... you've missed two:
My first note is exactly about the name. In the intro text (lines 1 to 42), sometimes you refer to it as "Open Video Info", and other times as "Open Media Format". Is that intended, or should it always be "Open Video Info"?
About the name, any conclusion?

Lines 541:
UINT de>>>gre<<<dationFlags; // See VIDEODEGRA>>>DEA<<<TIONFLAGS enumeration for possible values
should be: "gra" and "DA"
You've missed the variable name, which is still degredationFlags instead of degradationFlags.
Now it's in line 543.

And I've just found one more:
Line 40:
// The open media format is discussed on the >>>follow<<< Doom9 thread and
should be: "following"

[pwimmer]

The name of the project is "Open Media Format", which comprises several definitions of new data types, many of them are named after existing types with the "Open" prefix. Examples: IOpenMediaSample extends Microsoft's IMediaSample, OPENVIDEOINFOHEADER supersedes VIDEOINFOHEADER2. Although there is a OPENVIDEOINFOHEADER structure, it is just one out of many new types. We have choosen "Open Media Format" as project name because it is a generic one. Parts of the specification like the IOpenMediaSample might be use for other types than video in the future.

And finally, we also have a SourceForge project:
http://openmediaformat.svn.sourceforge.net/viewvc/openmediaformat

[madshi]

Wonderful, thanks for all the work you've put in this, pwimmer, I appreciate it!

Could you add me as sourceforge committer/admin? I will not do any changes on my own, I'll run everything through you. But I think it would be good to have a backup, just in case something ever happens to you.

[pwimmer]

Could you add me as sourceforge committer/admin?

Done.

[pwimmer]

There's one think I'm worried about: The current specification is based on the ideas of just a few people. There are certainly many more features that should be added. Maybe we should keep the format more generic, not defining any fields for stereo or degredation at all, but just define space for a property/value table that would be fully generic? We would pre-define some properties (for the fields that we have in the spec right now), but everybody else could also define his own private properties.

The obvious advantage is that it would never be necessary to update the format structure. To add new features, only new properties must be defined. There are also disadvantages, however: Accessing fields is not as straightforward anymore, because you have to search the property table and the format block would not have a fixed size. Some helper function can simplify accessing properties, of course.

A table item could have the following form:


typedef struct TOpenMediaFormatItem {
GUID guidProperty;
union {
BOOL boolValue;
BYTE byteValue;
INT intValue;
UINT uintValue;
GUID guidValue;
...
}
} TOpenMediaFormatItem;

Instead of the union, a VARIANT could be used, however, it must not contain pointers because it would make it difficult to copy the format block.

[madshi]

Maybe we should keep the format more generic, not defining any fields for stereo or degredation at all, but just define space for a property/value table that would be fully generic? We would pre-define some properties (for the fields that we have in the spec right now), but everybody else could also define his own private properties.
I like the general idea of a fully flexible format. However, I'm afraid of making it too complicated. I think with the current OpenMediaFormat spec we've found a very simple structure which is very easy to understand and very straightforward to use. No pointer arithmetic necessary. No danger of buffer overruns. 99% compatible with VIDEOINFOHEADER2. All these factors will make it very easy for programmers to add support for OpenMediaFormat.

I think we should extend the current spec a bit to add your idea of fully flexible items. But I would really hate to remove any of the already defined fields. I think they're all important and should stay in the non-flexible part. How would you like this?

typedef struct tagOPENVIDEOINFOHEADER {
EXTRAVIDEOINFO extraVideoInfo; // A couple of added fields
FLEXVIDEOITEMS flexVideoItems[32]; // can be used to store anything
VIDEOINFOHEADER2EX videoInfoHeader2; // Compatible with VIDEOINFOHEADER2
} OPENVIDEOINFOHEADER;
This would keep the basic structure simple. One item could then look e.g. like this:
typedef struct tagFLEXVIDEOITEM {
GUID guidProperty;
union {
BOOL boolValue;
BYTE byteValue;
INT intValue;
UINT uintValue;
GUID guidValue;
WCHAR strValue[MAX_PATH];
BYTE binaryValue[512];
}
} FLEXVIDEOITEM;
We could later define some common "FLEXVIDEOITEM" properties, when needed. But devs could also use these flex fields to transport private data between their filters.

What do you think?

[madshi]

P.S: I would be ok with moving the stereo stuff to FLEXVIDEOITEMs, if you prefer it that way. But I would really like the other 4 fields in EXTRAVIDEOINFO to stay, because if we make them "flexible", too, most devs probably won't ever use them. Having them spelled out in EXTRAVIDEOINFO gives them higher priority/importance. Moving the stereo stuff to FLEXVIDEOITEMs might make sense because these fields are really only useful for stereo movies, but not for the majority of movies. The other flags in EXTRAVIDEOINFO are useful for every movie, though.

[yesgrey]

The name of the project is "Open Media Format", which comprises several definitions of new data types, many of them are named after existing types with the "Open" prefix. Examples: IOpenMediaSample extends Microsoft's IMediaSample, OPENVIDEOINFOHEADER supersedes VIDEOINFOHEADER2. Although there is a OPENVIDEOINFOHEADER structure, it is just one out of many new types. We have choosen "Open Media Format" as project name because it is a generic one. Parts of the specification like the IOpenMediaSample might be use for other types than video in the future.
It makes sense. I think it might be a good idea to put this explanation on the sourceforge site...

And finally, we also have a SourceForge project
Great. It would be a good idea to also put the file in the Files area, so it would be easier for people to get it. If you want to add me as commiter/Admin feel free to do it

Thanks for adding me to the copyright. My e-mail is: yesgrey (at) gmx (dot) com

[pwimmer]

P.S: I would be ok with moving the stereo stuff to FLEXVIDEOITEMs, if you prefer it that way. But I would really like the other 4 fields in EXTRAVIDEOINFO to stay, because if we make them "flexible", too, most devs probably won't ever use them. Having them spelled out in EXTRAVIDEOINFO gives them higher priority/importance. Moving the stereo stuff to FLEXVIDEOITEMs might make sense because these fields are really only useful for stereo movies, but not for the majority of movies. The other flags in EXTRAVIDEOINFO are useful for every movie, though.

I have no problem to move the stereo stuff to the FLEXVIDEOITEMs, but I would move everything else to FLEXVIDEOITEMs as well.

Easy access to a flag doesn't mean that all developers use it correctly. I'm afraid that most developers will set it to zero, or even worse, write some garbage to it.

I see two options: Moving nothing or all to FLEXVIDEOITEMs. In any case, I don't want any string or other large data in FLEXVIDEOITEMs. The GUID type should be the largest supported type, else it would bloat FLEXVIDEOITEMs too much.

[madshi]

I see two options: Moving nothing or all to FLEXVIDEOITEMs.
Then let's move nothing to FLEXVIDEOITEMs. That's my preferred solution, anyway.

In any case, I don't want any string or other large data in FLEXVIDEOITEMs. The GUID type should be the largest supported type, else it would bloat FLEXVIDEOITEMs too much.
In what way would it hurt to allow larger types? As far as I can see it, larger types would allow even greater extension flexibility, without making the structure any more complicated.

Are you so worried about consuming a few KBs of memory? If you're totally opposed to that, how about a compromise? E.g. I'd be happy enough with maybe 64 ANSI chars / 32 WIDE chars. That would increase one FLEXVIDEOITEM to a size of 64 byte. That's not so much. A GUID consumes 16 bytes, too. If memory consumption is so important to you, we could replace the guidProperty GUID with a simple dword. Would save 12 bytes... :p

[pwimmer]

I just see no need to make the data field that large. 16 bytes are fine to hold all numeric data types, including GUIDs.

The format block should only contain data that is important during format negotiations. Other data, including large ones like language, stream name or descriptive text shouldn't be in the format block but retrieved via an interface like IPropertyBag.

The property itself must be a GUID so that it is unique. If it is a (small) number, we would need an authority that assigns properties. I don't want to create a mess like the FOURCC system. Therefore, we use GUIDs.

[madshi]

The format block should only contain data that is important during format negotiations.
The format block already contains information which is not important during format negotiations, e.g. dwPictAspectRatioX/Y or parallaxX/Y.

Let me give you an example of where string properties would be useful: Let's suppose a new MPEG5 video standard is created, supporting different profiles, just like MPEG4 does. And let's suppose that decoders do not support all profiles, just as most h264 decoders today do not support all h264 profiles. Let's further suppose the profiles in MPEG5 are not specified by numbers, like in MPEG4, but instead by strings. In this situation it would be quite useful to be able to transport the profile name through a FLEXVIDEOITEM string property. Of course this is a constructed situation. But my point is: We do not know for sure whether string properties might be useful in the future or not. So not supporting string properties would unnecessarily limit flexibility.

You seem to like GUIDs a lot. You use them to differ between stereo modes and between flex properties. Some future video, audio or subtitle codecs might use strings instead of GUIDs. We can't know that. Actually, personally, I prefer strings to GUIDs, because with strings the name already tells you something. With GUIDs, the number doesn't tell you anything. You have to lookup the GUID first before you know what it means. E.g. if instead of "GUID layout" we would use "CHAR layout[32]", we could use the real layout names like "unknown", "monoscopic", "left", "right", instead of some cryptic GUIDs. Same flexibility, easier to debug. Because of this reason I personally prefer strings to GUIDs, and I think allowing GUIDs as FLEXVIDEOITEM data type, but not strings, would mean that we design the Open Media Format to your personal preferences (using GUIDs), instead of allowing the greatest possible flexibility.

Even if you don't agree with me, please tell me: What would it *hurt* to allow strings as a data type? Where's the big disadvantage? I don't see any worth mentioning. As I said before, it doesn't have to be MAX_PATH.

[Loomis]

Guids rawk. Strings blow.

With strings you have to worry about upper/lower case issues, with leading and trailing spaces, with locale issues, with string length and termination issues, and with name collisions.


We do not know for sure whether string properties might be useful in the future or not. So not supporting string properties would unnecessarily limit flexibility.That type of reasoning is absurd, and an insult to anyone skilled in critical thinking.


Some future video, audio or subtitle codecs might use strings instead of GUIDs. We can't know that.Fear meme.

In the future the Son of Man might return in the clouds with His myriad of angels, and execute judgment on all of us. And in that case this whole guid v. string issue will be moot.

[madshi]

With strings you have to worry about upper/lower case issues, with leading and trailing spaces, with locale issues, with string length and termination issues, and with name collisions.
These are all well understood "problems".

That type of reasoning is absurd, and an insult to anyone skilled in critical thinking.
??? Is that the style you're discussing in? How about discussing with real arguments?

Loomis, you also fail to answer a question I've asked pwimmer twice:

What would it *hurt* to allow strings as an additional optional data type? Where's the big disadvantage?

[madshi]

I've done some more thinking:

(1) We don't really know right now what purpose the "flex" items will be used for. The very purpose of these items is that we want them to be useful for virtually *any* (reasonable) purpose. So what data types might make sense? I think in most cases a single int or dword will do. But I could also imagine that points (2 ints), rects (4 ints) or palettes (16 dwords) might be used. I don't really see why GUIDs would be useful as a datatype, more than strings. But I think we should allow as many datatypes are could be eventually useful. With the current FLEX structure, storing rects or palettes would be very painful to transport, because e.g. for a rect you'd have to define 4 GUIDs and you'd have to consume 4 flex items.

(2) The current FLEX specification is not very debug friendly. MPC-HC outputs detailed pin information, but in order to output it in a nicely readable way, MPC-HC would need to know which kind of data is stored in the FLEX items.

Let's imagine a new video codec stores crop windows for different aspect ratios. It might make sense to store these in FLEX items. With the current structure MPC-HC would output:

FLEXITEM[0]:
property: {403eed06-a58f-4eb8-a2bd-a48ecfda4fa0}
value: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FLEXITEM[1]:
property: {a0a3f800-194d-4b4d-a299-4919b2e24002}
value: 8C 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FLEXITEM[2]:
property: {47c57add-a3e3-4cc6-9f74-7f447d0c9a99}
value: 80 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FLEXITEM[3]:
property: {114957c5-8978-4317-8d1d-cf1d07ae25ab}
value: AC 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00
I think you'll agree that this is painful to read and hard to understand. You'd have to lookup the GUID to find out what they means. And if you've bad luck, the GUID might not be easily available, anywhere.

So I'd suggest to change the FLEX items to this:

typedef struct tagFLEXVIDEOITEM {
CHAR strProperty[32];
FLEXTYPEENUM enumValueType;
INT intValueCount;
union {
BOOL boolValue[16];
BYTE byteValue[64];
INT intValue[16];
UINT uintValue[16];
GUID guidValue[4];
CHAR ansiStrValue[64];
WCHAR wideStrValue[32];
}
} FLEXVIDEOITEM;
I understand that using a string as the key field has its problems, but in this case I prefer it to make debugging easier, because due to FLEX items being open for extension by anyone without any specification, the FLEX GUIDs would in most cases not be known to anyone. With my suggested changes, MPC-HC would output in its pin information:


FLEXITEM[0]:
property: scope_crop_rect
valueType: int
valueCount: 4
value: [0, 140, 1920, 940]
Isn't that *much* better than consuming 4 flex fields with 4 unknown GUIDs?

[nevcairiel]

Having dealt with Media Foundation a bit some time ago, i can't help but being reminded of IMFAttribute (http://msdn.microsoft.com/en-us/library/ms704598%28v=VS.85%29.aspx). MF got rid of the concept of static media types, and instead the filters exchange these attribute stores, which can contain an unlimited amount of data. (See here (http://msdn.microsoft.com/en-us/library/aa376629%28v=VS.85%29.aspx) for a list of attributes defined by MS)

Now, this uses GUIDs as keys, and i don't think thats too complicated.
The only problem to take care of is documenting, but anyone using the Open Media Type (was that what we're calling it now?), should know where to find that documentation, and everyone should be asked to submit the GUIDs for any custom attributes to you, so you can add them to the documentation. (This would equally be required for strings as keys, even if i can read the identifier, i can't be sure that i'm interpreting it 100% correct without documentation)

Going further, mostly focusing on madshi's suggested structure here:

An extra field for "type" is just overhead for debugging, it does not serve a real purpose for the filters implementing it, they will know to check for flex item with id "foobar", and expect it to have type DWORD. Same mostly goes for the "count" field, in most cases you'll know to expect exactly 4 values, and if the corner case would come up where there is a dynamic number of items, you could just have the first item be the count variable.

I would not add those just to get better debug output. The tool should be improved if its lacking, not the format being forced to compensate.

Finally, alot of the types in the union are redundant.

- BOOL is an alias for INT (at least in C/C++)
- UINT and INT are both 32-bit types and can be converted freely between each other.
- CHAR and BYTE both are 8-bit types (again only sign being different)
- WCHAR is meh. I would vote for only allowing CHAR, if someone needs to transport locale-sensitive data, convert it into a Multi-Byte UTF-8 String and put it into CHAR/BYTE

After slimming that down, i would keep:
- INT, for any form of number or bitmask (including BOOL) -- possibly make this a DWORD instead, as its commonly used as a number type in DShow
- BYTE, for binary data, including strings.
- GUID

Additionally, i could potentially see a floating point type being added as well, not sure if that would ever be used, however.

I do however like having the arrays of values in the flex type.

[madshi]

Thanks for your input, nevcairiel, I appreciate that. Using only ansi chars is ok with me (thanks to UTF8). I personally would like to see "type" and "count" items even though they're mainly useful only for debugging. But I can live without them if there's a majority for that. I like the idea to add doubles. Any opinion on max ansi string length? Maybe 64 chars?

IMFAttribute does indeed look similar in concept. Interesting is that IMFAttribute does support strings and also byte arrays and doubles. It also supports IUnknown, which I find bad because anything pointer like is asking for trouble, IMHO.

Any further comments, anybody? Would be nice to have this finally completed.

[nevcairiel]

A GUID is 16bytes, isn't it? If you allow 4 GUIDs in the union, having 64 bytes for the char/byte field makes sense. Should try to keep the arrays equal in size, targeting for 64 bytes is a good goal.

[nevcairiel]

I've been thinking about implementing the OPENVIDEOINFOHEADER in my video decoder (not 3D, just 2D, but still) to export transfer matrix and video primaries to the renderer, if it wants that information.

Is anyone still working on implementing the latest suggestions? Is anyone else using this? Is it dead? :o

[madshi]

I'd like to use it, but we need to officially finalize the spec first. @pwimmer, are you there?

[pwimmer]

I've updated the specification for H.264 MVC media types in the first posting of this thread.

[madshi]

You *are* still alive it seems. What is your comment on the last couple of posts regarding the new flex items?

About those FourCC media types listed on the first page: Why do you use the "H264" FourCC (AnnexB format)? Why not using "AVC1" (NAL format), which is used by most splitters these days?

[pwimmer]

Is anyone still working on implementing the latest suggestions? Is anyone else using this? Is it dead? :o

Nobody is currently working on the spec for OPENVIDEOINFOHEADER and it is not used so far by any DirectShow filter.

The project is not dead, but for the H.264 MVC decoder it turned out to be the better choice to have two separate output pins, thus there was no need to finalize the spec quickly.

The H.264 MVC FOURCCs (AMVC, EMVC) are already used by MPC HC (patches required) and will be used by the next releases of the Haali splitter and the CoreAVC decoder.

[pwimmer]

About those FourCC media types listed on the first page: Why do you use the "H264" FourCC (AnnexB format)? Why not using "AVC1" (NAL format), which is used by most splitters these days?

My mistake, I've corrected it. The implementations used AVC1 from the beginning.

[madshi]

Nobody is currently working on the spec for OPENVIDEOINFOHEADER and it is not used so far by any DirectShow filter.

The project is not dead, but for the H.264 MVC decoder it turned out to be the better choice to have two separate output pins, thus there was no need to finalize the spec quickly.
Are you still interested in the OPENVIDEOINFOHEADER spec? If not, I can take over, because I am still interested. I'd really like to finally finalize the spec, so I can add support for it to my video renderer.

[pwimmer]

Are you still interested in the OPENVIDEOINFOHEADER spec? If not, I can take over, because I am still interested. I'd really like to finally finalize the spec, so I can add support for it to my video renderer.

I am interested.

Let me summarize the outstanding decisions based on the recent suggestions. I've also added my optinion, let's see what others say...

1) Should the format be extensible?

Yes.

2) Should we use a GUID to identify extensions?

Yes.

3) What should be the max. data size per extension?

16 bytes (the size of a GUID). I don't see any reason to allow strings in the format block.

4) Should there be a type flag?

No, it is redundant, the GUID implicitely specifies the type.

5) Should there be a fixed number of extensions?

I'm not sure. A fixed size format block would be easier to deal with, a variable size block would be more flexible.

6) Should there be a non-flexible part?

I would put either everything or nothing in the flexible part.

[pwimmer]

Btw, we have some time to finalize the spec because I don't want to make any final decision before the first beta of Windows 8 has been released and we see what Microsoft has added to DirectShow.

[nevcairiel]

I doubt Microsoft will still extend DirectShow, their money is on Media Foundation.
I'm secretly hoping they kill DirectShow, even when it means nothing will work anymore, it would for the first time be a clean cut. :p

[madshi]

Ok, let me sum up the opinions of you, nevcairiel and me, let me also extend the list of decisions a bit:

1) Should the format be extensible?

[pwimmer] Yes.
[nevcairiel] Yes.
[madshi] Yes.
Yes.
[clsid] Yes.
[Matt] Yes.

[B]2) What data type should be used to identify extensions?

[pwimmer] GUID.
[nevcairiel] GUID.
[madshi] String (for better debugging).
GUID, if registered in the OS, String otherwise.
[Matt] GUID or String, no preference noted.

[B]3) What should be the max. data size per extension?

[pwimmer] 16.
[nevcairiel] 64.
[madshi] 64.
[Matt] "Worrying about size is silly. So allocate more space than you think you'll need."

4) What data types should be allowed in the extension?

[pwimmer] BOOL, BYTE, INT, UINT, GUID.
[nevcairiel] DWORD (= int), BYTE (= char), GUID, float or double.
[madshi] BOOL, BYTE, CHAR, INT, DWORD, GUID, double.
[Matt] int64 instead of int/byte, double instead of float, no comment on GUID and "char" for strings

5) Should the extension use a single item (e.g. "BOOL boolValue"), or should it use arrays (e.g. "BOOL boolValue[64]")?

[pwimmer] array (?).
[nevcairiel] array.
[madshi] array.
[Matt] array.

6) Should there be a type flag and an item count?

[pwimmer] No.
[nevcairiel] No.
[madshi] Yes. For better debugging.
[Matt] No.

7) Should there be a fixed number of extensions?

[pwimmer] No.
[nevcairiel] No.
[madshi] Yes.
Yes.
[clsid] Yes.
[Matt] Yes, if there's an advantage to it. No otherwise.

[B]8) Should there be a non-flexible part?

[pwimmer] ?
[nevcairiel] ?
[madshi] ?

Can you clarify what exactly you mean with question 8)? I'd like the fixed parts to stay as they are and only use the flexible parts for future extensions.

[nevcairiel]

If you use GUID extensions, they are by definition variable, having a fixed number in there would be kinda odd, so put me down for no on those.

I agree with madshi on 8. There can easily be fixed parts for the very common types, like outlined in the definition already, and have flexible GUID identified parts for future extensions.

[pwimmer]

I agree with madshi on 8. There can easily be fixed parts for the very common types, like outlined in the definition already, and have flexible GUID identified parts for future extensions.

Basically I agree, but which are the very common fields? It would be an endless discussion, because each developer has different priorities.

If the structure has a variable size, I don't see any need to have fixed parts anymore. Helper functions to access the parameters can make it nearly as easy to access the variable part as the fixed part.

[madshi]

If you use GUID extensions, they are by definition variable, having a fixed number in there would be kinda odd, so put me down for no on those.
Oh, I just see that I mis-stated my opinion on that one. I think having a fixed number of "flex items" would be preferable, because it would allow very easy use of the whole structure. I could then simply use:

- openInfoHeader->extraVideoInfo.someField
- openInfoHeader->flexItems[i].someField
- openInfoHeader->videoInfoHeader2.someField;

If we allow the numbers of flex items to be dynamic, the videoInfoHeader2 structure will not have a fixed position in the overall structure, anymore, making adressing more complicated. Of course we could use helper functions, but it would still be more complicated. Especially for non-C++ users. IMHO e.g. 16 or maybe 32 flex items should be plenty for future extensibility.

Basically I agree, but which are the very common fields? It would be an endless discussion, because each developer has different priorities.
We had already pretty much agreed on the "final" structure with all fields in it etc. Only then you had the idea to add flexible items. So now my opinion is that we should keep the original "final" structure as it is, and just add flexible items for future expansion capability. There'd be no need for discussion, we'd just leave the original structure as it is with no changes. So the "common" fields are already defined.

[Blight]

Madshi asked me to give my feedback on the process, so here goes:

As a programmer, I mostly care about 'easy to code, flexible for the future'.
So with that view in mind, I recommend calling the structure "OPENVIDEOINFOHEADER1" (note the '1').

I would prefer a fixed number of extensions, but keep the structure simple enough so if/when "OPENVIDEOINFOHEADER2" is introduced, you can easily add extensions, requiring minimal code changes.

With regards to GUID vs. Strings. Are you going to register the GUID with the system, or are you just using them internally? If you plan to register with the system and possibly allow for custom values (saved in the registry), GUID may be the way to go.
If you're only planning to use the GUID internally, it may actually be clearer to use strings (possibly with some pre-defined structure).

[pwimmer]

The variable size of VIDEOINFOHEADER2 requires that it is at the end of our structure, making it inconventient to add a variable size property block before.

However, VIDEOINFOHEADER2 usually has a fixed size unless there is a color palette. I doubt anybody is using a color palette these days, so I suggest to assume VIDEOINFOHEADER2 has a fixed size. Under this assumtion, we could define the following:



typedef struct tagOPENMEDIAPROPERTY {
GUID guidProperty;
union {
BOOL boolValue[4];
BYTE byteValue[16];
INT intValue[4];
UINT uintValue[4];
GUID guidValue;
LONGLONG longlongValue[2];
ULONGLONG ulonglongValue[2];
FLOAT floatValue[4];
DOUBLE doubleValue[2];
};
} OPENMEDIAPROPERTY;

typedef struct tagOPENVIDEOINFOHEADER {
VIDEOINFOHEADER2EX videoInfoHeader2; // Compatible with VIDEOINFOHEADER2 (no color palette allowed)
EXTRAVIDEOINFO extraVideoInfo; // All extra stuff not part of VIDEOINFOHEADER2
UINT propertyCount;
OPENMEDIAPROPERTY properties[];
} OPENVIDEOINFOHEADER;

[madshi]

In the last couple of days/weeks I've added support to my video renderer for some compressed video. For compressed video, the bitstream sequence headers are usually stored in the media type information, following VIDEOINFOHEADER2, or more commonly, following MPEG2VIDEOINFO. See here:

http://msdn.microsoft.com/en-us/library/dd390707%28v=vs.85%29.aspx

MPEG2VIDEOINFO already defines that VIDEOINFOHEADER2 can not have a color palette. Now the big question is: Do we want Open Media Format to be used for transporting compressed video bitstream, too? If so, we may need to alternatively support VIDEOINFOHEADER2 or MPEG2VIDEOINFO. Furthermore the sequence headers will then follow. So that again would require VIDEOINFOHEADER2 (or MPEG2VIDEOINFO) to be the last element in the structure.

Thoughts?

[nevcairiel]

I'm really not a fan of that pure-dynamic OPENVIDEOINFOHEADER.

Anyhow, if it should be used for compressed data, it does need some sort of space for a arbitrary length sequence header. This doesn't necessarily have to be right after VIH2/MP2VI - although it would make things easier - it could be like in MP2VI, just at the end of the wrapping structure. The more complicated question is how that can be easily combined with flex-elements.