DirectShow Media Types for Stereoscopic Content

[pwimmer]

I'm opening this thread to discuss media types and media formats to be used by DirectShow filters for stereoscopic formats.

[pwimmer]

There is a need to specify media types for two categories of stereo content. This posting will be updated periodically to reflect the outcome of the discussions in this thread.


1) Compressed H.264 stereoscopic/multiview content.

The following FOURCCs have been specified and are supported by the MPC HC MP4 and MPEG-TS splitters after applying the 3D patches (http://www.3dtv.at/OpenSource). Upcoming versions of the Haali splitter and CoreAVC will support these media types, too.

The AVC1 FOURCC should be used for 2D content. If a decoder input pin uses the AVC1 FOURCC, the decoder should also offers a second input pin that supports the EMVC FOURCC only.

The AMVC FOURCC should be used for MVC streams where both the base view NALs and the MVC extension NALs (coded slice extension NAL units and prefix NAL units) are delivered on the same DirectShow filter pin. If the AMVC FOURCC is used, there must be a subset SPS and a second PPS in the format block.

A DirectShow splitter filter pin offering the AMVC FOURCC can also offer the AVC1 FOURCC as second media type so that it also connects to legacy 2D decoders. However, when the decoder input pin is connected using the AMVC FOURCC, the decoder must not offer a second pin for EMVC FOURCC media types.

The EMVC FOURCC should be used if the pin delivers MVC extension NALs only. The format block must contain a subset SPS and the corresponding PPS.

NAL units of the base view must be delivered on another pin using the AVC1 FOURCC. Timestamps of the base view and MVC extension samples must be derived from the same clock, else the decoder fails to synchronize them.


2) Uncompressed stereoscopic/multiview content.

The open media format is an extension to Microsoft's DirectShow technology to enable interoperability between stereo-capable DirectShow filters of different vendors. Click here to see the draft of the open media format specification:
DirectShow Open Media Format SourceForge Project (http://openmediaformat.svn.sourceforge.net/viewvc/openmediaformat)
OpenMediaFormat.zip on 3dtv.at server (http://www.3dtv.at/Downloads/OpenMediaFormat.zip)

[pwimmer]

Ad 1) I'm currently working on the MVC decoder and thinking about possible solutions. Things currently evolve...

[pwimmer]

Ad 2)

The Stereoscopic Player that I'm working on already uses a special stereo media type for uncompressed stereo samples sent from the 3dtv.at Stereo Tranformation filter to the 3dtv.at Stereo Renderer filter and from the 3dtv.at Stereo Image Source to the 3dtv.at Stereo Transformation.

Edit: Removed the code to avoid confusion with the new Open Media Format definition. If you are interested in the media format currently used by the Stereoscopic Player, please contact me offlist.

[pwimmer]

I propose a different IMediaSampleEx interface that derives from IMediaSample and adds just two methods:

HRESULT GetPointerEx(BYTE **ppBuffer, int iView);
long GetSizeEx(int iView);

This allows to use separate buffers for each view, which I would prefer over a single buffer for all views.

The format type adds a dwNumViews field to the VIDEOINFOHEADER2

DEFINE_GUID(STEREOLAYOUT_MONOSCOPIC, ...); // Monoscopic
DEFINE_GUID(STEREOLAYOUT_MULTIBUFFERS, ...); // Use StereoMediaSampleEx to get buffers != 0
DEFINE_GUID(STEREOLAYOUT_SIDEBYSIDELEFTFIRST, ...);
DEFINE_GUID(STEREOLAYOUT_SIDEBYSIDERIGHTFIRST, ...);
DEFINE_GUID(STEREOLAYOUT_OVERUNDERLEFTTOP, ...);
DEFINE_GUID(STEREOLAYOUT_OVERUNDERRIGHTTOP, ...);
DEFINE_GUID(STEREOLAYOUT_INTERLACEDLEFTFIRST, ...);
DEFINE_GUID(STEREOLAYOUT_INTERLACEDRIGHTFIRST, ...);
DEFINE_GUID(STEREOLAYOUT_FRAMESQUENTIALLEFTFIRST, ...);
DEFINE_GUID(STEREOLAYOUT_FRAMESQUENTIALRIGHTFIRST, ...);


typedef struct tagSTEREOVIDEOINFOHEADER {
RECT rcSource;
RECT rcTarget;
DWORD dwBitRate;
DWORD dwBitErrorRate;
REFERENCE_TIME AvgTimePerFrame;
DWORD dwInterlaceFlags;
DWORD dwCopyProtectFlags;
DWORD dwPictAspectRatioX;
DWORD dwPictAspectRatioY;
DWORD dwControlFlags;
DWORD dwReserved2;

// Stereo related fields
GUID guidStereoLayout; // See supported STEREOLAYOUT_xxx GUIDs above
DWORD dwNumViews; // Number of views (only valid for STEREOLAYOUT_MULTIBUFFERS)

BITMAPINFOHEADER bmiHeader;
} STEREOVIDEOINFOHEADER;

[madshi]

A few comments:

(1) Let's prepare for *Multi* views and not just stereo. After all it's h264 M(ulti)VC and not h264 S(tereo)VC.

(2) HDMI 1.4a not only knows left/right, but it also supports funny things like "L + Depth". Not sure if the new media type should support such things, too?

(3) I'd like to use the opportunity to add more information to the header structure. Which means that the purpose of the new media type might not be limited to 3D. I'd also like to see the new media type being used for 2D. So maybe we should not even name it 3D or MVC. Maybe we should name it completely different, e.g. "FORMAT_VideoInfo3". However, that name might collide with a future Microsoft format, so maybe we should use "FORMAT_Doom9VideoInfo" or "FORMAT_OpenVideoInfo" or whatever. I don't think there should be "Stereo" in the name. The new format should be stereo capable, but that would only be one benefit of the new media type. I want to have more benefits. And if the new format is used for 2D, the standard IMediaSample interface should suffice.

(4) I strongly suggest reusing the same FOURCC values that also exist for conventional 2D frames, instead of defining new ones. According to Microsoft, there already exists FOURCC definition for YCbCr 4:2:0, 4:2:2 and 4:4:4, with 8bit, 10bit and 16bit each. See here:

http://msdn.microsoft.com/en-us/library/bb970578%28VS.85%29.aspx#_420formats

(5) I don't really like the idea of locking the buffer for different "components". After all, depending on the FOURCC, the buffers may not be planar, and thus locking separate component buffers doesn't always make sense. I think there should be a way to get the pointer/size for every view, but it should be one simple pointer, similar to how the standard IMediaSample interface works. How the data is stored in the buffer depends on the FOURCC.

(6) Let's try to name the methods of the extended IMediaSample interface similar to the original functions. E.g. instead of "LockBuffer" I'd suggest "GetPointerEx/GetSizeEx".

(7) I'd suggest to start the structure with VIDEOINFOHEADER2, and to just append more data to it. This way it would be extremely easy for existing software to add support for the new media type. They'd just need to interpret the structure as VIDEOINFOHEADER2. They'd only need to do extra coding if they want to actually support the new fields of the structure.

(8) Additional fields I want to have in the header, all taken from the h264 specification:

- video_format
- video_full_range_flag
- colour_primaries
- transfer_characteristics
- matrix_coefficients
- chroma_format_idc
- chroma_sample_loc_type_top_field
- chroma_sample_loc_type_bottom_field
- bit_depth_luma
- bit_depth_chroma

Maybe more. The fields above might have to be adjusted to be more flexible. We don't want to limit ourselves to what the h264 spec supports. Other video formats may support more variations than h264.

[nevcairiel]

(7) I'd suggest to start the structure with VIDEOINFOHEADER2, and to just append more data to it. This way it would be extremely easy for existing software to add support for the new media type. They'd just need to interpret the structure as VIDEOINFOHEADER2. They'd only need to do extra coding if they want to actually support the new fields of the structure.


While this idea sounds good in theory, there is one problem with it.
Some formats require codec/format specific data that gets appended after the BITMAPINFOHEADER in VIDEOINFOHEADER2.
If you want to append data to the end of VIDEOINFOHEADER2, you would occupy this space.

I don't see a clear way to preserve compatibility in a way like this without crippling some functionality of the format, tbh.

Another question that i just asked myself..
This new format is mainly designed for data from the decoder to the renderer, not from the source/splitter to the decoder, right?
Would decoders benefit from any additional information we could provide on the splitter side of things?

I'm currently working on a DS source/splitter and just wondering if on this side anything new is required.

[kieranrk]

(1) Let's prepare for *Multi* views and not just stereo. After all it's h264 M(ulti)VC and not h264 S(tereo)VC.


Agreed. Just because Stereo is all the rage now doesn't mean everything else should be excluded.

You should also include Side-by-side MPEG-2 and H.264 broadcasts. There are also 720p120 H.264 3D broadcasts.

There's a special SEI in H.264 for indicating side-by-side 3D but not many people use it yet. Presumably there's something similar in MPEG-2.

[pwimmer]

(1) Let's prepare for *Multi* views and not just stereo. After all it's h264 M(ulti)VC and not h264 S(tereo)VC.


Agreed. The dwNumView field already allows for more than two views.


(2) HDMI 1.4a not only knows left/right, but it also supports funny things like "L + Depth". Not sure if the new media type should support such things, too?


I don't think so. It is not used in practice and would make the new media type much more difficult. The only 2D+Depth files are the ones for Philips Wow 3D displays, but they not only use 2D+Depth but some of them 2D+Depth+Background 2D+Background Depth.

I suggest to define a separate media type for 2D+Depth and do not cover it in our new media type.


(3) I'd like to use the opportunity to add more information to the header structure. Maybe we should name it completely different, e.g. "FORMAT_VideoInfo3". However, that name might collide with a future Microsoft format, so maybe we should use "FORMAT_Doom9VideoInfo" or "FORMAT_OpenVideoInfo" or whatever.


They name can always be changed without breaking binary compatibility. Only in the source code a search&replace must be performed. The GUID is unique in any case, even if there is a collision of the names.

FORMAT_OpenVideoInfo is my personal preference.


(4) I strongly suggest reusing the same FOURCC values that also exist for conventional 2D frames, instead of defining new ones.


Right. I did this because I needed FOURCCs for formats where each plane resides in different formats. I've published this so that everybody can see how the existing Stereoscopic Player works, but it should no be part of the new standard.


(6) Let's try to name the methods of the extended IMediaSample interface similar to the original functions. E.g. instead of "LockBuffer" I'd suggest "GetPointerEx/GetSizeEx".


Agreed. I forgot the GetSizeEx method in my posting above. Will fix it later.


(7) I'd suggest to start the structure with VIDEOINFOHEADER2, and to just append more data to it.


Agreed. That's what I did.


(8) Additional fields I want to have in the header, all taken from the h264 specification:

- video_format
- video_full_range_flag
- colour_primaries
- transfer_characteristics
- matrix_coefficients
- chroma_format_idc
- chroma_sample_loc_type_top_field
- chroma_sample_loc_type_bottom_field
- bit_depth_luma
- bit_depth_chroma



VIDEOINFOHEADER2 already has support for parts of this values. Microsoft's doc says: If the AMCONTROL_COLORINFO_PRESENT flag is set in the dwControlFlags member, you can cast the dwControlFlags value to a DXVA_ExtendedFormat structure to access the extended color information.

http://msdn.microsoft.com/en-us/library/ff563967(VS.85).aspx

[BetaBoy]

I'm bringing in squid_80 and Haali to follow/discuss.

[madshi]

While this idea sounds good in theory, there is one problem with it.
Some formats require codec/format specific data that gets appended after the BITMAPINFOHEADER in VIDEOINFOHEADER2.
If you want to append data to the end of VIDEOINFOHEADER2, you would occupy this space.
True, so it would not be *perfectly* compatible to FORMAT_VideoInfo2, but only "mostly". But isn't 90% compatability better than none at all? Or in other words: What do we lose if we start with VIDEOINFOHEADER2?

This new format is mainly designed for data from the decoder to the renderer, not from the source/splitter to the decoder, right?
Would decoders benefit from any additional information we could provide on the splitter side of things?

I'm currently working on a DS source/splitter and just wondering if on this side anything new is required.
There's been some 3D related discussion on the Matroska mailing list in the last couple of days/weeks, too. So I think this new media type could also be very useful to connect splitters with decoders. If you have any splitter related new fields in mind that would be useful, we can consider them.

You should also include Side-by-side MPEG-2 and H.264 broadcasts. There are also 720p120 H.264 3D broadcasts.
Yes, absolutely.

Agreed. The dwNumView field already allows for more than two views.

[...]

I don't think so. It is not used in practice and would make the new media type much more difficult. The only 2D+Depth files are the ones for Philips Wow 3D displays, but they not only use 2D+Depth but some of them 2D+Depth+Background 2D+Background Depth.
Why would it make the media type more difficult? We could simply add one more "guidStereoLayout" (we should change the name of that field, though) for 2D+Depth etc. The first view index would then be 2D, index 1 would be Depth. I don't see how that would make anything more complicated?

I suggest to define a separate media type for 2D+Depth and do not cover it in our new media type.
Why? I'd have to implement 2 different media types in my renderer, then. That makes things only more complicated. I'd much prefer to have only one new media type which can handle all new formats.

Agreed. That's what I did.
No, you've inserted your new fields *before* the bmiHeader field. As a result your structure definition is not compatible to VIDEOINFOHEADER2.

VIDEOINFOHEADER2 already has support for parts of this values. Microsoft's doc says: If the AMCONTROL_COLORINFO_PRESENT flag is set in the dwControlFlags member, you can cast the dwControlFlags value to a DXVA_ExtendedFormat structure to access the extended color information.

http://msdn.microsoft.com/en-us/library/ff563967(VS.85).aspx
Cool, I didn't know that! However, does any current splitter/decoder actually fill in these values? I doubt it. I think if we add dedicated fields to the structure and make them mandatory (although they may be set to "not specified"), that would very much increase the chance of splitters & decoders actually filling in proper values. Furthermore, the DXVA_ExtendedFormat structure does not contain all the information I'm looking for.

I'm bringing in squid_80 and Haali to follow/discuss.
That's great - thanks!

Here comes my suggestion:

typedef struct tagOPENVIDEOINFOHEADER
{
VIDEOINFOHEADER2 VideoInfoHeader2;
MULTIVIEWHEADER MultiView;
VIDEOPROPSHEADER VideoProps;
DWORD Flags[8]; // Flags[0] & 0x1 = Full Range
} OPENVIDEOINFOHEADER;

typedef struct tagMULTIVIEWHEADER
{
DWORD ViewType; // enum to be defined
DWORD NumViews;
}

typedef struct tagVIDEOPROPS
{
BYTE ColourPrimaries; // enum to be defined
BYTE TransferCharacteristics; // enum to be defined
BYTE MatrixCoefficients; // enum to be defined
BYTE ChromaFormat; // enum to be defined
BYTE ChromaLocTopField; // enum to be defined
BYTE ChromaLocBottomField; // enum to be defined
BYTE LumaBitdepth;
BYTE ChromaBitdepth;
}

[pwimmer]

I've written a C++ header file that includes all the suggestions made so far. The link is in the second message of this thread.

It was possible to use a union including the VIDEOPROPS structure and dwControlFlags (that can be casted to DXVA_ExtendedFormat). This means I didn't break compatibility with the VIDEOPINFOHEADER2. New fields are only required for the stereo stuff.




typedef struct tagVIDEOPROPS
{
BYTE ColourPrimaries; // enum to be defined
BYTE TransferCharacteristics; // enum to be defined
BYTE MatrixCoefficients; // enum to be defined
BYTE ChromaFormat; // enum to be defined
BYTE ChromaLocTopField; // enum to be defined
BYTE ChromaLocBottomField; // enum to be defined
BYTE LumaBitdepth;
BYTE ChromaBitdepth;
}

ColourPrimaries, TransferCharacteristics, MatrixCoefficients, ChromaFormat and FullRange are already covered by DXVA_ExtendedFormat. I just added a few new items to the enumerations. To ensure that we do not have a conflict if Microsoft should add their own items in the future as well, I used the highest possible values for the new items instead of the next available value. ChromaLocTopField and ChromaLocBottomField are imho not required, they are covered by DXVA2_VideoChromaSubsampling. LumaBitdepth and ChromaBitdepth is implicitely defined by the FOURCC, there is no need to store it twice. Consequently, all required fields are already present in DXVA_ExtendedFormat so we can stay compatible.

[madshi]

I've written a C++ header file that includes all the suggestions made so far.
We're working together to define the structures. No offense, but does it make sense to put your name as the copyright holder above it? If at all, shouldn't this thread be the copyright holder?

Redistributions in binary form must reproduce the above copyright
No, thanks!! Your copyright conditions are worse than using media types defined by MS!

It was possible to use a union including the VIDEOPROPS structure and dwControlFlags (that can be casted to DXVA_ExtendedFormat). This means I didn't break compatibility with the VIDEOPINFOHEADER2. New fields are only required for the stereo stuff.
I would still prefer dedicated fields in the header, because this whole logic with "AMCONTROL_COLORINFO_PRESENT" makes the VIDEOPROPS fields look very much optional. I want them to be mandatory. If we make them optional, then once again nobody will bother using them.

LumaBitdepth and ChromaBitdepth is implicitely defined by the FOURCC, there is no need to store it twice.
FOURCC defines 8bit, 10bit or 16bit. Intermediate values are not defined (at least not by MS). Also the existing FOURCCs always define the same bitdepth for Luma and Chroma. The h264 spec allows *any* bitdepth between 8bit and 14bit, and it allows different bitdepths for Luma and Chroma. So let's consider a video track with 9bit Luma and 8bit Chroma. What will a renderer output? Probably 10bit for both Luma and Chroma. But now the video renderer does not know the *native* bitdepth of the source. And that information could be very useful for specific processing algorithms (e.g. anti-banding post processing).

BTW, your structure is still incompatible with VIDEOINFOHEADER2, because the bmiHeader field is in the wrong place, as I said before.

For example, for a 1920 x 1080 video containing squeezed side-by-side format
// the dwPictAspectRatioX should be set to 32 and dwPictAspectRatioY to 9.
This is debatable. The 32:9 aspect ratio would describe what is transfered downstream. But dwPictAspectRatioX/Y actually does not describe what is transfered, but what should ultimately be visible on the screen. And that is 16:9. Whether the side-by-side is squeezed or not is very obvious from the bmiHeader information. Actually it could even be half squeezed, or it could be anamorphically squeezed in addition to the side-by-side squeeze, no problem at all. No need to misuse the dwPictAspectRatioX/Y fields.

// ----------------------------------------------------------------------------
// Stereoscopic layout information
// ----------------------------------------------------------------------------
As said before, please don't use "stereo", please replace it with "Multiview" or something not stereo-specific.

GUID guidStereoLayout; // See supported STEREOLAYOUT_xxx GUIDs above
union {
STEREOLAYOUTPARAMS1 stereoLayoutParams1; // Layout-specific parameters
STEREOLAYOUTPARAMS2 stereoLayoutParams2; // Layout-specific parameters
DWORD dwReserved3[2]; // Set to zero if neither of the STEREOLAYOUTPARAMS structures is used
};
The union solution is somewhat "clever", but I'd prefer a simple flat structure with all fields in it. Just imagine you want to log out the information fields of the media type and the multiview GUID is not known to you (it might be a new GUI defined after your software was written). In this situation you don't know which of the unions is the correct one to use. Because of that I'd prefer a flat structure with all fields in it.

// Video transfer matrix. Values are identical to the
// DXVA2_VideoTransferMatrix and MFVideoTransferMatrix enumerations.
I'd like the media type to have a way to signal that the original source was encoded in RGB and not YCbCr. Maybe we could add a transfer matrix for RGB (basically a passthrough matrix) for that purpose? The reason why I want to know which format the original source had is that my renderer insists on doing all video processing, so it refuses a connection which would result in the decoder doing any kind of conversion. Which means that I'd like to be able to see from the media type information whether the connection would be a "native" connection or not. If the original source was YCbCr, my renderer will not accept an RGB connection and vice versa...

Views must be ordered from left to right. For stereoscopic content (dwNumViews = 2), view 0 is the left and view 1 the right view. For multiview content (dwNumViews > 2), view 0 is the most-left view, view dwViewNum-1 is the most-right view.
If that is the case then why do you have different "STEREOLAYOUT_SIDEBYSIDELEFTFIRST" and "SIDEBYSIDERIGHTFIRST" GUIDs? I'd simply drop that quoted paragraph.

The IOpenMediaSample interface must be supported by media samples if the field guidStereoLayout in the STEREOVIDEOINFOHEADER structure is set to the value STEREOLAYOUT_MULTIBUFFERS.
Why only for MULTIBUFFERS? E.g. for 2DDEPTH IOpenMediaSample would also be needed.

Number of horizontal tiles (only for STEREOLAYOUT_TILEDxxx, zero otherwise)
I'd say "1 otherwise". Makes more sense to me.

[pwimmer]

I've addressed the issues you noticed.

* The license terms are less strict.

* A Doom9 thread cannot be a copyright holder, only individuals. Of course I'll add other contributers as well. Let me know name & email address.

* VIDEOPROPS will always be optional because in many causes the information is unknown. In this case I prefer the value "unknown" instead of forcing a filter to write some mess in these fields. If it is "unknown", it gives the application the chance to the ask the user or apply application-specific defaults.

* You should define new FOURCCs for currently unsupported bit depths. Using existing FOURCCs with wrong bit depths will introduce more problems than it solves because developers will not expect other bit depths than those defined by Microsoft. Is there really content with unusual bit depths? The H.264 spec is very flexible, but that doesn't mean that all possible combinations are used in practice. Lets see the optinion of other people on this issue...

* Using dwPictAspectRatioX and dwPictAspectRatioY is a very convenient way to make squeezed stereo files display properly in 2D media players. Although the output is not stereo but still two views, at least the image is not distored. Imho it is a clean solution, it is pretty much the same situation as with anamorphic 2D content. dwPictAspectRatioX and dwPictAspectRatioY are designed for the purpose of making the video display in the proper aspect ratio, so why not use it for stereo content as well? Anyway, I added an alternative that allows to leave dwPictAspectRatioX and dwPictAspectRatioY untouched: StereoFlags_HalfHorizontalResolution and StereoFlags_HalfVerticalResolution. But I would allow both methods.

* I modified the text to mention "multiview" in addition to "stereoscopic". As multiview content is typically used for autostereoscopic displays, I do not consider the term "stereoscopic" wrong or inappropriate. The specification currently doesn't cover any other use of multiview content than for autostereoscopic displays. If somebody is familiar with other usage scenarios of multiview content (e.g. multi viewpoint video) and has suggestion what could be added to better support such scenarios, then let me know.

* I completely gave up the union stuff in favor of a flat structure.

* The concept of a passthrough matrix is rather confusing and I doubt any filter developer would support it. I added a much simpler feature that allows to find out if a media type is the native one: The flags AdvancedFlags_NativeFormatUnknown, AdvancedFlags_NativeFormatTrue and AdvancedFlags_NativeFormatFalse.

* I have different "STEREOLAYOUT_SIDEBYSIDELEFTFIRST" and "SIDEBYSIDERIGHTFIRST" GUIDs because I want the GUIDs to be as precise as possible. "Views must be ordered from left to right" only applies to the STEREOLAYOUT_MULTIBUFFER GUID. A filter can easily change the order by swapping pointers while it would mean memcpys for other layouts. Thus I only created a single GUID for the multi buffer layout but not for the other layouts. I updated the comments to clarify the issue.

* The latest version contains many new stereo layouts and desciptions for all layouts. Most notably, I added STEREOLAYOUT_UNKNOWN that should be used instead STEREOLAYOUT_MONOSCOPIC if the actual content is unknown.

* The latest version also includes many new fields in the new STEREOLAYOUTPROPS structure, e.g. orientation (rotation and flipping), cropping, parallax adjustment. All these stuff is used and required in practice, e.g. for content recorded with mirror rigs, beam splitter or other optical stereo attachments.

[madshi]

I like some/most of the changes, but I still see some issues. You seem to know a lot more about 3D/multiview related things than I do, so there's not much I can say about that.

(1) Again: Your structure is not compatible with VIDEOINFOHEADER2, because bmiHeader is in the wrong place.

(2) The license terms are much better. But IMHO we don't need any copyright at all. But let's hear what other people have to say about that.

(3) About VIDEOPROPS being optional: The same argument you're using could also be used for the multiview properties - yet all those fields are there and must be filled. There's no "this_is_multiview_content" flag which must be set first to activate the multiview structures. I want the same for VIDEOPROPRS. Of course there should be an "undefined" or "unknown" value for every VIDEOPROPS field. But having to activate the VIDEOPROPRS structure first by setting an additional flag is not good enough for me. If the VIDEOPROPS information is unknown, then filters can set the fields that way. But very often the information is known (or could easily be retrieved) and it's just not set because of lazyness.

(4) I just noticed that one original field I was asking for, namely a "ChromaFormat (4:2:0, 4:2:2, 4:4:4)" is missing. Yeah, I already hear you say: "You can see that from the FOURCC". No, you can't, because an intermediate filter (e.g. ffdshow raw video processor) could already have applied chroma upsampling. Or the decoder could have internally upsampled chroma for e.g. YUY2 output. I want to know what format the original source had, not what the upstream filters have converted the original source data to.

(5) Right now (4) made me think: Maybe there should be 2 VIDEOPROPS structures: One for the native format. And another one for the "current" data format. Something like "NativeVideoProps" and "CurrentVideoProps".

(6) "Using existing FOURCCs with wrong bit depths will introduce more problems than it solves". I never meant to suggest using incorrect bitdepths. Of course if a filter uses a 10bit FOURCC then the data must really be 10bit. But if the original video source was only 9bit then the least significant bit of that 10bit data will always be 0. And that is an important thing to know for the video renderer. It makes no sense at all to define dozens of different FOURCCs for any wild bitdepth combination. No decoder writer would ever implement dozens of FOURCCs for that purpose. Too much programming time needed. Too much time needed for testing. It makes *much* more sense to e.g. always have the filter use a 16bit FOURCC with the video data "upconverted" to 16bit, with additional information fields that indicate how many least significant bits are zeroed out.

(6) About dwPictAspectRatioX/Y. I was just starting to write a long text about how we should set it to 16:9, but then I thought: If there's a video renderer which doesn't really support/know 3D at all, then if we set aspect ratio to 16:9 for "side-by-side (full)" content, the video renderer will draw the image incorrectly. So I have to change my opinion and agree with your original suggestion to use 32:9 for side-by-side (full), for better compatability to non-3D-aware video renderers. Which also means that I vote for removing those half/full flags again.

(6) What happens if the decoder wants to fill the VIDEOPROPS fields, but the content is originally RGB? In that case the current transfer matrix and chroma subsampling options don't fit. The correct transfer matrix for RGB is "passthrough". You could also name it "no matrix (RGB)" or something like that.

(7) I fear that the native flags might not work as intended. E.g. if a decoder programmer finds that his decoder does not connect to my renderer if he sets "NativeFormatFalse", then he might misunderstand that as a bug in my renderer and as a "fix" he may decide to always set the flag to "True" or "Unknown". As a result I'd be screwed. I'd prefer to have decoder programmers fill in the correct values in the VIDEOPROPS structure, then I can check myself what they've done.

So my suggestion would be this:

typedef struct tagOPENVIDEOINFOHEADER {
VIDEOINFOHEADER2 videoInfo2;
VIDEOPROPS nativeVideoProps;
VIDEOPROPS currentVideoProps;
STEREOLAYOUTPROPS stereoLayoutProps;
UINT reserved[8]; // Set to zero when writing and ignore when reading
} OPENVIDEOINFOHEADER;

I also like to suggest using "videoInfo2" like that, because it makes the structure look much smaller and easier to understand. If you list all VIDEOINFOHEADER2 fields one by one in your structure, your structure looks much more intimidating.

[nevcairiel]

You should forget the compat with VIH2. Devs need to modify their code anyway for the new format, and as long as the fields are the same, plus new ones, you can still parse it with the same code.
Having a VIH2 at the top is rather confusing, because the BMI in the VIH2 can actually grow with extradata. How would you handle this? Move it to the end of the OVH? Your compat is already gone, then.

I'm with pwimmer on the basic design, copy VIH2 and add new fields, keep BMI at the end.

[pwimmer]

Having a VIH2 at the top is rather confusing, because the BMI in the VIH2 can actually grow with extradata. How would you handle this? Move it to the end of the OVH? Your compat is already gone, then.


Having the full VIH2 at the top would be nice, but as you say it is not possible because BITMAPINFOHEADER is a variable size structure, so it must be at the end.

All the other fields should be identical, which is the reason why I would like to keep the VIH2-compatible VIDEOPROPS. Anyway, the VIDEOPROPS is big improvement over the dwControlFlags, because it's now obvious what information it contains. Many developers didn't know that dwControlFags can be casted to DXVA_ExtendedFormat. At least this issue is solved, which should increase the chance that the structure ios actually used. By keeping it compatible we even increase the change that developers even use it for VIH2, too.

[pwimmer]

(3) About VIDEOPROPS being optional: The same argument you're using could also be used for the multiview properties - yet all those fields are there and must be filled. There's no "this_is_multiview_content" flag which must be set first to activate the multiview structures. I want the same for VIDEOPROPRS. Of course there should be an "undefined" or "unknown" value for every VIDEOPROPS field. But having to activate the VIDEOPROPRS structure first by setting an additional flag is not good enough for me.


I agree it is not good that there is a separate flag to activate the VIDEOPROPS structure. But that's how Microsoft specified it and since we can reuse the structure, it makes sense to stay compatible.

There is no flag to active the STEREOLAYOUTPROPS, but it is optional, of course. If all byte are set to zero (thus, if the developer ignores it), this defaults to STEREOLAYOUT_UNKNOWN (which is deliberately defined as GUID_NULL). All the other fields in the structure are defined in a way that ensures that 0 always means the default. A renderer that ignores STEREOLAYOUTPROPS and a renderer that obeys STEREOLAYOUTPROPS will output the same image when all fields of STEREOLAYOUTPROPS are zero.


(4) I just noticed that one original field I was asking for, namely a "ChromaFormat (4:2:0, 4:2:2, 4:4:4)" is missing. Yeah, I already hear you say: "You can see that from the FOURCC". No, you can't, because an intermediate filter (e.g. ffdshow raw video processor) could already have applied chroma upsampling. Or the decoder could have internally upsampled chroma for e.g. YUY2 output. I want to know what format the original source had, not what the upstream filters have converted the original source data to.


Ok, now I understand what you mean. But I don't believe the fields are well-thought-out yet. What do you do if the filter delivers 4:4:4 or RGB and says the original chroma format was 4:2:0? You can't do anything, because you do not know how the upsampling was performed. You cannot undo it. It is simply impossible to restore the original data. To find out if it is not the native format, the flags I've already introduced are sufficient.


(6) But if the original video source was only 9bit then the least significant bit of that 10bit data will always be 0.


Actually padding with 0 is the wrong way to convert to higher bit depths. This would make the image slightly darker than it should be.

EDIT: I was wrong. http://msdn.microsoft.com/en-us/library/bb970578%28VS.85%29.aspx#_420formats specifies that bit shifting should be used!


(6) About dwPictAspectRatioX/Y. I was just starting to write a long text about how we should set it to 16:9, but then I thought: If there's a video renderer which doesn't really support/know 3D at all, then if we set aspect ratio to 16:9 for "side-by-side (full)" content, the video renderer will draw the image incorrectly. So I have to change my opinion and agree with your original suggestion to use 32:9 for side-by-side (full), for better compatability to non-3D-aware video renderers. Which also means that I vote for removing those half/full flags again.


There are pros and cons for both approaches. To con of dwPictAspectRatioX/Y is that the display size in a 2D player depends on the decoder being used. A old decoder not supporting the spec would output a squeezed image while a new decoder would output a full-size image. The flags avoid this issue, while still allowing a 3d-enabled renderer/player to properly decode the side-by-side format and display at correct aspect ratio.

Imho, there is no clear winner which approach is better, so I would keep both and let the developer decide.


(6) What happens if the decoder wants to fill the VIDEOPROPS fields, but the content is originally RGB? In that case the current transfer matrix and chroma subsampling options don't fit. The correct transfer matrix for RGB is "passthrough". You could also name it "no matrix (RGB)" or something like that.


For RGB content, filters should write VideoPrimaries_Unknown and VideoTransferMatrix_Unknown; when reading, a filter should ignore videoTransferMatrix and videoPrimaries.

This is the way it is already handled in VIH2 and I see no need to change it.


(7) I fear that the native flags might not work as intended. E.g. if a decoder programmer finds that his decoder does not connect to my renderer if he sets "NativeFormatFalse", then he might misunderstand that as a bug in my renderer and as a "fix" he may decide to always set the flag to "True" or "Unknown". As a result I'd be screwed. I'd prefer to have decoder programmers fill in the correct values in the VIDEOPROPS structure, then I can check myself what they've done.


But if the programmer correctly fills in nativeVideoProps and currentVideoProps, you filter would refuse connection as well (because it is not the native format), resulting in exactly the same problems.

It's more likely that developers are willing to properly set a single flag than a complex structure.

[madshi]

Having a VIH2 at the top is rather confusing, because the BMI in the VIH2 can actually grow with extradata. How would you handle this?
How big can it grow? As far as I can see, the max size would be "RGBQUAD bmiColors[256]", right? So the problem could be solved like this:

typedef struct tagOPENVIDEOINFOHEADER {
VIDEOINFOHEADER2 videoInfo2;
RGBQUAD bmiColors[256];
VIDEOPROPS nativeVideoProps;
VIDEOPROPS currentVideoProps;
STEREOLAYOUTPROPS stereoLayoutProps;
UINT reserved[8]; // Set to zero when writing and ignore when reading
} OPENVIDEOINFOHEADER;
Violà, perfect compatability with VIDEOINFOHEADER2. Or am I missing something?

[madshi]

I agree it is not good that there is a separate flag to activate the VIDEOPROPS structure. But that's how Microsoft specified it and since we can reuse the structure, it makes sense to stay compatible.
You say yourself that many devs didn't know about DXVA_ExtendedFormat. So I don't see a big loss if we don't use that, but define our own fields instead. Using our own fields would not be incompatible, either, because they'd be outside of VIDEOINFOHEADER2.

Ok, now I understand what you mean. But I don't believe the fields are well-thought-out yet. What do you do if the filter delivers 4:4:4 or RGB and says the original chroma format was 4:2:0? You can't do anything, because you do not know how the upsampling was performed. You cannot undo it. It is simply impossible to restore the original data.
Correct. But I can inform the user, saying that the decoder has upsampled chroma and that he should modify the decoder settings to disable the chroma upsampling, or switch to another decoder.

Actually padding with 0 is the wrong way to convert to higher bit depths.

EDIT: I was wrong. http://msdn.microsoft.com/en-us/library/bb970578%28VS.85%29.aspx#_420formats specifies that bit shifting should be used!
Doing anything else would introduce banding, so bit shifting is the only reasonable solution. (Unless you want to go the full mile with floating point and dithering etc).

To con of dwPictAspectRatioX/Y is that the display size in a 2D player depends on the decoder being used. A old decoder not supporting the spec would output a squeezed image while a new decoder would output a full-size image.
I don't understand what you mean here. Can you explain?

For RGB content, filters should write VideoPrimaries_Unknown and VideoTransferMatrix_Unknown
Wrong. The primaries are very important for RGB content, too!!

But if the programmer correctly fills in nativeVideoProps and currentVideoProps, you filter would refuse connection as well (because it is not the native format), resulting in exactly the same problems.
With nativeVideoProps/currentVideoProps, the programmer would have to actually lie to make my filter connect. With the flags you have suggested he simply has to leave it at "0", which is even the default value (provided that he uses memset(0))! Furthermore, if I have nativeVideoProps and currentVideoProps, I can see exactly what the filter chain has done and so I can give exact tips to the end user about what he has to change to improve image quality.

[pwimmer]

How do you like this solution for > 8 bpp?


// 10 bit and 16 bit YUV can contain content that has been converted from 9
// to 10 bit or from 11-15 to 16 bit by using bitshifting. To allow a
// renderers to use optimized algorithms that minimize banding, the following
// fields define the original bit depth of the luminance and chrominance
// channels. If native format is 10 or 16 bit, respectively, the fields can
// either be set to 0 or 10/16. If the native format is unknown or the FOURCC
// is a full range format, which means the maximum value is 2^n-1 and not
// 255*(2^(n-8)), the fields should be set to 0. If only bit shifting has
// been applied on the native data, AdvancedFlags_NativeFormatTrue should be
// set. The following link contains a list of bit-shifted YUV formats:
// http://msdn.microsoft.com/en-us/library/bb970578(v=VS.85).aspx
UINT originalLumaBitCount;
UINT originalChromaBitCount;

[madshi]

How do you like this solution for > 8 bpp?
I like it, but I'd suggest to shorten the comment a bit. How about this?

"10 bit and 16 bit YUV can contain content that has been converted from a lower bitdepth by using bitshifting. To allow the video renderer to use optimized algorithms that minimize banding, the following fields define the original bit depth of the luminance and chrominance channels. These fields should be set (and should only be set) if a lower bitdepth source was upconverted to the output bitdepth by using bitshifting."

[nevcairiel]

How big can it grow? As far as I can see, the max size would be "RGBQUAD bmiColors[256]", right? ?

Not really. Compressed formats send private data in that part thats not specifically limited by any length. For example H264 decoders expect a SPS and a PPS NALU in there.
While i haven't seen a format with really long private data, limiting it here seems wrong.

[madshi]

Not really. Compressed formats send private data in that part thats not specifically limited by any length. For example H264 decoders expect a SPS and a PPS NALU in there.
How do these filters calculate the offset where the private data begins?

[pwimmer]

A new suggestion that replaces the nativeVideoProps and the native format flags: Degradation flags that are passed from the source downstream to the renderer in the OPENVIDEOINFO structure. Each filters adds flags if it performs lossy operations on the content.


// Degradation flags. These flags specify all kinds of degradation that
// occured in the DirectShow filter graph pipeline. Filters should add the
// appropriate flags if they do any modification to the content. Flags
// already set by upstream filters must be preserved. Muxers that combine
// two video streams (e.g. left and right stream to a stereo layout) should
// use binary-or on all input stream flags to create the output stream flags.
typedef enum tagVIDEODEGRADATIONFLAGS {
VideoDegradation_FlagsValid = 0x00000001, // Set if the degradation flags are valid
VideoDegradation_Other = 0x00000002, // A degradation other than the ones below occured
VideoDegradation_Incomplete = 0x00000004, // An upstream filter does not support this flag, so the flags might be incomplete
VideoDegradation_ChromaUpsampled = 0x00000008, // Chroma component upsampled e.g. 4:2:0 to 4:2:2
VideoDegradation_ChromaDownsampled = 0x00000010, // Chroma component downsampled e.g. 4:2:2 to 4:2:0
VideoDegradation_Upsampled = 0x00000020, // Image has been upscaled to higher resolution
VideoDegradation_Downsampled = 0x00000040, // Image has been downscaled to lower resolution
VideoDegradation_Cropped = 0x00000080, // Image has been cropped
VideoDegradation_Rotated = 0x00000100, // Image has been rotated
VideoDegradation_Flipped = 0x00000200, // Image has been flipped
VideoDegradation_PrecisionIncreased = 0x00000400, // The bits per pixels have been increased (do not set if filter can undo the operation by bitshifting, see originalLumaBitCount and originalChromaBitCount)
VideoDegradation_PrecisionDecreased = 0x00000800, // The bits per pixels have been decreased (do not set if YUV is converted to RGB and the bit depth per component, e.g. 8 bpp remains the same)
VideoDegradation_FramerateIncreased = 0x00000800, // Interpolated frames added
VideoDegradation_FramerateDecreased = 0x00001000, // Frames dropped
VideoDegradation_YUVtoRGB = 0x00002000, // Colorspace conversion from YUV to RGB
VideoDegradation_RGBtoYUV = 0x00004000, // Colorspace conversion from RGB to YUV
VideoDegradation_ColorToGrey = 0x00008000, // Chroma channel removed or cleared
VideoDegradation_ColorspaceConversion = 0x00010000, // Colorspace conversion other than YUV to RGB, RGB to YUV or color to grey (e.g. RGB to YCgCo)
VideoDegradation_RemovedAlpha = 0x00020000, // Removed alpha channel
VideoDegradation_PrimariesChanged = 0x00040000, // Image converted to use other primaries
VideoDegradation_TransferFunctionChanged = 0x00080000, // Gamma correction applied
VideoDegradation_VideoProcAmp = 0x00100000, // Brightness, contast, saturation, hue and similar adjustments
VideoDegradation_LossyStereoFormatConversion = 0x00200000, // Loss stereoscopic format conversion, e.g. conversion to checkerboard or anaglyph layouts
VideoDegradation_StereoFormatInterpolation = 0x00400000 // Additional interpolated pixels added during stereo format conversion but existing pixel values remained untouched, e.g. from Sensio to side-by-side layout

// Future flags might be added in the future and must be preserved by filters.
} VIDEODEGRADEATIONFLAGS;

[madshi]

:p I still prefer my suggestion, but I guess I could live with yours. A few comments on those flags:

(1) Rotation (if 90° wise) and flipping is probably lossless? So should we consider that a degradation?
(2) "Cropped" needs a clear definition. E.g. I don't consider applying cropping specified by the h264 bitstream as a degradation, but rather as a must, because without h264 cropping h264 Blu-Rays would be 1920x1088.
(3) "YUV" is usually used for analog content. The correct term for digital data is "YCbCr", AFAIK.
(4) I think there needs to be one more degradation flag for changing of the "transfer matrix"?

I like how you named the flags "degradation" and not "modification" or something. That shows nicely that such changes are usually considered to be a bad idea... :)

[pwimmer]

:p I still prefer my suggestion, but I guess I could live with yours. A few comments on those flags:

(1) Rotation (if 90° wise) and flipping is probably lossless? So should we consider that a degradation?
(2) "Cropped" needs a clear definition. E.g. I don't consider applying cropping specified by the h264 bitstream as a degradation, but rather as a must, because without h264 cropping h264 Blu-Rays would be 1920x1088.
(3) "YUV" is usually used for analog content. The correct term for digital data is "YCbCr", AFAIK.
(4) I think there needs to be one more degradation flag for changing of the "transfer matrix"?

I like how you named the flags "degradation" and not "modification" or something. That shows nicely that such changes are usually considered to be a bad idea... :)

ad 1)

Agreed, the flag should only be set for rotations != 90° and 180°. Maybe it should even be named "resampled" to cover other kinds of distortions as well. Flipping should indeed be removed as it is lossless.

ad 2)

The flag should indeed not be set in the special case 1088 pixels height as no valid content is lost.

ad 3)

Micosoft also uses the term YUV in the DirectShow SDK, so it is not completely wrong. I admit that I'm not sure about the actual difference...

ad 4)

Use VideoDegradation_ColorspaceConversion if the conversion is from one YUV to a different YUV space, or VideoDegradation_YUVtoRGB if the destination is RGB.

[pwimmer]

:p I still prefer my suggestion, but I guess I could live with yours.

The degradation flags are more flexible and also easier to implement by filter developers.

For example, you couldn't detect if a decoder performs contrast/brightness adjustments (which you certainly want to do yourself in your renderer) with your approach, while it is possible with the degredation flags.

[madshi]

The flag should indeed not be set in the special case 1088 pixels height as no valid content is lost.
Why special casing 1088? Should we not "allow" the full h264 cropping to be executed without a degradation flag having to be set? I think the cropping degradation flag should only be set if a filter crops more than the video bitstream asks for.

Micosoft also uses the term YUV in the DirectShow SDK, so it is not completely wrong. I admit that I'm not sure about the actual difference...
I think I was wrong. It's "YPbPr" which is for analog and "YCbCr" for digital. "YUV" seems to be a valid alternative name for "YCbCr". I've confused "YUV" with "YPbPr".

For example, you couldn't detect if a decoder performs contrast/brightness adjustments (which you certainly want to do yourself in your renderer) with your approach, while it is possible with the degredation flags.
True.

[nevcairiel]

How do these filters calculate the offset where the private data begins?

Its just plainly appended after the BMI, and in the BMI is a field that holds the actual size of the thing. (Actually with VIH2 there is a extra null byte between the BMI and the extradata). The formats do not hold color tables, for that matter.

If you want to go with VIH2 as the first member of your new structure, i would do it similar to how MPEG2VIDEOINFO looks, and provide your own place to store extradata (as well as a field that indicates its length)
It would still break compat with VIH2 if the extradata is used, but at least you won't be re-creating the structure, and your concerns of it looking like a "beast" are at least adressed.

[madshi]

If you want to go with VIH2 as the first member of your new structure, i would do it similar to how MPEG2VIDEOINFO looks, and provide your own place to store extradata (as well as a field that indicates its length)
It would still break compat with VIH2 if the extradata is used, but at least you won't be re-creating the structure, and your concerns of it looking like a "beast" are at least adressed.
Ok, I like the MPEG2VIDEOINFO logic. @pwimmer?

[pwimmer]

I think the easiest solution is to place VIH2 at the end of the structure...

[madshi]

Hmmmm... Yes, I like that even better! It's a bit different to what MS is normally doing, but I like it.

[pwimmer]

Updated the header file (link is in second posting).

The structure is very simple now:


typedef struct tagOPENVIDEOINFOHEADER {
EXTRAVIDEOINFO extraVideoInfo; // All extra stuff not part of VIDEOINFOHEADER2
VIDEOINFOHEADER2EX videoInfoHeader2; // Compatible with VIDEOINFOHEADER2
} OPENVIDEOINFOHEADER;



The VIDEOINFOHEADER2EX is compatible with VIDEOINFOHEADER2, it just defines a union that allows easy access to the VIDEOPROPS.

[yesgrey]

We're working together to define the structures. No offense, but does it make sense to put your name as the copyright holder above it? If at all, shouldn't this thread be the copyright holder?
I've addressed the issues you noticed.
* The license terms are less strict.
* A Doom9 thread cannot be a copyright holder, only individuals. Of course I'll add other contributers as well. Let me know name & email address.
Before starting my participation on this I would like to clarify a little further the copyright idea...

In first place, I don't see why it should have a copyright at all... Are you thinking in getting any money from it? If you don't, all this copyright talking might only serve to keep away from it some potential users...

If we really want to create an open source format that would be used by a lot of people, we should target to a completely open format without any copyrights.

Let me show you an example... A while ago, in a doom9 thread, me, madshi, tritical and IanB have defined an open file format for 3D LUTs, and we haven't created any copyright. Recently, we even decided to create a sourceforge project to hold the entire format. You can take a look at it here (http://thr3dlut.sourceforge.net/).

I think that this idea would work better if you would do something similar. A sourceforge project is very simple to create and handle, and then other people involved would also be able to edit and/or update the file format whenever it would be needed.

What do you think?:)

[Wilbert]

Before starting my participation on this I would like to clarify a little further the copyright idea...

In first place, I don't see why it should have a copyright at all... Are you thinking in getting any money from it? If you don't, all this copyright talking might only serve to keep away from it some potential users...
That's not possible this way. Every released work is copyrighted (although that can be implicitly) unless disclaimed otherwise. You just need to release it explicitly in the public domain:

http://www.gnu.org/philosophy/categories.html#PublicDomainSoftware

[yesgrey]

That's not possible this way. Every released work is copyrighted (although that can be implicitly) unless disclaimed otherwise. You just need to release it explicitly in the public domain:

http://www.gnu.org/philosophy/categories.html#PublicDomainSoftware
Thanks for the clarification, I didn't know that.

[madshi]

So where does that leave us?

It would be great if we could get feedback from the commercial guys I've invited to this thread (e.g. CoreAVC, Nero etc). Are you guys ok with the current copyright? Or do you need the spec to be public domain for you to use it?

Personally, I'd prefer if the spec were public domain and if it would be managed through Sourceforge (or a comparable site), with at least 2-3 doom9 devs having edit rights.

[yesgrey]

Personally, I'd prefer if the spec were public domain and if it would be managed through Sourceforge (or a comparable site), with at least 2-3 doom9 devs having edit rights.
I would prefer this too.

[pwimmer]

I would prefer this too.

I'll setup a SourceForce project for it. It is certainly best that the spec is avaiable on a neutral place.

Before I put it on SourceForce, we should agree at least on a final name so that we don't have to rename the SourceForge project later on. Currently, the name is "Open Media Format". As it is not released under one of the typical Open Source licenses but allow unrestricted commercial, I'd rather prefer a name without "Open". Some people could associate it with Open Source, which it isn't. Here are other proposals:

"Extended Video Info"
"Flexible Video Info"
"Advanced Video Info"

[pwimmer]

It would be great if we could get feedback from the commercial guys I've invited to this thread (e.g. CoreAVC, Nero etc).

We need the feedback of the commercial guys indeed. Not only on the copyright issue (which I believe should be fine) but also on the technical aspects. The goal is interoperability between stereo-capable filters of different vendors, which can only be achived if it is adopted by most vendors. I'm also missing features beyond stereoscopy. Up to know, we only support the degradation flags and have improved the video props (gamma, transfer matrix, primaries, ...). Any other ideas?

The very best feedback would be one by Microsoft, in the ideal case we can make the structure compatible with the VIDEOINFOHEADER3 that might be part of Windows 8. I have the feeling that Microsoft will add stereo support in Windows 8, the question is, only in DirectX and Media Foundation or also in DirectShow.

[madshi]

I'll setup a SourceForce project for it.
Great!

Before I put it on SourceForce, we should agree at least on a final name so that we don't have to rename the SourceForge project later on. Currently, the name is "Open Media Format". As it is not released under one of the typical Open Source licenses but allow unrestricted commercial, I'd rather prefer a name without "Open". Some people could associate it with Open Source, which it isn't. Here are other proposals:

"Extended Video Info"
"Flexible Video Info"
"Advanced Video Info"
Hmmmm... That's a tough choice. Let me add a few alternatives:

- Enhanced Video Info
- Detailed Video Info
- Video Info 2010
- Video Info XT (for extended)
- Video Info Pro

It is tempting to use 3D or stereo in the name, but I think we should avoid that, because otherwise many devs will think the new media type is "only for 3D".

[yesgrey]

I vote for "Extended Video Info".

"Flexible" doesn't give the idea that is something new.
"Advanced", after awhile, is note advanced anymore, but simply current usage.
"Detailed" seems to indicate that previously it was just simple video info.
"Enhanced" seems too short, because I feel it's much more than just enhancing...
The 2010 would seem like it's info related only to this year, the XT would seem something related to ATI, and the Pro would seem that previously it wasn't nothing too serious...

Another option would be "Multiple Video Info", but I think I prefer the "Extended", even though it might sound too Microsoftish...

I just tried to explain my feelings about each name, I hope I didn't hurt anybody's feelings... ;)

[nevcairiel]

Open Source doesn't mean it cannot be used by Commercial Products. Just license it as Public Domain or MIT/BSD. The BSD license is a recognized Open Source license that basically allows anyone to use it for anything, they just have to somewhere ship the license notice. Public Domain doesn't require any of this, while its strictly not a license, it does qualify for the definition of open source.
What you think about as "open source" are probably licenses like the GPL that require any project that uses it to also be open source (and non-commercial), but thats only one of the recognized open source licenses, and many licenses actually do not require this. Anyone actually working professionally in this business would hopefully know the difference, and wouldn't be confused by the name.

I would stay with Open Media Format, its the best name yet.

[yesgrey]

I would stay with Open Media Format, its the best name yet.
Agreed.

[BetaBoy]

We need the feedback of the commercial guys indeed. Not only on the copyright issue (which I believe should be fine) but also on the technical aspects..

Give me a chance to ping everyone here internally.

[BetaBoy]

It is tempting to use 3D or stereo in the name, but I think we should avoid that, because otherwise many devs will think the new media type is "only for 3D".
I agree.

[pwimmer]

I would stay with Open Media Format, its the best name yet.

If we keep the word "Open", we should at least rename it to Open Video Info. Open Media Format sounds like a container or codec. Video Info is more appropriate.

[BetaBoy]

madshi and Peter.... I spoke to Haali and he is following this thread. He will help support OMF and what is needed in his splitter. CoreCodec can also provide the splitter source code to Peter as needed to help make it happen.

[madshi]

@BetaBoy,

thank you, I appreciate that. So you Core guys have no specific wishes for further enhancements to the media type record? Are you ok with the current copyright (see header file linked to in the 2nd post)?