Which containers support multiple video streams?

[DND]

I'm wondering which container formats can handle (and better yet have application playback support for) multiple video streams?

Specifically I am wondering about support for things like 3D with a left and right stream that must be kept synchronized as well as multiple camera angles such as found on some DVD's (and could multiple angles be played simultaneously?).

I realize that, to a degree, this is a subjective opinion question, but which container formats (and therefore also the tools that write them) provide the best interleaving of streams for easy synchronization and playback such that they could, for example, be streamed without needing to pre buffer huge amounts of one stream in order to find enough of a corresponding stream to play them in sync.? (or worse have to find an index at the end like AVI).

Thank-you :)

[Drachir]

Take some time and look here http://gpac.sourceforge.net/ ( http://gpac.sourceforge.net/auth_demos.php )

[DND]

whoa, too much information... ;)

Should I infer that you are recommending MP4? I gather that GPAC can do a bazillion things, most of which I'm not interested in, but how much of that alternate format stuff is officially comliant such that files will play anywhere (I realize most players don't handle all of the spec)? Would this also restrict codec usage to MPEG4 compliant one(s)?

I understand the documentation of the MP4Box options to mean that it can mux or interleave multiple streams together and the creator is responsible for specifying how it is done, is that correct?
I'm not clear though on whether multiple video streams would be concatenated for sequential playback, only selectable individually such as in the quality example, or if they could be played simultaneously?

[Drachir]

Should I infer that you are recommending MP4?
Yes, I recommend to use ISO/IEC 14496-14 (MP4 file format), because you asked for 3D and synchronisation of audio/video streams(media objects).


I gather that GPAC can do a bazillion things, most of which I'm not interested in, but how much of that alternate format stuff is officially comliant such that files will play anywhere (I realize most players don't handle all of the spec)? Would this also restrict codec usage to MPEG4 compliant one(s)?

GPAC is an implementation around ISO/IEC 14496-14 (MP4 file format) with a main focus at the scenedescription BIFS and LASeR. In MP4 files (ftyp set to "mp42") the visual and aural composition are done using the BIFS system.

In MPEG-4 both visual and aural composition are done using the BIFS system. Therefore structures marked as “template” in the ISO Base Media Format which pertain to composition, including fields such as matrices, layers, graphics modes (and their opcolors), volumes, and balance values, from the MovieHeaderBox and TrackHeaderBox, are all set to their default values in the file format. These fields do not define visual or audio composition in MPEG-4; in MPEG-4, the BIFS system defines the composition.
ISO Base Media Format=ISO/IEC 14496-12 or ISO/IEC 15444-12

Beside the ISO Standard GPAC support some alternate format stuff like: Nero chapter information or iTunes tags.
In my opinion the way Apple goes with m4a or m4v files is not that bad. This files don't claim to be a MP4 file, Apple set the main file brand to "M4A " or "M4V " if I remember right.


I understand the documentation of the MP4Box options to mean that it can mux or interleave multiple streams together and the creator is responsible for specifying how it is done, is that correct?
I'm not clear though on whether multiple video streams would be concatenated for sequential playback, only selectable individually such as in the quality example, or if they could be played simultaneously?
If you mux more then one audio and one video stream with MP4Box the main file brand will be set to "isom"( ISO/IEC 14496-12 ISO Base Media Format). In such a file the visual and aural composition is not defined.

I such a case you have to create your MP4 file over BIFStext or XMT-A. Take a look at: http://gpac.sourceforge.net/auth_tutorial_eng.php

About playback:
I would recommend to use Osmo4 from GPAC for playback.
Most players which call themselves a MP4 player are not able to handle BIFS.

Question:
If we don't use BIFS, why should I use the MP4 file format?
Couldn't we get similar feature with MOV files?

[zambelli]

I believe this company is already using ASF for stereoscopic video stream storage:

http://www.3dtv.at
http://www.3dtv.at/Movies/Index_en.aspx

[Cornucopia]

Sorry to dredge this slightly older topic up, but I hadn't been a registered user until recently, then I got busy for a while...

In answer:

ALL major container formats support multiple streams!

I have successfully tested:
AVI
WMV
MPG (1, 2, 4), incl. MP4
MKV
MOV

Haven't tested Real, OGG, or Flash yet.

This is an area I've been doing a good deal of research on lately, so I know what I'm talking about.

Now, support is the important word here, because people will, in the near future, need to decide how they want their applications & appliances to react to mutliple streams...

I advocate a Standard vs. Enhanced player model, where a standard player sees a 1st video and 1st audio stream and works only on those, while completely ignoring other streams (this includes not flaking out or crashing when given multiple streams--which some do). Enhanced players, OTOH, would recognize and be able to work with 1 or more additional streams (whether video or audio).

As I am currently primarily concerned with stereoscopic video, my emphasis would be on proper recorder/encoder/muxer and demuxer/decoder/player support and response to 2 video streams (L + R). Many hardware and software players already act correctly when given dualstream video files. There are some that play as if there were only 1 stream, a few that [ACCIDENTALLY] play both streams--in perfect sync--but with one window size/placement being relatively uncontrolled, and there is 1 application (so far) that has started on this same path and is CORRECTLY supporting dualstream WMV files. That is Peter Wimmer's excellent Stereoscopic Player. I have been in contact with him and he has assured me that he will be incorporating dualstream more in the future, so I'm hopeful that will include (at least) AVI and MPG formats.

The beauty of this method is that one saves/sends only 1 file, and those that aren't set up for stereoscopic just see a normal program, while those that are will get to choose how best to view a stereoscopic program on their particular system setup. The analogy with (analog) StereoFM and ColorTV broadcasting is very strong.
The only downside I see is that, for full quality, the overall filesize/bandwidth is twice a normal program, but in this day and age I don't see that as a major limiting factor anymore.

I will soon be adding more thoughts and goodies about this and other stereoscopic technologies to my new (under construction) website: cornucopiadm.com (http://www.cornucopiadm.com).

If you have more questions about muxers/demuxers, let me know...

HTH,

Scott

[Drachir]

ALL major container formats support multiple streams!


How do you define support for multiple streams? Sure you can place more than one video stream in many container, but its is up to the player how to play the file.

In MP4 files the author can define how the media object have to be presented. If you have more than one video stream, the author of the file could define that both streams should be played at once, or play the first video stream and after this stream the next video stream...

MP4 is a container which was developted to handle media objects.

At the time I answered the question from DND I thought about something different than stereoscopy.

Take this sample here: http://gpac.sourceforge.net/auth_demos_qtvr.php
For playback you will need the plug-in from GPAC. In this sample a still image is used, but you could also use an video stream.

In MP4 you can place every media object in 3D coordinates, and you can define the time at which it have to be schown.