Features that every new container format should implement

[pirata]

Hello to you all!

This one goes for the people working at matroska, MCF and OGM.
There are also requests that maybe should be player features rather than container features, people developing players, please, have a look at this thread and share your points.

First of all, I must apologize in advance, because even though I have read the threads that are devoted to new containers, I haven't been able to extract conclusions out of them. So maybe some of the matroska gurus could summarize (in human-readable english, pleeeeeease), what will be implemented (and how), and what will not.

This things I find important:


1-make playback more confortable:

-Less CPU overhead: OGM takes more CPU for playback than avi, at least on my PC (I have to start all players in a higher priority level for them to play fluently since I use OGM). All this, when the CPU is basically doing the same things (decode video and audio). I am no coder, but I guess this could easily (and should) be avoided.

So, please, try not to be like Micro$oft coders, who do not make the best software, but the most profitable software (that is, the one that will cause more upgrades)

-Movies split across several discs behave like one: I don't know whether this should be a container property or a player property, but we know that most of the rips today take more than one CD, sometimes up to three. Lots of people have more than one cd drive, sometimes up three or more. Confortable playback in such sceneraios would mean that the player should be multiCD-multiDrive-aware, meaning this that the movie chunks on each CD on each drive should be detected as belonging to the same film. Then, they should be parsed in order to gather all needed information (e.g. all chapters would be gathered and presented to the user, not only the chapters present in the current chunk; the player would jump to the proper CD automatically according to the selected chapter). Timings should be arranged in order not to begin at 0 in each chunk.

In multiCD-single drive scenarios, the player would just demand the proper CD as needed.

Finally, consider that DVD burning is taking still a little in some countries until it becomes mainstream ("mainstream" meaning that EVERYBODY can afford it).

-For the sake of seamless playback, when the player should detect that the current chunk is about to end, it should spin up the drive where the next chunk lies in advance, so that the user doesn't have to stare at a frozen picture at chunk joints until the next drive spins up and starts reading (well, maybe this bit is for player requirements...).

-when seeking, video must not run in order to catch up with the audio. This an issue in AVI, but not in OGM. It has to be continued so.

2-Make a straight-forward conversion procedure for DVD backups (you should consider that most of the people here are making DVD backups, so they are your major "costumers"):

-Multi compression format support: yerterday I found a mp3 comment track for Mulholland Drive. I don't happen to own that DVD, but if I did, I'd like to add that track to my backup. It would be useful to have a container providing properties similar to VOBs (AC3, MP2 audio, MPEG2 video, subs, chapters...) plus support for OGG, (VBR) MP3, MPC, WAV, DTS, MPEG4 video...). I think this is already attained. Matroska supports all that, doesn't it?

-direct support for DVD menus, subtitles and chapters: there should be some means to transfer the DVD contents to the container without too much trouble. Back to the MicroDVD days and its menu support, it was painful having to create the backup menu, when all the needed data was already defined in the DVD. This also applies to subs and chapters (maybe also extras?).

3-Other:

-Space overhead: well, I don't mind more overhead if results are worth it.

-Support for characters of multiple languages (for comments and subtitles). While OGM seems to have solved this, a player like Zoom Player does not manage good unfrequent characters in comments (like for instance, the language of a given audio stream).

-multiangle: well. Dunno, guys... make your best at this one. Just think that there are lots of multiangle DVDs out there, It would be nice.

-variable playback rate for audio or video: well this the last one, and maybe you will consider it a stupid one. But look: everytime I make a backup, I try to download other sound streams from Internet for that movie. I happen to be a language students, but unfortunately, having selected french and italian as main languages was kinda unfortunate, since both languages are not provided in most of the DVDs here. Converting them takes a big deal of time on my PC, when all I had to do would be playing the video at the proper bitrate for the audio. Yeah, I know you think it is silly, but the container could be so defined that I could play the video at different rates (25, 23'976), so the audio wouldn't have to be converted (you save time and audio quality).


OK! That's all. Please be patient if some proposal sounds ridiculous to you. Just let me know why it does.

[Atamido]

Originally posted by pirata
-Less CPU overhead: OGM takes more CPU for playback than avi.
Unfortunately, one of the trade offs of making Matroska so extendable was making it more difficult to parse. It is unknown how much precessing it will take until the parser has been released, but it will definately be more than AVI. If MCF is ever fully developed, it will require very little power to parse, as that was one of the design features.
-Movies split across several discs behave like one: Confortable playback in such sceneraios would mean that the player should be multiCD-multiDrive-aware, meaning this that the movie chunks on each CD on each drive should be detected as belonging to the same film. Then, they should be parsed in order to gather all needed information In multiCD-single drive scenarios, the player would just demand the proper CD as needed. Matroska has a tagging to say that streams are continued in another with ID xxxxxxxx. So, as long as a player supports it, you could have all of these features.
-For the sake of seamless playback, when the player should detect that the current chunk is about to end, it should spin up the drive where the next chunk lies in advanceThis is completely player dependant.
-when seeking, video must not run in order to catch up with the audio. This an issue in AVI, but not in OGM. It has to be continued so. This will depend on several factors, mainly on where keyframes occur, how much space is devoted to indexing, and if the parser/player is designed to get all data synched before starting playback.
2-Make a straight-forward conversion procedure for DVD backups. This isn't something covered by the container so much, but hopefully some better all-in-one software comes out.
-Multi compression format support: Matroska supports all that, doesn't it? Yes. You can fit however many of whatever kind of supported streams into Matroska simultaneously.
-direct support for DVD menus, subtitles and chapters: there should be some means to transfer the DVD contents to the container without too much trouble. Back to the MicroDVD days and its menu support, it was painful having to create the backup menu, when all the needed data was already defined in the DVD. This also applies to subs and chapters (maybe also extras?). Unfortunately a menuing system hasn't been developed yet, but once it is, this will most likely be posssible. The man that was working on the menuing system has had to stop, so if anyone else could offer a solution, that would be great. We're looking at a DVD-Subs to USF conversion utility for subtitles.
-Space overhead: well, I don't mind more overhead if results are worth it.The overhead has always been a big concern for the Matroska team. The overhead should be small for normal circumstances, but it is configureable. For instance, how many seek points do you want, how much CRC-32 and/or error correction do you want, data signatures, etc.
-Support for characters of multiple languages (for comments and subtitles). This is dependant on the subtitle format, the playback filter, and the the player.
-multiangle: well. Just think that there are lots of multiangle DVDs out there.I haven't seen very many multiangle DVDs, but it should be pretty simple to place the extra angles in seperate streams, synched to the main stream.
-variable playback rate for audio or video: the container could be so defined that I could play the video at different rates (25, 23'976), so the audio wouldn't have to be converted (you save time and audio quality).All video/audio 'frames' have a timestamp on them for when to play them back. For a variable playback, a player or parser would have to completely ignore the timestamp. So, it would be possible to change the framerate of the video on the fly, but this would be more classified as a hack, and I would really have to recommend against it. (A hack as far as Matroska is concerned.) On the other, the Matroska file could be edited for certain things to play at a certain rate beforehand without to much trouble.

I certainly hope the Xiph team reads this and posts their thoughts on each aspect.

[ChristianHJW]

Originally posted by pirata This one goes for the people working at matroska, MCF and OGM.
I apologize in advance if i cant answer in detail, this weekend is quite busy for me, but i'll do my best.
1-make playback more confortable:
-Less CPU overhead: OGM takes more CPU for playback than avi, at least on my PC (I have to start all players in a higher priority level for them to play fluently since I use OGM). All this, when the CPU is basically doing the same things (decode video and audio). I am no coder, but I guess this could easily (and should) be avoided.
Please bare in mind there are zillions of M$ programmers working on an AVI parser and its optimization :D ... i never heard of OGM parsing resulting in a high CPU load, but if i read correctly in another thread here you put quite a high number of streams into it. Please be aware that every additional stream will increase the necessary CPU power to parse a file, and also cause more read/write accesses for the HDD. While for AVI there are rarely more than 2, max 3 streams, you can put a much higher number into any OGM, and the same will be valid for matroska.
So, please, try not to be like Micro$oft coders, who do not make the best software, but the most profitable software (that is, the one that will cause more upgrades)
Dont worry ;) ... maybe one day an excellent coder like trbarry can optimize the code of OGMs and matroska parser filters ...
-Movies split across several discs behave like one: I don't know whether this should be a container property or a player property. Confortable playback in such scenarios would mean that the player should be multiCD-multiDrive-aware, meaning this that the movie chunks on each CD on each drive should be detected as belonging to the same film.
-For the sake of seamless playback, when the player should detect that the current chunk is about to end, it should spin up the drive where the next chunk lies in advance,
It affects both, but the main work is on the player side. Matroska files will contain a unique ID number, being calculated from the MD5 of the block headers. This way every file can be easily identified ( will be very useful for p2p also :D ). In addition to that matroska and MCF will support multi-segment files, means that every segment of a file will point to the file ID of the segment before and after it, so a player can easily find out if the next segment is already in one of the drives, or on the HDD ( only same folder possible i guess ) and start it automatically if it finds it.
-when seeking, video must not run in order to catch up with the audio. This an issue in AVI, but not in OGM. It has to be continued so.
OGM, MCF and matroska are based on timestamps attached to the blocks of data, so all of them should be able to keep perfect sync even for more than one tracks/streams.
2-Make a straight-forward conversion procedure for DVD backups (you should consider that most of the people here are making DVD backups, so they are your major "costumers"):
Not our job. GKnot, VdubMod and a couple of other programs do a brilliant job here IMHO ... if the authors of these programs decide to support another container than AVI than we came a big step forward to finally overcome the limitations of good old AVI .. one step closer to the one goal all container developers have.
-Multi compression format support: yerterday I found a mp3 comment track for Mulholland Drive. I don't happen to own that DVD, but if I did, I'd like to add that track to my backup. It would be useful to have a container providing properties similar to VOBs (AC3, MP2 audio, MPEG2 video, subs, chapters...) plus support for OGG, (VBR) MP3, MPC, WAV, DTS, MPEG4 video...). I think this is already attained. Matroska supports all that, doesn't it?
Both MCF and matroska can support every codec in principal, at least every digital codec ( and only those, as Emmett was pointing out to us ). If i say in principal, a few things are required for it to work for every audio/video compression or subtitle format :
- there has to be a defined way how the data have to be put into the container. For every frame based codec this is already the case, for other well known types the teams are either in the process of definition already or the codec developers are encouraged to contact their preferred container development team ( or all of them ) and discuss with them the best way of how to embed the codec data into the tracks.
- the encoder and decoder for the compression format have to be available as either a VfW/Dshow/UCI plugin or to be hardcoded into the encoders and parsers.
- the formats have to have a type identifier, either a FourCC/GUID/UUID or a matroska/MCF ID, so that its possible to identify the type from reading the track header. Those who dont have such already could choose one for themselves, but its much better to contact the matroska and MCF Teams and inform them about that, so the codec ID can be added to the list of codec IDs. Emmett considered to maintain these lists outside of a project like matroska and MCF because of the possible implications in case of a project split like the one we had recently, and i agreed to talk to Steve and Lasse about this.
It will certainly take some time until the most well known formats will be supported, but we will work hard on that.
-direct support for DVD menus, subtitles and chapters: there should be some means to transfer the DVD contents to the container without too much trouble. Back to the MicroDVD days and its menu support, it was painful having to create the backup menu, when all the needed data was already defined in the DVD. This also applies to subs and chapters (maybe also extras?).
I cant speak for OGM or MCF here, but the matroska Team is urgently seeking for input from people who know more about menues than we do :D !! With respect to chapters, they will be nicely supported in all new containers. Subtitles are a different subject, please read the USF thread here about that.
3-Other:
-Space overhead: well, I don't mind more overhead if results are worth it.
matroska will probably have the highest overhead, but all 3 should be better than AVI in this respect.
-Support for characters of multiple languages (for comments and subtitles). While OGM seems to have solved this, a player like Zoom Player does not manage good unfrequent characters in comments (like for instance, the language of a given audio stream).
Cant speak for OGM here, but both MCF and matroska allow UTF-8 .
-multiangle: well. Dunno, guys... make your best at this one. Just think that there are lots of multiangle DVDs out there, It would be nice.
Possible via different video streams ....
-variable playback rate for audio or video: well this the last one, and maybe you will consider it a stupid one. But look: everytime I make a backup, I try to download other sound streams from Internet for that movie. I happen to be a language students, but unfortunately, having selected french and italian as main languages was kinda unfortunate, since both languages are not provided in most of the DVDs here. Converting them takes a big deal of time on my PC, when all I had to do would be playing the video at the proper bitrate for the audio. Yeah, I know you think it is silly, but the container could be so defined that I could play the video at different rates (25, 23'976), so the audio wouldn't have to be converted (you save time and audio quality).
Timestamps make almost everything possible here, even variable framerates. But please be aware that you cant mix audio streams of different length, like audio for NTSC and for PAL. At least one of the two had to be adjusted in its pitch with a good WAV editing program before being compressed ..
OK! That's all. Please be patient if some proposal sounds ridiculous to you. Just let me know why it does.
Not at all. Thanks for the interest in our projects ...

[pirata]

I see now that many of my requests actually should be addressed to Blacksun (I think it's him who is developing a player with you).

How could I make him notice these feature requests? A new thread? E-mail?


It affects both, but the main work is on the player side. Matroska files will contain a unique ID number, being calculated from the MD5 of the block headers. This way every file can be easily identified ( will be very useful for p2p also ).


I guess in P2P you need an ID computed out of the entire file, in order to make it content-dependent. If you have different headers, but the content is exactly the same, you get 2 different IDs for the same movie.

Dont worry ... maybe one day an excellent coder like trbarry can optimize the code of OGMs and matroska parser filters ...


You talk about excellent coders... so what are you? I have always thought you are all professional coders, working for some software company, who have the knowledge and experience it takes to generate working code in a small amount of time (a portion of your precious and rare spare time). Nobody without that "powers" would have the b*lls to invest the nowadays scarce spare time to do such a difficult task. Maybe this profile I kept in mind is wrong, though. Aren't you all guys profis or quasi-profies??


Timestamps make almost everything possible here, even variable framerates. But please be aware that you cant mix audio streams of different length, like audio for NTSC and for PAL. At least one of the two had to be adjusted in its pitch with a good WAV editing program before being compressed ..

What do you mean with that? Do the streams have to be of EXACTLY the same length? That's weird. What about if you a comment track that lasts, say 10 minutes less (credits at the end not commented).

What I'd like to have is one stream of video, and the audio for PAL and NTSC. Then I set the frame rate in the player for the video at 25 or 23.976 or 30 or whatever. It is easy and you could save encoding time and quality (reencoding ain't no good). I guess, it is all work for the people coding the player. What's the best way to contact Blacksun for feature requests?

[-h]

I can't imagine that a byte-aligned container would take anything more than 1% of CPU time to parse, even if CRCs are being calculated. You're just reading bytes and memcpy'ing payloads.

-h

[Atamido]

Originally posted by pirata
What do you mean with that? Do the streams have to be of EXACTLY the same length? That's weird. What about if you a comment track that lasts, say 10 minutes less (credits at the end not commented).

Not at all. It means that a PAL video stream has been sped up slightly for use on 25fps PAL DVD's. So, if you want to put the PAL audio stream on a video stream that has been converted back to the original 24fps, then you will need to slow it down first using an audio editor before placing it alongside the video, If you don't, then every second your audio stream will get 1/25th of a second ahead. So, in 25 seconds, the audio would be one second ahead.

The audio and video streams can both be started and stopped at anytime.

[pirata]

Ok guys that of the frame rate varying from PAL to NTSC I learnt it back in my digital video kindergarden days (a few months ago! :-) ).

I know that the audio has to be time warped, etc...

My point was, that it could be cool not to have to time-warp the audio. I have the PAL streams from my DVD, got the portugese audio track from somewhere else and say (Brazil, 23.976fps), I happen to be learning portugese.

I don't want to touch the audio. Just mux it. On playback time, I set the player to display the frames at any given frame rate, in order to have it catch up with the audio. Maybe some (positive or negative) delaying would be needed, but that does not affect the audio. I would also save lots of time reencoding the audio in such a way that all can fit in 3CDs (that needs some attempts to get it right-- more time spent!). Only caveat: time stamps should be recomputed for each frame in real time.

How could I explain this to Blaksun to hear his thinking on it?



I also want to use this thread and this very post to congratulate all the people working at Matroska, VirtualDubMod, and TCMP (in any order). It is the first time that all aspects of everyday digital video are coordinated. With such a coordinated scene forums like this won't be as populated with problem-threads as they are.

By the way, and as always... WHEN will be all this wonder released to the public?!?!?! I begin to be unsure if I ever will taste this honey... and now OGM seems to be deceased, too...

[ChristianHJW]

Originally posted by pirata
My point was, that it could be cool not to have to time-warp the audio. I have the PAL streams from my DVD, got the portugese audio track from somewhere else and say (Brazil, 23.976fps), I happen to be learning portugese.
I don't want to touch the audio. Just mux it. On playback time, I set the player to display the frames at any given frame rate, in order to have it catch up with the audio. Only caveat: time stamps should be recomputed for each frame in real time.

Huh ! Recomputing time stamps in real time !! As this is affecting the basic playback setup it wouldnt be the job of the player this time, but of the DShow parser. Dont know if its possible at ll, but you could drop a mail to matroska-devel at freelists dot org to hear Steves opnion about it as soon as he's back from Reunion.

I also want to use this thread and this very post to congratulate all the people working at Matroska, VirtualDubMod, and TCMP (in any order). It is the first time that all aspects of everyday digital video are coordinated. With such a coordinated scene forums like this won't be as populated with problem-threads as they are.
By the way, and as always... WHEN will be all this wonder released to the public?!?!?!

You forgot to mention UCI and USF ;) ... yes, we have a bigger picture. Not sure if we ever will come to a point where all functionalities will work as expected, but i promise we will have 'basic' matroska files ( XviD + MP3, XviD + Vorbis ) comparably soon now.

I begin to be unsure if I ever will taste this honey... and now OGM seems to be deceased, too...

Unforunately it seems OGM development is done away from public, at least i havent seen Tobias show up on an of the public XipH IRC channels talking to Monty about how to progess now, so o cant really tell you anything about its future ....