Packed Bistream handling in mp4creator ?

[ChristianHJW]

Hi all,

i made a search on *mp4* , *packed* , *bitstream* and was reading through a few dozens threads on XviD sector, but without any sensible result, so i am turning to the MPEG4 experts here ( Hans-Jürgen, temporance, rjamorim ? ), as i do have a couple of questions about MP4 container :

1. I assume that both tricks to be able to put b-frames into AVI with a VfW codec, that is using packed bitstream, dummy frames or both ( causing lag ) are NOT MP4 spec compliant ? Anybody to confirm that ?

2. If mp4creator will open a XVID AVI with a packed bitstream, containing b-frames, how will it process that to make a spec compliant MP4 file from it ( assuming my assumption above is correct ) ?

3. Do current versions of mp4creator ( or any other mpeg4ip tools, i am not 100% familiar with them ) handle aspect ratio flags in MPEG4 codec header already ?

Thanks for clarifying

Christian

[ChristianHJW]

I found this link on the mpeg4ip homepage :

http://mpeg4ip.sourceforge.net/docs/

They worry about the MP3 audio coming with the DivX N MP4 tools, and also claim their muxer from the DivX Pro Codec doesnt produce totally MP4 compliant streams, but there is no mention at all about the handling of b-frames in mp4creator ?

How can i get in contact with these people to find out ? It seems they dont have a forums or mailing list ?

[MemTronic]

try:

http://mpeg4ip.sourceforge.net/developers/index.php


so long...

MemTronic

[ChristianHJW]

FYI, i dropped an email to Bill May, the main developer of the MPEG4IP project, asking about this and a few other things. I read on the sf.net News that he is away until 11th June, hopefully he will reply to me after that ...

[temporance]

1. I assume that both tricks to be able to put b-frames into AVI with a VfW codec, that is using packed bitstream, dummy frames or both ( causing lag ) are NOT MP4 spec compliant ? Anybody to confirm that ?Yes and no. An AVI file is not "MP4 spec" compliant. Nor will it ever be. AVI and mp4 are two different file formats. However, the tricks do not break compliance with the MPEG-4 video spec. Pristine compliant MPEG-4 video may be extracted from the AVI file, with correct timing information. When writing a mp4 file, the same tricks are not needed (mp4 has native support for B frames etc.) and indeed, packed bitstream would be illegal in mp4 samples. Dummy frames are more of a grey area, they are probably legal but I wouldn't include them in files I wanted playable everywhere.

2. If mp4creator will open a XVID AVI with a packed bitstream, containing b-frames, how will it process that to make a spec compliant MP4 file from it ( assuming my assumption above is correct ) ?
It would need to do several things: generate (one or more) decoderSpecificInfo blocks, repacketise the video stream, sort out timing information. I don't know whether mp4creator actually performs all the necessary functions to generate compliant mp4 files.

3. Do current versions of mp4creator ( or any other mpeg4ip tools, i am not 100% familiar with them ) handle aspect ratio flags in MPEG4 codec header already ?PAR is included with DSI (decoderSpecificInfo = VOSH/VO/VOL/etc.) which is private data as far as the file format is concerned. Support of these flags is more a matter for the player/decoder.

[sillKotscha]

Originally posted by temporance
AVI and mp4 are two different file formats.

Chris... did you knew that LOL :D

[ChristianHJW]

FYI : Message from Bill May, MPEG4UP Team. As a result of what he said, i'd make a bet the MP4 files made from AVIs with b-fra,es may be b0rked .. :(

------ Original Message --------
Subject: Re: Packed bitstream/dx50 VOP handling for AVIs with B-frames
in MP4creator ?
Date: Fri, 06 Jun 2003 09:39:47 -0700
From: Bill May <wmay at cisco dot com>
To: chris at matroska dot org
CC: David Mackie <cahighlander at yahoo dot com>, dave at leatherdale dot org, ed dot gomes at free dot fr, syskin at ihug dot au


Christian HJ Wiesner wrote:

>>
>>
>> Hi Bill,
>>
>> am sending this email to you, but copied to the other developers of the
>> MPEG4IP project also, the matroska development mailing list and also
>> some people helping us with MP4 support .


Okay - great.


>> Please allow me to introduce myself, i am one of the project admins of
>> the matroska opensource container project on http://www.matroska.org.
>> Our goal is too create a general use audio/video container format,
>> focussed on storing/editing and support for all existing codecs. More
>> info is on our homepage, as well as the specs and the tools to
>> play/create/edit matroska files.


Okay - that sounds fine to me. I personally like the QT library that
mp4 is based on, but if you've got a different idea, that's fine with me.


>> We have plans to make a transmuxing tool to be able to mux the content
>> of MP4 files into matroska files ( .MKV ), so they can easily be edited
>> in VirtualdubMod ( http://sf.net/projects/virtualdubmod ). This is
>> possible because matroska was designed for high editability, and it
>> doesnt even require the help of the codec for this. There are loud
>> voices from users asking us to do that, so they can transmux their MP4
>> files to MKV, edit them in VirtualdubMod and then remux the files to MP4
>> finally, in a lossless process. matroska, in this case, is only used as
>> an interims storage format for easy editing of MP4 files. We dont have a
>> problem with that, we understand MP4 has a huge advantage as there will
>> be hardware support for MP4 soon, while this is unlikely to ever happen
>> for MKV.


Okay, still with you.


>>
>> Based on what is said above, i'd like to ask you the following questions :
>>
>> 1. To remux the MKV files into the MP4 format, after editing them in
>> VirtualdubMod or any other editing tool ( we will maybe have a Vegas
>> Video plugin soon ), your tool MP4creator would need the ability to
>> parse them. Could you think of adding libmatroska ( GPL/QPL ) to
>> MP4creator, and allow transmuxing of MKV files also ? We will gladly
>> help you on how to read them, the library is quite easy to use and has a
>> lot of intelligence built in menahwile to assist developer who want to
>> use it.


No. I can't do the work, but I'd be glad to have it in the package. It's
a time thing. I've got a pregnant wife and a full plate of work things.

I would worry a bit about the licensing - we've tried to stick mostly
with LGPL or Mozilla for our libraries that can't be seperated out
easily.

The best thing might be to have a seperate executable that use the
mpeg4ip code, as a seperate project, if you want to stick with GPL.


>> 2. When parsing AVI files for muxing into MP4, how do you handle
>> B-frames ? As AVI doesnt normally have support for b-frames, DivX
>> Networks introduced a hack with their so-called dx50 VOP mode, inserting
>> dummy frames into the AVI as placeholders and packing the b-frame itself
>> into the same chunk together with the I/P frame it is referencing to.
>> XviD does that also, but if offering a so-called 'packed bitstream' mode
>> also as alternative. IN both cases an AVI parser, reading the MPEG4
>> frames , would have to get rid of the dummy frames and reorder the
>> b-frames such that coding order is achieved in the final MP4 file. How
>> do you handle that from mp4creator, if at all ? How do you overcome the
>> lagging problem ? Would MP4 files, when created with mp4creator from AVI
>> files with B-frames, be fully MP4 spec compliant if no frame reordering
>> is done ?


I'm really not sure. I believe that we expect all the frames in decoding
order, and reset the timestamps accordingly. I believe that the b frames
would need to be in seperate frame chunks.

I'm not sure we even have avi support for b frames. It might just be
for elementary stream raw files, and then, like I said, we expect it
to be in decoding order.

I've copied Dave Mackie on this - he worked on this more than I did, and
would be able to give an answer.

Bill

[mf]

Judging on the reset the timestamps thingie, I think standard XviD dummy frame coding will be properly handled by mp4creator. Packed Bitstream (DivX5 approach) might not be.
Oh and btw, here is a summary of what the terms all mean, because you seem to confuse some of them.
DX50 Compatibility: Closed GOP. This is only logical, ME-wise and bitstream-wise. This also has nothing to do with AVI container, except that disabling it screws up decoding.
Dummy frames: Standard procedure for B-Frame handling in AVI, is MPEG4-bitstream compliant, and is handled by libavcodec as well. In normal decoding (ie via DirectShow) no lag or desynch will be introduced, but in 1-frame-in-1-frame-out schemes (VfW) this causes problems.
Packed bitstream: DivX5 procedure for B-Frame handling in AVI. Basically only works properly with 1 consecutive B-frame (afaik). Is not MPEG4-bitstream compliant, but does retain proper synch (no decoder lag) on 1-frame-in-1-frame-out schemes (VfW).

[ChristianHJW]

Originally posted by mf Oh and btw, here is a summary of what the terms all mean, because you seem to confuse some of them.
Thanks mf, you are right, i never get it it seems :( .... but your listing clarified a few things for me i hope, in the end ;) ...

[temporance]

Originally posted by mf
Packed bitstream: DivX5 procedure for B-Frame handling in AVI. Basically only works properly with 1 consecutive B-frame (afaik)...No, it would work with multiple B frames. The first B frame is appended to the P frame, subsequent B frames live in their own AVI chunk, e.g. [I0] [P3B1] [p3].
... Is not MPEG4-bitstream compliant ...I don't see why a packed bitstream is not MPEG-4 compliant. It is just a different packetization of the video ES. If you are saying that packed bitstream in an AVI file (a la DivX5) is not compliant with the MPEG-4 *video* spec, then neither is a .m4v file containing just raw video ES. [A .m4v file just comprises concatenated VOPs].

[outlyer]

I'm not sure if I understood well, if I did there are different ways used to put b-frames in avi files, so, as mp4creator simply puts the frames in the avi order, and afaik the fourcc is lost, when working with the mp4 file, can the mode used be detected?

[ChristianHJW]

FYI :


-------- Original Message --------
Subject: Re: Packed bitstream/dx50 VOP handling for AVIs with B-frames in MP4creator ?
Date: Mon, 9 Jun 2003 14:10:19 -0700 (PDT)
From: David Mackie <cahighlander at yahoo dot cm
To: Bill May <wmay at cisco dot cm>, chris at matroska dot org


When I wrote the avi import code for mp4creator, Divx
wasn't doing B frames. Hence there is no avi B frame
support.

I suspect your easiest path would be to extract the
MPEG-4 video bitstream from the .avi to a .mp4v Then
you can just feed that to the existing mp4creator.

Dave

So you better not use AVIs with b-frames in mp4creator, at least for the time being. Maybe we can even unite efforts, as the matroska team has to 'unpack' DivX N's packed bitstreams also .... :( ..

[temporance]

Unpacking is incredibly easy - just step through the stream until you find the VOP startcode 00 00 01 b6. That's the first four bytes of the packed frame.