Aspect Ratio flag in MPEG4-Video?

[Chibi Jasmin]

Does MPEG4-Video support setting the display aspect ratio in the stream? If yes, is there any tool to set it?

Or do I have to live with using square pixels?

[ProfDrMorph]

I'm not sure but I think the problem is that AVI ( the container most people use for MPEG4 video ) only supports square pixels. The upcoming MCF format will support an aspect ratio flag as far as I know.

[DeXT]

It's true that AVI only does support square pixels. But it's true also that the MPEG-4 video stream has an aspect ratio flag. The problem is, no current MPEG-4 CODEC makes use of this flag (it's always set to 1) and no player currently available support it.

I made my own tests, both with a DivX file and a MP4 one. The first one, when I changed it to an alternate aspect ratio value (using an hex editor and some MPEG-4 specs), the DivX decoder totally messed up the picture, so I guess it doesn't like this very much.

With the MP4 file, once changed, the MPEG4IP's wmp4player simply ignored this field when playing.

I put a request in DivX forums asking support for this in future. But I doubt anyone in the DivX team may have any interest in this.

[Chibi Jasmin]

Thanx for the information...at least, it's there... :-)

[Chibi Jasmin]

Originally posted by DeXT
The problem is, no current MPEG-4 CODEC makes use of this flag (it's always set to 1) and no player currently available support it.



Well, mp4encode from the mpeg4ip utils supports setting an aspect ratio for encoding from raw (uncompressed) avi to MPEG-4...strange that wmp4player doesn't support it then...so there is one mpeg-4 codec using it...but I guess this one is out of the question due to quality...and if the player really doesn't play it anyway....

[DeXT]

Well I tried with a somewhat old release I think (0.9.2 or 0.9.3). Perhaps the new ones may support this. I'll try to do a test when I get some free time. But anyways having this option supported in the encoder and the player is a different thing, because the first one is much easier (it's just a flag set) and the latter requires image resizing.

[gabest]

Any video decoder (dshow) can set the aspect ratio on the output media type and the video renderer will make use of it automatically. Unfortunately, none of the current decoder filters are using this feature.

(see dwPictAspectRatioX/Y at http://msdn.microsoft.com/library/en-us/dshow/htm/videoinfoheader2structure.asp)

[Chibi Jasmin]

Fine...but my goal was to have the aspect ratio in the mp4-stream...

[gabest]

Hehe, ok. But once it is there, it will be easy to handle.

[Chibi Jasmin]

Well, a tool to set the flag and an MP4-DSF and we're set.

[ChristianHJW]

Why use MP4 instead of MCF ( if ever ) ? Because of hardware support !
But : Will MP4 hardware players support aspect ratio flags ??

[Chibi Jasmin]

Who knows... :)

[Gawen]

For editing mp4 files try MP4Edit from ENST in france. Very nice Interface.

http://www-elec.enst.fr/~dufourd/mpeg-4/tools.html

There is also an alternative muxing prg., mp4tool, but it uses IM1 compatible scripts to compile and those are not so easy to create.

An alternative would be Apples dumpster.exe, wich is for .mov qt files, but also accepts .mp4 as .mov with some unknown tags. You dont even have to rename the files. Is shows the high similarity between .mov and .mp4. Its almost the same.

Afaik mp4 uses the same sizes as VCD, the cif size family, (for instance 352*288 PAL + the well known DVD format wich is called CCIF if i remember correctly.) Dont forget, its an MPEG organisation product, they are conservative with formats. The Apple suggested formats for streaming are 320*240 and 640*480, but not limited to.

[Chibi Jasmin]

Thanx for your hints...I never heard of these tools :-) Will take a look...but even if I should be able to set the flag, it won't get played with aspect ratio other than 1 at the moment, will it?

Apart from that, what bugged me a bit is this:

"AAC (mono 0x40 / stereo 0x41: OK this is not standard, this is just the way we use it in IM1 currently) or MPEG-4 video (IM1 format, 0xC4: again, not standard, just the way in IM1)"

taken from the above mpeg-4 tools description... I am not familiar with most aspects of the mpeg-4 specs so I prefer a tool, which at least claims to be spec-compliant (like the mpeg4ip tools), until I understand what such things mean...

[Gawen]

@Chibi

Hi, i wouldnt have posted, if MP4EDIT.EXE did not do what you want, please give that URL a second chance. Dont mix it up with mp4tool.exe. *g*

Happy RTFMing! :cool:

P.S.: You know, i am researching in the same area *g*, but i am only 1 or 2 steps ahaed of you. MP4EDIT gives you an easy to use flag editor, please dont ask ME about wich app uses wich flag how. Even MPEGLA is still doing interoperability tests. They dont know themselfes.:D

Link to dumpster:
ftp://ftp.apple.com/developer/Quicktime/Windows_Tools/Programmers_Tools/Dumpster/Dumpster.zip

Theres a new player in town:
http://www.mpegable.com/

[Chibi Jasmin]

Originally posted by Gawen
@Chibi

Hi, i wouldnt have posted, if MP4EDIT.EXE did not do what you want, please give that URL a second chance. Dont mix it up with mp4tool.exe. *g*

Happy RTFMing! :cool:

P.S.: You know, i am researching in the same area *g*, but i am only 1 or 2 steps ahaed of you. MP4EDIT gives you an easy to use flag editor, please dont ask ME about wich app uses wich flag how. Even MPEGLA is still doing interoperability tests. They dont know themselfes.:D

Link to dumpster:
ftp://ftp.apple.com/developer/Quicktime/Windows_Tools/Programmers_Tools/Dumpster/Dumpster.zip

Theres a new player in town:
http://www.mpegable.com/


I will try to find something :-) Any hints appreciated...but as long as there is no player playing the files with modified aspect ratio, it's useless...maybe the mpegable one will do that? I will see, if I ever find out how to change the flag :-)

What I would still need is a muxer that can reliably mux 23.976 fps avi into mp4...mp4creator has its problems (as posted in http://forum.doom9.org/showthread.php?s=&threadid=23751 - in the meantime I also modified mp4creator V0.9.3.9 the same way) and I don't too much trust in my self-made solution (see that thread).

Will check out the mpegable players capabilities later...while we are at it, it would still be interesting to find out, which resolutions and aspect ratio are supposed to be standard in mpeg4-video...you posted some above...maybe simply the same as in dvd today can be used!? Many things to be researched...

[Chibi Jasmin]

Hey ! :) This player seems half-way usable! My first quick tests result in...

mpegable Player V2.0 plays...

muxed with mp4creator (all 25 fps):

XVid h263: yes
XVid mpeg: yes
DivX5: yes
DivX5_b: no (choppy, seems to simply skip the b-frames)
DivX5_gmc: yes
DivX5_qpel: no (color artifacts)

muxed with DivX5Creator
DivX5 (23.976 fps): yes
XVid mpeg (23.976 fps): no (too fast)
XVid mpeg (25 fps): yes

directly written with DivX5 (all 23.976 fps):
DivX5: yes

Note that DivX5Creator muxes 23.976 fps correctly (unlike mp4creator)!

Much better than mpeg4ip player, that only plays xvid h263 and divx5 without any mpeg-tools...I personally like to see that xvid with mpeg quantizer is supported...great stuff!

The only problem with this player is it does not create overlay windows for playback so my G400-DVDMax TVOutput doesn't work :(

You said:

"Afaik mp4 uses the same sizes as VCD, the cif size family, (for instance 352*288 PAL + the well known DVD format wich is called CCIF if i remember correctly.) Dont forget, its an MPEG organisation product, they are conservative with formats. The Apple suggested formats for streaming are 320*240 and 640*480, but not limited to. "

Can you provide some more detailed info or some links about this? Is there some standards? Can't be too wrong simply keeping the original dvd-resolution (would it be legal to cut black borders?)...would just have to set the aspect ratio (good hope the mpegable player supports it...???)...and as I said a muxer for 23.976 fps avis....

BTW: I didn't find out how to set aspect ratio with mp4edit or some other way, can you tell me how to do it, please?

[Gawen]

@Chibi

And i alway thought googeling belonged to the base lessons for a researcher...:D

Just try a search for "mp4 aspect ratio" at google.com.

As stated in the m4if mail list archive there is a flag (atom) aspect_ratio_info in mp4, but it is the player programmers choice to use it or not. Default ratios are 12:11 and 40:33.

http://lists.m4if.org/pipermail/technotes/2002-March/000573.html

Common Intermediate Format - CIF
A video format used in videoconferencing systems that easily supports both NTSC and PAL signals. CIF is part of the ITU H.261 videoconferencing standard. It specifies a data rate of 30 frames per second (fps), with each frame containing 288 lines and 352 pixels per line.

4CIF 704*576, comp. H.263

http://www.tele.ntnu.no/signal/courses/musibe/foiler/image-formats.pdf

Best supported DSP Hardware Platform i found for MP4 is TMS320C62x:

http://focus.ti.com/docs/tool/list.jhtml?FamilyIndex=0&familyId=132&toolTypeId=32&_DARGS=/main/docs/tool/listbodydroplet.jhtml.24_A&_DAV=Generation

Most encoders for those DSPs use CIF family sizes, so i would assume they are good for future player compatibility.

[Chibi Jasmin]

I'm sorry for not searching myself and am thankful for the info you provided...it's just been late at night, forgive me being lazy...

[Gawen]

The .mov atoms are a lot better described than .mp4, but the "mvhd" and "mask" atom values for .mp4 should imho be the same.

Load your .mp in dumpster and hack it!

* 8+ bytes movie header atom = long unsigned offset + long ASCII text string 'mvhd'
-> 4 bytes version/flags = byte hex version + 24-bit hex flags (current = 0)
-> 4 bytes created mac date = long unsigned value in seconds since beginning 1904 to 2040
-> 4 bytes modified mac date = long unsigned value in seconds since beginning 1904 to 2040
-> 4 bytes time scale = long unsigned time unit (default = 600)
-> 4 bytes duration = long unsigned time length (in time units)
-> 4 bytes decimal user playback speed = long fixed point rate (normal = 1.0)
-> 2 bytes user volume = short unsigned level (mute = 0; 100% = 255; 300% max = 767)
-> 10 bytes reserved = 5 * short values set to zero
-> 4 bytes decimal window geometry matrix value A = long fixed point width scale (normal = 1.0)
-> 4 bytes decimal window geometry matrix value B = long fixed point width rotate (normal = 0.0)
-> 4 bytes decimal window geometry matrix value U = long fixed point width angle (normal = 0.0)
-> 4 bytes decimal window geometry matrix value C = long fixed point height rotate (normal = 0.0)
-> 4 bytes decimal window geometry matrix value D = long fixed point height scale (normal = 1.0)
-> 4 bytes decimal window geometry matrix value V = long fixed point height angle (normal = 0.0)
-> 4 bytes decimal window geometry matrix value X = long fixed point positon (left = 0.0)
-> 4 bytes decimal window geometry matrix value Y = long fixed point positon (top = 0.0)
-> 4 bytes decimal window geometry matrix value W = long fixed point divider scale (normal = 1.0)
-> 8 bytes preview = long unsigned start time + long unsigned time length (in time units)
-> 4 bytes still poster = long unsigned frame time (in time units)
-> 8 bytes selection time = long unsigned start time + long unsigned time length (in time units)
-> 4 bytes current time = long unsigned frame time (in time units)
-> 4 bytes next/new track id = long integer value (single track = 2)

* 8+ bytes optional clipping (mask) atom = long unsigned offset + long ASCII text string 'clip'

http://www.geocities.com/xhelmboyx/quicktime/formats/qtm-layout.txt

[BlackSun]

I have already asked Ingo about this, and MCF will definitely support the Aspect Ratio flags via a custom interface which will provide a DAR for size, and pixels...

[Chibi Jasmin]

Originally posted by Gawen
The .mov atoms are a lot better described than .mp4, but the "mvhd" and "mask" atom values for .mp4 should imho be the same.

Load your .mp in dumpster and hack it!

* 8+ bytes movie header atom = long unsigned offset + long ASCII text string 'mvhd'
-> 4 bytes version/flags = byte hex version + 24-bit hex flags (current = 0)
-> 4 bytes created mac date = long unsigned value in seconds since beginning 1904 to 2040
-> 4 bytes modified mac date = long unsigned value in seconds since beginning 1904 to 2040
-> 4 bytes time scale = long unsigned time unit (default = 600)
-> 4 bytes duration = long unsigned time length (in time units)
-> 4 bytes decimal user playback speed = long fixed point rate (normal = 1.0)
-> 2 bytes user volume = short unsigned level (mute = 0; 100% = 255; 300% max = 767)
-> 10 bytes reserved = 5 * short values set to zero
-> 4 bytes decimal window geometry matrix value A = long fixed point width scale (normal = 1.0)
-> 4 bytes decimal window geometry matrix value B = long fixed point width rotate (normal = 0.0)
-> 4 bytes decimal window geometry matrix value U = long fixed point width angle (normal = 0.0)
-> 4 bytes decimal window geometry matrix value C = long fixed point height rotate (normal = 0.0)
-> 4 bytes decimal window geometry matrix value D = long fixed point height scale (normal = 1.0)
-> 4 bytes decimal window geometry matrix value V = long fixed point height angle (normal = 0.0)
-> 4 bytes decimal window geometry matrix value X = long fixed point positon (left = 0.0)
-> 4 bytes decimal window geometry matrix value Y = long fixed point positon (top = 0.0)
-> 4 bytes decimal window geometry matrix value W = long fixed point divider scale (normal = 1.0)
-> 8 bytes preview = long unsigned start time + long unsigned time length (in time units)
-> 4 bytes still poster = long unsigned frame time (in time units)
-> 8 bytes selection time = long unsigned start time + long unsigned time length (in time units)
-> 4 bytes current time = long unsigned frame time (in time units)
-> 4 bytes next/new track id = long integer value (single track = 2)

* 8+ bytes optional clipping (mask) atom = long unsigned offset + long ASCII text string 'clip'

http://www.geocities.com/xhelmboyx/quicktime/formats/qtm-layout.txt

Sounds interesting... :D Not that I understand everything...still thinking of how to modify the matrix parameters to achieve a certain pixel aspect ratio...and how to modify duration to achieve a certain frame rate...are you sure this all is spec-compliant after modifying freely?

BTW: With dumpster it looks easy...but I didn't find out what MP4Edit could be useful for...

[Gawen]

You asked about different aspect ratio values than 1.0, those 4 bytes seem to be your answer, i didnt play around with them yet and i also never intended to, this is your game. I am just pointig you to a direction ;):

-> 4 bytes decimal window geometry matrix value W = long fixed point divider scale (normal = 1.0)

- values use big endian (network) byte order
- general terms: integer = signed value
- general values: byte = 8-bit value; short/word = 16-bit value; long = 32-bit value
- fixed point values: value made up of an integer for whole numbers and an unsigned value for the decimal
- octal values: base-8 long unsigned values (values containing 8 and 9 are invalid)
- atom offsets: values relative to atoms only and are used to skip to the next atom

[Chibi Jasmin]

Originally posted by Gawen
You asked about different aspect ratio values than 1.0, those 4 bytes seem to be your answer, i didnt play around with them yet and i also never intended to, this is your game. I am just pointig you to a direction ;):

-> 4 bytes decimal window geometry matrix value W = long fixed point divider scale (normal = 1.0)

- values use big endian (network) byte order
- general terms: integer = signed value
- general values: byte = 8-bit value; short/word = 16-bit value; long = 32-bit value
- fixed point values: value made up of an integer for whole numbers and an unsigned value for the decimal
- octal values: base-8 long unsigned values (values containing 8 and 9 are invalid)
- atom offsets: values relative to atoms only and are used to skip to the next atom

I am happy I just found out, how fps seems to be calculated...duration/timescale = length in seconds...should be easy to adjust, still have to try...
Can you tell me what big endian (network) byte order is? I will try to find out the rest myself...thanx for your help! Why do you think I have to play with that particular matrix value and not with one of the others that are at 1.0 by default? Can't we set for example for 6:9 width scale 16 and height scale = 9 and leave geometry? Well, I will play and see...

[Gawen]

@chibi

Anything is allowed:

"+> > +> > The bitstream syntax allows to either use one of the pre-defined
+> > pixel aspect ratios (e.g. 12:11 or 40:33) or to explicitly specify
+> > a private one by setting 'aspect_ratio_info:=extended_PAR'.
+> > This allows to transmit any aspect ratio that can be represented
+> > as a fraction p:q with p,q in [1;255]."

But nobody knows what will happen:

"Decoders must decode weird bitstreams that conform, but what to do for display is entirely up to the decision of the decoding system."

"Can't we set for example for 6:9 width scale 16 and height scale = 9 and leave geometry?"

There seem to be flags for almost every value, but how do you know the viewing partys screen resolution and dimensions when creating? If you produce for fixed 1024*768 displays, anything is possible ... But if someone uses one of those cute new 16:9 22inch TFT displays?

MP4edit is good for understanding the basic structure of .mp4, dumpster is great for hacking every byte.


Big Endian Byte Order:
http://www.webopedia.com/TERM/b/big_endian.html

(Also a google position 1 hit)

Maybe you should begin with basic structures before hackin big endian bytes :D ?

[Chibi Jasmin]

Well, entries for matrix and duration and stuff are in tkhd, mvhd and mdhd and so on....hacking around didn't lead to any change in playback speed, also I got another fps displayed with mp4v1getfps..and about aspect ratio...I didn't even start...maybe it makes sense to study structures first :) But maybe it makes more sense to simply wait for some fully functional encoder and player implementations so you know better what you are playing with...if now a modification doesn't work I don't know, is it the player? Is my mod wrong? I won't go too deep into it now...

[Gawen]

This is my opinion too, i think we should first evaluate what ist there. Wich software combination makes sense from a usability point of view? And than we should begin tweakin it. Nobody knows yet wich player supports wich flags. But there are tools to find out if we needed to.

Also many options are redundantly stored in mp4v headers and mp4 system atoms. Wich are interpreted? Its a young technology, we are actually screening its birth i think. Lots of things will change when it grows up.

[Chibi Jasmin]

I am glad, you agree with me... :)

[Gawen]

Of cause. You asked a specific question, i tried to give you an exact answer.
You didnt ask if it makes sense to try...:cool:

I think, if we use standard sizes like CIF and 4CIF we are on the secure side. Players will know from internal logic how to display them best. Or on what would you put the focus as a programmer of a new player? Standard sizes or flexibility? I would try implementing standards first and put flexibility in afterwards in order to have good results from the start.

[Chibi Jasmin]

Originally posted by Gawen
Of cause. You asked a specific question, i tried to give you an exact answer.
You didnt ask if it makes sense to try...:cool:

I think, if we use standard sizes like CIF and 4CIF we are on the secure side. Players will know from internal logic how to display them best. Or on what would you put the focus as a programmer of a new player? Standard sizes or flexibility? I would try implementing standards first and put flexibility in afterwards in order to have good results from the start.


:)

Well, the only thing I think we can be sure of is, they will be able to display square pixels...and that's not meant as a joke...if they will support aspect ratio flags and which ones, or if they will 'know' what aspect ratio to change a certain size to is yet to be seen..

[Gawen]

@chibi

"if they will 'know' what aspect ratio to change a certain size to is yet to be seen..."

As stated before, it can be seen today in already existing specs, have a look at the Sigma Design hardware decoder specs pdf for instance:
http://www.sigmadesigns.com/pdf_docs/em847Xpb.pdf

The Sigma 8400 is often used in todays high end DVD players (line doubler). Take the new Sigma 8470 (wich is 8400 pin compatible), put it on a National Semiconductor Geode SC1200 Reference Board, attach a dvd drive, patch the National linux software kit to MPEG4 and you have your tomorrow DVD Player today.

http://www.national.com/appinfo/solutions/0,2062,796,00.html

Sigma references to the ISMA format specs and claims to be envivio player compatible so you should be on the secure side with CIF (352*288 ISMA streaming), CCIR601 (720*576 ISMA broadcast) format and testing with the envivio player.

http://www.envivio.com/solutions/elb/

Data-Book Sigma EM8400
http://www.ecn.purdue.edu/~laird/Linux/iDVD3036/cdrom/Document/MPEG2-Em8400/EM840x10a.PDF

[Chibi Jasmin]

You are simply more into this than I am :) Thanx for teaching me...

A pity, I never got anything to work on envivio :(

[Chibi Jasmin]

Originally posted by DeXT
[B]I made my own tests, both with a DivX file and a MP4 one. The first one, when I changed it to an alternate aspect ratio value (using an hex editor and some MPEG-4 specs), the DivX decoder totally messed up the picture, so I guess it doesn't like this very much.


Would you mind explaining the process of changing the aspect ratio with the hex editor a bit? You changed it in the AVI before creating an MP4-File?

[DeXT]

Well since the aspect ratio is inside the video stream itself (actually the VOL header) I changed it after it was multiplexed into AVI and MP4 files. I used some MPEG-4 sources as a reference. From MPEG4IP sources, VOL header structure:

vol.GetBits(32); // start code

vol.GetBits(1); // random accessible vol
vol.GetBits(8); // object type id
u_int8_t verid = 0;
if (vol.GetBits(1)) { // is object layer id
verid = vol.GetBits(4); // object layer verid
vol.GetBits(3); // object layer priority
}
if (vol.GetBits(4) == 0xF) { // aspect ratio info
vol.GetBits(8); // par width
vol.GetBits(8); // par height
}
...


Just search the VOL start code (= 0x00000120) inside the header, then the aspect ratio info which is some bits ahead (follow the same procedure as described in the code).

The aspect ratio value takes 4 bits, 0x1 means square pixels and it's the default value for most encoders. 0xF means extended PAR, this uses two additional 8 bit values which are in the x:y form. I have not exact information about the aspect ratio values but I guess it uses a table very similar to that of MPEG-1, so there are 14 fixed aspect ratio values (apart from the optional extended PAR).

So to make tests just change this 4 bit value to a different one (2, 3, 5... nof F though, since this would need 2 additional bytes).

You also have a good reference in mp4live/video_util_mpeg4.cpp, divx/mpeg4_header.c and others.

[Chibi Jasmin]

Interesting...thanx for describing... :)

[Gawen]

@chibi

Yesterday i read an article about a trade show where Apple, Sun and Sigma demonstrated interoperability. QT delivered the MP4 files, Sun streamed them and Sigma displayed with a reference STB unit. So this should also be Apple QT6 compatible.

Installing Envivio on a non US OS platform with QT is tricky. I managed to work this way:

- Deinstalled RealOne and Envivio.
- Installed the engl. language Version of QT5.2 into "C:\program files\quicktime" even if it wanted to install into "programme".
- Than installed Envivio TV and, o what a wonder, it found QT and installed with it.
- Stuff encoded with divxenc/faac/mp4creator works fine with Envivio/QT combination.

Seems Envivios installer is green banana software. We should visit their forum and flame them ;)

[Chibi Jasmin]

The new envivio version actually detects my quicktime :)