16:9 AR flagging within an Mp4 container. Is it possible?

[SeeMoreDigital]

Well, I guess the title says it all!

I've read many posts on the forum regarding the lack of 16:9 header/flagging implementation within an AVI container. But wonder if the same is true for the MP4 container!

I would like to generate some short 'video only' .mp4 test encodes at both 720x480 and 720x576 pixels, so I can check some equipment.

Can anybody help please?

[Neo Neko]

There is no such thing for the MP4 container IIRC. The AR header is stored in the video bitstream itself. So if you encode with ffmpeg, transcode, or ffvfw you can already put the AR in the video stream. Problem is this. The story goes that directshow is rather porly designed. Whodathunkit! Microsoft being poor software designers? :p Anyhow you are supposed to only resize the video frame before you read the video header. After you read the headder you can't resize the frame. But you have to read the headder to get the AR flag. This is the reason for the necesity to have them at the container level.

If you encode with ffvfw you can set the flag and your files will play back with the correct AR no matter what container it is in. The only catch is that the player can't be VFW/Directshow based! Mplayer for Windows works kinda. And it is totally slick under *NIX.

[SeeMoreDigital]

Hi Neo Neko,

Wow, there appears to be so much to learn about the MP4 container. I've read the MP4 FAQ posted by Bond and must have missed a bit!

Originally posted by Neo Neko
...If you encode with ffvfw you can set the flag and your files will play back with the correct AR no matter what container it is in. The only catch is that the player can't be VFW/Directshow based! Mplayer for Windows works kinda. And it is totally slick under *NIX. Do you think such files would work with hardware based players such as my Xcard?

Cheers

[Sulik]

Actually, if decoders were smart enough (hint for DivX and other developers), it would be possible to use the bi{X|Y}PelsPerMeter field in the VideoInfo header in order to signal the aspect ratio.
Unfortunately, this field is usually ignored by decoders, and current encoders set it to zero.

[Neo Neko]

Originally posted by SeeMoreDigital
Do you think such files would work with hardware based players such as my Xcard?


Good question. One I am not qualified to answer though. I will never buy an Xcard myself for personal reasons due to sigma designs past actions. I would check their website and tech support if I were you. But considering that the Xcard is mostly targeted at Windows users and most Windows encoders don't use the flag I have serious doubts that it would. In fact since Sigma Designs outright coppied most of Xvids code to make their MPEG4 codec in violation of Xvids license I suspect that Xvid has a strong influence on the programs burned on the e-prom of the card. And since Xvids decoder does not support the AR header I don't think Sigma will have implemented it on their own. If the company is lazy enough to try passing off ctrl-a+ctrl-c+ctrl-v as programming their own implementation I have no confidence in them implementing an almost currently useless feature. :(

[SeeMoreDigital]

Hi Neo,

Yes I have to agree with you with regard to Sigma being lazy and by ripping off XviD's code to create their own 'badged' version of an Mpeg4 codec!

The trouble is I had already bought a number of Xcards quite a few months before the sh!t hit the fan with XviD!

In fact, I had not purchased the Xcards for their Mpeg4 capability at all. They were just replacements for our Hollywood+ cards and were used primary to view high-res Mpeg1 encodes with our plasma screens!

I ask about the 16:9 flagging because I've been giving Sigma a really hard time (on their forum) about the lack of new driver support and the fact that when you set the card to 16:9 TV out and feed it an 720x480/576 Mpeg4 encode, the picture is all sqashed up on the screen!

Nagui Stephanos (of Sigma) has hinted that the card is waiting for flagging information from the Mpeg4 file. But like you, I can't imagine Sigma have been pro-active enough to implement such a feature. Although I have to admit when playing back m2v streams (ripped from a DVD) they are displayed correctly!

So I'm looking for a way to generate or download an suitable short (very short) Mpeg4 encode so I can test his suggestion!

I understand your personal reasons and will fully accept them if you are unable to help further.

Cheers

[bond]

latest 3ivx alpha version supports setting an AR during encoding (in the bitstream), so i guess their directshow filter also supports resizing during playback

the bad thing is this 3ivx version isnt publicly available, and i also dont know a tool that can change the AR on already existing mp4s...

[SeeMoreDigital]

Originally posted by bond
latest 3ivx alpha version supports setting an AR during encoding (in the bitstream), so i guess their directshow filter also supports resizing during playback

the bad thing is this 3ivx version isnt publicly available, and i also dont know a tool that can change the AR on already existing mp4s... Thanks for the info guys.

I don't know, 16:9 flagging is not possible within an AVI container. And it's also not possible with an MP4 container!

So it looks like we will all have to wait. Shame!

Can I ask you Bond, if you've been able to test 3ivx's alpha version that supports AR flagging?

Cheers

[Neo Neko]

Originally posted by bond
latest 3ivx alpha version supports setting an AR during encoding (in the bitstream), so i guess their directshow filter also supports resizing during playback


Really? Interesting. I wonder how they did it. Perhaps Shitowax could enlighten us if he is lurking! Because that would be quite a feat! Maybe he would send me a copy if I pestered him enough. :D Naaaaaaah.

Originally posted by bond
and i also dont know a tool that can change the AR on already existing mp4s...

It does not exist afaik. But it should not be that hard to code . I suspect a decent coder with a free day and a hord of mountain dew could get it done. ;)

[Neo Neko]

Originally posted by SeeMoreDigital
Thanks for the info guys.

I don't know, 16:9 flagging is not possible within an AVI container.


It is possible. It's just that no AVI parser that I know of handles it. And that unless you magically switched everyone elses parser to this new supporting parser it would cause nothing but problems and dificulty. A standard is only as good as it's standard implementation.

Originally posted by SeeMoreDigital
And it's also not possible with an MP4 container!

There is no need in MP4. All MP4 compatable video bitstreams support the AR flag in the video bitstream itself. Which is where it makes the most sense to have it anyway.

Originally posted by SeeMoreDigital
Can I ask you Bond, if you've been able to test 3ivx's alpha version that supports AR flagging?
Cheers

I am just dying to know how they did it and when can I have it. 3ivx is not at all a bad decoder. I still preffer ffdshow. But that would be handy indeed! Till now I was afraid Mplayer was the only option.

[RadicalEd]

I'm pretty sure you can resize the video stream with XMT, but that's getting into rich content which is probably beyond the scope of what most people are creating around here.
But at least it is possible to do on the container level.

[shitowax]

Hi guys,

It's true that the next 3ivx has some funky features, and AR is one of them. You won't have to wait that long to test I think ;). Btw, DirectShow is FAR from being badly designed (no, I don't bill m$ to write that), BUT it's poorly documented, especially DX9 ... Another good one, it's not because DXN put the AR info in the video bitstream in .avi that it 's the ISO way to do it in .mp4 =)


Originally posted by Neo Neko
Really? Interesting. I wonder how they did it. Perhaps Shitowax could enlighten us if he is lurking! Because that would be quite a feat! Maybe he would send me a copy if I pestered him enough. :D Naaaaaaah.

It does not exist afaik. But it should not be that hard to code . I suspect a decent coder with a free day and a hord of mountain dew could get it done. ;)

[SeeMoreDigital]

Originally posted by Neo Neko
There is no need in MP4. All MP4 compatable video bitstreams support the AR flag in the video bitstream itself. Which is where it makes the most sense to have it anyway. Hmmm.

My main intension is to view 720x480/576 Mpeg4 encodes using an Xcard (or stand-alone player) connected to an 16:9 TV/plasma. I can't imagine I'm the only person who's doing this now (or want's to do it in the future). And having a problems!

I tried using Sorenson Squeeze 3.5 (some months ago) to encode 16:9 m4v streams from an 16:9 m2v source. But this did not work either!

At present, I am able to view DVD's and raw 'Mpeg2' streams perfectly on my 16:9 TV when 16:9 TV out selected on my Xcard. But in order to view 'Mpeg4' 720x480/576 anamorphic encodes correctly (on my 16:9 TV) I have to select 4:3 TV out!!!

So ths makes me wonder, how well dedicated stand-alone player users, with 16:9 TV's, are getting on?

It's a mystery.... Cheers

[symonjfox]

I really was surprised that MPEG2 has AR flag, while the first implementations of MPEG4 video haven't.

Instead they have, just nobody implemented them before. Everytime I talked about this, I heard people sayng "In AVI we can't, use Matroska".
Yes, I could use it off course, but I like MPEG4video (DivX Xvid) in MP4 container.

AFAIK also Xvid have AR flags, but they weren't used in VFW Gui. I read somewhere that can be used just under linux, and the first tests worked ok.

I hope to see a good implementation for Win of course :D

PS: My signature is the same from about 1 year ...

[SeeMoreDigital]

Nero Digital in the form of Nero Recode2 arrives in finally arrives in December.

Lets hope that it's capable of 16:9 flagging and uses the MP4 container only!

Can somebody remind me if mp3 (and multiple mp3 audio) streams can be placed in an MP4 container?

[Neo Neko]

Yes.

[bond]

the avi specs also allow setting an AR flag, but as m$ splitter doesnt support it, this part of the specs is like non-exisiting

with the following exception, i think:
i now tested the 3ivx AR feature and resizing during playback (not during encoding) works great (you have to set the AR during encoding with 3ivx)
the only bad thing is that resizing seems only to work with the 3ivx decoder currently (well i think ffdshow can also handle resizing, but it doesnt do so automatically)
i always thought resizing is done by the splitter :confused:

though setting the AR in 3ivx is a great feature, we will definitely need a tool too which allows setting the AR on existing avis/mp4s (perhaps mp4creator can do that, looks like we will have to make some preasure on the mpeg4ip guys :D )

Originally posted by SeeMoreDigital
Wow, there appears to be so much to learn about the MP4 container. I've read the MP4 FAQ posted by Bond and must have missed a bit!
---
Can somebody remind me if mp3 (and multiple mp3 audio) streams can be placed in an MP4 container?hey, read the faq more precise ;)

[shitowax]

Originally posted by bond
the avi specs also allow setting an AR flag, but as m$ splitter doesnt support it, this part of the specs is like non-exisiting


the AVI format is totally unaware of any AR flag, using the m$ splitter or a house-made AVI splitter is the same. You have no standard way to put AR info in AVI independantly of the codec.

i always thought resizing is done by the splitter :confused:


The splitter outputs compressed frames ... how do you want it to resize anything ?!? ;)

[bond]

Originally posted by shitowax
the AVI format is totally unaware of any AR flag, using the m$ splitter or a house-made AVI splitter is the same. You have no standard way to put AR info in AVI independantly of the codec.hm, i think alexnoe told me that the avi specs support that!?

The splitter outputs compressed frames ... how do you want it to resize anything ?!? ;) :p

[SeeMoreDigital]

Originally posted by bond
hey, read the faq more precise ;) OK you got me on that one!

But I'm still does not clear about multiple audio streams. I assume it should be possible in theory. As should multiple video streams and subtitles etc. But only you guys 'in the know' so to speak, can answer these questions for sure!

I realize too, that most of the above functions may only be available to PC media player users first. So it may be time to put pressure on hardware player software manufactureres too!

Cheers

[bond]

Originally posted by SeeMoreDigital
But I'm still does not clear about multiple audio streams. I assume it should be possible in theory. As should multiple video streams and subtitles etc. But only you guys 'in the know' so to speak, can answer these questions for sure!well, you can put as much audio, video and subtitle streams into mp4 as you need (multiple mp3, mixed mp3 and aac...)
i am pretty sure that mp4ui/mp4creator can already put multiple audio streams into mp4 (never tried multiple video)

bst (bsplayer developer) already asked me for multiple video samples to implement video switching in his player

I realize too, that most of the above functions may only be available to PC media player users first. So it may be time to put pressure on hardware player software manufactureres too!yep, but before hardware manufacturers implement mp4, people will have to use it ;)
or mp4 (with h264) really becomes the next standard for hd-dvd

[SeeMoreDigital]

Thanks bond,

So tomorrow we will see the launch of 3ivx that will allow some form of AR flagging. But if understand correctly, this will only be available for PC media player viewers and not hardware player viewers like me!

AR flagging whether it be in an AVI or MP4 container, seems to be the wholly grail! I really find it difficult to understand why it has taken so long, and is continuing to take so long, to come to fruition. And why it was never implemented from the start!

As most people (on the forum) are probably aware by now, I am a Sigma / JovePlayer user. So I'm happy to announce that the makers of JovePlayer have now made it possible for users who want to view their anamorphic encodes 'automatically' on their 16:9 TV's, to do just that. All you have to do is create an .xml file first!

Yes, it may be a compromise. But it works 100%. And you can make it work 100%, with any weird sized encode!

Cheers

[symonjfox]

Originally posted by SeeMoreDigital
I really find it difficult to understand why it has taken so long, and is continuing to take so long, to come to fruition. And why it was never implemented from the start! Me too. And I also don't understand why everybody have to make boring calculations to crop, and resize to 1:1 AR.

With AR flag, we just can take an anamorphic source, crop it, (horizzontally resize if needed to gain compressibility) and set the AR to 16:9, 4:3 and so on.

And also I don't understand why MPEG4 streams come out in AVI container. MPEG4 specs are out from long time. And also, why MPEG1 and 2 streams are created and stored in MPG container ONLY, while MPEG4 has avi, mkv, ogm, mp4 ... ?

And again, why doesn't anybody created a standard for MPEG4 video CD? For example encoding to MPEG4 Simple profile (or A S P), audio AAC, MP3; Container MPEG4 with kind of Error protection, Cd burnt in MODE2 (just like (S)VCD, but using MPEG4 streams)? Instead in the real world what appened? MPEG4 streams in Avi files; standalones play DivX only material (DivX Certification ... let's create a FULL MPEG4 certification instead) and there's still NO MPEG4 Advanced Simple Profile Standalone Player on the market!

[SeeMoreDigital]

Originally posted by symonjfox
Me too. And I also don't understand why everybody have to make boring calculations to crop, and resize to 1:1 AR.

With AR flag, we just can take an anamorphic source, crop it, (horizzontally resize if needed to gain compressibility) and set the AR to 16:9, 4:3 and so on. I have to admit, as an MPEGmediator user I find it very easy to generate 1:1 encodes. Even 1:1 cropped encodes. In fact after a bit of practice, it's quite a simple task to generate encodes at any image pixel frame size. And resize the image to fit!

Originally posted by symonjfox
And also I don't understand why MPEG4 streams come out in AVI container. MPEG4 specs are out from long time. And also, why MPEG1 and 2 streams are created and stored in MPG container ONLY, while MPEG4 has avi, mkv, ogm, mp4 ... ? Me too!

I've mentioned this many times. OK sure, it was maybe a good idea to do this in order to get things going. But why are there still no simple applications that can generate MP4 container compliant encodes and why can't Mpeg4 video be stored in an MPG container?

Originally posted by symonjfox
And again, why doesn't anybody created a standard for MPEG4 video CD? For example encoding to MPEG4 Simple profile (or A S P), audio AAC, MP3; Container MPEG4 with kind of Error protection, Cd burnt in MODE2 (just like (S)VCD, but using MPEG4 streams)? Instead in the real world what appened? MPEG4 streams in Avi files; standalones play DivX only material (DivX Certification ... let's create a FULL MPEG4 certification instead) and there's still NO MPEG4 Advanced Simple Profile Standalone Player on the market! I would like to think that Mpeg4 video in an MPG container could resolve some of these questions too!

Anyway. That's my little protest finished!

Cheers

[symonjfox]

Originally posted by SeeMoreDigital
Anyway. That's my little protest finished! Mine too :)

I'm happy to see that I'm not alone in my small battle :D well, if I was a programmer, I surely won't complain about anything, I would spend my spare time to code my whishes ...

Let's wait tomorrow and see what appends with 3ivx and let's hope that ALL MPEG4 codecs will support this feature (that is in the specifics).

Cheers

[Atamido]

Originally posted by shitowax
the AVI format is totally unaware of any AR flag, using the m$ splitter or a house-made AVI splitter is the same. You have no standard way to put AR info in AVI independantly of the codec. This is incorrect. AVI does have a space to store the AR, but no splitter takes advantage of it. If someone made a new splitter, it could adjust the AR just like the Matroska splitter does.

typedef struct {
DWORD VideoFormatToken;
DWORD VideoStandard;
DWORD dwVerticalRefreshRate;
DWORD dwHTotalInT;
DWORD dwVTotalInLines;
17
Required Information
DWORD dwFrameAspectRatio;
DWORD dwFrameWidthInPixels;
DWORD dwFrameHeightInLines;
DWORD nbFieldPerFrame;
VIDEO_FIELD_DESC FieldInfo[nbFieldPerFrame];
} VideoPropHeader; Active Frame Aspect Ratio
The aspect ratio is stored as a DWORD value with a word each storing the x:y ratio. For
example, 1 to 1 is 0x00010001. Standard values for television is 4:3 or 16:9. This value can be
used with the frame width and height to calculate the pixel aspect ratio.

[alexnoe]

I really wonder why people want to kill a format only because M$ has written shitty filters for it. It has most things you need, including aspect ration :)

[Atamido]

I want people to move on to Matroska so that we can have proper VFR support in both audio and video. (By VFR audio I mean varying the number of data blocks per second like Vorbis does.)

I also want subtitles to be stored as a stream. You can see from the Matroska Stream Editor how this kind of idea is beneficial. Anything that is time synched can be stored as a stream.

And having synch issues is almost impossible. Using the MSE, I can create gaps in the video or audio and the file plays back fine with the gap there.

I also like Matroska's ability to encrypt and/or compress streams independant of codec type.

There are a lot of very good features that come along with Matroska, and if everyone began to focus on building support for it, I think we'd all be better off.

[Neo Neko]

Originally posted by SeeMoreDigital
AR flagging whether it be in an AVI or MP4 container, seems to be the wholly grail! I really find it difficult to understand why it has taken so long, and is continuing to take so long, to come to fruition. And why it was never implemented from the start!

It's quite simple really. All you have to do is think back on your history. Microsoft released the first MPEG4 based codec avalible to the general public. Being Microsoft they were never one to fully implement anything. Half assed or die. They saw no reason to implement AR flagging on a PC only codec. Like I said no one ever said the guys at MS were bright. They locked their codec to their proprietary container and viola gej and friends hacked it to unlock it again. Divx 3 was born. No AR flags. No advanced profile. Like I said very incomplete but it was the first. And it set the trend for a long long time. Including using AVI as it's container.

[SeeMoreDigital]

Originally posted by Neo Neko
...Like I said no one ever said the guys at MS were bright. They locked their codec to their proprietary container and viola gej and friends hacked it to unlock it again. Divx 3 was born. No AR flags. No advanced profile. Like I said very incomplete but it was the first. And it set the trend for a long long time. Including using AVI as it's container. So is there any reason why the AVI container can't be 'hacked' again!

I was kinda hoping something like this might have already happened, now that proper 16:9 anamorphic widescreen DV.avi camcorders are available.

But all this still does not explain why DivX decided to abandon the MP4 container altogether (after DivX 5.0.2), given it's obvious flexibility. Surely this action has only confused matters further.

I think it's fair to say that if DivX had stuck with the MP4 container we could all be enjoying 16:9 flagging right now!

I did point this fact out to Gej some months ago but never received a reply.... Strange. Very strange!

Cheers

[RadicalEd]

Well DXN is after all moving in the direction of making DivX totally independent of Mpeg 4 (a stupid move IMO). They never added level compatibility (instead came up with DivX profiles :rolleyes: ) and I'm not even too sure any of the ASP tools comply the way they're supposed to :\

So yeah, I suppose it's time to stop associating the name DivX with Mpeg 4, although I guess you could say it never fully was Mpeg 4 to begin with :P

[SeeMoreDigital]

That's very true RadicalEd. I could not quite believe it when I read Gej's statement myself!

But what ever direction DXN (DivX) decides to go, it's still got to fit it's codec's video stream into an container of some kind. So why not shove it into MP4.

Why should it matter if it does not conform to the current MP4 spec. This doesn't seem to have stopped the rules being bent before.

Anyway my gut felling is that DivX will be relying more and more on post-processing features/algorithms in the future anyway. Which could be achieved using either pure software or embeded software within a hardware players chipset. Hopefully this won't eventually lead to a form of DivX encryption... but you never know!

Now that's what I call radical Ed!

Cheers

[bond]

well, lets get back on topic:

shitowax,
how does the 3ivx codec store the AR info in the outputted avi, which than gets muxed into mp4? how is the info stored in mp4?

[Atamido]

I am going to take a wild guess on this one.

I think that the MPEG-4 bitstream is probably similar to the MPEG-2 bitstream. In this case you would have sequence headers repeated every so often that contain the AR. If you just package these sequence headers with the I-Frame in AVI you should be fine. Then when you transfer the data to MP4, you toss the data packets together to form an MPEG-4 Elementery Stream. Once that is done it should be a simple step to store in the MP4 container.

This is just a partially educated guess though.

BTW, this would be post 500. If anyone thinks I'm breaking rule #11, you might just be right. Of course I would never tell. ;)

[shitowax]

There are A LOT of differences between MPEG-4 bitstream stored into .avi (basically a big hack to make things work) and stored into .mp4 (ISO documented). I don't want to bug you with them, but, anyway, PAR info are stored into VOL header. This header is only present in MPEG-4 codecs and is repeated with all keyframes in a .avi.

Because of the way AVI and DirectShow works, there is no simple way to know PAR info at connection time with .avi, but it's possible (and the format was even built with this idea in mind) with .mp4 ...

hope that helps.

Originally posted by Pamel
I am going to take a wild guess on this one.

I think that the MPEG-4 bitstream is probably similar to the MPEG-2 bitstream. In this case you would have sequence headers repeated every so often that contain the AR. If you just package these sequence headers with the I-Frame in AVI you should be fine. Then when you transfer the data to MP4, you toss the data packets together to form an MPEG-4 Elementery Stream. Once that is done it should be a simple step to store in the MP4 container.

This is just a partially educated guess though.

BTW, this would be post 500. If anyone thinks I'm breaking rule #11, you might just be right. Of course I would never tell. ;)

[SeeMoreDigital]

Well I do'nt know about anybody else but I can't get the AR flagging to work with 3vix.

Please remind me of the software players are supposed to work!

Cheers

[shitowax]

ok, I think there will be some documents on the 3ivx web site soon. Anyway, you need to encode to .avi using 3ivx VfW with a custom PAR. Then you need to remux your .avi into .mp4 (using our mp4 muxer and graphedit should do it). The produced .mp4 should be playable with all directshow players and 3ivx. While 3ivx is the only directshow MPEG-4 decoder supporting PAR, I suggest you let the splitter "locked" to avoid using ffdshow ;)

hope that helps.


Originally posted by SeeMoreDigital
Well I do'nt know about anybody else but I can't get the AR flagging to work with 3vix.

Please remind me of the software players are supposed to work!

Cheers

[The Edge]

Ahhh....I totally forgot that ffdshow could interfere with the PAR.
I'll check this later tonight :)

Edge