Embedded MPEG4 video stream aspect.

[Neo Neko]

@-h;koepi;suxendrol;and the rest of those contributing to Xvid.

Mencoder under Linux using libavcodec and ffmpeg alow you to embed an aspect ratio in to the video stream and respect it on play back. What is the status of this feature in Xvid. It is part of the official MPEG4 specs AFAIK. It has been in all other MPEG video streams. I have recently come to the conclusion that encoding at resolutions like 640x480 for the most part is a waste of bitrate. Especially when it comes to TV captures. 480x480 generally looks just as good and takes considderably less bitrate to look good.

FFdshow of course does allow manual aspect ratio adjustment. But I would much preffer to have it embedded on a per stream basis as it will be the standard in the future any way.

If it is not in progress or on the to do list I ask that you might nudge others to help get it there. Perhaps the code could be taken from ffmpeg and libavcodec in a simple fassion. Gordian Knot is great for calculating the propper rez to resize to and keep aspect ratio. But Gordian Knot will loose almost all relevance with the inception of this feature. You can encode at any rez you like and get the right aspect every time with embedded aspect ratio. All it costs is a bit of CPU overhead for real time resizing.

I will likely mention this over at Xvid.org in the next few days but I would like to pick your brains on this and get some consensus form others as well. I have made up my mind that which ever codec does it first whether Divx or Xvid will be the one getting my backing for the forseeable future. It would be a great thing to add and another distinction from Divx and another thing to rub in the Divx 3.xers faces. :D

[Teegedeck]

Hi,

hasn't this actually been dicussed on the mailing-list before? I think it was called a good idea but apparantly isn't high on the priority list. Would be lovely...

[Neo Neko]

I have not had time to keep up with the mailing list. :( so it is quite possible it has. For the most part I have heard very little about it. But for me a feature like that is as important if not more so than B frames Q-pel etc. It would stand to make a big difference in things!

[Herske]

Yep, it would be a great idea, it would save encoding time (lower res) and space.


Just take a look at how many dvb channels broadcast at resolutions such as 544,528,480x576.

[ChristianHJW]

Its no so easy to do though, at least with AVI and using DirectShow for playback ( mplayer doesnt have this problem ;) ) !

Milan Cutka explained to us that under DirectShow you need to knw the ouput AR before you are calling the filter graph ( player's job ), but you cant read the AR flag from the codec header ( thats where mencoder puts it, according to MPEG4 specs ) before the graph is rendered and you can read any data from the video stream :( ...

The workaround he was suggesting was to use the AVI comment fields ( like HuffYuv does for some Huffman table info ) to store the AR, that way the player could read the AR flag before trying to render the stream, and set the correct output AR.

Better alternatives :

Wait for proper MP4 container support for both muxing and playback, including AR flags and Vorbis/AC3 audio support, or hope that Suiryc will be able to finish a first alpha of matroskadub this weekend, and that Milan will join the matroska Team and help us improve our DShow parser, like with including a proper resize filter to properly support the codec independant AR flag we have in our spec since we started as MCF :D ....

[BoNz1]

Well, I have been bugging Divx Networks about supporting this for several months, of course nobody seems to listen to me, so I hope you will have more success trying to convince people how great of an option this would be. It is beyond me why DXN did not support this in last release since they have profiles for high def and home theater it would only make sense to have this, since anamorphic playback on a 16:9 TV would be best done using this embedded stream aspect. You could keep more of the vertical and horizontal information and it will look better, although you must use more bit rate in a lot of cases. Plus, something that nobody has mentioned yet but, encoding this way would really speed up our encodes because we could eliminate resizing. Playing video encoding this way takes up virtually zero cpu because the stretching is all done by the graphics card. But, I guess what we really must wait for proper MP4 support or wait for a month or two for Chris' beloved matroska. ;)

[Neo Neko]

@ChristianHJW

So what you are saying is that under Dshow this would not work. Even though directshow filters can resize on the fly like FFdshow?

For instance Directshow inspects a file and sees that it is an AVI. The AVI parser parses the AVI and sees the fourcc XVID, DX50, etc and tells directshow to insert that filter. The filter then begins to parse the video stream and recognises the aspect ratio flag and stretches as apropriate. It all seems plausable to me. Or is my sequence of events wrong?

At worst it would mean Nic would have to update his filter or FFdshow would need to be programmed to recognise it. No? If someone has the time to elaborate I would be interested to read.

[Chibi Jasmin]

Not sure if and how you get this to work, but I wanted this feature for a long time...a definite "YES, PLEASE!" from me!

BTW: In case you ask me, what kind of flag to implement, Display Aspect Ratio or Pixel Aspect Ratio, I vote for PAR!

[fraatz]

hmm. this is called rrv encoding which is already fully implemented in the xvid dev api 3 source and working flawlessly for several weeks now (;P divx.com). problem is that it`s not part of the mpeg4 advanced simple profile, so bframes an qpel are not allowed in rrvops....
you can activate support while compiling. take a look at the xvid mailing list for more infos..
greetings
fraatz

[ChristianHJW]

Originally posted by Neo Neko @ChristianHJW
So what you are saying is that under Dshow this would not work. Even though directshow filters can resize on the fly like FFdshow?

I dont have the time to find the appropriate link here, but do a search for posts from Milan ( he is no too active here ), i guess it would be clearer then than with my bad explanation :).

For instance Directshow inspects a file and sees that it is an AVI. The AVI parser parses the AVI and sees the fourcc XVID, DX50, etc and tells directshow to insert that filter. The filter then begins to parse the video stream and recognises the aspect ratio flag and stretches as apropriate. It all seems plausable to me. Or is my sequence of events wrong?

The problem, as i see it, is that you have to know the output AR before calling the filter graph, but apprently you have no chance to retrieve this information as long as the filter chain is not built and you can start to read codec headers.

Thats why Milan suggested to store the flag in the AVI comments, as those can be accessed by the filter before the filter chain is built, so you know what AR to set on output.

At worst it would mean Nic would have to update his filter or FFdshow would need to be programmed to recognise it. No? If someone has the time to elaborate I would be interested to read.

Yes, and of course VdubMod needed to find a way to set the flag in the MPEG4 codec header, and this is maybe hard to do with the current VfW API ? Not sure here, but for sure XviD API 3 does support writing this flag ...

[Teegedeck]

Originally posted by fraatz
hmm. this is called rrv encoding which is already fully implemented in the xvid dev api 3 source and working flawlessly for several weeks now (;P divx.com). problem is that it`s not part of the mpeg4 advanced simple profile, so bframes an qpel are not allowed in rrvops....
you can activate support while compiling. take a look at the xvid mailing list for more infos..
greetings
fraatz

I guess you're confusing aspect with reduced-resolution P-VOPs, here? We're talking about anamorphous encoding and the like; i.e. non-square pixels.

[Neo Neko]

@Christian
Basically what you are saying is that he was implying that Directshow does not use the fourcc parsed from the AVI to determine the codec used for video in an AVI. But instead samples the video headder setting the stream position later or after the headder. So that when the actual codec gets inserted it never gets a chance to read the headder?

That just seams really like a wrong thing to do. So I could easilly see Microsoft screwing up and doing something like that instead of using the old stuff they already had that worked.

I will have to contact Milan on this. If we give the decoder the ability to read this flag and perform resize itself I fail to see the issue. The MPEG4 codec has to read the header to get info like stream type, dimensions, frame rates, etc. This would just be another field in the header stating the video conforms to a 4:3 aspect ratio or a 16:9 one and the decoder(especially ffdshow) should be able to read that and accomidate.

[MrDarcy]

Originally posted by Chibi Jasmin

BTW: In case you ask me, what kind of flag to implement, Display Aspect Ratio or Pixel Aspect Ratio, I vote for PAR!
Me too!

But from a few tests I did with mencoder/mplayer (win32 porting) it seems that they implement DAR (as for MPEG-2), while MPEG-4 header aspect_ratio_info is about PAR.

Where am I wrong?

[ChristianHJW]

Originally posted by Neo Neko @Christian : Basically what you are saying is that he was implying that Directshow does not use the fourcc parsed from the AVI to determine the codec used for video in an AVI. But instead samples the video headder setting the stream position later or after the headder. So that when the actual codec gets inserted it never gets a chance to read the headder?
Sorry, i seem to have missed that reply of yours Neo.

The Dshow AVI parser will read the BITMAPINFOHEADER structure in the AVI header to learn about FourCC, framerate, colourspace, etc. It will build the graph then based on the info it has from this, but unfortunately there is no AR flag in it.

After the grpah is built, it can start reading the streams, and thats when it can have a look at the MPEG4 codec header, being hidden in the stream.

Milan's idea is to abuse the AVI comment fileds, a substructure following BITMAPINFOHEADER, and hide the information there. The player, when calling AVI parser, can force it to pass this information to it and then the player can set the AR accordingly, in a semi-automatic grpah building process. Still, WMP 6.4 couldnt display the AR correctly this way, the player had to support this solution.

matroska could help here, but with some effort that we will likely not be able to make ....

[LoKi128]

Don't know much about this... but here goes.

Back when I looked into compressing DVDs with XviD, I read a lot of the threads around here on how to optimize the compression. One of the things I remember is that XviD worked better with a mod16 horizontal and vertical size. BTW, not totaly sure abou the mod#, been a while.

Anyway, there is a VDub plugin (Smart Resizer, I think) that allows you to modify the AR to conform with a certain modulus. For situations like this, the correct AR could then be saved in the file.

Another use would be for low-bandiwdth previews or similar? Whatever the original AR is, compress it down to 320x240, then use the saved AR to play it back properly. A very wide original might look bad when played back with very non-square pixels, but it would save some bits.

[ChristianHJW]

On a sidenote :

A team around spyder and Blacksun is working to get the matroska CDL plugin for The Core Media Player working ( TCMP ), so maybe by the end of this week we will be able to encode and play non 1:1 Aspect Ratios ( like anamorphic encodings ) with correct DAR from TCMP automatically, even before the matroska DSF parser will support this generally for every DShow player.

This means that, when making a 2 CD backup, it will not be necessary to resize any longer, just cropping the picture and tagging it as 16:9 AR will suffice.

We have plans also to read the AR flag in the MPEG4 header on muxing XviD into matroska, but this requires that the dev-API 4 finds its way into any win32 applications, which currently is not the case AFAIK ...

[UGAthecat]

I'm wondering if anyone can give an overview of the process that directshow is going through when it creates the graph, at what point it would fail if AR information was in the video, and why it would fail at that point.
It really should not be difficult at all to make a working solution, after all what is trying to be done here is not really that different to what is done in yv12 and similar color spaces where you have multiple pixels being generated from one stored pixel value. Its just a matter of telling the system what the true output frame size should be, and resizing the available (smaller) frame to match that true output size. Since I'm not sure where the breakdown should be happening, I won't bother posting my hypothesis on how to fix it yet :)

[ChristianHJW]

Ok, AR flag is working now. The next TCMP release ( RC4 ) will come with the matroska CDL, now we only need to bug Cyrius about adding the AR tag field in VdubMod .... our current testfiles are made on Linux using Mosu's mkvmerger ....

Mosu should also obtain some c0ode from sysKin soon that will allow him to read the MPEG4 codec header in XviD to parse it for an AR flag that was set there.

[MrDarcy]

Originally posted by ChristianHJW

Mosu should also obtain some c0ode from sysKin soon that will allow him to read the MPEG4 codec header in XviD to parse it for an AR flag that was set there.

Christian, excuse me if I bother with this question: what kind of AR ar you talking about, Pixel Aspect Ratio (certainly present in MPEG4 header) or Display Aspect Ratio (I don't know if present in MPEG4 header, but used by mencoder/mplayer )?

Bye!

[ChristianHJW]

Originally posted by MrDarcy what kind of AR ar you talking about, Pixel Aspect Ratio (certainly present in MPEG4 header) or Display Aspect Ratio (I don't know if present in MPEG4 header, but used by mencoder/mplayer )?

matroska is using a new type of AR storage system offering high space efficiency. You can define the PAR, but using another number than actual PAR, it actually has to be calculated with a small formula from the matroska AR tag.

You can also set a desired output width x height for a movie, e.g. if an artist decides to output the movie or sequence only in a small windows with limited resolution, for whatever purposes ....

[symonjfox]

Is it there anybody working on Aspect Ratio for REAL MPEG4 files?

I know that:
AVI: is a problem
Matroska: No problem
MPEG 4: should support AR ... instead ... Xvid is an MPEG 4 implementation, but there's no chance. Also DivX 3ivx and others can't.

PS: it should be nice also to save the video stream to AVI-Xvid and then set it to 16:9 (for example) while multiplexing in an MP4 stream. You understand.

Thanks

[ChristianHJW]

Xvid does support setting the AR flag in the codec header since dev-API 3 already .... just there are no Windows tools to support this feature, as all are using the VfW interface of XviD and this doesnt have means to use it ...