About containers and audio in the codec comparison

[Doom9]

I always thought it's best to make a full rip so that you really get to see how large the resulting file is.. imho a quite important property. However, that also makes my life a lot more complicated so I'm asking what you think via this poll. Should I make my life easier by cutting down certain parts of the comparison and normalize others so that for the video codecs it's more consistent, but for your overall scenario it's farther away from what you'd do when making a backup?

Here are the notes for all options, please make sure you read them before voting.


(1) Encode audio, and calculate the video bitrate taking into account the container overhead

(2) Assume a perfect 128 kbit/s audio stream, calculate the video bitrate so that with that audio stream the final file (in the given container, the format's native one if possible)
size matches the target file size.

(3)All video codecs need to reach a certain target size in their given container where the target size has been compensated for the audio track (so it's not e.g. 700 MB but
700 MB minus the size of a perfect 128 kbit/s audio track, without taking into account overhead of the audio in the container. Not that this still means different video bitrate (due to the fact that each container has a different overhead), but any difference in audio filesize and audio overhead are no longer considered.

(4)All video codecs need to reach a certain target size with regards to their raw video bitstream. The output size will not be considered as it will be different,
but all raw video streams will be extracted from the end container and the sizes compared then. As in option 3, the video target size will be e.g. 700 MB minus
the size of a perfect 128 kbit/s audio trac,k without taking into account the overhead of the audio in a container.

Option 3 would be the easiest for me. Option 4 would be the most leveled, but also the furthest from what I consider a realistic backup scenario.

If you select option 5 I expect to find your option in a post.. I'll match the number of votes to the number of options in posts, and whatever doesn't match doesn't count.

Also note that target sizes are picked as an example only and not up for discussion at this point.

[SeeMoreDigital]

I went for option 3.

As I think it most closely represents what the majority of end-users still want to achieve.... ie: backing up their movies to a 700MB CD~R (or even 800MB CD~R).


Cheers

[stax76]

As I think it most closely represents what the majority of end-users still want to achieve.... ie: backing up their movies to a 700MB CD~R (or even 800MB CD~R).

Honestly I don't know what the majority of users is aiming for. Only thing I know for sure is I got DVD-R for years and a +200 GB hard drive and I aim to reach a certain quality and not a certain filesize. I probably have to watch out for good polling/survey php scripts so at least I know what StaxRip users want.

[Herske]

In my opinion: ignore audio and containers; when testing video quality, why should you bother about other unrelated factors?

Voted option 4.

[Doom9]

why should you bother about other unrelated factors?Because the end user has, too? The farther away your test scenario is from a scenario you experience in the real world, the less applicable any results will be to the user.

I aim to reach a certain quality and not a certain filesize. You can't use that in a comparison because you cannot measure quality. You could aim for a certain PSNR, but that is something I'll do over my dead body as I don't trust PSNR one bit, and the same goes for every other metric I've seen to date. And even when you use DVD-R.. you have a certain amount of bytes available and you can't go beyond that ;) And note that neither option mentions anything about what that given size will be so there's no point in discussing that. 700MB is just there as an example.. it remains to be seen what that size will be in 2006.

[Herske]

Yes, but a user has the guides, has software available to encode in whatever container he chooses to; the hard part is using the right video codec with the right options.

IMO, container choice and audio encoding is much less costly (in user time) than video encoding.

[Elias]

I think that it should be kept as it is. The tiny container overhead isn't really a problem. Most containers that are new have pretty much the same overhead, except for avi which is old and results in slightly bigger filesize.

[Tuesday]

700mb, or 1400mb is still a very popular filesize with most rippers i know, the main reason being that with modern codecs you can get a film on there at very watchable quality (for most people), with 5.1 sound too (if you uses HE-AAC), quite easily and as it fits on a CD (or 2) its very easily portable.

DVD-R maybe all well and good, but what codec doesn't come up with decent video at that kind of bitrate? even mpeg2 cuts it!

For the poll, i vote option 3 as it is the best "representative vs. pain in the ass" balance

[Mug Funky]

i vote target video size/bitrate.

muxing overhead and audio bitrate should be left to the user, as ultimately we don't know what container they're likely to use. maybe encode raw streams if possible?

[b0b0b0b]

If someone does an encoding with avc, would they be more likely to use aac? Similarly, with Theora, vorbis? xvid, mp3?

I voted to ignore the container and ignore the audio for a video codec shootout. This clearly isn't the most realistic approach, but I just want to see what the encodings look like at low-ish bitrates and then at high-ish bitrates.

[unmei]

i voted 4 because i liked it to be a codec comparision and not a "backup solution comparision".

[Doom9]

i voted 4 because i liked it to be a codec comparision and not a "backup solution comparision".If you're going to decide between say AVC and VC-1, you cannot use the same container, audio codec, and thus those things will matter into your decision.. you can't have the codec without the container. Same thing with RealVideo.. it's RMVB or MKV. With Theora, it's AVI or OGG but Theora in AVI seems to be kind of a hack so effectively you should be using OGG. With Dirac.. right now there's basically no option.
Bottom line, while you can separate for a comparison, it's artificial and forgets about the fact that certain codecs only go with certain codecs. Personally I like things to reflect reality, be reproducible not in some constructed scenario that you'll never encounter, but something you encounter in daily life.

[Elias]

i voted 4 because i liked it to be a codec comparision and not a "backup solution comparision".Damn good point.If you're going to decide between say AVC and VC-1, you cannot use the same container, audio codec, and thus those things will matter into your decisionWhat about Matroska? I thought it could handle anything. Gabest's *.dsm container can handle everything too (according to Gabest, anything that Matroska can handle).Personally I like things to reflect reality, be reproducible not in some constructed scenario that you'll never encounter, but something you encounter in daily life.Also a good point.

[Doom9]

how do you get dirac into mkv? vc-1?, even theora might be a problem.

[Elias]

how do you get dirac into mkv? vc-1?, even theora might be a problem.How about containerless then? Just raw video bitstream(s).

[Doom9]

@elias: Option 4 would be the most leveled, but also the furthest from what I consider a realistic backup scenario. Realism and fairness are often not the same thing. By all accounts I'm an academic but while academically pure things are very likeable for people like me, they translate rather bad to people who don't have a nack for numbers and procedures. Sure, it's the most academically perfect to compare raw stream sizes, but it rather badly translates into the real world where people have to consider real issues like what bitrate to pick with a certain audio codec and muxed together in a certain container. With MP4 we have a 10.4 byte/frame overhead. With AVI and VBR audio it's 60 byte/frame. For a 200k frame movie, that's an overhead difference of 9.5MB and that's something you're going to notice. Let's take my matrix bitrate, and in case of Matrix1, it translates into an 11 kbit/s bitrate difference, or 1.8% of the bitrate. A 9.5 MB oversize is going to matter very much if you don''t have space to waste (with 6 700MB movies per CD you have a bit of leeway, but if you're amining for CDs you don't). So in the end, you cannot just discard containers.. that 1.873% may not appear to be much, but can you rule out it could make a difference? Perhaps not tip the scales all the way over but even them out a little? And there are people encoding at much lower bitrates.. if your target size is only 350MB, suddenly that 1.8% becomes much larger and will definitely start to matter.

And the audience is not only tech savvy people.. the comparison is read by a lot of people who have no idea what a raw bitstream is to begin with.. they'd be rather confused to see raw stream size comparisons and the comparison would left out the question that is important to joe average: how big is my file going to be? Try explaining raw bitstreams to your parents, see how far you'll get. Then try, "the movie is going to fit on a 700 MB or not" and see which one they're more likely to grasp.

[Elias]

@Doom9: since the use of different containers give different bitrates for every codec, wouldn't that be an unfair codec comparison in terms of quality? I mean, some codecs get to perform better depending on what container they're in. And you're right: realism and fairness are often not the same thing. But still, it's not an exact codec comparison if they are in different containers. All codecs should utilise the same bitrate imho. Still though, I'm not going to complain too much. I'm actually happy and satisfied that you're giving us these codec comparisons, and I voted on number 1.

Oh and by the way, you should've used the latest XviD 1.2 CVS in this test imho, because it is more up to date. :)

[Doom9]

well.. I used an older build to be nice to the qualificants. Of course the main round will include a bleeding edge build. Keep in mind.. qualification is foreplay.. the serious part is only beginning now.. the real contenders are only entering the picture now ;) If a codec doesn't even make it to the main round.. don't bother with it.

[Elias]

Sounds sweet! :D Bleeding edge kicks ass.

[foxyshadis]

I back movies up to CDs, and series to DVDs; I simply don't have enough or buy enough movies to rip often enough to fill DVDs. So yeah, I might be able to tweak overburning and stuff like that but then I risk a bad burn (my drive is iffy on the subject). So that size is still important to me. Whether you use AVI or MP4 doesn't matter - if a codec blows up oversize it'll do the same to both, just with less overhead.

Doesn't AAC in AVI have the same overhead regardless of what video stream it's paired with? In that case find the size of 128kbps AAC in AVI, and just add it to every codec's output. Audio oversize/undersize probably isn't so much of a problem... A couple percent either way on audio is only a meg or so at most. So you might as well just assume.

Oh, and the latest version of mkvtoolnix does theora, just fyi.

[Elias]

Doesn't AAC in AVI have the same overhead regardless of what video stream it's paired with?AAC cannot be placed in *.avi because they're not compatible.how do you get dirac into mkv? vc-1?, even theora might be a problem.Theora (from ffdshow) works great to encode into Matroska with VirtualDubMod.

[Wilbert]

AAC cannot be placed in *.avi because they're not compatible.
I'm sure you know this is possible.

[SeeMoreDigital]

AAC cannot be placed in *.avi because they're not
I'm sure you know this is possible.Yep.... AVI-Mux has no problems at generating such muxes ;)

[Elias]

I'm sure you know this is possible.I'm not talking about avi hacks... to me that's not really avi. And no, I didn't know that, you learn something new everyday :)

[Doom9]

In that case find the size of 128kbps AAC in AVI, and just add it to every codec's output. Doesn't work in case of Dirac, Theora, VC-1 ;) I guess that's the main problem here.. I have to consider 16 different codecs (that's the entire number of participants this year, qualification and main round combined, plus there are some that won't be in the comparison because there's not enough change in the quality department this year.. so make that 18), 4 different containers, 3 different audio formats.. any solution must be able to accomodate all the possible combinations. So you gotta take about 10 steps back and look at the whole table, not just the 2-3 codec you are using ;)

Theora (from ffdshow) works great to encode into Matroska with VirtualDubMod.Besides XviD, no open source team working on Linux feels comfortable if I use a VfW codec, so effectively it's the commandline encoder for theora, snow, lmp4 and x264 so they can use the same tools I'm using. For instance, ffdshow has another 2 pass ratecontrol than mencoder/ffmpeg.. ffdshow uses the XviD ratecontrol. That's also just another of these little things you are likely forget unless you look at the whole plate.

It's not so easy if you have to consider everything, not just your little corner of the world. It much reminds me of administrative decisions on this board.. people might not understand because they do not have to consider so many aspects of the same problem. It is much easier just being a normal user and care about your set at the table instead of the whole table.

[Wilbert]

I'm not talking about avi hacks... to me that's not really avi.
According to Alexnoe (author of Avimux) they are 100% spec compliant to the AVI file specification. That doesn't sound like a hack to me.

http://www-user.tu-chemnitz.de/~noe/Video-Zeug/AVIMux%20GUI/en_myths.html

[Doom9]

either way guys... aac audio doesn't go in every case so it's a moot point..

[mg262]

IMO real-world settings are nice. But not nice enough to be worth causing you a major headache. (Plus, all options give an idea of which codecs overshoot/undershoot, which is the part we are likely to remember anyway.) So I would say go with 3.

Really looking forward to the main round!

[Elias]

According to Alexnoe (author of Avimux) they are 100% spec compliant to the AVI file specification. That doesn't sound like a hack to me.

http://www-user.tu-chemnitz.de/~noe/Video-Zeug/AVIMux%20GUI/en_myths.htmlWell I'll be damned. I just remembered why I thought AAC didn't go with avi, and that's because VirtualDub uses ACM for audio encoding, and AAC isn't compatible with that, right?

[Wilbert]

I just remembered why I thought AAC didn't go with avi, and that's because VirtualDub uses ACM for audio encoding, and AAC isn't compatible with that, right?
Yup ...

[SeeMoreDigital]

Well I'll be damned. I just remembered why I thought AAC didn't go with avi, and that's because VirtualDub uses ACM for audio encoding, and AAC isn't compatible with that, right?Yup again!

In-fact.... I'm hoping to convince a hardware player manufacturer to upgrade their player firmware so it can recognise MPEG-4 with 6Ch AAC within .AVI, along with MPEG-2 with 6Ch AAC within .TS ;)


Cheers

[Inventive Software]

I voted 1, because that's what I do. I generally use AVI because it keeps things simple. ;)

[Doobie]

The container is pretty much part of the codec, as far as I'm concerned. If I'm making a Nero AVC, it's going into the mp4 container. If I'm using xvid with mp3, it's going into an avi. The same reasoning applies to most video codecs.

If I'm using Nero AVC, I'm almost certainly using a lower audio bitrate than 128Kb/s (HE AAC). And, if I'm using Xvid, I just might be using a much higher bitrate than 128Kb/s (the original AC3 stream). If I'm using the Ogm container, I'll be using Ogg Vorbis audio which will give me a bit savings greater than the Ogg container itself over avi. But, adding these factors into a codec comparison is impractical.

So, I'm voting for dump the audio but consider the container. #2, I always vote for the loser, 'cause I vote 3rd party. :rolleyes:

I can mentally factor in the effects of audio on bits I'll have left for video. I know if I go with HE AAC or Ogg over MP3, I can give the video more bits. And, if I keep the AC3 stream, the video will get less bits.

[Atamido]

Most containers that are new have pretty much the same overhead
In certain circumstances, Matroska will have a negative overhead. IE, placing the raw content in Matroska will give you a smaller total filesize than the raw content alone.
how do you get dirac into mkv? vc-1?, even theora might be a problem.
I think it is safe to assume these will be available at some point in the future. So comparing raw video data would be the only fair option. Then you could include notes about relative container overhead for the default output of each codec. That would give the user much more information.

[Doom9]

well, you know well enough why Matroska will never be an option for me, so please don't start. It's the same every year.

[Atamido]

well, you know well enough why Matroska will never be an option for me
Seriously, I have no clue what you're talking about. But I do have a sneaky suspicion that this post could be a rule 1a (http://forum.doom9.org/forum-rules.htm) violation.

I've been really busy at work for the past many months, so maybe something happenned that I missed?

[Caroliano]

I like doom9's comparisons because it is more realistic and holistic. That is the point where it differs to MSU comparisons for example (despite the quality measure) that are pure cientifical. A codec is not only it self, but all the things around it. For example: VC-1 can be faster and have an greater quality than Xvid (hipotetical situation, not necessarily real), but Xvid is still a better codec than VC-1, because it is free, is based in an open-standart, it is more suported in edition tools, players, hardware, etc, etc....

It is not good take the things isolated, so I voted keep everything as it is, though 3 option would not be so bad.

[alchemy]

While I didn't had dvdrw (shame on me for waiting so long) I did all my movies backups with x264/ac3 or mp3 in avi files, becose they are more supported and easy to handle. Yes I do know mp4/others might be more effective, but hey I was lazy to get dvdrw drive, and you expect me to change entire container? :)

To the topic issue : yes, my backups were being 700 or 1400 mb,
so yes, container overhead is important - since its too often "part of codec itself". While I don't want to mix in differect audio codecs, since it will really mess things up.

P.S: Are going to see x264 in next round? Imo its one of the best out there.
(If not THE BEST)

P.S.S: Can't wait for AVC/AAC 7.1 on HDDVD-R DL media :)

[zambelli]

I think Option 3 - encoding just video within the native container - is the best option. Audio isn't the focus of this codec comparison so there's no reason for it to be included. Right now it's serving only as a placeholder. Although ultimately achieving a perfect file size matters, in a video codec shootout it's the rate control accuracy of the video codec that should be judged. Packet and container overhead do matter in the end, but ultimately they're not a property of the video codec.

[foxyshadis]

For example: VC-1 can be faster and have an greater quality than Xvid (hipotetical situation, not necessarily real), but Xvid is still a better codec than VC-1, because it is free, is based in an open-standart, it is more suported in edition tools, players, hardware, etc, etc....
But Doom9's comparison isn't about free vs. commercial, all it measures is video quality, speed, and reliable bitrate, with an emphasis on the first... Mostly it's so useful because free vs. pricey and supported vs. proprietary can be found anywhere, but what he measures is otherwise hard to come by (and trust).

[Doom9]

well.. it's safe to say VC-1 isn't exactly fast at this point ;)

[Audionut]

In the end, doesn't an video encoder have to hit a certain bitrate/filesize no matter audio, subs, etc, etc.

So why make things complicated in including these things into the equation.
I'll say again, in the end the encoder has to hit a certain bitrate/filesize, reguardless.

That leaves the container. Preferbly as you are talking about an video encoder comparision, it would be best to have no container.

Do all encoders in your comparison output raw streams?

Or, do they all output say, mp4, Do you know the exact overhead of said container.

[Caroliano]

But Doom9's comparison isn't about free vs. commercial, all it measures is video quality, speed, and reliable bitrate, with an emphasis on the first... Mostly it's so useful because free vs. pricey and supported vs. proprietary can be found anywhere, but what he measures is otherwise hard to come by (and trust).
OK. My example was realy bad and too far of what I was saying. The point is that we can't put every codec in every container with every audio codec. This limitation should be put because we will have these problems too. I think it is not very fair compare codecs with an fixed bitrate for all because with some codecs you can't use this bitrate if you want to reach a fixed final size, whith audio and everything.

Anyway, the diference in bitrate is relatively small, less than 10kbps.

[leowai]

i voted 4 because i liked it to be a codec comparision and not a "backup solution comparision".
Supported here. Comparison should be made ground to ground else it will be meaningless.
That leaves the container. Preferbly as you are talking about an video encoder comparision, it would be best to have no container.

Do all encoders in your comparison output raw streams?

Or, do they all output say, mp4, Do you know the exact overhead of said container.
I'm also wondering whether the mp4 container is suitable for all video codec? Is the overhead same for all video codec? If yes, I will choose option 4.

With option 4: Why not left the container overhead problem to the encoding program (GUI) to meet the desired target size? You know what? MeGUI's did a good job. ;)

[Doom9]

can I please ask you guys to read all posts.. or at least all of mine? Lots of things mentioned on page 3 has already come up, like what kind of output various codecs do and do not support, or whether all codecs can output raw streams or not.

Comparison should be made ground to ground else it will be meaningless.So all my work in 5 years has been meaningless and I must be the number one moron on this board for having wasted so much of my time in the past. Am I perhaps wasting more time and should thus take down the site and forum as well while I'm at it, actually get a life for a change? Oh, and what about all the people that bothered to read the comparison in the past... I guess they must be morons too.

[leowai]

can I please ask you guys to read all posts.. or at least all of mine? Lots of things mentioned on page 3 has already come up, like what kind of output various codecs do and do not support, or whether all codecs can output raw streams or not. OK, I read through but not in details (working hours :o ). Sorry for that.

So all my work in 5 years has been And I must be the number one moron on this board for having wasted so much of my time in the past. Am I perhaps wasting more time and should thus take down the site and forum as well while I'm at it, actually get a life for a change?Why say that? I think you get me wrongly. What I'm trying to say is the comparison between codec should ignore container overhead in order to make the comparison REALLY fair (ONLY). Don't fire me just like this... :devil:

In another word: Since different codec requires different container, REALLY fair codec comparison should take in same Average BitRate (ABR) regardless of the container overhead/size. This is the ideal case.


I always have the first impression that you've are more concern on the overhead of a container for hitting a desired target size. But when it comes to a backup solution (this is a realistic case):
in case of Matrix1, it translates into an 11 kbit/s bitrate difference, or 1.8% of the bitrate.I don't think 1.8% bitrate hurts the quality significantly for a good codec @~610kps. Correct me if I'm wrong.

[Doom9]

This is the ideal case.That's the ideal academic case.. but by far not he most realistic one. You are faced with containers.. you just are, there's no way around that. None. You can decide to ignore as many factors that have nothing to do with the raw bitstream, but in the end, nobody will just have to deal with that. And you can translate the final size back to a raw bitstream size if you like.. the numbers are available after all. Any size discrepancy of the final output directly translates back to the size discrepancy of the raw video bitstream versus what size I asked for. I could probably put 10 different tables with all kinds of container and bitstream information.. but it's not interesting for most of the audience (which, keep in mind, has no idea what a raw bitstream is).

I always have the first impression that you've are more concern on the overhead of a container for hitting a desired target size.That's where you are mistaken as so many others.. last year I've gone back to prove that we can directly derive the raw bistream sizes from the final sizes.. they match almost perfectly (I think 6KB was the maximum difference).. we know what the container overhead is. It's more about how realistic to you keep the scenario. You say the 1.8% bitrate difference shouldn't matter, and it probably doesn't.. then why shouldn't I care about the whole surroundings too if the bitrate change it incurrs is insignificant? Why shouldn't the scenario be as realistic as possible as the imperfections this approach incurrs are insignificant? That's the question I think everybody who votes for raw bitstreams needs to explain.

And don't forget.. the final size table allows you to directly figure out the raw bitstream and container ovherhead. Many details are already there.. if you want more info about container overhead, MeGUI has all the calculations inside, you just need to have a look at BitrateCalculator.cs..

[leowai]

You say the 1.8% bitrate difference shouldn't matter, and it probably doesn't.. then why shouldn't I care about the whole surroundings too if the bitrate change it incurrs is insignificant? Why shouldn't the scenario be as realistic as possible as the imperfections this approach incurrs are insignificant? That's the question I think everybody who votes for raw bitstreams needs to explain.This is true. However, in my opinion, when it comes to "Codec Comparison @xxxkps", the average bitrate of the raw bitstreams should hit xxxkps rather than (xxx-yy)kps (where yy is the cost of container overhead).

Unless the comparison says, "Codec Comparison @700MB". Then I will agree to take the container overhead into account.

[Caroliano]

Who use the codecs to hit an determinated bitrate use this because they want an determinated final size, not a determinated raw size....

[Dizzy]

And the audience is not only tech savvy people.. the comparison is read by a lot of people who have no idea what a raw bitstream is to begin with..
I think most of doom9 readers are quite savvy ;-) and they know what "raw bitstream" means. Anyway, I voted for option 3 - I think container does matter.