About containers and audio in the codec comparison

[Tuesday]

I think some people are missing the point (or maybe i just got it wrong), I always thought the Doom9 backup comparisons where about which format it is best to use *when backing up a dvd*, ie - the whole process is a factor, not just a "codec 1 vs codec 2 at bitrate x".

I realise the results will often be similar, but they are not the same thing.

[Inventive Software]

@Doom9: I'm curious. When you're doing the comparison, is there any reason why you measure the first 10000 frames, as opposed to all the frames? This may disadvantage those that are faster in say high bitrate scenes than others and maybe the credits as well.

It makes sense to me to do the measure across the whole bitstream, and not a selection of it. At least that's how I'd do it. ;)

[Doom9]

When you're doing the comparison, is there any reason why you measure the first 10000 frames, as opposed to all the frames? Can you spare your machine for that long? Any measurement requires that you do not touch your machine.. It would also mean I could only use one machine as opposed to the other one I have standing by. If you consider what hardware sites do in terms of speed measurement.. what I'm doing is a considerable improvement, and it's not like the speed changes over time except for the ateme codecs which use some proprietary technology for the first pass.

[JoaCHIP]

Doom9: I'm really looking forward to your test! And you've got an interesting point about taking the container overhead into account, thus assessing the result an end-user would have. But if the purpose of your test is to test the over-all efficiency of the combination of codecs and the container, then i think that compatibility is also something that must be mentioned in such a test - i.e. it's definately worth noting that there are a lot more hardware players that support the .avi container than any other container, and the issues they have with e.g. avi files containing audio that isn't CBR.

[Inventive Software]

Can you spare your machine for that long? Any measurement requires that you do not touch your machine.. It would also mean I could only use one machine as opposed to the other one I have standing by.
Yes I can, because I can find other things to do. I have a couple of books on the go, or I can watch TV, or clear some video tape. Please don't take that the wrong way! Either that, or set it up with job queues and go to work. I do.

If you consider what hardware sites do in terms of speed measurement.. what I'm doing is a considerable improvement, and it's not like the speed changes over time except for the ateme codecs which use some proprietary technology for the first pass.
Can you elaborate more? My understanding is a little limited. Try and be more verbose, I have trouble understanding some things that are explained to me. This isn't to do with the person explaining the subject, it's to do with me. Honest (I have Asperger's Syndrome).

[Alwayscamp]

hi long time i have not been here.
i voted for number one, and i think that makes the most sense. persanolly i don't use any fancy programs with all those fancy options.. just the plain and great autogk. i could use a different codec and waste time trying to get the right compression and what not but i am looking for a clean and easy solution. i think that most guys out there (maybe not those that spend hours of their day looking at doom9 when they want to backup there movie its so that it can fit on some type of medium and played easily. look at any torrent site or go to the kazaa or edunkey network, check irc and you'll see most video files are a set size usually for a full lengh film 700 - 1400 mb (1 or 2 cd's. if its a tv show it usually is around 175mb or 350mb something that can fit nicly on medium wether it be cd / dvd or in the future what ever may come to market.
therefore i think that the best thing to do would be to get the file size to be a round number something that goes on medium easily. 700 mb is a great number now as you can fit it perfectly on a cd or 1400 for two cd's. i think this makes the most sence and also seems to be what the magarity of the world does too.

just my two sense.

p.s. Doom9
i want to say thank you very much for this site and for all the work and effert you put into it every day. its a very great site and i am loking forward to the finnished comparison however you do it.

thanks and keep up the good work.

[Audionut]

Unless the comparison says, "Codec Comparison @700MB". Then I will agree to take the container overhead into account.

I agree.

That to me means, which codec is best at backing up a dvd to a desired file size with audio, subs, whatever, in the codec's chosen container.

Codec comparision at "x" bitrate, to me means, which codec is best at compressing video to the desired bitrate, "without" any overheads. (audio, subs, container, etc.)

edit: you asked for opinions, and this is my opinion. Nothing personal. Take it or leave it.

Either way the comparision is done, I appreciate the effort put in by yourself, and would like to think that I have the knowledge to be able to take your comparision, and to provide my own conculisions as to which codec is best.

Thanks.

[Buggle]

I go for option number 3. Comparing the codecs alone would not be complete; then there should also be a container comparison along with it.
The complete package is what makes or breaks a codec. Thats why, for instance, I do not use Quicktime, because I have not been able to work with MOV at any time I attempted it. (for instance, comparing different kinds of fuel (octane95,98,super benzin) by using different cars... does not make sense). You cannot use the one without the other.
Only if, in the future, matroska is mature enough to hold everything you throw at it, the container would be a free choice. By that time number 4 would be my choice.

[Doom9]

Can you elaborate more? Speed tests at THG and Co involve encoding one chapter with AutoGK and posting the last FPS number VDub reports. The problem is.. VDub reports current FPS, not average FPS.. so if the end of the chapter just happens to be particularily easy to encode, the FPS will be considerably off. I'm measuring the average FPS over two passes.. the way it's meant to be if you want a meaningful value. It is simply not possible for me to measure speed in a fair way over the whole movie.. keep in mind we're talking several days of non stop encoding. I don't like TV (boring), and even when working, at some point you get home and you want to read your prive emails, read the forum, post news to the site, etc. 10k frames is as good as it will ever get on my end. And by the way.. speed measurements are not up for discussion ;) Only what's in the poll is. If I want to hear opinions on other comparison related matters, I will ask (we're mostly guys here.. and you hate it when other people tell you what to do just as much as I do... so always keep in mind this one very important little difference between the sexes).

[darkpepe]

The farther away your test scenario is from a scenario you experience in the real world, the less applicable any results will be to the user.

Most users will choose a format that will play on their standalone players, does that mean you won't test any codec that won't play on one of those?

The purely academic results seem more interesting to me...

[Buzz Lightyear]

Just my 2 cents:
Since this is a VIDEO codec comparison, the best thing would be to test the Raw-Video-Streams without audio + container.
That is scientifically the "cleanest" approach.
But since the container overheads are pretty small, compared to the size of the video-stream, adding containers doesn't hurt a lot.

[dragongodz]

That is scientifically the "cleanest" approach.
hmm but is the target audience for this scientists or people going to backup their dvds or home footage etc ?

i personally prefer a realistic scenario when possible.

[stax76]

I aim to reach a certain quality and not a certain filesize.

You can't use that in a comparison because you cannot measure quality. You could aim for a certain PSNR, but that is something I'll do over my dead body as I don't trust PSNR one bit, and the same goes for every other metric I've seen to date. And even when you use DVD-R.. you have a certain amount of bytes available and you can't go beyond that ;) And note that neither option mentions anything about what that given size will be so there's no point in discussing that. 700MB is just there as an example.. it remains to be seen what that size will be in 2006.

I assume you encode using same file size, same image size etc. and use multipass to hit the file size, for a codec comparison that seems to make most sense (to my limited knowledge).

regarding realistic scenarios (maybe slightly ot):

I don't know if a quality mode encode looks different than a multipass encode with same size, according to what I found about this topic the quality mode is at least equal if not better (if anybody knows this here please correct me). Storing a few movies on a DVD or some more on a HD imho quality mode (e.g. quant 2.5) is the only thing that makes sense (again to my limited knowledge). You cannot say I divide the DVD let's say by four and make every movie 1175 MB because some movie might need 5 times more space (in extreme cases) than other for equal quality (compressibility might vary by factor > 3 and on top of that duration might be twice or more).

[Doom9]

Most users will choose a format that will play on their standalone players, does that mean you won't test any codec that won't play on one of those?I have to disagree with you on this one. Most standalone DVD players being sold for this Christmas, as well as most that are already out there, just play plain old DVDs, and perhaps VCD if you're lucky. Only a small percentage is capable of handling MPEG-4 content. The number of people using MPEG-4 is considerably larger than the number of people who have an MPEG-4 capable standalone. Hence, playback of MPEG-4 content most often happens on some form of PCs, and since you can play everything on a PC....

[Kurtnoise]

@Doom9 : is there a deadline to vote for this poll ?

Personally I voted for [4]. For me [3] is interesting but not with this target size.

[SeeMoreDigital]

Maybe we should also have a survey to find out how users watch their encodes....

For-sure many people on Doom9 will sit behind their PC's monitor screens (usually 4:3).

But I'm sure there'll also be quite a few forum members who started with PC only set-up's who've since migrated to watching their encodes via dedicated stand-alone players connected to their TV's.


Cheers

[Sagittaire]

It's only video codec comparison : only the real bitrate of the video elementary stream is importante (very high overhead with avi and low overhead with mp4)

In fact I think that choose the same container for all video codec is the best/simple way and by far

Little example in my last Metric test: If you choose container size x264/mp4 is better than VP7/avi but if you choose VES size VP7/mkv is better than x264/mkv ... :eek:

For MPEG4 you can choose mp4 or avi and it isn't the same bitrate for VES because overhead are totaly different ...

conclusion: (3) is good if you choose the same container for all situation and else (4) is the only way acceptable to make good comparison ...

[cathars]

I personally backup to personal sizes.
I like 40+ minute TV eps (eg: House MD) to 350mb with XviD and Ogg Vorbis.
I like 20+ minute TV eps (eg: Simpsons) to 175mb with XviD and Ogg Vorbis/mp3.
I backup movies to DVD based on quality. I like 1.1Gb (1/4 DVD) using XviD and AC3.
If the video bitrate is projected to less than 1000, I use 1.4Gb (1/3 DVD) eg: Lord of the Rings.
Movies I prefer using AC3 ripped straight from the DVD. The movies are played through an Xbox with optical output to a Dolby Digital amp.
But these are just personal preferences...not a standard :^)

[keys]

By all means use 128kbps for the sound if we're talking about a 700mb filesize - but if it's anything bigger like 1gb-1.4gb could you please consider 192kbps. Thanks.

[Revgen]

I believe that the container should be .mkv. It supports all codecs and framerates (vfr, cfr) and should equaly represent all codecs.

I also believe that the bitrate (audio + video) should be 1737 kbps (or whatever fills 6 hrs. on a DVD-5). I typically burn multiple movies on DVD's. I can't understand why anybody would want to burn 700mb CD's in this day and age.

[Doom9]

I can't understand why anybody would want to burn 700mb CD's in this day and age.Board an airplane going south, get out at the next stop.. the situation isn't quite as clearcut in countries south of the US, south of the mediterranean, east of... hmm... not the ural.. it begins closer to home than that, etc.

It supports all codecsAnother one not reading all my posts ;)