Log in

View Full Version : Doom9's "Codec shoot-out 2004" test results are out!


Pages : 1 [2] 3 4

SeeMoreDigital
29th December 2004, 15:29
Actually one of the reasons why I bought this particular plasma was it's panel life... which is supposed to equal/exceed most CRT's!

It offers a vertical resolution of 1024 pixels which should be more than good for most HD content... There's no doubt though that a good quality LCD screen (such as Doom9's) will offer better contrast and deeper blacks ;)


Cheers

SeeMoreDigital
29th December 2004, 15:35
Originally posted by babayaga
Anyway, flat screens are for sure the trend ... Happily they're not just a trend... they are the future. And once everything becomes fully digitized.... There will be no need to mention the ITU specs again ;)

robUx4
29th December 2004, 15:35
Originally posted by celtic_druid
The codec doesn't need to know the container/settings, the bitrate doesn't take that into account, that is upto the user.

Well, you could have made the test with a container that supports all the codec in this comparison. This way the audio and overhead would be the same. Only the codec and the frames it produces would differ... So in the end here you compare pears and apples. The container and the audio differs (in size and quality). It's not just a codec comparison. Maybe the title should be changed.

edit: BTW using the proper container could also solve that "frame finding" problem. Since when seeking to a frame in DShow, it would seek to the previous I frame and decode all the needed frames up to the point.

Doom9
29th December 2004, 16:05
Well, you could have made the test with a container that supports all the codec in this comparison. And the fact that the only container that would be a viable solution would be MKV and you're a member of the Matroska team did have nothing to do with that comment, right? :devil: But can you mux raw AVC streams, or transmux files created with WME into Matroska? And then, you cannot directly put RV into Matroska either, you need an RMVB file to transmux if I'm not mistaken.. and in order to find the proper video bitrate, you first have to encode RV once, transmux to Matroska, then extract the AAC audio track, and from that do the bitrate calculations again taking into account the actual audio size and finally encode again. And RV is slow.

It's a simple fact that it is simply not possible to use one input, one tool and have one type of output and it never will be possible. These times have passed.

And when talking about audio sizes, the lower overhead of the MP4 using containers was compensated with larger audio files, so you cannot say they had an unfair advantage. And if anybody actually had the right to complain, it would be the representatives from the various contestants. Yet, I'm not hearing anything negative from them regarding the matter in which this comparison was conducted.

edit: BTW using the proper container could also solve that "frame finding" problem. Since when seeking to a frame in DShow, it would seek to the previous I frame and decode all the needed frames up to the point.That I'm not going to try out until I have your personal assurances that it'll actually work. RV10 and WMV9 are freely available.. so go ahead, try and report back. And one thing I can tell you for sure: the codecs that caused the most problems don't act the way you described. they start decoding from the keyframe AFTER the frame that I'm looking for. I very much doubt the container has anything to do with that.. imho it's the playback filter that determines that behaviour. The fact that seeking wise DivX5 and XviD behave exactly the same when decoded by ffdshow seems to confirm that.

darth rosenberg
29th December 2004, 16:23
And when talking about audio sizes, the lower overhead of the MP4 using containers was compensated with larger audio files, so you cannot say they had an unfair advantage. Nope, but you still can say that there is some uncertainty as to where the difference to the desired file size came from
they start decoding from the keyframe AFTER the frame that I'm looking for. Scroll back a bit :p Though you should contact the "creators" of such bugs when you find them...

robUx4
29th December 2004, 16:24
Originally posted by Doom9
And the fact that the only container that would be a viable solution would be MKV and you're a member of the Matroska team did have nothing to do with that comment, right? :devil:

The fact that you are running this site has nothing to do with the fact that you publish comparisons ? Now instead of talking about the one word I won't pronounce, why don't you tell me how wrong is my statement ?

Originally posted by Doom9
But can you mux raw AVC streams, or transmux files created with WME into Matroska?

Yes.

Originally posted by Doom9
And then, you cannot directly put RV into Matroska either, you need an RMVB file to transmux if I'm not mistaken..

I think you used AutoRV which can output Matroska direclty (but I never tried any such program). There are tools that could do that in one pass. Maybe you should spend more time searching for what is happening out of here.

Also the target bitrate and possible try and test doesn't matter to me. In the end what any user wants is something that fits on 1 or 2 CDs. In the end you compare encodings that have different bitrates just because you didn't want to run another encoding.

Originally posted by Doom9
And when talking about audio sizes, the lower overhead of the MP4 using containers was compensated with larger audio files, so you cannot say they had an unfair advantage.

So you mean making 2 mistakes (using different audio and different container) equals 0 mistake just because luckily the size fits on a CD ?

edit: I lost at my own game, I used the word Matroska once.

darth rosenberg
29th December 2004, 16:35
A new point: Doom9, you say if the user does the bitrate calculations wrongly, it's his fault, and you don't care. So far so good :)

Well, if the user cannot handle his burning app or is using a bad one, it's his fault as well, and you shouldn't care either. CDs don't really need the leadout, and every drive supporting RAW writing (in other words: every CD writer besides very few very old ones) can overburn at least 12 MB. In other words, if you don't take stupid users into account, the "tolerance" before giving the red card should be 12 MB

celtic_druid
29th December 2004, 16:50
The fact is though that if you are encoding to fit a given filesize then you would obviously want a codec with acurate rate control, therefor codecs that aren't acurate should be penalised, whether it be under or oversized, because who knows how it will turn out next encode?

Do the video stream overheads with mkv vary much between say VFW, RV, native AVC/ASP, etc.?

I don't really see that it matters much though about containers and overhead. If you are purely comparing video codecs then surely the only important thing is to make sure that the video bitrate is the same (all raw video streams the same size). For instance if RV stored in MKV has a lower overhead then does that mean that it gets to use a higher bitrate? Or instead you use the same bitrate as the others and have an undersized file?

Then again the purpose of the test is really more about the practical use isn't it? So if you can get away with a higher bitrate for the video of one codec and still have the actual output fit on a CD then that is what you would do.

Here's another one. Wouldn't XviD without packed bitstream have an advantage over say DivX with packed bitstream?

Doom9
29th December 2004, 16:50
Though you should contact the "creators" of such bugs when you find them...What makes you think I didn't? And what makes you think I was not eventually going to find out about your double registration? 14) Multiple registrations are prohibited and are grounds for immediate account deletion.

Now instead of talking about the one word I won't pronounce, why don't you tell me how wrong is my statement ?It's a perfect example of using whatever chance comes along to promote Matroska. I admit, using it for a comparison would be a major PR boost for you. But then, any over or undersize could be conveniently blamed on the container by codec makers. If I stick to the container they use by default, they have no such recourse.

I think you used AutoRV which can output Matroska direcltyI don't find that option.. perhaps it was only in AutoRV9? I saw there's a producer plugin though that allows MKV output from producer.

Maybe you should spend more time searching for what is happening out of here.You just gotta love that, you spend countless of hours and nightshifts and that's what you have to listen to? Inconsiderate is not even a strong enough word for that. I don't ask that you like what you do, but I don't go around dissing Matroska, do I?

So you mean making 2 mistakes (using different audio and different container) equals 0 mistake just because luckily the size fits on a CD ?One strike away from permanent ban.. what can I say.

babayaga
29th December 2004, 16:54
Originally posted by Doom9
And if anybody actually had the right to complain, it would be the representatives from the various contestants. Yet, I'm not hearing anything negative from them regarding the matter in which this comparison was conducted.

My breaking point toward VP6 for starting thinking of a complain was at about 753MB on Revolutions, my first try when encoding :) and only if a small set of scenes were reviewed.

Since a representative set of scenes were reviewed and the oversize is smaller, from ATEME point of view, the test is as fair as it can be (as it was last year).

I'm not saying that all remarks are pleasing (some of them are ;), but since I don't have the same eyes/taste than Doom9 there is nothing I have to complain about.

Pierre Larbier (aka babayaga)
ATEME

Doom9
29th December 2004, 16:56
My breaking point toward VP6 for starting thinking of a complain was at about 753MB on Revolutions, my first try when encoding Well.. they got theirs already on the first page (the recommendation not to use it for DVD backup purposes weights rather strongly imho), and if I had gotten a build with a rate control that doesn't oversize, I'd have definitely used it.

Well, if the user cannot handle his burning app or is using a bad one, it's his fault as well, and you shouldn't care either. CDs don't really need the leadout, and every drive supporting RAW writing (in other words: every CD writer besides very few very old ones) can overburn at least 12 MB. In other words, if you don't take stupid users into account, the "tolerance" before giving the red card should be 12 MBWrong, when we consider that, I'd have to set the target size to 712MB with no headroom. Or say 710MB with 2 MB headroom. It's just a matter what you're aiming for. And overburning is dangerous and tends to produce that CDs that cannot be read everywhere and are more error prone. And I'd like to invoke my r16 right and end the container and size discussion right now.

Manao
29th December 2004, 16:58
darth rosenberg : your new point is unvalid : what if the user wants a final size of 708 MB, because he knows its CD burner can do overburning ?

Robux4 : AVC into MKV wasn't possible when the test was done.

SeeMoreDigital
29th December 2004, 17:11
Oh well,

Maybe it would have been fairer to have had all the Mpeg4 codecs output their content into MP4 container. And all the other video codecs stream types into their generic containers ;)

Personally I can't see why this "container overhead issue" is being made into "such an issue".

As mentioned earlier, this kind of discussion should really be confined to another thread. Especially as such discussions seem to involve the "usual suspects" and detracts from the spirit of Doom9's tests!


Cheers

Doom9
29th December 2004, 17:13
@Manao: thank you, beat him with his own weapons :) And according to the latest posts in the mkvtoolnix thread in the container forum there are still playback issues and an mkvtoolnix build that would support AVC muxing will only be released in January, at the earliest.

Sharktooth
29th December 2004, 17:14
Originally posted by Doom9
Well.. they got theirs already on the first page (the recommendation not to use it for DVD backup purposes weights rather strongly imho), and if I had gotten a build with a rate control that doesn't oversize, I'd have definitely used it.

Well VP7 is on the way and from what i heard it will be "astonishing". But we kwow how those things are influenced by marketing... However i like the VP6.2 (or 6.4) codec, expecially the fact it has a VFW interface and i can use it inside virtualdub. Sure now im playing with AVC but i dont like Recode at all.
I wish ateme had released the commandline encoder with a separate GUI.
Seriously, that's the worst encoding app i've ever seen (buggy and stuff...) and doesnt unleash the true potential of the codec.
Unfortunately i think it wont happen for commercial reasons but i hope there will be other AVC codecs (comparable in quality with NeroDigital) with DS, VFW or commandline interfaces that could be used in other encoding apps.

Tommy Carrot
29th December 2004, 17:30
Originally posted by Sharktooth
Seriously, that's the worst encoding app i've ever seen (buggy and stuff...) and doesnt unleash the true potential of the codec.
Unfortunately i think it wont happen for commercial reasons but i hope there will be other AVC codecs (comparable in quality with NeroDigital) with DS, VFW or commandline interfaces that could be used in other encoding apps.
Yep, i have the same opinion about nero, the codec is awesome, but i don't like recode (and the whole "the codec is only usable with our own encoder app" concept), so i'm reluctant to encode any important stuff with it, although the quality is superior.

jggimi
29th December 2004, 17:36
Once again, Doom9, you've done an admirable job of trying to produce a fair comparison of different scenes with equivalent bpp values.

Nice job!

Your container detractors should have read FAQ Q7 (http://www.doom9.org/codecs-203-6.htm) before posting.

The Edge
29th December 2004, 18:10
Just finished reading the whole shoot-out. Nice work Doom9! ;)
Cheers for all the effort put in.

DeathTheSheep
29th December 2004, 18:54
Mmm... so ND AVC wins here too. Wow, it must truly be the best codec ever. However, because it is only usable with recode and MP4, I simply ... well, can't use it, especially with VDub(mod), because, as Tommy Carrot pointed out:

"i don't like recode (and the whole "the codec is only usable with our own encoder app" concept)"

VFW? That would be Absolutely Perfect, but might rely on "hacks" to use B-frames and will "never utilize the true potential of AVC." Maybe some sort of controllable filter-based encoder (like in graphedit maybe?)...
I wish ateme had released the commandline encoder with a separate GUI.

Mmm...NICE...
Well, just a thought. A... request, if you will.

Cheers, baa

babayaga
29th December 2004, 19:01
Originally posted by DeathTheSheep
Mmm... so ND AVC wins here too. Wow, it must truly be the best codec ever. However, because it is only usable with recode and MP4, I simply ... well, can't use it, especially with VDub(mod), because, as Tommy Carrot pointed out:

"i don't like recode (and the whole "the codec is only usable with our own encoder app" concept)"

Ok guys, we get the message :p

DeathTheSheep
29th December 2004, 19:13
Eh, sorry for the bluntness -- I dont wanna anger anyone... baa, baa...

PlazzTT
29th December 2004, 19:47
Great job, Doom9!

Always a great read!

Atamido
29th December 2004, 20:36
Good review for the most part. There were a few things that I was a little disappointed on.

1. The codecs that missed their marks by a long shot should have been reencoded. Granted, there should definately be notes to the effect, but why would you compare two clips with different bitrates?

2. What is up with the credits? Maybe I missed the segment where it was talked about? Were they encoded seperately? If not, why not? How much bitrate was devoted to the credits for the different codecs?

3. There should be a listing of exact bitrates for the different video formats. The information for the video in the AVIs is available within this thread, but still not on the shootout. The RV9 can be put into Matroska to determine the exact size of the video data. I guess the AVC could also be determined the same way, although I would imagine there is another way to do it.

4. Not a flaw, but a nice touch would have been to know how close to the previous I frame the image was for each codec.

Other than that, good job. The layout of the images is superb and the option to view the lossless PNGs is an excellent option.

SeeMoreDigital
29th December 2004, 20:52
There's no denying that "arguably" Nero's Mpeg4/AVC codec sets a new benchmark for other codecs to follow...

But for me, what I would have liked to have had confirmed was how well Nero's Mpeg4/ASP codec performed against other Mpeg4/ASP codecs such as, XviD, DivX and 3ivx!


Cheers

niamh
29th December 2004, 21:34
2. What is up with the credits? Maybe I missed the segment where it was talked about? Were they encoded seperately? If not, why not? How much bitrate was devoted to the credits for the different codecs?
Though I can see the point of this Pamel,and somehow agree with it, my opinion is that in practical life, credits will be encoded with the movie, and if a codec is going to waste bits on credits, well then it should be penalized in the test, because it will penalize itself in a real encoding.
A codec should know where to apply more bits and where to save some IMO, otherwise tests would eventually be reduced to encoding stills at a fixed bitrate, and is there a sense in that? :)

I think there are 2 trends of people here, (as there are in every single codec test that appears).... the ones who want a totally scientific codec test, precisely precise (and in theoretical theory, they are in the right ;)), but quite removed from encoding reality IMO, and the others who just want something that's immediately applicable to real encoding and DVD back-ups ... I lean in the others camp, so I'm fine with the test, and it seems fair to me ; besides, it looks like no matter how one will redo that test, the winner will be the same regardless.... and yes, we still love Xvid to bits ;)

Neo Neko
29th December 2004, 21:53
I think the reason the credits were done as they were was simply because not all codecs offer such features as zones. Which are less of a codec improvement and more of a user end tool since the user must decide how to use them. Also it is more than likely the way most people will encode. Thereby giving the most indicative results for most people. You or I may use Zones or EKG etc but alot of people don't. What doom did was a base standardised test using only general codec options. Not sitting for a few hours tweeking a single source as I have been known to do in the past. ;)

temporance
29th December 2004, 21:55
Originally posted by niamh
Though I can see the point of this Pamel,and somehow agree with it, my opinion is that in practical life, credits will be encoded with the movie, and if a codec is going to waste bits on credits, well then it should be penalized in the test, because it will penalize itself in a real encoding.
Credits can make a BIG difference to the rest of the movie. I replicated one of the encodes from the previous codec shootout and found that the % filesize occupied by the credits was disproportionately high. And codecs (DivX and xvid) were so different in their treatment of credits that the effective bitrate available for the remainder of the movie was different between codecs. The average bitrate of doom9's scrutinised scenes also varied. (see my original post at http://forum.doom9.org/showthread.php?s=&postid=423390)

Codecs do not know that a user is not interested in the quality of his credits unless they are told about them. If credits are encoded along with the rest of the movie then, IMHO, a segment should be take from the credits for comparison ;)

Btw, it seems QPel's forte is scrolling credits.

SeeMoreDigital
29th December 2004, 22:14
Personally, I can't see the point of encoding the credits at all.

With most movies, it's quite a straight forward task setting say, DVDdecrypter to rip a movie without the chapter that contains the credits!

Jeez, if you want to know who's made the film, either rip and encode "that chapter" as a totally separate file, look on imdb.com, or wait for the movie to appear on TV :)


Cheers

niamh
29th December 2004, 22:15
Credits can make a BIG difference to the rest of the movie.
I donŽt disagree with that in the least. Personally I encode credits though, so if a codec test was made without credit encoding, and gives such and such results, and it turns out that in effect, codec x has a good result in that case, but uses up a lot of bitrate on credit encoding (which is basically black stuff), making a BIG;) difference to the total encode, well then,IŽd feel cheated.

If credits are encoded along with the rest of the movie then, IMHO, a segment should be take from the credits for comparison
Very good sweat-effective solution :) I would be interested to know,actually.
Btw, it seems QPel's forte is scrolling credits.
Well the settings were given by the respective devs of the codecs, you canŽt blame them for giving relevant settings,can you? :)

Ark
29th December 2004, 22:41
Originally posted by SeeMoreDigital
Personally, I can't see the point of encoding the credits at all.

With most movies, it's quite a straight forward task setting say, DVDdecrypter to rip a movie without the chapter that contains the credits!

Jeez, if you want to know who's made the film, either rip and encode "that chapter" as a totally separate file, look on imdb.com, or wait for the movie to appear on TV :)


Cheers

Well, i like to hear the music that plays in the end credits, so i try to maintain them, and i don't like very much to suddently cut the audio right after the last scene, it destroys all the atmosphere :p

niamh
29th December 2004, 23:23
Maybe we should make a poll " do you encode credits or not?" ;) this way weŽd know whoŽs in the majority....I agree with you Ark.......nothing worse than a movie going PloP suddenly.

aaar9800
29th December 2004, 23:37
Originally posted by Ark
Well, i like to hear the music that plays in the end credits, so i try to maintain them, and i don't like very much to suddently cut the audio right after the last scene, it destroys all the atmosphere :p

Couldn't have said it better

SeeMoreDigital
29th December 2004, 23:53
Originally posted by aaar9800
Couldn't have said it better Oh well.... Each to their own!

Doom9
30th December 2004, 00:00
@DeathTheSheep: You should be able to use the ND filters in graphedit.. just rename graphedit.exe to recode.exe (at last that's how it worked with the asp filters).. I just never really figured out the PAR flag so I'd be grateful if somebody could eventually explain that to me.. I get DAR but PAR seems weird.. they use really weird numbers that I cannot put in any relation to the encoded and playback resolution.

I'm sure we'll find plenty of people who fit into one of the three end credit categories:
1) credits are cut
2) credits are encoded as part of the movie
3) credits are encoded separately, normally using special curve scaling features.

I fully understand the reason for the use of a lower bitrate for end credits (though personally I don't care much to see them at all), and I would use end credit treatment, if it were not for this: end credits bitrate is not a feature of many codecs. This can be compensated for with special tools (like in case of DivX5 encoding in GKnot), but in most of the cases, this would required a special encoding session, and thus basically a lot of extra work and a whole can of worms that can bit you at the wrong moment and give me even more to do. I hope some of you read the last point of my FAQ.. making a codec comparison is a huge effort, and my resources are scarce. Hence, encoding credits separately is off my list for now until I can get this done automatically and have the tools that take care of bitrate calculations and such for this special occasion.

But, in the absence of use of any special end credits treatment, I think it's reasonable to assume that no codec will cut the bitrate by 50% and use it someplace else. If it did that, it would also do the same for the intro with the scrolling letters, would it not? And I'd not miss that because the intro is a scene I review.

1. The codecs that missed their marks by a long shot should have been reencoded. Granted, there should definately be notes to the effect, but why would you compare two clips with different bitrates?This mostly concerns VP6. I agree that it's not quite fair, that's why even on page one, just after presenting the sizes, I clearly state I cannot recommend VP6 for DVD backups. Though, your comment does raise an interesting question: What is the point at which I cannot accept an entry because it gets an unfair advantage or disadvantage over the others? I did that with the Moonlight codec. So, I think that's one point we can go on discussing: How much oversize is allowed? I tend to rather disqualify a codec, than take a blind shot and encode again (especially VP6 was slow as hell), it could potentially waste a lot of time. But, next time I will ask if I can use the standard bitrate calculations provided by GKnot (for codecs using the AVI container), to reach the size target. And there needs to be some cutoff. Where would you place that? 10 MB? 15MB? 20MB? And, should the same apply for undersize, or should undersized entries be allowed as long as the quality isn't abysmal?

3. There should be a listing of exact bitrates for the different video formats. The information for the video in the AVIs is available within this thread, but still not on the shootout. The RV9 can be put into Matroska to determine the exact size of the video data. I guess the AVC could also be determined the same way, although I would imagine there is another way to do it.There is another way. Here are the exact video bitrates for the two NeroDigital codecs:

ND Main Profile: (bitrate asked for, bitrate received)
SPR: 1025091 bits/s - 1025.08 kbit/s
Matrix3: 627266 bits/s - 627.25 kbit/s
Futurama: 1000094 bits/s - 1000.03 kbit/s

ND High Profile
SPR: Forgot to save them, but you can derive them knowing the overhead per frame (final size and audio size are known).
Matrix3: 627266 bits/s - 627.27 kbit/s
Futurama: 1000094 bits/s 1000.00 kbit/s

The problem with this is the following: It serves very few people, and it can actually be derived on your own given the info in the comparison (okay, I need to add the exact number of frames each source has). Knowing full video size, number of frames, framerate and audio file size you can derive all those values (GKnot can calculate the overhead for you). But the derivation is time consuming. Page one of the comparison already takes more than 3 hours as it is, and is probably already overloaded. The more info I add there, especially info that you can derive by yourself, takes my own time away, and since I tend to think it's not something most people are interested in, I prefer not to spend that additional time. And, being given all the info you need to duplicate an encoding session with RV10, you can derive those values as well.. just takes you an encoding session.

4. Not a flaw, but a nice touch would have been to know how close to the previous I frame the image was for each codec.Here also I have to question how many people actually are interested in that info. And, I do not know how to find this out for all codecs. The ones that have a VFW interface are no problem, that's what VDUB is for, and even if there's only a DS interface, as long as the filter supports all forms of seeking (there's a seek to the next/previous keyframe), it could be done, but what do you do if the DS filter doesn't support that kind of seeking and if there's no VFW interface?

babayaga
30th December 2004, 00:05
Originally posted by temporance
Credits can make a BIG difference to the rest of the movie.
On Revolutions, end credits represent about 7% of all movie duration (~13 000 frames out of ~186 000).

And 7% more bitrate makes a big difference on the quality of the video : something like the difference between VHQ1 and VHQ4 in Xvid and more than the difference between our Main Profile and our experimental High profile.

SeeMoreDigital
30th December 2004, 00:08
Originally posted by Doom9
... I just never really figured out the PAR flag so I'd be grateful if somebody could eventually explain that to me.. I get DAR but PAR seems weird.. they use really weird numbers that I cannot put in any relation to the encoded and playback resolution. In my opinion, when it comes to generating encodes from 4:3 or 16:9 Mpeg2/DVD sources, PAR calculations are less important than DAR calculations!

For those interested, this is why I wrote this: -

http://www.SeeMoreDigital.net/03_Video_Only_Info/Why_I_don't_follow_ITU601_specification.html


Cheers

Doom9
30th December 2004, 00:43
I'm just uploading a few FAQ addendums picking up some of the issues discussed this time. But I guess I'll have to keep adding things for every new comparison. I have also added the number of frames, as well as the framerate of each source. That way you should be able to derive stats on your own. I'm also looking for ways to determine RMVB overhead and if it's possible to encode only video and add an AAC stream to it (and how to figure out what kind of overhead that would add)

ut for me, what I would have liked to have had confirmed was how well Nero's Mpeg4/ASP codec performed against other Mpeg4/ASP codecs such as, XviD, DivX and 3ivx!you're not the only one. I'd also have liked to know how HDX4 performs with the film grain reconstruction on (so that the grain would be encoded normally and not filtered out)... but the number of codecs...

I've been thinking, that beside a new animated source (we'll get to that in 2005), there needs to be another way to conduct a test more locally, and so that it'll take less time (so it can be done more often).. throwing in all codecs will get excessively hard and that much free time I spent this time is a rarity for me. So I was thinking what if I separate the codecs in categories like ASP, AVC and proprietary, face them off against each other all in their own category, then face off the winners? Or perhaps you have other ideas as well?

babayaga
30th December 2004, 01:37
Originally posted by Doom9
I've been thinking, that beside a new animated source (we'll get to that in 2005), there needs to be another way to conduct a test more locally, and so that it'll take less time (so it can be done more often).. throwing in all codecs will get excessively hard and that much free time I spent this time is a rarity for me. So I was thinking what if I separate the codecs in categories like ASP, AVC and proprietary, face them off against each other all in their own category, then face off the winners? Or perhaps you have other ideas as well?
Another way is to have a pool of reviewers, not just you.
Assuming there are 3 reviewers and 3 movies :
- each reviewer encodes one movie with all codecs
- he extract let's say 7 2mn video samples out of its encoded movies (this requiers a tool to extract a sample out of an encoded movie but i'm sure all companies have this kind of tool).
- then all reviewers swap their samples and vote.

As a result, each reviewer is only responsible for extracting representative samples out of his movie (and taking the pictures). The work load is divided by the number of reviewers and the final word is still yours.

BTW, is it possible to have the extra-old codec comparisons you made in 2001 ?

Doom9
30th December 2004, 01:50
BTW, is it possible to have the extra-old codec comparisons you made in 2001 ?I'm not sure but I'll check.

krackato
30th December 2004, 02:03
I'm just a little confused about the Nero AVC codecs. In the shootout they were referred to as Main Profile and High Profile (MP and HP)

But all I see in NeroRecode 2.2 is the following:

Maximum Definition AVC
Mobile AVC
Portable AVC
Standard AVC
Cinema AVC
HDTV AVC

So which on of those did Doom9 use? Also, I'm a little confused as to which supposedly offers the best quality between Maximum Definition, Cinema and HDTV.

Or is everyone using a totally different version of Recode than I am?

Doom9
30th December 2004, 02:14
in Recode, you have an older version of the Main Profile codec. All those profiles in Recode are Nero's own invention.. if you switch between them and look at the settings, you'll notice that certain features are activated / deactivated, and certain settings differ... for instance, the number of reference frames, the number of b-frames, etc.

And, as it says on page one, I got a special commandline encoder that you cannot get.

babayaga
30th December 2004, 02:15
Originally posted by krackato
I'm just a little confused about the Nero AVC codecs. In the shootout they were referred to as Main Profile and High Profile (MP and HP)

But all I see in NeroRecode 2.2 is the following:

Maximum Definition AVC
Mobile AVC
Portable AVC
Standard AVC
Cinema AVC
HDTV AVC

So which on of those did Doom9 use? Also, I'm a little confused as to which supposedly offers the best quality between Maximum Definition, Cinema and HDTV.

Or is everyone using a totally different version of Recode than I am?
Doom9 did review the video codecs. This is not related to NeroDigital profiles which are designed for compatibility with standalones. Besides, he did not use Recode but beta internal versions of the codecs.

So AVC Main Profile (a generic MPEG-4 AVC definition) is the basically Recode codec with the same tools. The settings we did provide would fit in all NeroDigital profiles starting and above "Standard AVC".

The High profile codec (also a generic MPEG-4 AVC definition) is an early beta and the first High profile codec ever able to encode a full movie in a reasonnable time. Besides us at Ateme, Doom9 is the first in the world being able to write about the "High Profile experience" :)

dvd_maniac
30th December 2004, 02:34
@babayaga & Doom9

So when you say "We can not get"... You mean we will never get? Is this a tool that is designed for the next version of Recode? Does that mean that Recode will be noticably faster encoding AVC?

slavickas
30th December 2004, 02:40
Originally posted by dvd_maniac
@babayaga & Doom9

So when you say "We can not get"... You mean we will never get? Is this a tool that is designed for the next version of Recode? Does that mean that Recode will be noticably faster encoding AVC?

although i'm not babayaga || Doom9 :D

command line programs usefull for codec developers (easier to make things automated), while for end users there fancy one-click guis,

it's very unlikely to see cmdln version of codec for us "standart" users

babayaga
30th December 2004, 02:44
Originally posted by dvd_maniac
Is this a tool that is designed for the next version of Recode? Does that mean that Recode will be noticably faster encoding AVC?
Doom9 did receive command-line encoders in the form of the beta package .
Those encoders are beta versions scheduled to be integrated in Recode.
The Main Profile encoder will be integrated shortly, the High Profile is an early beta that will need some polish.
The Main Profile is faster and better than the current Recode version but share the same features.

You mean we will never get?
I honestly don't know.

Joe Fenton
30th December 2004, 02:51
Great comparison. I appreciate all the time you spent on it. Personally, I almost went nuts just flipping back and forth between the finished frames in the article. I don't see how you stayed sane while putting it all together. :D

The only real question I have is - who suggested that Futurama be encoded without b-frames in xvid? I've never seen a good encode of cartoons or anime with b-frames off. The difference is night and day on all my encodes. Sure, a b-frame tends to be lower quality, but it gives you a BIG savings on bits which can be reused elsewhere where they are actually needed. So some frames look worse, but overall the quality is TREMENDOUSLY better. I ask who told you not to use b-frames so I can complain to the right person. ;)

I usually encode credits the same as the rest of the movie. It's easier than trying to separate out the credits and process them differently. I would guess more people encode the credits the same as the movie than don't. What I wish for in the next year or two is proper variable frame rates. Right now, if you inverse telecine a movie, the credits are jerky because they were done via computer at 30Hz instead of 24 like the rest of the movie. Or worse, you get a mix of 24 and 30 Hz all through the movie due to special effects. This is becoming a particular problem in anime where more of the anime is being done by computer than ever before while the hand drawn sections are STILL shot on film at 24 Hz.

When you get to the point where VFR becomes part of the shootout, I suggest you use Cowboy Bebop as a test source. It has a good mix of hand drawn 24 Hz material and 30 Hz computer sequences throughout all episodes. Add in lots of pan and tilt shots, fight and chase sequences, and a wide dynamic range on the lighting and Cowboy Bebop will put any codec to the ultimate test.

dragongodz
30th December 2004, 04:27
@dragongodz: I have a 16 question FAQ for just that purpose so I don't waste my time replying.
and how many do you think actually read it ? not many by the looks of things. :)

infact that is why i put that bit, to try and get the message through. with the amount of B.S. and nonsense suggestions that followed from some its obvious they dont want to understand and have no intention of doing there own comparison so we can pick the crap out of it. :sly:

SeeMoreDigital
30th December 2004, 11:03
Originally posted by Doom9
So I was thinking what if I separate the codecs in categories like ASP, AVC and proprietary, face them off against each other all in their own category, then face off the winners? Or perhaps you have other ideas as well? I like this idea!

This would make it easier for us to determine which codec manufacturer has made the most progress from year to year, or from release to another release.

Speaking from personal experience, as I play most of my encodes in hardware I tend to generate them in Mpeg4 'Simple Profile' mode only (due to compatibility issues). And even in SP, the difference between say, NeroDigital, DivX, XviD, FFmpeg, Sorenson, 3ivx etc, at the same bit-rate, is quite staggering!

Yep... I'm all for this idea :)

EDIT: Also, once you've established your sources, generating encodes from (alpha, beta, pre-release) codecs as they appear throughout the year should save time... meaning only a "summing-up" is required at the end of the year.... Just a thought!


Cheers

Mr_Schizo
30th December 2004, 11:54
Thanks for this great comparison!

lamer_de
30th December 2004, 12:03
@Joe Fenton: First, you mixed up the terms Hertz (http://en.wikipedia.org/wiki/Hertz) and frames per second. Hz has nothing to do with framerates.

Second, variable framerate is pretty much codec independent and an issue of the container that is used. If you want vfr, you can already use matroska and Divx/xvid (probably any MPEG-4 codec will work, but I don't do DVD-"backups", so I don't care much for it. Another downside for me is that you can't edit the outcome (you can load it via directshowsource into vdub, but it's a major pain in the butt)). I think I read something that RV10 in rmvb supports it as well, but I don't use RV10. So this has nothing to do with the abilities of a codec.
CU,
lamer_de