New codec comparison of c't magazine

[BBugsBunny]

There's a new codec comparison of the german c't magazine (No. 10 2005).
It compares MPEG4 h.264/AVC codecs, Real Video 10, VP7 and VC-1.
Part of the article can be found online:
http://www.heise.de/ct/05/10/146/
The print version has a nice article about how the codecs work as well.

[Sirber]

English: here (http://translate.google.com/translate?u=http%3A%2F%2Fwww.heise.de%2Fct%2F05%2F10%2F146%2F&langpair=de%7Cen&hl=en&ie=UTF-8&oe=UTF-8&prev=%2Flanguage_tools)

[edit]

No results? :confused:

[BBugsBunny]

The detailed results can only be found in the magazine.
In short Nero Digital AVC was the best codec in the comparison regrading quality.
For all who speak german and can get the print magazine (library or buy) it's worth reading the article in the magazine - really good comparison.

[Lefungus]

OMG, I'm cited is c't. I need to get it even if I have no clue about german! :)

[unmei]

They DID test x264, i overlooked it in the online article..
I think it's a bit unfortunate they seem not to have tested x264.
I mean they explain how the AVC implementations make a large part of their tested encoders, then they even talk about how they did _not_ include snow and dirac, only to not spend a single word on x264.

IMO an AVC test is not of much interest without x264. It's "pretty clear" (excuse my ignorance) that Nero AVC wins at least among the AVC implementations this way. With x264 in, it could have been interesting.

I don't have the Nero encoder, but from what i hear and see it is pretty good - maybe "better" than x264 ..and maybe not, but definitly those two are _by_far_ the most mature AVC implementations i saw so far (and i saw more than those two ;) - even if most did not appeal to me enough to put the through a real test).

[communist]

A quick overlook seems to look promising (used tools, PSNR problems and approaches for quality measurement). As for the missing x264:
MainConcept, Sorenson, Videosoft (ehemals Vanguard Software) und das VideoLAN-Team schicken ihre Implementierungen ins Rennen.
They have a h.264 implementation? Or is it x264?

About Theora / Dirac , Snow and QT7, they say that they will test them in a later issue.

[Latexxx]

X264 is under VideoLAN's umbrella.

[Razorblade2000]

Just reading through the codec review :) (got the magazine)

they did test x264

[thed33p]

OT:

I wish I could read German - the Google translation is making my head spin for all the wrong reasons. Eg:

"Afterwards the candidates had to master NT sports club test sequences the videos quality Experts Group (VQEG)."

[Razorblade2000]

I could give it a shot if you gave me the german sentences you don't get in the translation :)

[Koepi]

x264 was tested and titled as being from the videolan team.

Also, mpeg4 asp codecs got tested, this is missing in the initial post. I hope you like the overview about how the codecs work ;)

Cheers
Koepi

[thed33p]

Thanx for the offer, Razorblade2000. Here's what I'm gonna do:

Sit a German collegue of mine down over lunch tomorrow and have him repay a favour he owes me :D

Unfortunately he can't tell a P-frame from a door-frame so I'll pass anything I still don't understand after that your way.

[SpaceV]

Anyone know if the CT is available in the US?
I speak German but live in MD...

[Koepi]

As far as I know it should be available in the US as well - but you have to search really well to find a shop which sells the c't.

Another option is to buy the article online, they have a "Kiosk" for that on their site.

Cheers
Koepi

[pest]

the tests being done where intensive
but as in the last one i don't like the way that they
mostly compare the quality with psnr-like functions

the reliability with ssim results are better than a straight
psnr compare but i only trust my eyes!

perhaps the average c't-reader likes fancy graphs
and imo c't isn't as good as 5years ago

just my 2ct

[Koepi]

They also did visual quality tests at that comparison - and if a visual result (in comparison to the other codecs) wasn't suitable to the results you would expect by the PSNR/.../<insert any quality metric here> values it's mentioned in the article.

To the OT about the c't quality: I thought about it and came to the conclusion that I can't learn as much as 5 years ago anymore. So the quality level didn't degrade necessarily, but my experience grew.

Cheers
Koepi

[lamer_de]

Wie schon beim vergangenen Test stellen wir alle komprimierten Sequenzen mit Erlaubnis der Rechteinhaber in einer Tabelle zusammengefasst unter dem Soft-Link zum Download bereit, damit sich Interessenten selbst von der resultierenden Videoqualität überzeugen können
All clips are online (except the vc-1 ones, because they can't be played back by any player yet according to their site), so have fun downloading all 128 clips and compare them by yourself:

http://www.heise.de/ct/05/10/146/testsequenzen.shtml

CU,
lamer_de

[Sharktooth]

Bah, they used some outdated codec versions...

[IgorC]

They´ve tested a final version of Divx6. Strange, it has a rotated image.

[ac-chan123]

@sharktooth:
They ask the developers about the setting and version they should use for best quality. When they say use version xyz, they was used. Some version(Nero) was spezial version so they can work with raw yuv files or avisynth as input.

[Sharktooth]

x264 rev 184...
VP7 ver 7.0.4.0...
etc etc...
those codecs have been improved a lot since those old versions.
maybe they should consider updating the comparison, and no... they didnt test the divx 6 final version (it doesnt exist, yet). they tested 1509, and it is also outdated...

[IgorC]

Files have diferent sizes.
They tested not last versions of x264 and vp7
to Test Fusion beta now? there´s no sence.

[Koepi]

It's a printed magazine. There you have a little more time _before_ the readers can read the article, it's not as fast as an internet page.

So at the time of testing, the codecs were even _earlier_ than the rest of you as some codec makers handed out betas etc. early to the c't.

And SO much didn't improve in the latest builds that the overall impression of each codec would be flawed, that - sorry for the harsh word - is plain BS. (not meant offensive!)

Cheers
Koepi

[Sharktooth]

well... vp7 did improve a lot in metrics test and if they based their rankings on metrics, VP7.0.6.0 should have won the match...
btw i dont consider those "reviews" reliable at all.

[IgorC]

yes, it was improved quite Vp7.06 for very low bitarte gives here a slightly better results than Nero H.264, while Vp7.04 was a bit inferior to H.264. x264 rev 219 is closed to H.264. Difference beween
rev219 and rev187 is visible. . No information abour date of Divx6, but Fusion2 is an old beta (there wasn´t big improvements since january first Fusion beta)

[Koepi]

So you suggest that codec develoeprs stop "public" development of their codecs until such a review got printed after the test? That would mean a delay of 4 weeks (average from all codec tests I know of. Even onlinecodec tests do have a delay there as the tester has to assemble an article around that which takes a serious amount of time).

I can't believe what I must read here :) Claims that a codec beats the crap out of every other codec after 1 month of further usual development is quite unrealistic if the codec wasn't very close before.

VP7 got a very good evaluation in the text. I really don't see the problem here.

Cheers
Koepi

[Sharktooth]

You misunderstood me. I didnt say developers should stop... i suggested to update the comparison, at least on the website.

[bobololo]

Originally posted by Sharktooth
well... vp7 did improve a lot in metrics test and if they based their rankings on metrics, VP7.0.6.0 should have won the match...
btw i dont consider those "reviews" reliable at all.

Just for your info, the version of ND/AVC used for this comparison was the official public version released just after the beta, back in september 2004 (7 months ago). So among all codecs vp7 isn't so much outdated ;)

[slavickas]

Originally posted by bobololo
Just for your info, the version of ND/AVC used for this comparison was the official public version released just after the beta, back in september 2004 (7 months ago). So among all codecs vp7 isn't so much outdated ;)
so what? we (users) don't have access to newer version either, unlike for vp7, x264
edit: many of us wish to have better app tehn recode and even beta test newer builds like we can for almost all codecs except wmv probably

[unmei]

oh, well i can't even read - edited my post above.
So i guess i should try to get a copy of the magazine, even if it were to have few information value, it will at least be entertaining.

[LigH]

The article "Effizientissimo" (page 158 ff.) was written by "Dirk Knop"; ahem... am I supposed to know him?! :rolleyes:

:goodpost:

[communist]

Originally posted by Sharktooth
You misunderstood me. I didnt say developers should stop... i suggested to update the comparison, at least on the website.
The online part is an excerpt from the printed version. Something a lot of magazines do. I dont see why they should hurry out and do this small part again if its outdated again in some weeks and isnt much of use on its own :|
You can always disagree with any kind of review - but atleast you should have a good argument than just "outdated", notice that the codec developers handed out the builds (all, maybe except for x264 due to its open source nature) and the settings to use.

[sillKotscha]

Originally posted by LigH
ahem... am I supposed to know him?! :rolleyes:

most of us do... but pssssssssssst!

:D

[Soulhunter]

Originally posted by sillKotscha

most of us do... but pssssssssssst!

Well, there is a "hint" on page 151... :D


Bye

[CruNcher]

I absolutely belive Dirk in his tests even if im absolutely drunk @ the moment (coming from a birthday party) i wish i could have meet him GomGom, Syskin, Gruel, Suxendrol or Isibaar years ago on 19C3 (2002) but sadly XVID wasn't their :( anyways thx Dirk and the rest still actice on XviD, X264 or Open Source for all your eforts :) and i get the C't immediately today when im on my way home btw fefe rules also ;)

PS: Eh ok geting it today could be a little problematic ;) saw the comparsion charts now very impressive results :) geez hopefully the High Profile test can be done in the future maybe a Speed/Quality Chart could be made from the results Koepi ?

[On2Tech]

With respect to using the latest versions of VP7 and X.264 I have some sympathy with the author as he had a publishing deadline to meet. However, the pace of improvements / change for both these codec’s is still quite fast and certainly 7.06 would do better.

What concerns me more, however, is that the data rates for the clips he has put up (http://www.heise.de/ct/05/10/146/tabelle_engel.shtml) seem to vary wildly, which would appear to render many of the comparisons pretty meaningless. I do not have a full translation so perhaps this is explained elsewhere.

There are some footnotes about overshoot but even these seem inconsistent (assuming that the published clips and results correspond). For example he notes that for the 30K cbr clip VP7 overshoots by 10% (in fact the ES rate for the clip he has up is < 30K but I assume he has just based his numbers on the file size coming out of VDUB including container overhead). Again, for X.264 he says that the 30k clip has massive overshoot but there is no footnote for the ND-AVC clip which appears to have a far bigger overshoot (> 50%).

File sizes / approximate data rates for the “30K” clips on the web site (assuming 1k = 1000) are as follows:

VP7 231,424 bytes (32.6 K/s)
X.264 241,664 bytes (34.1 K/s).
ND-AVC 328,690 bytes (46.4 K/s).


Even at the higher data rates there seem to be quite big file size discrepancies. For example for the nominal 250K cbr clip VP7 (247.6K) vs ND-AVC (269.8K). Can someone who has seen a translation of the full article shed any more light on this because at face value comparing clips coded at such disparate data rates would seem to be a bit silly.

On2Tech

[LigH]

They just entered a desired bitrate, and took the resulting filesize - if the files grew bigger than desired (footnotes 2 and 3 in this table), it was just reported, but not compensated by reducing the entered bitrate.

[On2Tech]

Originally posted by LigH
They just entered a desired bitrate, and took the resulting filesize - if the files grew bigger than desired (footnotes 2 and 3 in this table), it was just reported, but not compensated by reducing the entered bitrate.

OK but doesn't this renders the results pretty meaningless. Also, as I have pointed out, he does not appear to have footnoted consistently. For example the ND AVC clip at 30K appears actually to be about 46K!!! but there is no footnote.

[SpaceV]

Did anyone download those high res clips from
the c't site?

I tried the VP7 and the x264 clip, not a chance :(
Who can run them with 25fps (I can't - AMD64 3500+)?

Wonder what it takes and with NTSC you need 30fps!

On2Tech, do you know what kind of system requirements
VP7 has to run 1920x1080 30fps with full post processing
enabled? More than 4GHZ?

[LigH]

:D A Cube processor! :D

Okay - seriously: The more advanced codecs developed to a complexity that requires either a more powerful CPU than available at the moment, or dual CPU systems, or - even better - a custom, specialized decoding processor, to be able to decode such huge video dimensions.

For the current technology, you'll have to either limit/reduce the dimensions, or the complexity of the used compression.

[Wilbert]

@Koepi,

Also, mpeg4 asp codecs got tested, this is missing in the initial post. I hope you like the overview about how the codecs work :)
I also bought the magazine (they sell it here in the Netherlands). I can follow most of it. Very interesting! Is there any chance you could translate it in English, and perhaps doom9 can put it on his guides page? That would be great :)

[ac-chan123]

What say you the word copyright?

[Wilbert]

Yeah, it's probably not possible. I thought perhaps he can arrange something because he's the author :)

[Koepi]

Sorry, but that's indeed not possible :-/

But you can always cite from an article... ;-)

Cheers
Koepi

[ac-chan123]

How much cite you need the translate this article? ;)