New PC Magazine Codec Test Flawed

[aharden]

Originally posted by karl_lillevold in the "New A/V Formats" forum
In the big "Play Anything" Media Player test in PC Magazine,
http://www.pcmag.com/article2/0,4149,805233,00.asp
...
For music ( http://www.pcmag.com/article2/0,4149,806392,00.asp ), MP3 and MP3Pro come out on top. RealAudio scores higher than WMA, especially for lower bitrates. Vorbis scores about the same as RealAudio and higher than WMA for low bitrates but much worse for higher bitrates.

I'm a Vorbis fan and am disappointed when testers (especially those with a large audience) insist on encoding audio at a constant bitrate. From the article at the second URL:

We tested each format using three music genres—classical, pop, and rock. We digitally encoded each selection to MP3, MP3Pro, Ogg Vorbis, Real, and WMA formats at a low-bit-rate setting (64 Kbps) and high-bit-rate settings (typically 128 Kbps, although we used 96 Kbps for MP3Pro—the format's maximum bit rate—and 132 Kbps for Real files, which was the closest available setting). The tracks were encoded with constant bit-rate settings.


Aside from low-bitrate streaming and MP3 players of limited ability, I can't understand what usefulness CBR has over VBR, especially at bitrates of 128kbps and higher. Based on my own experience and other tests I've read about and participated in, Vorbis probably would have fared better if the samples were encoded in a "quality" mode (natively VBR).

I'm glad PC Magazine included Vorbis in their tests, but what they tested doesn't reflect what I think is what the majority of users do for their high-bit-rate MP3 and Vorbis encoding.

[alexnoe]

Your headline sounds as if that magazine would have done at least once in the past something useful....actually, many German PC magazines never have written the least bit of sense...

[aharden]

Originally posted by alexnoe
Your headline sounds as if that magazine would have done at least once in the past something useful....actually, many German PC magazines never have written the least bit of sense...
Regardless of our estimation of their savvy or usefulness, PC Magazine has a broad readership. In my opinion, this kind of negative mindshare for Vorbis hampers the effort to try to get hardware manufacturers to produce Vorbis-compatible players.

[alexnoe]

PC-Welt, Computer-Bild or Tomshardware are read by so many people as well, but spread the same disknowledge. PC-Welt still believes in "games protected by EFM"...and CHIP is making "reviews" of drives without installing them.
And all of these are read by (far too) many users.

There is nothing you can do about it. Even if any of their authors would be willing to learn, they would not have the time. Writing disknowledge is more profitable, as long as most people don't see that it is disknowledge.

Since cdrinfo has introduced their C2-error-report-accuracy-nonsense and even C2-error-report-accuracy-score, people go to forums and write "but this writer has low C2 accuracy...." and all of them just believe it, because it sounds good (in their "article" about error correction, they have demonstrated that they don't even know bits from bytes :scared: )

Again, you can't fight them. None of them. No chance.

[JohnMK]

It's a damn shame, that's for sure.

[bb]

At least there's one German computer magazine still standing tight. One that you can still trust. You probably know I'm talking of c't.

bb