MP3 at 128kbps public listening test - OPEN

[rjamorim]

Hello.

I'd like to announce the opening of my MP3 at 128kbps public listening test.

The test consists of comparing how 6 MP3 encoders behave compressing 12 different samples representing a wide spectrum of musical styles.

People interested in participating are invited to visit this page:
http://www.rjamorim.com/test/mp3-128/presentation.html

The test will end on January 25th and results will be posted soon after.

Best regards;

Roberto Amorim.

[bond]

there is only one thing to say:

join it!!!

[Gabriel_Bouvigne]

Test is still running (extended by 1 week), so do not forget to participate.

If you do not participate, perhaps you will never know if you should use FhG or Lame to encode movies...

[bond]

good idea to extend it by a week

the more people who participate the better the results!


and not to forget i will strike every doom9 user who doesnt participate! :D

[bond]

ok guys, today is the last chance to send your results in!

hurry up!

[duartix]

I've seen a roadmap of rjamorim's future listening tests but I can't seem to find it anywhere.
I believe there was a new AAC 128 kbps test in the works. Does anyone have any info on this?
By the way, is this test closed already?

[rjamorim]

Originally posted by duartix
[B]I've seen a roadmap of rjamorim's future listening tests but I can't seem to find it anywhere.
I believe there was a new AAC 128 kbps test in the works. Does anyone have any info on this?

Starting February 18th: AAC at 128kbps
In March: Multiformat at 128kbps (with the MP3 and AAC winners)
In April: Multiformat at dial-up bitrate (32 or 48kbps)
And then I'll probably retire :)

By the way, is this test closed already?

Not only closed, the results are already up ;)

Here is the results page:
http://www.rjamorim.com/test/mp3-128/results.html

Here is the final plot for those in a hurry. (zoomed version, you can find non-zoomed at the test page)

http://rarewares.hydrogenaudio.org/test/mp3-128/plot12z.png

Thanks to everyone that participated.

[Andrey]

Why mp3enc by Fhg does never participate ? :)
This encoder was especially tuned for 128 Kbit, as I remeber...

[rjamorim]

Originally posted by Andrey
Why mp3enc by Fhg does never participate ? :)
This encoder was especially tuned for 128 Kbit, as I remeber...

mp3enc uses FhG's slow codec, which FhG stopped developing in favour of their fast codec, because it is faster (heh) and higher quality.

Proof of that is that you can't find the slow codec on Adobe Audition/CoolEdit Pro and MusicMatch (the two sources for up-to-date FhG codecs) anymore. Even Audition's High Quality - Slow codec is fastenc with different tunings.

FhG never implemented VBR on their slow codec either.

slowenc stoped being developed around 1998 or 1999. fastenc is developed to this day, although FhG mostly does bugfixes now and very rarely does tunning.

Hope that answered your question :)

Best regards;

Roberto.

[Andrey]

>>because it is faster (heh) and higher quality.
I doubt the second statement very musch :)
But the slow codec is extremely slow, I agree... :(

>>FhG never implemented VBR on their slow codec either.
Who cares ? :)

>>Hope that answered your question
Mostly, thanks !

BTW: It seems that later codecs were developed by Opticom, not Fhg inst. themselves. At least, Opticom stated it for a very long time, but some years ago this phrase disappeared from their advertisment...
I think, only Ivan know the truth... :) :)

[rjamorim]

Originally posted by Andrey
BTW: It seems that later codecs were developed by Opticom, not Fhg inst. themselves. At least, Opticom stated it for a very long time, but some years ago this phrase disappeared from their advertisment...
I think, only Ivan know the truth... :) :)

http://www.opticom.de/1_company/1profile-set.html
OPTICOM was founded by it's CEO Dipl.-Ing. Michael Keyhl in 1995 as a "spin-off" company of the Fraunhofer-Institute

So, that seems to indicate that Opticom ~= FhG

One company that I am sure that tweaked their encoder further after getting sources from FhG is Telos (that's why their encoder, AudioActive, is featured). And, AFAIK, Telos is not related to FhG.

[duartix]

Thank you Roberto.
I'm sorry you had so few tests in the mp3 test. I was one to blame for sure. :o
But I promise to participate in the AAC test. :) :) :)

[rjamorim]

Originally posted by duartix
But I promise to participate in the AAC test. :) :) :)

Thanks :)

I'll announce it here as soon as it starts (planned for February 18th)

[Sagittaire]

And MusicMatch jukebox encodeur for MP3 VBR ... ???

[rjamorim]

Originally posted by Sagittaire
And MusicMatch jukebox encodeur for MP3 VBR ... ???

MusicMatch uses the FhG codec, so it's the same as the one used in Adobe Audition.

[Sagittaire]

Thank you for this fast answer ... ;-)

In March: Multiformat at 128kbps (with the MP3 and AAC winners)

It will be possible to make this test with the WMA9 and the WMA9 Pro with 2 pass in WM9Enc ... ?

To also put the profiles of producer used for encoding in RA8 (cook surround) and RA10 (aac and aac+) ... ?

And finally to make a test with 5.1 format ... ?

Thank you for your preceding tests which are really very interesting ... :-)

[rjamorim]

Originally posted by Sagittaire
It will be possible to make this test with the WMA9 and the WMA9 Pro with 2 pass in WM9Enc ... ?

WMA9 Pro was already tested at my former 128kbps multiformat test.
http://www.rjamorim.com/test/128extension/results.html

So, I will only test WMA9 Standard in this one.

To also put the profiles of producer used for encoding in RA8 (cook surround) and RA10 (aac and aac+) ... ?

RA8 only goes up to 96kbps, IIRC, so it's of no use for my test. And only one AAC codec will be featured, the winner of my next test.

And finally to make a test with 5.1 format ... ?

That would actually be interesting - Vorbis vs. HE AAC vs. WMA Pro vs. Real Audio vs. AC3 at, say, 192kbps - but testing would be a big problem. I would have to limit it to listeners with 5.1 systems. Also, to the best of my knowledge the testing tools available (Win32 ABC/HR, Java ABC/HR) only support stereo. So, I think it's unlikely that I will conduce such test.

Regards;

Roberto.

[Sagittaire]

Last question ...

In your 64 Kbps test it's WMA9 or WMA9 Pro?

... sorry

[rjamorim]

Originally posted by Sagittaire
In your 64 Kbps test it's WMA9 or WMA9 Pro?

I tested WMA9 standard. It's quite hard to get WMA9 Pro at such low bitrates.

... sorry

No worries, I like to discuss my tests :)

[bond]

Originally posted by rjamorim
That would actually be interesting - Vorbis vs. HE AAC vs. WMA Pro vs. Real Audio vs. AC3 at, say, 192kbps - but testing would be a big problem. I would have to limit it to listeners with 5.1 systems. Also, to the best of my knowledge the testing tools available (Win32 ABC/HR, Java ABC/HR) only support stereo. So, I think it's unlikely that I will conduce such test.wouldnt it be also correct if you would downmix all 5.1 samples to stereo wavs for listening?

the quality would stay the same and also people with no 5.1 system could join the test...

to avoid problems with different downmixers for each codec you could use one that works for all, eg matrixmixer

[rjamorim]

Originally posted by bond
wouldnt it be also correct if you would downmix all 5.1 samples to stereo wavs for listening?

Wel, the problem is that it is postprocessing. Postprocessing usually affects different codecs in different ways. It can emphasize artifacts in codec X, and hide them in codec Y. So, I believe people would complain about the validity of the results.

[bond]

imho downmixing is not more than playing front left, rear left on the left side; front right, rear right on the right side and center and lfe (?) on both...

are you sure that this influences the quality?

[rjamorim]

Originally posted by bond
are you sure that this influences the quality?

I'm not sure. But other people that are not sure either will criticize the test because of that :)

[Gabriel_Bouvigne]

With downmixing you would not even be able to discern a stereo encoder from a 5.1 encoder.

[bond]

of course, but as the test is about quality it should not matter if you listen to it on a 5.1 system or with stereo equipment qualitywise imho

[Andrey]

>>http://www.opticom.de/1_company/1profile-set.html
>>So, that seems to indicate that Opticom ~= FhG

Thnaks for the link, Roberto ! :)
I 've never read any company profile before, no joke !!! :) :)

And about 5.1 listening test - it would be really interesting !
Especially to compare not only codecs, but also bitrates...
For example, is there noticable difference when using 160, 192 and 224 Kbit with HE aac 5.1 ...

[Sagittaire]

I read the other test which you made ... really interessant too
http://www.rjamorim.com/test/128extension/results.html

http://www.rjamorim.com/test/128extension/plot04.png

and

AAC Lame MPC Ogg WMApro Blade
41_30sec 129 125 166 143 128 128
ATrain 129 121 144 139 128 128
Bachpsichord 129 125 197 173 128 128
Blackwater 129 124 130 130 128 128
death2 129 133 115 114 128 128
FloorEssence 129 129 145 147 128 128
Layla 129 124 150 155 128 128
macabre 129 122 156 149 128 128
MidnightVoyage 129 121 142 141 128 128
thear1 129 123 132 128 128 128
TheSource 129 127 130 126 128 128
Waiting 129 121 147 137 128 128
----------------------------------------------
Average 129 124.5 146.1 140.1 128 128


For death2 all codec code easily with a low bitrate but not the WMA9 Pro. I think that it is a bug in the algo 2 pass in this particular case. To solve this problem it would have been necessary to make WMV9 Pro in quality mode ...

[rjamorim]

Originally posted by Sagittaire
To solve this problem it would have been necessary to make WMV9 Pro in quality mode ...

The problem with quality mode is that the bitrates vary wildly, with samples using less than 100kbps and others using more than 160. That would surely lead to issues of fairness later on.