64kbps listening test - OPEN

[rjamorim]

Hello.

I'd like to invite all Doom9 forum members to participate in my 64kbps public listening test, that starts now.

Please head to my listening tests page, instructions are there.
http://audio.ciara.us/test/

Test ends on September 21st.

Thanks for your attention.

Best regards;

Roberto.

[bond]

join it, NOW!

[rjamorim]

Thanks, Bond

Update: A Java version of ABC-HR has been released. Now, users of MacOS, Linux, Solaris, etc. can also participate.

Further details are available at the presentation page.

Regards;

Roberto.

[rjamorim]

Hello.

The test is now finished. Here are the results:

http://audio.ciara.us/test/64test/results.html

And the final plot:



Thanks to everyone that participated and helped!

Best regards;

Roberto.

[bond]

hm, interesting results...

imho it somewhat confirms my guess that he-aac isnt ready enough qualitywise at the moment to be used for movie soundtracks @64kbps (and Ivan never responded to my suggestion to allow higher bitrates with he-aac)

[JohnV]

Originally posted by bond
hm, interesting results...

imho it somewhat confirms my guess that he-aac isnt ready enough qualitywise at the moment to be used for movie soundtracks @64kbps (and Ivan never responded to my suggestion to allow higher bitrates with he-aac)

Well, depends. It's better than Vorbis -q0 which some people use also..
But I think you have some point here. I'll poke Ivan about this.

[bond]

Originally posted by JohnV
I'll poke Ivan about this.

would be great

[JohnV]

Originally posted by bond
would be great

Ivan said that it's in his to-do list now.

[Doobie]

The two top codecs use SBR. They're really just severely frequency limited codecs with SBR fudging higher frequences. It really makes me question their quality when their high marks are a result of fudging. I'm sure people mistake a high frequency component with quality.

Maybe it doesn't matter as long as you think SBC sounds better, like you might think a cheap bookshelf speaker sounds better than an expensive audiophile speaker that has a little less thud at very low frequencies. But, don't confuse this with the codec's true quality.

As you give the codecs more bits to work with, they'll be preserving more high frequencies and thus the poor codecs will have to rely more on their quality and less on their "cheating." So, for example, don't expect MP3Pro to be any better than regular MP3 at 256Kb/s even though you think MP3Pro trounces MP3 at 64Kb/s. Vorbis, which falls below the SBR codecs at low bitrates could very will sound better at higher bitrates.

You learned from this test that all codecs are poor at 64Kb/s so you should be using them at higher bitrates anyway.

[Teegedeck]

I really think you're confusing things here; all participants are lossy codecs using psy-models. There's no sense in calling the techniques of one codec 'cheating'. There is nothing 'true' in the quality of an old codec at a steep bitrate. You could as well use a losless codec and call this 'true quality' with some justification.

But we are talking streaming-bitrates here, and LAME, especially at 256 kbps, isn't quite suited for that.

The setup of rjamorim's test is the only setup that has a meaning: Asking how close to the original the results sound subjectively.

[JohnV]

Originally posted by Doobie
The two top codecs use SBR. They're really just severely frequency limited codecs with SBR fudging higher frequences. It really makes me question their quality when their high marks are a result of fudging. I'm sure people mistake a high frequency component with quality.

Maybe it doesn't matter as long as you think SBC sounds better, like you might think a cheap bookshelf speaker sounds better than an expensive audiophile speaker that has a little less thud at very low frequencies. But, don't confuse this with the codec's true quality.

As you give the codecs more bits to work with, they'll be preserving more high frequencies and thus the poor codecs will have to rely more on their quality and less on their "cheating." So, for example, don't expect MP3Pro to be any better than regular MP3 at 256Kb/s even though you think MP3Pro trounces MP3 at 64Kb/s. Vorbis, which falls below the SBR codecs at low bitrates could very will sound better at higher bitrates.

You learned from this test that all codecs are poor at 64Kb/s so you should be using them at higher bitrates anyway.

Nobody is suggesting SBR for higher bitrates. Just a vbr profile or two to cover maybe 70-110kbps..
I agree with Teegedeck, whole lossy audio is based on different kind of psychoacoustic and other techniques to cheat the ears. SBR is just one technique among others, vorbis has its own "cheating" tricks (for example its noise normalization)..

[duartix]

Olá rjamorim.

Can you please post filesize details, as you have done with the 128kbps test, so we can draw some conclusions on how the VBR codecs stay within the bandwith?

I'm asking this because if you take the 128kbps test into perspective, while MPC was a only litle above AAC, it was doing it at the expense of 146 kbps while AAC stayed close to the 129 kbps objective.

This 11% diference in bitrate at almost the same quality could be enough to make me think on switching from MPC to a good AAC codec if only there was some kind of tagging suport.

[bond]

Originally posted by JohnV
Nobody is suggesting SBR for higher bitrates. Just a vbr profile or two to cover maybe 70-110kbps..
-
Ivan said that it's in his to-do list now.

yummie

[nemoxnine]

i'm sorry, maybe i'm stupid but this makes no sense to me. lame is in first place a 1.94 and HE AAC is in second at 4.29? can somebody please explain why you jump from the lowest score to the highest and then continue down?

[JohnV]

Originally posted by nemoxnine
i'm sorry, maybe i'm stupid but this makes no sense to me. lame is in first place a 1.94 and HE AAC is in second at 4.29? can somebody please explain why you jump from the lowest score to the highest and then continue down?

You are reading it wrong. Lame 128 abr (the high anchor, pretty must the best what MP3 can do at average of 128) scores 4.29. Nero HE AAC scores 3.68.

[rjamorim]

Originally posted by duartix
[B]Olá rjamorim.

Can you please post filesize details, as you have done with the 128kbps test, so we can draw some conclusions on how the VBR codecs stay within the bandwith?

Saudações, duartix!

I just uploaded the bitrate table, it's at the results page just after the individual plots.

Enjoy.

Regards;

Roberto.