Repeated lossy-encoding test: AAC, Opus, MP3, and MP2

[GeoffreyA]

In reference to a recent discussion (https://forum.doom9.org/showthread.php?p=2021163#post2021163), I gave the repeated lossy-encoding test a go. The results are interesting, not that it matters for encoding in practice.

Except for QAAC, all encoding was done with FFmpeg, using the relevant libraries. 101 iterations were performed. QAAC and Opus seem to be the winners, though neither escapes unscathed. Warning for your ears in all cases except QAAC, FDK, and Opus!

https://workupload.com/archive/ARup4FAjnQ

ffmpeg -i REF.flac -c:a aac -b:a 128k a.m4a

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.m4a -c:a copy tmp.m4a
ffmpeg -y -i tmp.m4a -c:a aac -b:a 128k a.m4a

)


/////////////////////////////////////////////////////////////////////////

zzffmpeg -i REF.flac -c:a libfdk_aac -b:a 128k a.m4a

for /l %%x in (1, 1, 100) do (

zzffmpeg -y -i a.m4a -c:a copy tmp.m4a
zzffmpeg -y -i tmp.m4a -c:a libfdk_aac -b:a 128k a.m4a

)


/////////////////////////////////////////////////////////////////////////

d:\tmp\av\qaac\qaac64 --cbr 128 REF.flac -o a.m4a

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.m4a -c:a copy tmp.m4a
d:\tmp\av\qaac\qaac64 --cbr 128 tmp.m4a -o a.m4a

)


/////////////////////////////////////////////////////////////////////////

d:\tmp\av\qaac\qaac64 --tvbr 64 REF.flac -o a.m4a

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.m4a -c:a copy tmp.m4a
d:\tmp\av\qaac\qaac64 --tvbr 64 tmp.m4a -o a.m4a

)


/////////////////////////////////////////////////////////////////////////

ffmpeg -i REF.flac -c:a libopus -b:a 128k a.opus

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.opus -c:a copy tmp.opus
ffmpeg -y -i tmp.opus -c:a libopus -b:a 128k a.opus

)


/////////////////////////////////////////////////////////////////////////

ffmpeg -i REF.flac -c:a libopus -b:a 128k -vbr constrained a.opus

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.opus -c:a copy tmp.opus
ffmpeg -y -i tmp.opus -c:a libopus -b:a 128k -vbr constrained a.opus

)


/////////////////////////////////////////////////////////////////////////

ffmpeg -i REF.flac -c:a libmp3lame -b:a 128k a.mp3

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.mp3 -c:a copy tmp.mp3
ffmpeg -y -i tmp.mp3 -c:a libmp3lame -b:a 128k a.mp3

)


/////////////////////////////////////////////////////////////////////////

ffmpeg -i REF.flac -c:a libtwolame -b:a 128k a.mp2

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.mp2 -c:a copy tmp.mp2
ffmpeg -y -i tmp.mp2 -c:a libtwolame -b:a 128k a.mp2

)


The test is not rigorous, and these were separate scripts that I pasted together above.

[tebasuna51]

It has always been said that it is a bad idea to recode what has already been recoded, but it is comforting to know that the free software opus and the recommended qaac acceptably withstand the challenge.

[GeoffreyA]

Today, I found that using FDK in VBR mode brings the quality up to QAAC. But when both are CBR, QAAC does have the edge.

FFmpeg's encoder is concerning: even after 10 iterations, it degrades considerably.

[Z2697]

FFmpeg's native AAC is even worse than FAAC, in terms of generational loss. Maybe the overall quality as well.
It's extra concerning that it's probably very often used by, um, "average joe" (amonst FFmpeg users)... which are exactly the ones that are more likely to re-encode lossy audio over and over...
Yes, commercial softwares (which probably have more user base) probably gonna have their own or licensed encoder, but the lack of high quality FOSS AAC encoders is still disappointing.

[GeoffreyA]

I think many videos "out there" have used FFmpeg's AAC,* and this test shows there are considerable issues when it comes to generational loss. With a single encoding, it sounds all right these days, but distortion can often be detected in challenging parts. Anecdotally, I've found that older encodings, likely FFmpeg AAC when it was worse, tend to give me a headache because of the strange stereo distortions.

* Doesn't YouTube use FFmpeg for their AAC tracks?

[SeeMoreDigital]

Out of interest...

Where does Nero's v1.5.4.0 AAC-LC encoder (from 2010) stand in the rankings when compared the other AAC encoders?

[GeoffreyA]

Out of interest...

Where does Nero's v1.5.4.0 AAC-LC encoder (from 2010) stand in the rankings when compared the other AAC encoders?

Using a binary from Free Codecs (https://www.free-codecs.com/nero_aac_codec_download.htm) and the following script:


neroAacEnc -q 0.4 -if REF.wav -of a.m4a

for /l %%x in (1, 1, 100) do (

ffmpeg -y -i a.m4a tmp.wav
neroAacEnc -q 0.4 -if tmp.wav -of a.m4a

)

It doesn't fare too well, falling behind FDK but ahead of FFmpeg, displaying pervasive "tinkling."

QAAC > FDK > Opus > Nero > MP3 > MP2 > FFmpeg AAC

Here are the updated files (https://workupload.com/archive/z9J4UT4Yct), where all are using VBR except for FFmpeg AAC and MP2.

This is perhaps consistent with HydrogenAudio's ranking of AAC encoders (https://wiki.hydrogenaudio.org/index.php?title=Advanced_Audio_Coding) for ordinary encoding (unlike here):

QAAC > FhG > FDK > Nero > FFmpeg > FAAC > Libav

[j7n]

I am curious how things would change without latency compensation or random delay where the audio blocks in input and output no longer precisely overlap, which would happen if you cut the stream per some requirements and edited it in any way.

[GeoffreyA]

I am curious how things would change without latency compensation or random delay where the audio blocks in input and output no longer precisely overlap, which would happen if you cut the stream per some requirements and edited it in any way.

Here is QAAC with and without delay: https://workupload.com/archive/nf9VmJMkKg.

I used the --no-delay setting. Not sure if that would be enough.

[VoodooFX]

QAAC > FDK > Opus > Nero > MP3 > MP2 > FFmpeg AAC

Is this from ABC/H blind ranking?

[GeoffreyA]

Is this from ABC/H blind ranking?

No. The differences were audible enough that it was reasonable, if not ideal, to rank them without blind testing, which I'm bad at.

[VoodooFX]

No. The differences were audible enough that it was reasonable, if not ideal, to rank them without blind testing, which I'm bad at.

Mind can play tricks on you, always do blind testing.
Here is ABC/H tool if you don't have it already:https://www.swisstransfer.com/d/892c20b7-998a-4ff3-96e4-d64102e7a747

[GeoffreyA]

Mind can play tricks on you, always do blind testing.
Here is ABC/H tool if you don't have it already:https://www.swisstransfer.com/d/892c20b7-998a-4ff3-96e4-d64102e7a747

That's right. I've used fb2k for blind testing in the past, but my brain tends to get lost. I'll give the linked tool a try when I get a chance. Thanks.

[VoodooFX]

When you rank them do ABX test [you'll see the button] 8-10 times for 1st vs 2nd places.