ffmpeg ac3 built-in encoder quality ?

[Ely]

Hello everyone,

I noticed that ffmpeg has a built-in ac3 encoder, but before I start converting my medias, I'd like to know if you guys know about its quality ?

I haven't been able to find anything on google regarding the efficiency of this encoder. Should I stick with it or do you recommend I use another tool to encode the audio tracks ?

Thanks !

[Ely]

I went ahead and converted some movies to try it out, so far I can't tell the difference between the source (dts, 5.1, 1536 kb/s) and the resulting encode (AC3, 5.1, 448 kb/s). Then again I don't have a super high-quality surround sound system, so take my words with a grain of salt..

[tebasuna51]

If you need AC3 for compatibility, ffmpeg is the recommended free AC3 encoder.

If source is DTS 1536 Kb/s the recommended output AC3 bitrate is 640 Kb/s, use 448 Kb/s for DTS 768 Kb/s.

[manolito]

If source is DTS 1536 Kb/s the recommended output AC3 bitrate is 640 Kb/s

Not if you want to use it for DVD creation, then the max AC3 bitrate is 448 Kb/s


Cheers
manolito

[tebasuna51]

Not if you want to use it for DVD creation, then the max AC3 bitrate is 448 Kb/s

Yeah..., but I can't understand for what burn DVD's with MPEG2 video today.

[manolito]

Maybe for folks like me who refuse to buy a BD player and who own a CRT TV which has no HD capabilty? :p

Cheers
manolito

[Ely]

Thank you for your inputs ;) .

I'm actually at a shortage of disk space so I'm converting most of my HQ movies from H.264/DTS to HEVC/AC3. I tried AC3 640 kb/s instead of 448 but couldn't tell the difference at all on my sound system, so I'm gonna go for 448.

[tebasuna51]

@manolito
Many DVD players, or others cheap standalone players, can play AVI with Xvid (MPEG4-ASP better than MPEG2) in CRT TV's.

@Ely
If your sound system can play AAC (Qaac or NeroAacEnc) or E-AC3 (ffmpeg) you can use 320 Kb/s better than AC3 448 Kb/s.

[manolito]

@manolito
Many DVD players, or others cheap standalone players, can play AVI with Xvid (MPEG4-ASP better than MPEG2) in CRT TV's.


MPEG4 is more efficient than MPEG2, sure. But better? I have never seen an XviD clip which looked better than MPEG2. (Plus you do not have menus and chapters in XviD)


Cheers
manolito

[nevcairiel]

More efficient also implies better. If you can reach the same quality at a lower bitrate, then its more efficient (or if its the same bitrate, quality improves).
Efficiency for a video codec is just quality per bits. Thats really all that makes a video codec "good" in the first place. :)

[pandy]

Use this topic as i have same question only different point of view - any explanation why ac3 codec in ffmpeg have poor SNR? (currently looks like SNR is limited somehow to approx 15 bits).

And as such any open source ac3 codec with SNR approx 100dB or better?

[nevcairiel]

15-bit is already around 90dB SNR, so the difference isn't really that big. Should stick to the same unit when comparing.
When testing ffmpeg, make sure to use the floating point ac3 encoder, the fixed-point encoder takes 16-bit input, which could easily explain such results.

[pandy]

15-bit is already around 90dB SNR, so the difference isn't really that big. Should stick to the same unit when comparing.
When testing ffmpeg, make sure to use the floating point ac3 encoder, the fixed-point encoder takes 16-bit input, which could easily explain such results.

Well - i used float ac3 to encode and default ac3 to decode (to 24 bits with HP TPDF).
15 bit is too low when you testing 16 bit system and native ac3 encoder from Dolby providing over 20 bit accuracy (more than 120dB).

[tebasuna51]

When testing ffmpeg, make sure to use the floating point ac3 encoder, the fixed-point encoder takes 16-bit input, which could easily explain such results.
How we can be sure of that?

[Ely]

How we can be sure of that?

-acodec ac3 will use floating point math, while -acodec ac3_fixed will use fixed point.

[pandy]

-acodec ac3 will use floating point math, while -acodec ac3_fixed will use fixed point.
Yep, by default float ac3 encoder is used, integer need to be called explicitly but there is no 100% sureness as this is internal ffmpeg life - don't feel so advanced to verify real codec selection...

For sure there is something wrong with ac3 SNR implementation in ffmpeg - it is comparable to ancient mpeg 1 layer 2 (both codecs).

mp2 -ac 2 -b:a 192k
libtwolame -mode stereo -ac 2 -b:a 192k

they give similar SNR results when compared to ac3

I need sufficient SNR (100dB or more) as i need to create pure sine wave for measuring SNR in hardware device - you can't verify 24 bit DAC with 15 bit quality signal... (yes, frequencies are selected in a way to exploit unique characteristic of codec - so called critical frequencies to give pure spectrum).

[tebasuna51]

@pandy
Please explain your test to measure the SNR.
Is know than AC3 is not a lossless encoder, then I don't understand what is a 15 bit SNR.

Have you tried a commercial AC3 encoder?
What means "native ac3 encoder from Dolby providing over 20 bit accuracy"?
You say than is lossless until 20 bit?

[pandy]

@pandy
Please explain your test to measure the SNR.
Is know than AC3 is not a lossless encoder, then I don't understand what is a 15 bit SNR.

Have you tried a commercial AC3 encoder?
What means "native ac3 encoder from Dolby providing over 20 bit accuracy"?
You say than is lossless until 20 bit?

Ok, i will try to explain but please accept apologies for my English.

For simple sine wave lossy encoding is quasi lossless - for example -20 dBFS 1kHz sinewave shall be lossless with exception related to level (i.e. it may be not perfect -20dBFS at the output as quantization is not uniform as LPCM but use semitresholds).

SNR quality od encoder is easy to measure - this is noise floor produced at the decoder side with assumption that decoder is bitexact (i can use native Dolby decoder which is part of Dolby test suite) - so based on this looks like noise floor for ffmpeg encoder is somehow limited to approx 15 bit.

Lossy encoding is only for complex spectral signal - sinewave should produce (especially with particular frequencies) very accurate lossy free function - DCT can be seen (in inverse DCT) as signal generator - similar concept as for vocoders (bank/set of bandpass filters) - output signal is re-synthesized combination of sinwaves.

To verify dynamics i use Dolby test streams called:
2_aswp4ka.ac3 and 2_200a60.ac3.

Based on Dolby description:
2_aswp4ka.ac3 - 4 kHz sine wave amplitude sweep from 0 to –120 dBFS. 20-bit - Tests THD vs. Level at 4 kHz. Also used for linearity check.

2_200a60.ac3 - Single tone 200 Hz –60 dBFS. 20-bit precision. - These signals are provided to test the ability to decode to the correct spectrum with various input frequencies and levels. Additional uses are spot checking compression, distortion, or linearity.

So yes - i believe DCT coefficients should be accurate enough to provide more than 100dB precision (20 bit is approx -120dB and even going for audio safe 24 bit int everything should work correctly) and as such at the output noise floor shall be way better than -90dB.

Please remember that i'm talking not about complex spectral signal but pure sine which after DCT in special conditions will produce single value and this value feed to iDCT should provide exact (spectrally) sine wave (and there will be only PCM quantization problems).
--
Made some test:

@ffmpeg -f lavfi -i "sine=frequency=200:sample_rate=48000:duration=60,volume=-41.94dB,pan=stereo|c0=c0|c1=c0" -c:a pcm_s32le -y 200a60_ffmpeg.wav
@ffmpeg -i "200a60_ffmpeg.wav" -vn -c:a ac3 -b:a 192k -y 200a60_ffmpeg.ac3
@ffmpeg -i "200a60_ffmpeg.ac3" -vn -c:a pcm_s32le -y 200a60_ffmpeg_32.wav


and with sox

@set dyna=145
@set /a clut=%dyna%/5
@sox --multi-threaded --buffer 131072 -S -V -D %1 -n spectrogram -z %dyna% -w Kaiser -q %clut% -y 1025 -x 2048 -s -o %1_%dyna%.png stat stats -b 16

analyzed results:

http://s18.postimg.org/7liy3co3d/2_200a60_ac3_32_wav_145.png
http://s18.postimg.org/v9yfymkmx/200a60_ffmpeg_wav_145.png
http://s18.postimg.org/ulplfolx5/200a60_ffmpeg_32_wav_145.png

It looks like i was wrong - lopping inside ffmpeg is not bad but i have strange (significantly worse) results on real HW when comparing both streams....

[tebasuna51]

I don't know how you use sox to know SNR.

My test using your source and encoder:
ffmpeg -f lavfi -i "sine=frequency=200:sample_rate=48000:duration=60,volume=-41.94dB,pan=stereo|c0=c0|c1=c0" -c:a pcm_s32le -y 200a60_ffmpeg.wav
ffmpeg -i "200a60_ffmpeg.wav" -vn -c:a ac3 -b:a 192k -y 200a60_ffmpeg.ac3

To decode I use libav but with:
eac3to 200a60_ffmpeg.ac3 output.wav -full
To obtain the full native libav output 64 float.

Compared the output.wav with input 200a60_ffmpeg.wav the max differences are at -100.8 dB.

With Sonic AC3 encoder (max quality input 24 bit int), same decoder, the differences are at -94 dB. Seems ffmpeg is better with that sample.

[Groucho2004]

A little test I ran with SpectraPlus:

Source file is a 24 Bit 48 KHz PCM with a 0 dB, 1 KHz signal:
http://s22.postimg.org/nc9zr7zmp/pcm24.png

Convert to 192 Kbps stereo AC-3:
ffmpeg -i "pcm24.wav" -vn -acodec ac3 -ac 2 -ar 48000 -ab 192k "ffmpeg.ac3"

And back to PCM:
eac3to.exe "ffmpeg.ac3" "ffmpeg.wav" -down24

Result:
http://s15.postimg.org/6ogxmfb7v/ffmpeg192.png

Result with Aften:
http://s28.postimg.org/e0gglbb8d/aften.png

[pandy]

I don't know how you use sox to know SNR.

THis is simple - spectrogram shows levels as color - on right graph side there is stripe with colors and scale in dBFS - just compare color, black means there is floor (lowest possible scale value). Original pictures are bigger but after upload they was downscaled and as such blurry - not so important as they don't change overall result - seem that ffmpeg encoder use some loudnes contour and can provide more than 24 bit quality at parts where ear are the most sensitive.

However picture change with IMD stimulus (two high level sine waves separated by some fixed distance for example 1kHz - so for example 18 and 19 kHz - there is real mess on spectrum plot - but i need to try to play with lower level and see if this is maybe level related distortion or change frequencies - real frequencies are selected to be equal to critical layer II frequencies - in my case sine 17625Hz and sine 19875Hz - combined and limited to -3.103dBFS to avoid clipping in filter bank).



My test using your source and encoder:


To decode I use libav but with:
eac3to 200a60_ffmpeg.ac3 output.wav -full
To obtain the full native libav output 64 float.

Compared the output.wav with input 200a60_ffmpeg.wav the max differences are at -100.8 dB.

With Sonic AC3 encoder (max quality input 24 bit int), same decoder, the differences are at -94 dB. Seems ffmpeg is better with that sample.

Well, yes - thanks to loudness contour seem it is better...

Don't use float as my samples have fully predictable dynamics and INT are more stable than float - but once again - picture for IMD signal change dramatically - not sure if this is codec limitation or not - need to understand how AC-3 MDCT theoretically will affect IMD signal...
Sadly to say - HW is capable to accept PCM but SW support only MPEG 1 Layer II, AC-3, EAC-3 and AAC and freeware AAC is really disastrous on terms of the SNR.

[nevcairiel]

Note that sine waves are a bit special and a bunch of encoders behave a bit weirdly with them, since that's not what they are optimized for. Just something to keep in mind.

[Ely]

freeware AAC is really disastrous on terms of the SNR.

Even Fraunhofer FDK ? (libfdk_aac)

That's my go-to AAC encoder and I love it. Haven't done any SNR test on it though, only listening tests.

[raffriff42]

Haven't done any SNR test on it though, only listening tests.
No need to feel apologetic about that - it is how the codecs are developed in the first place. MP3/History/Development (https://en.wikipedia.org/wiki/MP3#Development) (wikipedia)
The song "Tom's Diner" by Suzanne Vega was the first song used by Karlheinz Brandenburg to develop the MP3. Brandenburg adopted the song for testing purposes, listening to it again and again each time refining the scheme, making sure it did not adversely affect the subtlety of Vega's voice.

OK that's MP3 - ancient history - but I doubt listening tests have been replaced as the final judge of quality. There are precise measurements being made in the development labs, certainly, but they have to do with auditory masking, response of the human ear, etc - psychoacoustics (https://en.wikipedia.org/wiki/Psychoacoustics). Measuring SNR while ignoring auditory masking will not tell you very much about how good the codec sounds.

Oh, BTW, the Hydrogen Audio wiki (http://wiki.hydrogenaud.io/index.php?title=AAC#Current) lists the following encoders in order, best to worst:
1. Apple AAC
2. Fraunhofer FDK AAC
3. Nero AAC
4. FAAC
5. FFmpeg/Libav AAC encoder

[pandy]

Note that sine waves are a bit special and a bunch of encoders behave a bit weirdly with them, since that's not what they are optimized for. Just something to keep in mind.

How they are special (except they are way simpler to encode)?

If your spectrum is simple and something wrong happen with codec this mean that your codec design is faulty...

Sine wave is same signal like other - main difference is that MDCT coefficients will be null except particular one... i know that some decoders introduce spectral dither substituting null data with random low level but... i see nothing wrong in this.

Even Fraunhofer FDK ? (libfdk_aac)

That's my go-to AAC encoder and I love it. Haven't done any SNR test on it though, only listening tests.

Well... seem that AAC perform worse than simpler codecs and as a result lower SNR (higher noise floor) is offered - i need to check all AAC encoders available to compare - currently have no time for AAC as most of hardware i testing support by definition MP2 and AC3 but in IMD test MP2 provide better results than AC3...
Also be aware that testing of codec is not my intention - i need most transparent codec to provide sinewave clean as possible to test physical HW.
Problem is not related to bitrate or similar things - obviously some encoders perform worse when compared to others on same signal (IMD is high stress from HW perspective but it should be not a problem from codec perspective).

[nevcairiel]

How they are special (except they are way simpler to encode)?

If your spectrum is simple and something wrong happen with codec this mean that your codec design is faulty...

Sine wave is same signal like other - main difference is that MDCT coefficients will be null except particular one... i know that some decoders introduce spectral dither substituting null data with random low level but... i see nothing wrong in this.

Lossy codecs are designed/tuned for common use-cases, voices, music, etc. Sine waves do not fall into this pattern, and can trip up these optimizations.

[pandy]

Lossy codecs are designed/tuned for common use-cases, voices, music, etc. Sine waves do not fall into this pattern, and can trip up these optimizations.

This is simply untrue as codec doesn't understand nature of signal - lossy codec works under same, strictly mathematical criteria and sine wave is same type of signal like music - it is only simpler to encode as with even low bitrate there is plenty of available to use bits to store compressed signal...

Accordingly to my knowledge there is no codec on market that use neural networks trained to ignore simple tones...

And on a side - some instruments are near to sine spectrum... i bet that i can add white noise to those two tones and result will be the same... faulty codec implementation have nothing to do with spectral masking and perceptual tuning.

[Balling]

User on Reddit has posted the quality of eac3 ffmpeg encoder vs "Dolby Media Encoder" application, took it a while until -drc_scale 0 was used to remove DRC on encoded by Dolby encoder (ffmpeg encoder does not support DRC and thus there is no need to use drc_scale 0 on decoder) and 256 sample priming gap that both encoders insert and unless put in mp4 by ffmpeg it is not removed... Google test zimtohrli suite was used.


https://www.reddit.com/r/ffmpeg/comments/k3z9i9/comment/m3zjt4h/?force-legacy-sct=1


Quote here in full.

My past zimtohrli measurements were all somewhat inaccurate, because ffmpeg decodes (e)ac3 with all samples shifted by 256 samples forward in time, adding ~11ms of silence at the beginning, basically the same issue that formats like MP3 and AAC have as well, so I should have expected that. But with `-drc_scale 0` for decoding and then filtering with `-af aresample=first_pts=256`, the scores should be accurate now.
While zimtohrli should highly correlate with human perception, it's worth noting that it is a pure per-channel metric, so it might unfairly punish coding techniques that involve, for example, reducing quality on one channel if the sound is overshadowed by louder sound from the other channels.

Anyway, here are the zimtohrli scores for the full audio of the Tears of Steel short movie; I removed the LFE channel, and the higher score of each FL/FR and BL/BR pair (which never deviated much from each other anyway):

fdk-aac 384k cbr aac (44.1 kHz):
4.9816 (FL FR)
4.99923 (C)
4.97722 (BL BR)

fdk-aac 384k cbr he-aac:
4.96607
4.99963
4.98321
(this high score surprised me, I would have assumed the static SBR threshold of 12 kHz to be a bit more limiting, but maybe the sample here just didn't contain much very high frequency harmonics)

Dolby 384kbps ac3:
4.72085
4.84867
4.72297

FFmpeg 384kbps ac3:
4.70113
4.80505
4.69726
(FFmpeg holds up extremely well here compared to the Dolby encoder, but looking at the AAC scores, I assume that a significantly better AC3 encoder is possible)

fdk-aac 256k cbr aac (44.1 kHz):
4.89187
4.98928
4.88925

fdk-aac 256k cbr he-aac:
4.93956
4.99519
4.94692
(again very impressive results from the fdkaac encoder)

Dolby 256k ac3:
4.65848
4.79297
4.65313

FFmpeg 256k ac3:
4.62544
4.72272
4.60867

Dolby 256k eac3:
4.62671
4.29229
4.57617
(the center channel gets a rather low score here, listening to it in isolation, the issue is very clear, there is a significant amount of bleeding from the other channels)

Dolby 256k Blu-ray eac3:
4.65497
4.60828
4.58956
(surprisingly, the Blu-ray eac3 profile from the Dolby encoder is more efficient across all channels)

FFmpeg 256k eac3:
4.6278
4.74468
4.62329
(very surprisingly, the FFmpeg eac3 encoder holds up very well with the Dolby competition here!)


fdk-aac 192k cbr aac (44.1 kHz):
4.84574
4.96756
4.84471

fdk-aac 192k cbr he-aac:
4.83493
4.97991
4.83677
(I expected HE-AAC to perform slightly better than AAC-LC at such a lowish bitrate for 5.1, not worse, but maybe it's just the sample...)

Dolby 192k eac3:
4.53272
3.75264
4.38356
(severe bleeding into the center channel)

Dolby 192k Blu-ray eac3:
4.57572
4.44936
4.43095
(again, the Blu-ray profile performs better than the online media profile eac3 on all channels for some reason)

FFmpeg 192k eac3:
3.26063
4.34128
3.69626
(this is some abysmal quality, especially on the most important FL/FR channels; and listening to it, the main issue quickly becomes very clear, the sound is just extremely muffled, and most likely very unnecessarily so)

So, turns out the FFmpeg eac3 encoder is not necessarily bad, it's just really broken at lower bitrates, it seems.

[tebasuna51]

Thanks for the tests. I was unaware of the Zimtohrli method for verifying quality.
This method seems to confirm the higher quality/size of the AAC (fdkaac) codec compared to AC3/EAC3. I still have some doubts about the supported bandwidth, so I ran some tests.

Using Tears of Steel.ogg 840 Kb/s as the source (the best I could find), I analyzed the M4A/AC3/EAC3 encodings with Spek to see the cutoff frequencies they retain.

I focused on bitrates around 384 Kb/s, which seem close to transparency.

File Size Kb/s min./avg./max KHz
----------------- ----------- ------------------ ----
tears.wav 634.181.828 22
tears_4.m4a 36.386.939 12 / 397 / 927 16.6
tears_384.m4a 35.379.365 312 / 386 / 533 16.2
tears_HE_384.m4a 35.313.900 306 / 385 / 470 22 SBR
tears_dd.ac3 35.232.768 384 18.2
tears_ff.ac3 35.232.768 384 18.2
tears_ff.eac3 35.232.768 384 18.2
tears_dd.ec3 35.232.768 384 18.2

As you can see, the AC3/EAC3 formats all maintain the same acceptable cutoff frequency.

The HE format is surprising because, with its SBR replication, it retains the full bandwidth, even though it was never recommended for high bitrates. It seems the Zimtohrli method disagrees with that.

However, I would request an analysis using that method for VBR encoding at quality 4 with fdkaac. Tears of Steel maintains demanding audio quality throughout its duration, but most of the soundtracks have many silences and only dialogue, which would recommend VBR encoding to save bitrate.

[Balling]

The original person was using this sample, forgot to mention it. https://media.xiph.org/tearsofsteel/tearsofsteel-surround.flac
I also opened the bug for this. It is likely that slightly worse quality in AC3 in ffmpeg is what leads to Blu-ray profile being better in EAC3+AC3 Blu-ray profile in Dolby encoder.
https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23317

The most important point here is that already on 256k eac3 in ffmpeg is MUCH BETTER than Dolby's unless you use Blu-ray profile. This puts the use of Dolby encoder at all under question.

[j7n]

I think bleeding is an expected result of channel coupling in the high frequencies when the target bitrate is low, which would not be as noticeable when listening to the complete mix.

Last two images at 384 kbit/s:
https://imgur.com/a/Vq5vl7u

[Balling]

"fdk-aac 192k cbr he-aac:
4.83493
4.97991
4.83677
(I expected HE-AAC to perform slightly better than AAC-LC at such a lowish bitrate for 5.1, not worse, but maybe it's just the sample...)"

This will be fixed by https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20672

[tebasuna51]

The most important point here is that already on 256k eac3 is MUCH BETTER than Dolby's unless you use Blu-ray profile. This puts the use of Dolby encoder at all under question.
It doesn't make sense that the Blu-ray profile (core AC3) has higher quality than pure EAC3; pure EAC3 encoding should be more efficient.

Balling, could you please run the Zimtohrli test for fdkaac -m 4?

[DJ Bobo]

That the industrial EAC3 encoder can achieve near transparency at 448kbps, is an established fact since 2007 (!), as per the study done by the EBU back then (screenshot)
They didn't test 512kbps, but it is clear that the industrial Dolby Digital Plus encoder achieves transparency at this bitrate.

Testing 384k DD against 384k AAC is again pointless, as you can see that even 320k AAC beats 384k DD by a comfortable margin (and 448k DD by a small margin), but 256k AAC can't. So these Zimtohrli scores don't seem to be very accurate.
But I wouldn't care much about AAC 5.1 encoding quality anyway, as virtually no soundbar can handle AAC streaming, so it is absolutely pointless imho.
Most important is AC3 quality, as this is the most compatible setting (followed by EAC3, which is supported by newer soundbars, and can achieve near 384k DD quality at 256k already as per the aforementionned EBU study)

The question is: is there a consumer AC3 encoder that can achieve transparency at 640kbps or not?
I am emphasizing "consumer" here, as industrial encoders seem to apply all kinds of transformations to input levels, with all their (complicated) settings (apart from the fact that they are too expensive), where consumer encoders don't tamper with the dB levels at all - at least not the TMPGEnc encoder: I check the dB and RMS levels, they are identical for the input and the output.
Problem with the TMPGEnc Dolby encoder is a cut-off at 18 KHz, even when I choose 640k; so obviously not transparent (even if I don't hear the difference with the DTS-HD source, as I can't seem to hear beyond 18 KHz :rolleyes: )
Last time I checked FFMPEG quality, I wasn't very pleased (kinda muffled). TMPG seemed better to me.

[Balling]

It doesn't make sense that the Blu-ray profile (core AC3) has higher quality than pure EAC3; pure EAC3 encoding should be more efficient.

Balling, could you please run the Zimtohrli test for fdkaac -m 4?

Yes, it does make sense because a) clearly bugs in pure EAC3 in Dolby encoder, b) perfectly polished AC3 encoder means that in 5.1 EAC3+AC3 Blu-ray profile the front left, front right and center are reproduced well c) there is no EAC3 dependent stream support for 7.1 in ffmpeg anyways.

Dolby 384kbps ac3:
4.72085
4.84867
4.72297


"But I wouldn't care much about AAC 5.1 encoding quality anyway, as virtually no soundbar can handle AAC streaming, so it is absolutely pointless imho"

Modern TVs can decode 7.1 flac or 5.1 flac correctly and aac 5.1 too and send it over eARC back in LPCM. They use ffmpeg often too, you know. And Lavfilters can bitstream truehd+Atmos.

"Last time I checked FFMPEG quality, I wasn't very pleased (kinda muffled). "

Yes. But this is on 192k. At 256k already nice quality. DRC on the industrial encoders can be disabled on modern AVRs.

[DJ Bobo]

Modern TVs can decode 7.1 flac or 5.1 flac correctly and aac 5.1 too and send it over eARC back in LPCM
I noticed that my TV can indeed decode AAC 5.1 and send it as Dolby Digital 5.1 to my soundbar if I read from a USB drive connected directly to the TV, but the moment you read from disc, it's over, unless you have eARC as you said to get multichannel PCM, but there is still a solid base of TVs and soundbars that can't do eARC.
But even then, I would rather take Dolby Digital Plus, as it is much better than AAC, and does not require eARC (as you can see on the graph above, huge difference between DD+ and AAC at 256kbps) - but from what I understood, the FFMPEG DD+ encoder is nowhere near that quality?
I would like to add, some soundbars sound worse with PCM than with Dolby. That's the case with Panasonic soundbars at least, where if I bitstream the Dolby Digital track, it sounds more dynamic than the same track decoded by the player and sent as PCM. Probably some DSP routines that kick in only when fed a Dolby or DTS track.

"Last time I checked FFMPEG quality, I wasn't very pleased (kinda muffled). "
Yes. But this is on 192k. At 256k already nice quality.
To be fair, TMPGEnc is not very good at 192kbps Stereo either: it cuts off at 16KHz already at that bitrate, and I need to go up to 256k to get the full 20KHz spectrum.
I asked the Pegasys support as to why that is, even though industrial encoders achieve full 20KHz at 192k already (and at 448k for 5.1); they told me, they don't have any influence on that, the codec being given by Dolby as a black box.
So Dolby is probably doing this on purpose so that you don't get a HQ encoder at a fraction of the cost of their professional encoders.

[tebasuna51]

Using the FLAC (thanks, Balling), I repeated the tests with similar results.
File Size Kb/s min./avg./max KHz
----------------- ----------- ------------------- ----
tears.flac 346.994.847 3782 21.8
Tears_HE_384.m4a 35.313.900 304 / 385 / 484 21.8 SBR
Tears_f384.m4a 35.379.365 317 / 386 / 532 16.2
Tears_fm4.m4a 36.069.737 12 / 394 / 895 16.6 m 4
Tears_q384.aac 36.427.926 13 / 397 / 756 17.6 CVBR
tears_dd/ff.e/ac3 35.232.768 384 18.2

I added a 384kbps CVBR QAAC encoding, which yields similar results to fdkaac -m 4. As you can see, using variable bitrate provides twice the maximum bitrate of the average bitrate, which undoubtedly optimizes quality.

It seems clear that AAC should be the preferred format. This is also supported by the EBU study, the only one we've seen for some time on multichannel audio.

Another issue is compatibility with our sound systems (soundbars, AVRs) and players (TVs, PCs, DVD/BD players, etc.).

And the next question, and the subject of this thread, is whether, if we only have a system compatible with AC3/EAC3, the open-source ffmpeg encoder offers comparable quality to commercial encoders.

According to the Zimtohrli test proposed by Balling, it seems to offer comparable quality, but a more thorough study (like the one by the EBU) would be needed to evaluate it. Let's see if they're willing to do it at https://hydrogenaudio.org/.

Regarding TMPGEnc, it appears to be based on a Dolby-closed codec, like Audition and others, which shouldn't be different from Dolby's official codecs (Dolby Encoding Engine, etc.).

[Balling]

"but from what I understood, the FFMPEG DD+ encoder is nowhere near that quality?"

No? Ffmpeg is already better quality than Dolby's eac3 encoder at 256k, though j7n was complaining that Dolby Media Encoder is worse quality than older Audition or eae.exe encoders of Dolby, in fact enhanced coupling is supported in Audition, even ffmpeg decoder does not support that. There is no yet support for dependent stream (Blu-ray profile) in FFmpeg, it is an upcoming patch since eac3 without Atmos patents expired anyways.

Dolby 256k eac3:
4.62671
4.29229
4.57617
(the center channel gets a rather low score here, listening to it in isolation, the issue is very clear, there is a significant amount of bleeding from the other channels)

Dolby 256k Blu-ray eac3:
4.65497
4.60828
4.58956
(surprisingly, the Blu-ray eac3 profile from the Dolby encoder is more efficient across all channels)

FFmpeg 256k eac3:
4.6278
4.74468
4.62329
(very surprisingly, the FFmpeg eac3 encoder holds up very well with the Dolby competition here!)


Now compare this, this is why I am saying that fronts encoding is worse in ffmpeg because ac3 is worse (well numbers are close 4.62544 and 4.6278 vs 4.65848 and 4.65497) so if this is fixed the encoders can be on par
Dolby 256k ac3:
4.65848
4.79297
4.65313

FFmpeg 256k ac3:
4.62544
4.72272
4.60867



"TMPGEnc is not very good at 192kbps Stereo either: it"

Oh, no, it is not stereo. It is all tested for 5.1 FLAC source.

[Balling]

Regarding TMPGEnc, it appears to be based on a Dolby-closed codec, like Audition and others, which shouldn't be different from Dolby's official codecs (Dolby Encoding Engine, etc.).

About that: Audition 2017 encoder has support of enchanced coupling which is not supported by ffmpeg decoder. And the encoder is actually provided by Rovi. https://trac.ffmpeg.org/ticket/10958

[DJ Bobo]

"TMPGEnc is not very good at 192kbps Stereo either: it"
Oh, no, it is not stereo. It is all tested for 5.1 FLAC source.
TMPG does not support anything below 384k for 5.1, and as I am a bit of an audiophile, I wouldn't dare go below that anyway! 192k with 5.1 is a blasphemy for me, unless it is AAC-HE, as it comes close to Dolby Digital 448k (!) according to the 2nd phase of the EBU study (see screenshot*). But as said earlier, I don't like using AAC because I almost always play from disc.

* _DTS in the legend means it was transcoded to DTS 1500 by the player to accomodate bitstreaming.

[tebasuna51]

About that: Audition 2017 encoder has support of enchanced coupling which is not supported by ffmpeg decoder.
Still unsupported, but enhanced coupling is applied with bitrates less than 48 Kb/s per channel and, like DJ Bobo, I can't never rtecommend less than 384 Kb/s for 5.1 Dolby audio.

It seems DJ Bobo has found an updated version of the EBU studio (please provide a link), it would be interesting to learn more about it.

[DJ Bobo]

updated version of the EBU study (please provide a link)
It's Document Tech3324 (https://tech.ebu.ch/docs/tech/tech3324.pdf)

As it gives completely different results than what this zimtohrli thingy is spitting (especially the 256k AAC VS 384k DD), I wouldn't trust the zimtohrli score at all for any comparison, be it between Dolby encoders or different codecs.
I saw that the newest version on Github is 0.2.1, so it is in a very early stage and not refined well yet. The development seems to have stalled anyway, as there was no update since last year.

Very low bitrates are not interesting anymore imho, as storing movies on CDs is a thing of the past, and we don't need to look for every last bit to save to squeeze a movie on 1 or 2 CDs. I usually burn my recordings on DVD today; that's 4 Mbit/s for video on average. Even at 640k for audio, this is only 16% of the video bitrate, where MP2 on VCD takes over 19%. So as long as it does not exceed 19%, it's perfectly adequate imho.

So I would be definitely be more interested in (reliable) comparisons at the usual bitrates: 384, 448 and 640k.
If I need to go under 384, I would prefer encoding a Pro-Logic II version instead anyway.

I checked Adobe Audition; the only free version available is Audition 3.0, and it does not seem to support AC3 encoding :-S
New versions are only on a subscription basis. No way in hell am I gonna pay Adobe a subscription for anything!! haha

[Balling]

This thread is not even about aac vs eac3? It is about eac3 quality in FFmpeg.

fdk-aac 256k cbr aac (44.1 kHz):
4.89187
4.98928
4.88925


Dolby 384kbps ac3:
4.72085
4.84867
4.72297
What is the problem, I don't see it?

[DJ Bobo]

What is the problem, I don't see it?
The Zimtohrli score of the 256k AAC is higher than the score of AC3 384k, which is the complete opposite of what the EBU study says: roughly 76 for DD384 vs 61 for AAC256.

[tebasuna51]

It's Document Tech3324 (https://tech.ebu.ch/docs/tech/tech3324.pdf)

Thanks for the link, but that second version is also very old (2007) and lacks modern encoders, which I hope have been substantially improved.

It doesn't make much sense that it shows worse results for DD+new256 than for DD+old256, and that AAC HE encodings are so highly valued.

[DJ Bobo]

that second version is also very old (2007) and lacks modern encoders
I thought about that too, but it is still very representative imho, as AAC was released in 1997. After 10 years of optimization, there is very little headroom left for improvement.

[Balling]

The Zimtohrli score of the 256k AAC is higher than the score of AC3 384k, which is the complete opposite of what the EBU study says: roughly 76 for DD384 vs 61 for AAC256.

Did you consider that AAC is reencoded from 48000 24 bit original flac to 44100 16 bit wav, while eac3 supports 24 bit flac input?

[DJ Bobo]

@ Balling
Yes, I did consider this.
It shouldn't matter, as both AAC and AC3 are lossy codecs. Lossy compression does not have an intrinsic bit depth afaik. It just throws out whatever it deems unhearable by humans.
As most input has a dynamic range of 70dB or less (meaning that almost all 24-bit content is fake 24-bit anyway :D ), lossy codecs probably work with 13 bits or less internally, and zero the remaining bits, depending on the detected dynamic range.
And thanks to the cut-off, the sampling rate becomes more of a number than hard reality.
So basically, FFMPEG would work internally with a 37KHz sampling rate on 384k AC3, as it cuts off at 18 KHz, and fills the rest of the spectrum with zeros to accomodate the selected sampling rate, whatever it may be, 44,1 or 48KHz
TMPGEnc cuts off at 17 KHz on 384kbps btw, so I am not sure if it is better than FFMPEG at this bitrate, as an earlier cut-off means more bits that could be allocated to the lower frequency band. I need to do further testing on this; I rarely use 384k.
What I know for sure, is that TMPG's 640k sounds better than FFMPEG's 640k.

What I also know for sure, 192k is borderline, even for Stereo AC3, even with professional encoders. I was just watching a DVD of the 6th Season of the X-Files yesterday, and I was not pleased at all, where I was very pleased with the sound in Season 5. I checked the bitrate, Fox used 192kbps on both seasons, but Season 5 rolls off around 16KHz, where Season 6 comes close to 20KHz (they probably started using better master tapes since Season 6?). Season 6 sounded awful, despite (or because of?) the wider spectrum (reminded me of DTS 5.1 @754k, annoying fatiguing kinda scratching mid-range?)
So the Dolby encoder used in TMPGEnc, cutting off at 16KHz with 192kbps instead of 20KHz like with industrial encoders, might actually be a good choice?!
Definitely sounds better to me than FFMPEG's implementation anyway. FFMPEG has more compression artefacts at 192k Stereo.

[j7n]

Where did you get the "13 bits" figure from? The dynamic range is adaptive to the current level in lossy codecs much like hearing. But if you place test tones spaced far apart in different bands where they don't mask each other (even if in practice they would), they can even be encoded simultaneously at high bitrate. For example, 8K at -2 dB, 2K and 14K at -96 dB, and 60 at -30 dB. Breaking the audio into bands is the core principle of transform codecs.

AC-3 at 192 kbit/s with the Dolby encoder is only discrete stereo up to 14 kHz, as is revealed by AC3Filter. This was a good decoder, showing various technical information right at playback. Above that the channels are coupled in joint-stereo. Independent stereo needs 256 kbit/s for full bandwidth.

https://imgur.com/a/U228aPX

[tebasuna51]

What I know for sure, is that TMPG's 640k sounds better than FFMPEG's 640k.
How can you prove that?

My old ears can't tell the difference between ffmpeg encoding and Dolby Encoder Engine encoding.

https://www.sendspace.com/file/m23d9g