View Single Post
Old 7th November 2009, 17:50   #1016  |  Link
Abradoks
Registered User
 
Join Date: Mar 2008
Posts: 71
Objective comparison of AC3 encoders

Source: DTS 1510 kbit/s 48kHz 6ch (Pirates of the Caribbean: At World's End)
Decoding into WAV (PCM 24 bit) — eac3to (libav).
SNR measurement (bigger is better) and padding detection — CompAudio
PEAQ measurement (bigger is better) — PQevalAudio
Padding — delaycut
WAV splitting — BeSplit
Spectrograms — Sound Forge

aften r832 (from MeGUI)
Dolby Digital Encoder 6.2.2 (Sonic Foundry Soft Encode 1.0.19): output.DD6448.ac3 (448 kbps), output.DD6224.ac3 (224 kbps)
Dolby Digital Encoder 7.0 (Sony Vegas Pro 9.0.563): output.DD7448.ac3 (448 kbps), output.DD7224.ac3 (224 kbps)

Difference between aften_wide and aften_spec:
- aften (wide) encodes full frequency range: -w 60
- aften (spec) encodes same frequency range as DD encoders:
448 kbps -> 20.34 kHz -> w=48
224 kbps -> 9.05 kHz -> w=8

Preprocessing in Dolby Encoder was disabled:


Comparison:
Code:
Name		|  SNR	| PEAQ

448 kbps, ch L

DD7		| 26.297| -0.226
DD6		| 26.268| -0.222
aften		| 20.653| -0.478
aften (spec)	| 18.504| -0.288
aften (wide)	| 19.368| -1.539

Aften gets less points than proprietary Dolby Digital Encoder (here referred to as "DD") both in SNR and PEAQ.
DD7 has slightly better results than DD6.

Average frequency graphs:

As you can see, aften has bigger frequency cutoff by default than DD.
With "-w 48" (aften_spec) it matches DD and more than that, it saves high frequency average energy closer to original.
But in all three cases aften has higher energy on low frequencies then original track.

DD6 vs DD7:


Time-frequency spectrograms
Original:		aften_wide:	aften_spec:		DD7:		DD6:



448 kbps, ch C

DD7  		| 26.039| -0.211
DD6		| 26.003| -0.208
aften		| 20.349| -0.379
aften (spec)	| 18.274| -0.263
aften (wide)	| 19.108| -0.975

Average frequency graphs:


Time-frequency spectrograms
Original:		aften_spec:	DD7:



224 kbps, ch L

DD7  		| 21.187| -2.266
DD6		| 20.845| -2.275
aften		| 15.009| -2.584
aften (spec)	| 14.830| -2.596
aften (wide)	| 6.6604| -3.527

Average frequency graphs:



224 kbps, ch C

DD7		| 19.303| -2.227
DD6		| 19.213| -2.239
aften		| 14.542| -2.601
aften (spec)	| 14.378| -2.583
aften (wide)	| 6.5879| -3.139


448 kbps, all channels (SNR only)

aften L		| 20.653 ||  DD7 L	| 26.297
aften R		| 20.525 ||  DD7 R	| 26.490
aften C		| 20.349 ||  DD7 C	| 26.039
aften SL	| 21.137 ||  DD7 SL	| 25.819
aften SR	| 21.003 ||  DD7 SR	| 25.541
aften LFE	| 22.176 ||  DD7 LFE	| -0.039484 ||  DD6 LFE	| 29.818

When running tests on all channels this strange problem with LFE was discovered.
I have rechecked everything several times, but DD7 was still getting bad results on LFE.
As you see DD6 hasn't this problem. More tests required to check this out.

Average frequency graphs (LFE):


Time-frequency spectrograms (LFE)
Original:		aften_spec:	DD7:
Commandline:
Code:
aften.exe -b 448 -pad 0 -w 60 -s on -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften448_wide.ac3
aften.exe -b 448 -pad 0 -w 48 -s on -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften448_spec.ac3
aften.exe -b 448 -pad 0 -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften448.ac3
aften.exe -b 224 -pad 0 -w 60 -s on -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften224_wide.ac3
aften.exe -b 224 -pad 0 -w 8 -s on -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften224_spec.ac3
aften.exe -b 224 -pad 0 -acmod 7 -lfe 1 -readtoeof 1 input.wav output.aften224.ac3

C:\comparison\eac3to\eac3to.exe output.aften448_wide.ac3 output.aften448_wide.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.aften448_spec.ac3 output.aften448_spec.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.aften448.ac3 output.aften448.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.aften224_wide.ac3 output.aften224_wide.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.aften224_spec.ac3 output.aften224_spec.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.aften224.ac3 output.aften224.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.DD6448.ac3 output.DD6448.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.DD6224.ac3 output.DD6224.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.DD7448.ac3 output.DD7448.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav
C:\comparison\eac3to\eac3to.exe output.DD7224.ac3 output.DD7224.wavs
del output.*.R.wav output.*.LFE.wav output.*.SL.wav output.*.SR.wav

delaycut.exe -start 5.33 -out input.L.pad256.wav input.L.wav
delaycut.exe -start 5.33 -out input.C.pad256.wav input.C.wav
delaycut.exe -start 13.25 -out input.L.pad636.wav input.L.wav
delaycut.exe -start 13.25 -out input.C.pad636.wav input.C.wav

C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften448_wide.L.wav > output.aften448_wide.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften448_wide.C.wav > output.aften448_wide.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften448_spec.L.wav > output.aften448_spec.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften448_spec.C.wav > output.aften448_spec.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften448.L.wav > output.aften448.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften448.C.wav > output.aften448.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften224_wide.L.wav > output.aften224_wide.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften224_wide.C.wav > output.aften224_wide.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften224_spec.L.wav > output.aften224_spec.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften224_spec.C.wav > output.aften224_spec.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften224.L.wav > output.aften224.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften224.C.wav > output.aften224.C.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 256: input.L.pad256.wav output.DD6448.L.wav > output.DD6448.L.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 256: input.C.pad256.wav output.DD6448.C.wav > output.DD6448.C.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 256: input.L.pad256.wav output.DD6224.L.wav > output.DD6224.L.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 256: input.C.pad256.wav output.DD6224.C.wav > output.DD6224.C.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.L.pad636.wav output.DD7448.L.wav > output.DD7448.L.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.C.pad636.wav output.DD7448.C.wav > output.DD7448.C.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.L.pad636.wav output.DD7224.L.wav > output.DD7224.L.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.C.pad636.wav output.DD7224.C.wav > output.DD7224.C.comp.txt

C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften448_wide.L.wav > output.aften448_wide.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften448_wide.C.wav > output.aften448_wide.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften448_spec.L.wav > output.aften448_spec.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften448_spec.C.wav > output.aften448_spec.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften448.L.wav > output.aften448.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften448.C.wav > output.aften448.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften224_wide.L.wav > output.aften224_wide.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften224_wide.C.wav > output.aften224_wide.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften224_spec.L.wav > output.aften224_spec.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften224_spec.C.wav > output.aften224_spec.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.L.wav output.aften224.L.wav > output.aften224.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe input.C.wav output.aften224.C.wav > output.aften224.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 256: input.L.pad256.wav output.DD6448.L.wav > output.DD6448.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 256: input.C.pad256.wav output.DD6448.C.wav > output.DD6448.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 256: input.L.pad256.wav output.DD6224.L.wav > output.DD6224.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 256: input.C.pad256.wav output.DD6224.C.wav > output.DD6224.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 636: input.L.pad636.wav output.DD7448.L.wav > output.DD7448.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 636: input.C.pad636.wav output.DD7448.C.wav > output.DD7448.C.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 636: input.L.pad636.wav output.DD7224.L.wav > output.DD7224.L.PEAQ.txt
C:\comparison\AFsp\PQevalAudio.exe -l 636: input.C.pad636.wav output.DD7224.C.wav > output.DD7224.C.PEAQ.txt

Measurement SNR of different channels:
delaycut.exe -start 13.25 -out input.R.pad636.wav input.R.wav
delaycut.exe -start 13.25 -out input.SL.pad636.wav input.SL.wav
delaycut.exe -start 13.25 -out input.SR.pad636.wav input.SR.wav
delaycut.exe -start 13.25 -out input.LFE.pad636.wav input.LFE.wav
C:\comparison\eac3to\eac3to.exe output.aften448.ac3 output.aften448.all.wavs
C:\comparison\eac3to\eac3to.exe output.DD7448.ac3 output.DD7448.all.wavs
C:\comparison\AFsp\CompAudio.exe input.L.wav output.aften448.all.L.wav > output.aften448.all.L.comp.txt
C:\comparison\AFsp\CompAudio.exe input.R.wav output.aften448.all.R.wav > output.aften448.all.R.comp.txt
C:\comparison\AFsp\CompAudio.exe input.C.wav output.aften448.all.C.wav > output.aften448.all.C.comp.txt
C:\comparison\AFsp\CompAudio.exe input.SL.wav output.aften448.all.SL.wav > output.aften448.all.SL.comp.txt
C:\comparison\AFsp\CompAudio.exe input.SR.wav output.aften448.all.SR.wav > output.aften448.all.SR.comp.txt
C:\comparison\AFsp\CompAudio.exe input.LFE.wav output.aften448.all.LFE.wav > output.aften448.all.LFE.comp.txt

C:\comparison\AFsp\CompAudio.exe -l 636: input.L.pad636.wav output.DD7448.all.L.wav > output.DD7448.all.L.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.R.pad636.wav output.DD7448.all.R.wav > output.DD7448.all.R.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.C.pad636.wav output.DD7448.all.C.wav > output.DD7448.all.C.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.SL.pad636.wav output.DD7448.all.SL.wav > output.DD7448.all.SL.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.SR.pad636.wav output.DD7448.all.SR.wav > output.DD7448.all.SR.comp.txt
C:\comparison\AFsp\CompAudio.exe -l 636: input.LFE.pad636.wav output.DD7448.all.LFE.wav > output.DD7448.all.LFE.comp.txt

C:\comparison\eac3to\eac3to.exe output.DD6448.ac3 output.DD6448.all.wavs
C:\comparison\AFsp\CompAudio.exe -l 256: input.LFE.pad256.wav output.DD6448.all.LFE.wav > output.DD6448.all.LFE.comp.txt

Padding detection:
BeSplit.exe -core( -input input.L.wav -prefix input.L.cut -type wav ) -split( 0 1 )
BeSplit.exe -core( -input output.aften448.L.wav -prefix output.aften448.L.cut -type wav ) -split( 0 1 )
BeSplit.exe -core( -input output.DD6448.L.wav -prefix output.DD6448.L.cut -type wav ) -split( 0 1 )
BeSplit.exe -core( -input output.DD7448.L.wav -prefix output.DD7448.L.cut -type wav ) -split( 0 1 )
C:\comparison\AFsp\CompAudio.exe -d 0:1030 input.L.cut01.wav output.aften448.L.cut01.wav > output.aften448.L.cut01.comp.txt
C:\comparison\AFsp\CompAudio.exe -d 0:1030 input.L.cut01.wav output.DD6448.L.cut01.wav > output.DD6448.L.cut01.comp.txt
C:\comparison\AFsp\CompAudio.exe -d 0:1030 input.L.cut01.wav output.DD7448.L.cut01.wav > output.DD7448.L.cut01.comp.txt
Conclusion:
1) There is a simple method of objective audio comparison that anybody can use.
2) Aften has much worse SNR than DD. Even 224 kbps AC3 encoded with DD beats aften 448 kbps in SNR.
3) When using PEAQ metric (which takes into account some psychoacoustic effects) aften is closer to DD, but still worse.
4) If you want to have frequency cutoff similar to DD encoders, you should use w=48 for 448 kbps and w=8 for 224 kbps.
5) Aften saves high frequency average energy closer to original. But it has higher energy on low frequencies then original track.
6) Aften has better SNR on surround channels than on front. With DD it is inversely.
7) DD7 has very bad SNR on LFE channel. More tests should be done on this issue.

P.S. It was originally written in Russian, so feel free to correct spelling.

Last edited by Abradoks; 8th November 2009 at 23:52.
Abradoks is offline   Reply With Quote