eac3to - audio conversion tool

[Music Fan]

I don't really understand your reaction because as your program is supposed to decode HDCD, and if you are curious about all that stuff (and I guess you are, otherwise you wouldn't have created this tool), you should be happy to learn these things and maybe thank people who take time to understand how your tool handle a particular format and make research about it.
People who use your program deserve to know its limits which should be clearly mentioned on this topic or in the help of eac3to.

[Thunderbolt8]

eac3tos was created to be able to decode/deal with HD movies and their audio tracks. HDCD is not really part of that.

[Yoshi]

Originally Posted by madshi

eac3to v3.30 released

http://madshi.net/eac3to.zip

* libDcaDec is now default for all DTS tracks except XSA / low bitrate

At least for two Blu-ray sources I already encountered after converting stuff for not even a day, this turns out to be a bad mistake:

In the case of the US release of "Ex Machina", the libDca decoder produces bad clipping and with the German release of "Still Alice", it produces white noise on all channels instead of the movie soundtrack.

Here you can see three times the left channel of the "Ex Machina" 7.1 DTS-HD MA source - 1. processed with eac3to and decoded by the ArcSoft decoder (1.1.0.7), 2. processed with eac3to and decoded by the libDca decoder and 3. processed with MakeMKV and decoded by the libDca decoder.



The first waveform is clipped as well, but this is most likely already contained in the source - the usual nowadays mastering stupidity. The second and third are identical, so it's obvious whose fault this is.

Maybe I missed something but did anyone actually test that libdca crap before deciding to use it?

Same question to the MakeMKV developers which seem to have fallen into the same trap. At least, eac3to can be forced to still use the ArcSoft-decoder.

[nevcairiel]

You should first stay calm and not result to insulting peoples hard work in providing a free and open decoder for a format thats been hard to handle completely for a long a time.
Of course its tested, but not every edge case can be thoroughly covered without access to every single disc on the planet, and if the source has baked in clipping, it could result in all sorts of problems - like this. Clearly clipping behavior can be improved by simply flat-lining the clipping instead of overflowing it, but I even saw a change related to that in newer libdcadec versions, so maybe its already resolved, just eac3to not updated yet.

On top of all that, if you can provide a short sample that illustrates such a problem, the developers will likely be happy to investigate and resolve these issues.

[ndjamena]

EAC3To should probably detect that and then run a second pass to normalise it, something that's probably not possible with Arcsoft.

libDcaDec is open source so can be modified, maybe to output 32 bit.

But the problem in this case is obviously in the audio stream itself.

[Yoshi]

Originally Posted by nevcairiel

You should first stay calm and not result to insulting peoples hard work in providing a free and open decoder for a format thats been hard to handle completely for a long a time.

My excuses, you're right. This was written right out of frustration about the screwed up rips which resulted in using the decoder which is supposed to replace the commercial add-ins required before. And to be fair, the ArcSoft decoder - depending on the version - isn't "perfect" either, at least with eac3to (the MakeMKV creators claim that by addressing the library directly, there are no issues at all with it, including 'strange 7.1 setups').

Originally Posted by nevcairiel

Of course its tested, but not every edge case can be thoroughly covered without access to every single disc on the planet, and if the source has baked in clipping, it could result in all sorts of problems - like this.

Sorry, but with all due respect to all the developers and their hard work - we are talking about a lossless codec system here. DTS-HD MA might me more complex due to its core and extension architecture, but for a proper decoder dealing with a lossless format, it must not matter one bit how screwed up the source is content-wise, meaning the raw PCM source, not damaged frames or headers which might lead to problems. In the latter case, one decoder might indeed be more tolerant than the other. Clipping in the source is not an excuse for a lossless decoder to mess up the resulting PCM completely.

If a decoder doesn't reproduce the original data of a losslessly encoded source, then it's faulty. There is absolutely no tolerance to that.

Originally Posted by nevcairiel

Clearly clipping behavior can be improved by simply flat-lining the clipping instead of overflowing it, but I even saw a change related to that in newer libdcadec versions, so maybe its already resolved, just eac3to not updated yet.

As I stated above, I don't see the tolerance of a lossless decoder regarding the reproduction of the PCM source.

Originally Posted by nevcairiel

On top of all that, if you can provide a short sample that illustrates such a problem, the developers will likely be happy to investigate and resolve these issues.

I'm certainly willing to contribute here, otherwise I wouldn't take the effort to compare and make screenshots.

[nevcairiel]

Originally Posted by Yoshi

Sorry, but with all due respect to all the developers and their hard work - we are talking about a lossless codec system here. DTS-HD MA might me more complex due to its core and extension architecture, but for a proper decoder dealing with a lossless format, it must not matter one bit how screwed up the source is content-wise, meaning the raw PCM source, not damaged frames or headers which might lead to problems. In the latter case, one decoder might indeed be more tolerant than the other. Clipping in the source is not an excuse for a lossless decoder to mess up the resulting PCM completely.

Unfortunately, its not quite as simple as that. Without having the PCM source, we don't know what is "lossless", and when there is baked in clipping like this, its possible that no decoder is going to produce an exact copy of the source, since before encoding, it may not have had this clipping (maybe because it was higher bitdepth, somehow).

The problem in your screenshots is a simple one really. The decoder decodes the signal perfectly, its just that the decoded signal does not fit into the defined integer range and instead "wraps" into negative (just how integers work, too high positive numbers suddenly turn negative).
You can actually see the waveform "continue" in the negative - which clearly sounds terrible, but it gives us strong hints to whats going on.

So, the answer is instead of decoding it "perfectly", to actually clip the signal after decoding, so you get flatlines instead. Unfortunately a lot of information about DTS-HD MA has to be reverse engineered or "guessed" based on the behavior of other decoders, like ArcSoft.

So in short, the best we can do is "lossless to ArcSoft", or any other reference decoder, since thats the only data points we have (unless someone can encode something with such a problem and provide the original PCM)
Clearly this is a bug, and should be fixed, but I find it important to clarify a bit on how it came to be.

Anyway, like I mentioned earlier, a change was already performed in libdcadec to perform more aggressive clipping, which may just handle this particular sample already.
If you can cut a small segment with the problematic areas from the two Blu-rays you have been having issues with, we can make sure, and/or get them fixed - and test losslessness to ArcSoft after any potential fixes.

Its everyones goal here to provide proper "lossless" decoding on all sorts of broken samples, of course.

[Yoshi]

Originally Posted by nevcairiel

Unfortunately, its not quite as simple as that. Without having the PCM source, we don't know what is "lossless", and when there is baked in clipping like this, its possible that no decoder is going to produce an exact copy of the source, since before encoding, it may not have had this clipping (maybe because it was higher bitdepth, somehow).

Very interesting remarks. I have to admit that I'm not familiar with the internal structure of the encoding and decoding process of DTS-HD(MA) in detail except for the core and extension concept. If I understand you correctly, there might be inputs, DTS-HD(MA) won't be able to handle properly. To my best understanding, this goes against all what makes up a lossless codec in the first place. Maybe a sacrifice of making DTS backward-compatible this way.

But let's leave source-clipping aside and have a look what libDca does to the calm and innocent intro of "Still Alice" instead.



Regarding the test files, I'll send you a PM shortly.

[nevcairiel]

That one sure is weird. Will be interesting to see.

[torturesauce]

Ex Machina has a new codec called DTS:X (not to be confused with DTS Headphone:X, which is also included on the disc). It is the rival format to Dolby Atmos. Therefore, the DTS-HD MA 7.1 is actually the core track. More info here: http://www.blu-ray.com/movies/Ex-Mac...128113/#Review

Maybe this is the reason libdcadec produces inaccurate results?

[ndjamena]

Turbo:

MKV, 1 video track, 1 audio track, 1:35:41, 24p /1.001
1: h264/AVC, English, 1080p24 /1.001 (16:9)
2: DTS Master Audio, English, 7.1 (strange setup) channels, 24 bits, 48kHz
(core: DTS-ES, 5.1 channels, 1509kbps, 48kHz)

With the core extracted:

MKV, 1 video track, 1 audio track, 1:35:41, 24p /1.001
1: h264/AVC, English, 1080p24 /1.001 (16:9)
2: DTS-ES, English, 5.1 channels, 1509kbps, 48kHz

So... when is DTS-ES still regular 5.1...

...or is this likely a bug in the DTS encoder suite?

[ndjamena]

Originally Posted by torturesauce

Ex Machina has a new codec called DTS:X (not to be confused with DTS Headphone:X, which is also included on the disc). It is the rival format to Dolby Atmos. Therefore, the DTS-HD MA 7.1 is actually the core track. More info here: http://www.blu-ray.com/movies/Ex-Mac...128113/#Review

Maybe this is the reason libdcadec produces inaccurate results?

Yeah, the audio is being mixed on the fly, so there isn't any lossless source.

[ndjamena]

Originally Posted by Yoshi

Very interesting remarks. I have to admit that I'm not familiar with the internal structure of the encoding and decoding process of DTS-HD(MA) in detail except for the core and extension concept. If I understand you correctly, there might be inputs, DTS-HD(MA) won't be able to handle properly. To my best understanding, this goes against all what makes up a lossless codec in the first place. Maybe a sacrifice of making DTS backward-compatible this way.

But let's leave source-clipping aside and have a look what libDca does to the calm and innocent intro of "Still Alice" instead.



Regarding the test files, I'll send you a PM shortly.

This second one looks like it's been amplified. Far too much gain has been applied.

[nevcairiel]

Originally Posted by Yoshi

But let's leave source-clipping aside and have a look what libDca does to the calm and innocent intro of "Still Alice" instead.

It seems to switch from 16-bit to 24-bit decoding here along the way, probably due to a change in the bitstream. It almost seems like eac3to doesn't handle this case properly, and not dcadec.
I have let madshi know so he can chime in here.

Other software using libdcadec like my own LAV Filters or ffmpeg can decode this file fine.

[NikosD]

Puzzled about this TrueHD sample:
https://www.sendspace.com/file/6cn8ub

I tried to convert it to AC3 using eac3to v3.30 source.thd dest.ac3 and the app crashed

TrueHD, 5.1 channels, 48kHz, dialnorm: -27dB
Removing TrueHD dialog normalization...
Decoding with libav/ffmpeg...
Remapping channels...
Encoding AC3 <640kbps> with libAften...
Initialization of the AC3 encoder failed.
Aborted at file position 262144.

Using eac3to v3.29 source.thd dest.ac3 works fine.

TrueHD, 5.1 channels, 48kHz, dialnorm: -27dB
thd, 48000, 5.1
Removing TrueHD dialog normalization...
Decoding with libav/ffmpeg...
Remapping channels...
Encoding AC3 <640kbps> with libAften...
Creating file "test329.ac3"...
The original audio track has a constant bit depth of 16 bits.
eac3to processing took 9 seconds.
Done.

[nevcairiel]

Originally Posted by Yoshi

In the case of the US release of "Ex Machina", the libDca decoder produces bad clipping

I checked a sample provided by Yoshi of this track, and the new libdcadec version produces a bit-identical result to ArcSoft, so as I suspected the clipping issue was already fixed. Just needs an updated version of eac3to with a new dcadec.

[SeeMoreDigital]

Originally Posted by NikosD

Puzzled about this TrueHD sample:
https://www.sendspace.com/file/6cn8ub

I tried to convert it to AC3 using eac3to v3.30 source.thd dest.ac3 and the app crashed

Using eac3to v3.29 source.thd dest.ac3 works fine.

I'm able to confirm this observation too, using your TrueHD sample...

[Boulder]

Originally Posted by nevcairiel

I checked a sample provided by Yoshi of this track, and the new libdcadec version produces a bit-identical result to ArcSoft, so as I suspected the clipping issue was already fixed. Just needs an updated version of eac3to with a new dcadec.

Is it possible to build libdcadec.dll outside of eac3to?

[nevcairiel]

Originally Posted by Boulder

Is it possible to build libdcadec.dll outside of eac3to?

I suppose it is, if its an unmodified version, which it probably is.

[madshi]

Yes, it is, but the dcadec interface changed a bit, allowing warnings now instead of errors. Anyway, I'll release a new build with an updated dcadec soon.