eac3to - audio conversion tool

[r0lZ]

Madshi, I have seen your reply in the dcadec bug tracker. Thanks.

To facilitate the decoding (at least while the origin of the "bug" is unclear), is it possible to turn off the DCADEC_FLAG_STRICT via eac3to's command line ? How ? Or is it only a flag that can be changed at compile time ?

If it's not possible, can you consider turning it off in a forthcoming version?

[madshi]

Please add an issue to the bug tracker to request an option to turn the strict decoding mode off, so that I don't forget it. It can currently only be changed in eac3to by recompiling.

[tebasuna51]

I don't know if that's help:

Decoding the sample with a recent ffmpeg, finish without errors, but using ffmpeg version N-71329-g235589e 07-Apr-2015 finish with:

libdcadec/exss_parser.c+395: Invalid EXSS size0kbits/s

The output for both versions are bit-identical.

Maybe you can output a <WARNING> but continue the decode.

[madshi]

I think at this point in time it's better to fail, because if dcadec sees a reason for a warning, this could still be a bug in dcadec (or a damaged audio file). It's better to force the user to double check, and maybe use ArcSoft instead, than to risk just writing a warning to the log which the user might simply not see, and then eventually produce a faulty audio file. After a couple more months or years, when we're more certain about dcadec being "bug free" or not, we can think about posting warnings and continuing decoding then. At least that's my opinion. Anyone having a different opinion?

[Boulder]

I propose a parameter to disregard the error, just because dcadec is the only freeware solution to decoding such files

[AYColumbia]

Thanks for the update madshi.

[jpsdr]

Thanks for the update madshi.
And, it's just to know what the actual statut is, do you also have update ffmpeg/flac with more recent version, or still haven't time ?

[r0lZ]

Originally Posted by madshi

I think at this point in time it's better to fail, because if dcadec sees a reason for a warning, this could still be a bug in dcadec (or a damaged audio file). It's better to force the user to double check, and maybe use ArcSoft instead, than to risk just writing a warning to the log which the user might simply not see, and then eventually produce a faulty audio file. After a couple more months or years, when we're more certain about dcadec being "bug free" or not, we can think about posting warnings and continuing decoding then. At least that's my opinion. Anyone having a different opinion?

Well, currently, only eac3to is unable to decode that picky DTSHDMA tracks. Even ffmpeg doesn't turn the strict decoding on, and has therefore no problem.

I agree that turning off the strict decoding mode is dangerous, but IMO, the "synchronisation error" reported (most probably erroneously) by DcaDec should be ignored. I don't know if it is possible to continue anyway after that error (and only that one), but if it's feasible, that could be a very good thing.

I have requested an option to turn the strict mode off via your bug tracker (here), but honestly, I would prefer to leave that flag on, but have a workaround for what seems to be a bug in the DTS decoder. After having double-checked the original DTSHDMA and the conversions with other decoders, it seems that the streams are perfectly normal. Therefore, DcaDec with the strict option on is at least way too picky, at least for this specific error.

Also, note that some decoders do NOT stop after an extremely great number of synchronisation errors. It's the case when demuxing subtitles from some 3DBD (as reported in another bug on your tracker). I see no reason to be much more tolerant with the subtitles than with the DTS tracks.

Anyway, a solution is necessary, but I'll be happy with just an option to turn the strict mode off, although I think it's not the best solution.

[73ChargerFan]

Originally Posted by r0lZ

I have a DTSHDMA track causing the problem but unfortunately it is way too big (2.7 GB) to be uploaded

It'd be really useful to get that file to the developers if there really is a problem (i.e., it isn't just a bad rip or BD decode.)

That large of a file can be easily transferred between two computers both using utorrent, see How to Share Personal or Public Files Using uTorrent. It would be a private pseudo tracker run from your computer, and won't be publicly shared.

Or see the NetworkWorld article 19 free cloud storage options.

[r0lZ]

I have already isolated the part causing the problem and posted it here. (See post #13582.) In this case, there is no need to analyse the entire track. But thanks for the hints about transferring large files. I didn't know that "pseudo tracker" possibility of uTorrent. (BTW, I like also Infinit to share large files, but it requires to install a little program on your computer.)

[ndjamena]

OK, so I think I've fixed MediaInfo's issues with FLAC default layouts. I've also added "VALID_BITS" as a recognised tag and have moved it to the audio stream as "Valid bits".

I've removed HDCD if the value is zero... otherwise I've called it "High Definition Compatible Digital" and set it to "Yes"... Is that OK? Is it important to know if it's specifically tagged as NOT HDCD?

Is there anything else I should attempt to prepare MediaInfo to encounter?

[madshi]

Originally Posted by ndjamena

OK, so I think I've fixed MediaInfo's issues with FLAC default layouts. I've also added "VALID_BITS" as a recognised tag and have moved it to the audio stream as "Valid bits".

I've removed HDCD if the value is zero... otherwise I've called it "High Definition Compatible Digital" and set it to "Yes"... Is that OK? Is it important to know if it's specifically tagged as NOT HDCD?

Is there anything else I should attempt to prepare MediaInfo to encounter?

Sounds good to me! FWIW, it might make sense to only list "valid bits" if it's less than the designated FLAC bitdepth. E.g. one common case is having 20bit in a 24bit container because FLAC doesn't support encoding a 20bit bitdepth, IIRC. I'd also ignore a 0 value for valid bits. Shouldn't occur in real life, but in theory it could be there for e.g. an audio track which has nothing but silence (zeroes) in it. I'm not a native english speaker btw, so maybe someone has a better idea how to name this in clear text? Not sure if "valid bits" is a good name.

I'd suggest to use "HDCD" as the name because that's the term everybody is familiar with. Writing "High Definition Compatible Digital" is like naming DVD "Digital Versatile Disc". Users would look at that three times and wonder what it means while "HDCD" is easy to recognize for anybody who knows what it is. eac3to always checks all audio tracks for HDCD, so if you find "HDCD" in the metadata, having it set to 0 usually means the track is *not* HDCD. But if you want to be extra safe that you're not reporting anything wrong then just reporting HDCD if it's set to "1" is fine, too.

IIRC, support for the "WAVEFORMATEXTENSIBLE_CHANNEL_MASK" is already available, right? At some point there was a problem with supporting "=0X" vs "=0x". Maybe you could double check that both is supported (you should simply ignore the case).

Thanks!

[Music Fan]

I read something astonishing about HDCD ;
http://forum.doom9.org/showthread.ph...18#post1612018

Originally Posted by MvB

HDCD can not be saved in .wav because sub channel will be lost.

Is it true ?

[ndjamena]

Originally Posted by madshi

Sounds good to me! FWIW, it might make sense to only list "valid bits" if it's less than the designated FLAC bitdepth. E.g. one common case is having 20bit in a 24bit container because FLAC doesn't support encoding a 20bit bitdepth, IIRC. I'd also ignore a 0 value for valid bits. Shouldn't occur in real life, but in theory it could be there for e.g. an audio track which has nothing but silence (zeroes) in it. I'm not a native english speaker btw, so maybe someone has a better idea how to name this in clear text? Not sure if "valid bits" is a good name.

I'd suggest to use "HDCD" as the name because that's the term everybody is familiar with. Writing "High Definition Compatible Digital" is like naming DVD "Digital Versatile Disc". Users would look at that three times and wonder what it means while "HDCD" is easy to recognize for anybody who knows what it is. eac3to always checks all audio tracks for HDCD, so if you find "HDCD" in the metadata, having it set to 0 usually means the track is *not* HDCD. But if you want to be extra safe that you're not reporting anything wrong then just reporting HDCD if it's set to "1" is fine, too.

IIRC, support for the "WAVEFORMATEXTENSIBLE_CHANNEL_MASK" is already available, right? At some point there was a problem with supporting "=0X" vs "=0x". Maybe you could double check that both is supported (you should simply ignore the case).

Thanks!

Actually, if I point mediainfo at a flac file before it's finish it DOES say '00" valid bits...

HDCD... What about the people who don't know what it is? I didn't and had no idea why that tag was in all my encodes. That's why I thought it would be a good idea the give the full name.

Jerome has already accepted the code, so I'd either have to issue another pull request to change things or pass on what you've said here.

[madshi]

Originally Posted by ndjamena

Actually, if I point mediainfo at a flac file before it's finish it DOES say '00" valid bits...

True, that's because eac3to only knows the final value after processing has run through. One more reason to ignore a value of 00...

Originally Posted by ndjamena

HDCD... What about the people who don't know what it is?

Well, that's like saying that some people don't know what FLAC is, so you write "Free Lossless Audio Codec" instead of FLAC everywhere. But if you do that, most people who *DO* know what FLAC is will be confused, because although they know FLAC they might not know what the acronym stands for.

Just my 2 cents, of course. It's not really important to me. Just wanted to provide some feedback.

[Zenitram]

Originally Posted by madshi

Sounds good to me! FWIW, it might make sense to only list "valid bits" if it's less than the designated FLAC bitdepth. E.g. one common case is having 20bit in a 24bit container because FLAC doesn't support encoding a 20bit bitdepth, IIRC. I'd also ignore a 0 value for valid bits.

Done.

Originally Posted by madshi

Shouldn't occur in real life, but in theory it could be there for e.g. an audio track which has nothing but silence (zeroes) in it.

Hum... Not a good example. I hoped that this is not the implementation.
Silence with 16 valid bits is not same than silence with 24 valid bits, and this is not 0 bit of something (whatever is the name). So the value is expected to be the count of bits really used during quantization else this is only a 0 measurement (not useful).
Said another way, silence quantized at 20 bits then stored in the file as 24-bit should still have 20 valid bits in the metadata (this is the count of bits used by the digitilization tool), not 0. having only zeroes in the lowest bit does not mean that the bit is not valid (99.99999% chances it is, but in theory it is not), it may be valid (no luck, the source is with zeroes...).

Originally Posted by madshi

True, that's because eac3to only knows the final value after processing has run through. One more reason to ignore a value of 00...

eac3to should know the bit depth of the input, scanning the lower bits of the full stream would be only a fallback if the input is known to provide invalid metadata (e.g. 24-bit FLAC which can be 20 or 24-bit in reality if I understand well). I understand the reason you prefer to scan the full file for checking lower bits but it may be misleading (e.g. in mathematics, 0.0000 means that it is between -0.00005 and 0.00004, it does not mean it is 0.000 or 0.00 or 0.0 or 0, which are true information but you lost the piece of information about precision).

Originally Posted by madshi

I'm not a native english speaker btw, so maybe someone has a better idea how to name this in clear text? Not sure if "valid bits" is a good name.

SMPTE uses "Quantization bits" for this purpose in MXF file format.

In MediaInfo, I use "Bit depth" for the real value (your "valid bits") and "Stored bit depth" for the bit size in the file.

Originally Posted by madshi

I'd suggest to use "HDCD" as the name because that's the term everybody is familiar with. Writing "High Definition Compatible Digital" is like naming DVD "Digital Versatile Disc".

I agree. Changed.

Originally Posted by ndjamena

HDCD... What about the people who don't know what it is? I didn't and had no idea why that tag was in all my encodes. That's why I thought it would be a good idea the give the full name.

As for FLAC or DVD, people put HDCD in Google and the first link is the Wikipedia page about "High Definition Compatible Digital".
Actually the acronym is more known by people able to help someone than the definition of the acronym.

Originally Posted by madshi

IIRC, support for the "WAVEFORMATEXTENSIBLE_CHANNEL_MASK" is already available, right? At some point there was a problem with supporting "=0X" vs "=0x". Maybe you could double check that both is supported (you should simply ignore the case)

I don't remember exactly the previous behavior but ndjamena definitely fixed an issue with MediaInfo previous behavior, I have now more files with channels position. So I take his patch!

[nevcairiel]

Originally Posted by Zenitram

Hum... Not a good example. I hoped that this is not the implementation.
Silence with 16 valid bits is not same than silence with 24 valid bits, and this is not 0 bit of something (whatever is the name). So the value is expected to be the count of bits really used during quantization else this is only a 0 measurement (not useful).
Said another way, silence quantized at 20 bits then stored in the file as 24-bit should still have 20 valid bits in the metadata (this is the count of bits used by the digitilization tool), not 0. having only zeroes in the lowest bit does not mean that the bit is not valid (99.99999% chances it is, but in theory it is not), it may be valid (no luck, the source is with zeroes...).

If you take a 20-bit signal, and you want to store it in a 24-bit format, what do you do? You pad it with zeroes.
Thats how its done everywhere, so you have no idea what the original bitdepth was when you get all zeroes, all the time.

Thats just a very common use-case when dealing with PCM, and eac3to wants to know if a signal was mastered wrong - since it occasionally happens that a 16-bit signal gets erroneously padded to 24-bit and encoded as such - with 8 LSBs all being zero.
Or in some cases, the format may not provide a way to signal some bitdepth - which is common for 20-bits. There is no harm to encode it as 24-bit in a lossless format, other than filesize, but the flag that it originally was 20-bit may be lost.

This is all long after quantization, and its all in perfectly accurate integer, not floating points.

[madshi]

Originally Posted by nevcairiel

If you take a 20-bit signal, and you want to store it in a 24-bit format, what do you do? You pad it with zeroes.
Thats how its done everywhere, so you have no idea what the original bitdepth was when you get all zeroes, all the time.

Thats just a very common use-case when dealing with PCM, and eac3to wants to know if a signal was mastered wrong - since it occasionally happens that a 16-bit signal gets erroneously padded to 24-bit and encoded as such - with 8 LSBs all being zero.
Or in some cases, the format may not provide a way to signal some bitdepth - which is common for 20-bits. There is no harm to encode it as 24-bit in a lossless format, other than filesize, but the flag that it originally was 20-bit may be lost.

This is all long after quantization, and its all in perfectly accurate integer, not floating points.

Exactly. The reason why I added the "VALID_BITS" tag in the first place was because many TrueHD and DTS-MA tracks are encoded as 24bit, but have a number of bits set to zero throughout the whole audio file. I've seen TrueHD/DTS-MA tracks detected by eac3to with anything between 16 and 24 bits (20 bits is common, but IIRC I've also seen e.g. 18bit or 22bit). Many tracks even have different bitdepths per channel! So this tag signals the number of bits that eac3to found to be set to non-zero anywhere throughout the audio file. Whether this information is considered useful by anybody or not is an entirely different question. But this is how VALID_BITS has always been "meant" by eac3to.

Maybe I should have named it "NON_ZERO_BITS". But to be honest, all those years ago when I added this tag I considered it a "private" eac3to flag and didn't think anybody would be interested in reading/displaying it.

[Zenitram]

Originally Posted by nevcairiel

If you take a 20-bit signal, and you want to store it in a 24-bit format, what do you do? You pad it with zeroes.

Please read again my comment (especially the example with silence), I don't say something else.
I just say that this is not bidirectionnal i.e. zeroes in the lowest bits does not means that this was a 20-bit signal for sure (it may be a 24-bit format with zeroes in reality), you can just say that there is 99.9999% chances that this is the case.
Silence is a good example: silence, all bits 0 everywhere, that does not mean that 0 bits are valid. Same with a square signal.

And as I also said, I understand the reason it is made, but this is not a reason for changing reality of mathematics, and relying on only this check is not perfect. If you can rely on something else (e.g. metadata) please priotize this method over zeroes checking (zeroes checking should be only a fallback in the case there is not such metadata)

[Zenitram]

Originally Posted by madshi

So this tag signals the number of bits that eac3to found to be set to non-zero anywhere throughout the audio file. Whether this information is considered useful by anybody or not is an entirely different question. But this is how VALID_BITS has always been "meant" by eac3to.

Maybe I should have named it "NON_ZERO_BITS". But to be honest, all those years ago when I added this tag I considered it a "private" eac3to flag and didn't think anybody would be interested in reading/displaying it.

Got it. "VALID" term was something misleading for me, I'll change the behavior of my own tool when I detect such tag.