LameXP v4.21 Final · Build #2382 (2023-12-29)

[LoRd_MuldeR]

Just because one decoder/encoder combination works this way, doesn't mean we can use it in general

It has to work with all decoders, all encoders as well as all filters that may be use optionally. Unfortunately, this very often does not work as intended, because of the various reasons explained before.

Also, how does the GUI determine the sample format in your example? As said before, many encoders are limited in what sample-rates, channel-count or bit-depth they accept as input...

(Thus the GUI must be able to determine what sample format the decoder is actually spitting out and it might required to insert an instance of SoX for the required conversion)

[Atak_Snajpera]

Just because one decoder/encoder combination works this way, doesn't mean we can use it in general

It works for all encoders i tried see here again

Also, how does the GUI determine the sample format in your example? As said before, many encoders are limited in what sample-rates, channel-count or bit-depth they accept as input...

mediainfo can extract frequenzy and channel count from source file. Then you can perform resampling or downmixing in ffmpeg directly! (-ac 2 -ar 48000)

[manolito]

Hey Atak,

I like your method using ffmpeg decoded output with a pipe...
Could you write a simple and small GUI which just takes the parameters and passes them to the command line?


Cheers
manolito

[LoRd_MuldeR]

Originally Posted by Atak_Snajpera

It works for all encoders i tried see here again

So it works with FFmpeg as the decoders and all encoders you have tested. That's one thing. But...

LameXP is using a variety of decoders (even some for rather "exotic" formats). There also are optional filters that might be between the encoder and decoder.

I'm not planning to add FFmpeg as decoder or to replace the current decoders with FFmpeg.

That's because FFmpeg is rather big and rather complex to build. I prefer to have individual decoders for each format, so I can maintain (update/patch) them more easily...

And of course switching over to FFmpeg would mean I had to rewrite large portions of the program, which I don't have the time/motivation for

Originally Posted by Atak_Snajpera

mediainfo can extract frequenzy and channel count from source file. Then you can perform resampling or downmixing in ffmpeg directly! (-ac 2 -ar 48000)

The info provided by MediaInfo is about the compressed source file. What I need is the info of the uncompressed data that the decoder spits out.

For example, most compressed audio formats don't have a "bit depth" per se. It's up to the decoder to output the decompressed audio as 16-Bit Integer or 24-Bit Integer or 32-Bit IEEE Float.

Thus for MediaInfo it is impossible to know which "bit depth" the decoder is going to put out. It's not a property that can be "detected" form the compressed file...

(But we have to know it, because some encoders don't like Float input. Or they even reject everything put 16-Bit Integer)

[Atak_Snajpera]

The info provided by MediaInfo is about the compressed source file. What I need is the info of the uncompressed data that the decoder spits out.

For example, most compressed audio formats don't have a "bit depth" per se. It's up to the decoder to output the decompressed audio as 16-Bit Integer or 24-Bit Integer or 32-Bit IEEE Float.

Thus for MediaInfo it is impossible to know which "bit depth" the decoder is going to put out. It's not a property that can be "detected" form the compressed file...

(But we have to know it, because some encoders don't like Float input. Or they even reject everything put 16-Bit Integer)

as well as channel count , frequency YOU CAN also FORCE BITDEPTH (-acodec pcm_s16le) in FFMPEG so I is no problem here.

final example 5.1 to stereo

ffmpeg.exe -i "6channels-24BIT.dts" -acodec pcm_s16le -ac 2 -ar 44100 -f wav - | "opusenc.exe" --bitrate 64 - "2channels.opus"

That's because FFmpeg is rather big and rather complex to build. I prefer to have individual decoders for each format, so I can maintain (update/patch) them more easily...

And who says that you have to do everything on your own . Just grab latest build from http://ffmpeg.zeranoe.com/builds/

There also are optional filters that might be between the encoder and decoder.

what filters? Normalization?

And of course switching over to FFmpeg would mean I had to rewrite large portions of the program, which I don't have the time/motivation for

This should be on todo list for LameXP 5.0 version

[LoRd_MuldeR]

LameXP v4.05 RC-2
http://sourceforge.net/projects/lame...29/2012-08-22/

Changes between v4.04 and v4.05:
* Added support for Opus Audio Codec, based on Opus-Tools v0.1.4 (2012-08-16) by Xiph.org/Mozilla
* Added Swedish translation, thanks to Åke Engelbrektson <eson57@gmail.com>
* Updated Qt runtime libraries to v4.8.2 (2012-05-22), compiled with MSVC 10.0
* Updated mpg123 decoder to v1.14.4 (2012-07-26), compiled with GCC 4.6.1
* Updated MediaInfo to v0.7.59 (2012-08-08), compiled with ICL 12.1.7 and MSVC 10.0
* Updated optional add-ins for QAAC encoder and FHG AAC encoder (see FAQ doc for details)
* Updated DCA Enc to v2 (2012-04-19), compiled with ICL 12.1.7 and MSVC 10.0
* Updated language files (big thank-you to all contributors !!!)
* Implemented multi-threading in file analyzer for faster file import (about 2.5x to 6.0x faster!)
* Implemented multi-threading in initialization code for faster application startup
* Fixed a potential crash (stack overflow) when adding a huge number of files
* Fixed a problem with Cue Sheet import and files that contain trailing dots in their name
* Workaround for a bug (feature?) of Qt's command-line parser that screwed up some arguments

If you found any regressions in version 4.05, please report now

[pururin]

Originally Posted by Atak_Snajpera

as well as channel count , frequency YOU CAN also FORCE BITDEPTH (-acodec pcm_s16le) in FFMPEG so I is no problem here.

(High)Quality may be the problem for those who concern.
Downmixing may be simple, but Sampling Rate resampling and Bit depth conversion can be rather complex.

I think FFMPEG resampler is quite poor compared to other famous (free) ones out there. (etc. eac3to, SoX, Foobar2000, r8brain)
See here: http://src.infinitewave.ca/ for comparison.
And does FFMPEG have proper dithering when converting bit depth? I don't know for sure.

If I understand correctly, LameXP use SoX for resampling and dithering, right? Does LameXP applies dithering when decoder output 24-bit(or more) intermediate file then having to encode to 16-bit final result?

[LoRd_MuldeR]

Originally Posted by pururin

If I understand correctly, LameXP use SoX for resampling and dithering, right? Does LameXP applies dithering when decoder output 24-bit(or more) intermediate file then having to encode to 16-bit final result?

That's correct. LameXP uses SoX for resampling and also bit-depth conversion, iff such thing is required (or requested by the user).

LameXP doesn't perfrom/request any dithering explicitly, but the SoX manpage says:

Specifically, by default, SoX automatically adds TPDF dither when the output bit-depth is less than 24 and any of the following are true:
• bit-depth reduction has been specified explicitly using a command-line option
• the output file format supports only bit-depths lower than that of the input file format
• an effect has increased effective bit-depth within the internal processing chain

[pururin]

Originally Posted by LoRd_MuldeR

For example, most compressed audio formats don't have a "bit depth" per se. It's up to the decoder to output the decompressed audio as 16-Bit Integer or 24-Bit Integer or 32-Bit IEEE Float.

As I've experimented so far, when convert from lossy format EVERY decoders just output to the same 16-Bit WAVE format
(with only one exception when convert mp3→mp3 which process directly within lame.exe). Converting from lossless format the intermediate wave's bit depth is as source, except few situations(e.g. 24-bit to DCA) where SoX comes when needed.

ac3, dts(AC3filter), aac(faad), mp3, vorbis, wma, musepack, opus and so on, everything got decoded to 16-Bit Integer, I think this is not very good.
Like when transcoding dts or ac3 track from blu-ray/movies it'd be better not to downconvert them to 16-bit int beforehand.
Because lossy audio has no bit depth so it usually got decoded at higher precision (often 32-bit Float) if I get it right, e.g. Eac3to internally works at 64-bit then dither to 32/24 bit depends on the encoder, meGUI using NicAudio reports "Bits per sample: 32" decoding, not sure about foobar2000's converter but IIRC it works 32-bit float internally.


Another thing I observed is when encoding 'medium-large?' audio files in multi-threading it got even slower than consecutive single-threading and CPU load is quite a bit low (about 13-24%). Like encoding 24-bit 5.1 flac tracks 24-minutes long each (~400MB) with either 2 or 4 instances running. I guess this is due to HDD bottleneck although my HDD is quite new and fast.

Consider 2 thing above, Atak does have a point I think. So far I really like LameXP the most but if there is an alternate way(?) to the (16-bit)intermediate wave file would be good. Also CPU power is not an issue these days.

-Also thanks a lot for the answer about dithering!

[LoRd_MuldeR]

Originally Posted by pururin

Another thing I observed is when encoding 'medium-large?' audio files in multi-threading it got even slower than consecutive single-threading and CPU load is quite a bit low (about 13-24%). Like encoding 24-bit 5.1 flac tracks 24-minutes long each (~400MB) with either 2 or 4 instances running.

I guess this can happen due to HDD thrashing. It's probably more a problem of IOPS when concurrently reading/writing multiple Temp files than a problem of throughput.

HDD's are rather slow for random access, because of the seeking time (for repositioning the read/write head) and the rotation delay.

For this reason, LameXP already uses a rather "conservative" choice for the default thread count. Using "num_threads = num_cpu_cores" can be bad in terms of HDD thrashing.

Fortunately this problem will be resolved soon, as more and more people are switching to SSD's nowadays, at least for the OS partition (including the TEMP folder).

Originally Posted by pururin

Consider 2 thing above, Atak does have a point I think. So far I really like LameXP the most but if there is an alternate way(?) to the (16-bit)intermediate wave file would be good. Also CPU power is not an issue these days.

I agree that using pipes (at least where possible) would be desirable. But, as discussed before, there are various practical problems to check and resolve. Most of them probably do have some kind of solution. But it still needs to be figured out, it needs to be implemented and then it needs to be tested. So that's not something you do on a rainy Sunday.

Furthermore LameXP currently uses several "stages" within each processing thread. These are "decoding", "analyzing", "filtering" and finally "encoding". And they are run one after the other, independently.

When using pipes, everything would essentially happen at the same time. Good in terms of parallelization. A nightmare for the design of the software, which currently tries to keep those tasks as isolated as possible. The more components of the software need to cooperate directly (and thus need to know about each other), the more dependencies we get. Those dependencies make maintenance and testing very difficult.

After all, such a fundamental re-design is probably not going to happen any time soon

[Atak_Snajpera]

i wonder if single ssd would be fast enough for 8 instances (amd fx 8xxx) during flac conversion?

[LoRd_MuldeR]

Originally Posted by Atak_Snajpera

i wonder if single ssd would be fast enough for 8 instances (amd fx 8xxx) during flac conversion?

FLAC of course is kind of a problematic case, because it not only needs to read the uncompressed PCM source, but also needs to write the only slightly compressed FLAC data.

But you can calculate:
hdd_bandwidth_needed = number_of_instances * flac_throughput_per_instance * 1.66

Here flac_throughput_per_instance is the maximum throughput of a single FLAC instance in "bytes encoded per second" on your system (regarding the input only).
Also I am assuming a compression factor of approximately 2/3, so we use the factor 1.66 for writing the output. hdd_bandwidth_needed is then the "bytes per second" we'd have to transfer from/to the SSD.
On my system flac_throughput_per_instance is about 9 MByte/s (a single instance took ~5 sec to encode a 45 MB Wav file), so we'd have to transfer ~15 MByte/s per instance.
Current SATA 6G SSD's should be able to handle at least 300 MByte/s. So, roughly, we would have room to run at least ~20 instances in parallel without having to worry about I/O bottleneck here...

[LoRd_MuldeR]

LameXP v4.05 RC-3

Changes between v4.04 and v4.05:
* Added support for Opus Audio Codec, based on Opus-Tools v0.1.4 (2012-08-16) by Xiph.org/Mozilla
* Added Swedish translation, thanks to Åke Engelbrektson <eson57@gmail.com>
* Updated Qt runtime libraries to v4.8.2 (2012-05-22), compiled with MSVC 10.0
* Updated mpg123 decoder to v1.14.4 (2012-07-26), compiled with GCC 4.6.1
* Updated MediaInfo to v0.7.59 (2012-08-08), compiled with ICL 12.1.7 and MSVC 10.0
* Updated optional add-ins for QAAC encoder and FHG AAC encoder (see FAQ doc for details)
* Updated DCA Enc to v2 (2012-04-19), compiled with ICL 12.1.7 and MSVC 10.0
* Updated language files (big thank-you to all contributors !!!)
* Implemented multi-threading in file analyzer for faster file import (about 2.5x to 6.0x faster!)
* Implemented multi-threading in initialization code for faster application startup
* Fixed a potential crash (stack overflow) when adding a huge number of files
* Fixed a problem with Cue Sheet import and files that contain trailing dots in their name
* Workaround for a bug (feature?) of Qt's command-line parser that screwed up some arguments

If you found any regressions in version 4.05, please report now

[Atak_Snajpera]

What do you think about lossyWAV as optional pre-processor for FLAC? BTW. I know how it works.

[LoRd_MuldeR]

Originally Posted by Atak_Snajpera

What do you think about lossyWAV as optional pre-processor for FLAC? BTW. I know how it works.

How does it work then? I haven't used it yet.

As far as I understand, this tool (dynamically) reduce the bit-depth of the PCM audio, which makes the audio file more "compressible" for lossless compression, such as FLAC.

So does it actually output a lower bid-depth Wave file (e.g. input is 16-Bit output is 8-Bit) or is the output Wave file still the same bit-depth as the input and only the "content" is changed?

Most important: What would be the most appropriate command-line to use lossyWAV as a pre-processor for FLAC ???

I may indeed add this to LameXP (optionally), if it really reduces the size of FLAC compressed files without any "bad" side-effects - except for the "subtle" quality loss that is to be expected.

But I wonder: If it works as nice as described in the Hydrogenaudio thread, why the author of FLAC hasn't integrated this yet? Having to apply it separately looks like a workaround...

[Atak_Snajpera]

So does it actually output a lower bid-depth Wave file (e.g. input is 16-Bit output is 8-Bit) or is the output Wave file still the same bit-depth as the input and only the "content" is changed?

bit-depth is still 16 but content is changed. Some bit will be zeroized (empty-bits).

Most important: What would be the most appropriate command-line to use lossyWAV as a pre-processor for FLAC ???

lossywav.exe input.wav --stdout --standard --silent | flac.exe - -b 512 -o "lossy.flac"

Important note

NB: when encoding using a lossless codec, please ensure that the block size of the lossless codec matches that of lossyWAV (default = 512 samples). If this is not done then the lossless encoding of the processed WAV file will (almost certainly) be larger than it would otherwise have been. This is achieved by adding the "Encoder Parameters" in the table above to the command line of the lossless codec in question.

I may indeed add this to LameXP (optionally), if it really reduces the size of FLAC compressed files without any "bad" side-effects - except for the "subtle" quality loss that is to be expected.

at least --standard is 99% transparent because noise is still below human hearing range.

But I wonder: If it works as nice as described in the Hydrogenaudio thread, why the author of FLAC hasn't integrated this yet? Having to apply it separately looks like a workaround...

Because lossy pre-proccesor might not look right for lossless encoder even as option?

[pururin]

Adding lossyWAV is very useful indeed! I do hope you will add this to LameXP, LoRd_MuldeR.

Bit depth to be reduced is determined block-by-block, 24→11, 24→20 ,etc. This is better/smarter than simple TPDF dither where every block got chop off from 24→16 bit. Sorta like VBR comparing to CBR I think.
This is very good for ones who don't want to store bloated hight bit depth lossless file(which has no perceptual benefit) but don't want to convert it to lossy codecs which prone to certain artefacts and unsuitable for transcoding.

[LoRd_MuldeR]

I had a quick look at lossyWave.

It's written in Delphi, and, which isn't much of surprise, doesn't support Unicode file names. I could repair this, I think. But only if lossyWave did compile with my Delphi 7 Pro, which it doesn't. There's some code that tries to write to a constant memory location. No idea if that is a bug or intentional. Maybe that was allowed in some older Delphi version - or has been made legal in a newer version. I have no idea. For my Delphi version it's a compiler error - which makes sense to me.

Further on, lossyWave tries to load libfftw DLL. At least on my system it crashed when libfftw was found. I probably had a wrong/incompatible version. Again, no idea. Anyway, lossyWave shouldn't try to use an incompatible version and crash. As using libfftw seems to be optional anyway, I would have to disable that "feature". Last but not least, FLAC complained about an "unknown" chunk in the Wave file produced by lossyWave and it exited with an error code. Didn't look closer what the cause is.

After all, there is much work left, before this tool can be integrated. Not sure if I am interested to do that (soon).

[Atak_Snajpera]

another issue is lossywav does not support wavs larger than 4gb. also it does not have --ignorelength switch for piping directly large flacs to lossywav.

another issue. lossywav is not multithreaded. basic idea is to process each channel at the same time.

good news is that user at hydrogenaudio almost finished porting to C++

[LoRd_MuldeR]

LameXP v4.05 has been released
http://sourceforge.net/projects/lamexp/files/

Changes between v4.04 and v4.05 [2012-09-03]:
* Added support for Opus Audio Codec, based on Opus-Tools v0.1.4 (2012-08-16) by Xiph.org/Mozilla
* Added Swedish translation, thanks to Åke Engelbrektson <eson57@gmail.com>
* Updated Qt runtime libraries to v4.8.2 (2012-05-22), compiled with MSVC 10.0
* Updated mpg123 decoder to v1.14.4 (2012-07-26), compiled with GCC 4.6.1
* Updated MediaInfo to v0.7.59 (2012-08-08), compiled with ICL 12.1.7 and MSVC 10.0
* Updated optional add-ins for QAAC encoder and FHG AAC encoder (see FAQ doc for details)
* Updated DCA Enc to v2 (2012-04-19), compiled with ICL 12.1.7 and MSVC 10.0
* Updated language files (big thank-you to all contributors !!!)
* Implemented multi-threading in file analyzer for faster file import (about 2.5x to 6.0x faster!)
* Implemented multi-threading in initialization code for faster application startup
* Fixed a potential crash (stack overflow) when adding a huge number of files
* Fixed a problem with Cue Sheet import and files that contain trailing dots in their name
* Workaround for a bug (feature?) of Qt's command-line parser that screwed up some arguments