Help with channel up- & downmixing using ffmpeg

[GrandAdmiralThrawn]

Hello,

I'm trying to remix some channels, e.g. layout "L R C LFE Ls Rs" 5.1 as well as "L R" 2.0 to the 5.1 channel layout "C L R Ls Rs LFE". On top of that, I need to remix "L R C LFE Ls Rs" 5.1 to "L R" stereo. I will also have to resample some parts.

The reason is that I need to stitch some audio tracks with different sample sizes and channel layouts together to match multiple main feature audio tracks. All files are PCM/Wav64 audio, which I intend to just concatenate. But for that to work, the layouts and sample sizes need to match. Resampling is super easy, but remixing is royally confusing me.

With one of the sources which comes in the layout "L R C LFE Ls Rs" I tried this for instance:

ffmpeg -i input.w64 -filter_complex "pan=5.1|c0=c2|c1=c0|c2=c1|c3=c4|c4=c5|c5=c3[a]" -map "[a]" -ac 6 -acodec pcm_s24le output.w64

Essentially, I'm trying to tell ffmpeg to do this: "Take channel 2 and put it on channel 0 (center), take channel 0 and put it on channel 1 (left)" and so on and so forth.

The result is confusing me, however, as mediainfo now claims the resulting file has the layout "L R C LFE Lb Rb". Huh?

And I haven't even started upmixing the stereo parts or downmixing the surround parts yet...

So in my helplessness, I ran to the great machine god (ChatGPT) and asked it to fix my mess, but it didn't produce anything that would've worked either. Most commands it gave me only produced errors. ;)

So here I am, asking for help. :)

Inputs that require remixing:

1.) 2.0: Layout "L R", sample size 16 bits
2.) 5.1: Layout "L R C LFE Ls Rs", sample size 24 bits

My desired outputs:

a.) 2.0: Layout "L R", sample size 16 bits
b.) 5.1: Layout "C L R Ls Rs LFE", sample size 24 bits

All files come with a 48 kHz sampling frequency. Every single input file needs to be remixed and partially resampled to match each given output format. So "1." needs to be converted to both "a." and "b." and "2." needs the same.

If possible, I'd like not to throw channels away entirely, so that the resulting streams lose as little audio information as possible.

For getting the channel layouts I use mediainfo, as mentioned, in the hope that it produces correct information.

It would be really nice if you could give me the correct commands for this, and even nicer if you could explain them to me as well, so I can finally understand this stuff. :)

Thanks a lot! :)

[tebasuna51]

I'm trying to remix some channels, e.g. layout "L R C LFE Ls Rs" 5.1... to the 5.1 channel layout "C L R Ls Rs LFE".

The channel layout "C L R Ls Rs LFE" does not exist in w64 format.

For what do you need that channel order?
Please don't mistake the internal order in any codec format with the order in wav/w64 format. Any encoder must translate the order in wav/w64 source to the internal codec order.

[GrandAdmiralThrawn]

The channel layout "C L R Ls Rs LFE" does not exist in w64 format.

For what do you need that channel order?
Please don't mistake the internal order in any codec format with the order in wav/w64 format. Any encoder must translate the order in wav/w64 source to the internal codec order.

Like I said, when I run mediainfo on my main feature audio, it shows "C L R Ls Rs LFE". That file is a .w64 one. It was originally some DTS audio and was encoded to .w64 with ffmpeg without specifying a channel layout. Basically just DTS in, W64 out, nothing else. Running mediainfo on the DTS source also shows "C L R Ls Rs LFE".

Additionally, there are some logos, notices and some epilogue for this feature, but they come as separate files (it's a community project). I think it's either linked MKVs, or the viewer was expected to just watch them one after another. Those additional files have either no audio, or audio that does not match the main features audio.

I want to stitch that all together doing simple concatenation. So the idea was:

Create audio paddings with correct runtimes for those smaller videos (notices, logos) with no audio streams, essentially just patches of silent audio, 5.1 and 2.0
Up / downsample and remix existing audio parts so that they match the main feature audio streams exactly to enable safe concatenation
Simply use "cat" to stitch the 5.1 .w64 files together in the correct order, same for the 2.0 ones
Take those single files containing all desired, original streams in the correct order with the correct, matching channel layouts and encode them to a single, lossy stream (AAC-LC, Opus, whichever)
Replace those six (in this case) existing MKVs with just one containing the entire content


I'm worried that concatenating multiple .w64 streams with different channel layouts ("L R C LFE Ls Rs" & "C L R Ls Rs LFE") will quite likely mess the resulting stream up.

Thank you very much!

[junh1024]

1. As Teba said, "C L R Ls Rs LFE" is an internal order useful mainly to developers, not editors. "L R C LFE Ls Rs" is used for editing & playback.

ED: If you're encoding to DTS, most encoders accept the latter input, or single channels as input. Please perform a full workflow test including listening to see what happens.

2. If you're upmixing, chances are you're going to mess it up (ruin balance, image, downmix compatibility) if you don't know what youre doing. Just pad 2ch to 5.1ch with silence on other channels.

3. cat may or may not work, audio files have headers at the start/end to tell the specifications. At best, you will end up with small glitches & playable files. At worst, you may end up with broken files. Best to use a media or audio app to join.

[tebasuna51]

Like I said, when I run mediainfo on my main feature audio, it shows "C L R Ls Rs LFE". That file is a .w64 one.
Then your MediaInfo software have a bug. The channel order in a wav (https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ksmedia/ns-ksmedia-waveformatextensible)/w64/rf64 file must be always: FL,FR,FC,LFE,BL,BR,FLC,FRC,BC,SL,SR,...
Read also about dwChannelMask in that link.

I put the channel names in Microsoft wav style, see here the equivalences (https://mediaarea.net/AudioChannelLayout) with MediaInfo or other names.

Running mediainfo on the DTS source also shows "C L R Ls Rs LFE".
Thats can be correct but must be ignored at all, because you want work with sources wav/w64

I want to stitch that all together doing simple concatenation.
The Unix command 'cat' can't be used because w64 headers.
Use ffmpeg:

ffmpeg -i 01.w64 -i 02.w64 -filter_complex "concat=n=2:v=0:a=1" output.w64

To convert parts 2.0 to the main audio 5.1 you can use also:

ffmpeg -i 20.w64 -af "surround=lfe_out=0" 51.w64

I'm worried that concatenating multiple .w64 streams with different channel layouts ("L R C LFE Ls Rs" & "C L R Ls Rs LFE") will quite likely mess the resulting stream up.

Like I say the wav/w64 5.1 always are FL,FR,FC,LFE,BL,BR or FL,FR,FC,LFE,SL,SR (the BackL,BR or SideL,SR pairs can be equivalents to Lsurround,Rs in 5.1 configuration)

[GrandAdmiralThrawn]

Thank you very much, I will just try it out and listen to my output before progressing further, especially at the positions where streams intersect.

Also thanks for the tip about padding the 5.1 stream with silent audio!

I was just assuming I could cat wav/w64 together because I tried this before and it just worked without any glitches. It may depend on the decoder I guess. I will follow your suggestion about doing this with ffmpeg instead though, better safe than sorry.

[saul01]

Hello,
ehsas programme (https://ehsaasprograms8171.com.pk/)
I'm trying to remix some channels, e.g. layout "L R C LFE Ls Rs" 5.1 as well as "L R" 2.0 to the 5.1 channel layout "C L R Ls Rs LFE". On top of that, I need to remix "L R C LFE Ls Rs" 5.1 to "L R" stereo. I will also have to resample some parts.

The reason is that I need to stitch some audio tracks with different sample sizes and channel layouts together to match multiple main feature audio tracks. All files are PCM/Wav64 audio, which I intend to just concatenate. But for that to work, the layouts and sample sizes need to match. Resampling is super easy, but remixing is royally confusing me.

With one of the sources which comes in the layout "L R C LFE Ls Rs" I tried this for instance:

ffmpeg -i input.w64 -filter_complex "pan=5.1|c0=c2|c1=c0|c2=c1|c3=c4|c4=c5|c5=c3[a]" -map "[a]" -ac 6 -acodec pcm_s24le output.w64

Essentially, I'm trying to tell ffmpeg to do this: "Take channel 2 and put it on channel 0 (center), take channel 0 and put it on channel 1 (left)" and so on and so forth.

The result is confusing me, however, as mediainfo now claims the resulting file has the layout "L R C LFE Lb Rb". Huh?

And I haven't even started upmixing the stereo parts or downmixing the surround parts yet...

So in my helplessness, I ran to the great machine god (ChatGPT) and asked it to fix my mess, but it didn't produce anything that would've worked either. Most commands it gave me only produced errors. ;)

So here I am, asking for help. :)

Inputs that require remixing:

1.) 2.0: Layout "L R", sample size 16 bits
2.) 5.1: Layout "L R C LFE Ls Rs", sample size 24 bits

My desired outputs:

a.) 2.0: Layout "L R", sample size 16 bits
b.) 5.1: Layout "C L R Ls Rs LFE", sample size 24 bits

All files come with a 48 kHz sampling frequency. Every single input file needs to be remixed and partially resampled to match each given output format. So "1." needs to be converted to both "a." and "b." and "2." needs the same.

If possible, I'd like not to throw channels away entirely, so that the resulting streams lose as little audio information as possible.

For getting the channel layouts I use mediainfo, as mentioned, in the hope that it produces correct information.

It would be really nice if you could give me the correct commands for this, and even nicer if you could explain them to me as well, so I can finally understand this stuff. :)

Thanks a lot! :)

Hello,

Remixing audio channels can indeed be complex. To achieve your goals using FFmpeg, you'll need to carefully map input channels to your desired output layouts. Here's how you can approach each conversion:​

1. Converting Stereo (L R) to 5.1 Surround (C L R Ls Rs LFE):

To upmix a stereo file to a 5.1 layout with the specific channel order, you can use the pan filter:​

ffmpeg -i stereo_input.wav -filter_complex \
"pan=5.1|c0=0.5*c0+0.5*c1|c1=c0|c2=c1|c3=0.5*c0+0.5*c1|c4=0.5*c0+0.5*c1|c5=0" \
-ac 6 -c:a pcm_s24le output_5.1.wav

Explanation:

pan=5.1: Specifies the output will have 6 channels.​
Stack Overflow
+1
FFmpeg Trac
+1

c0=0.5*c0+0.5*c1: Creates the center channel by mixing left and right channels at half volume.​
Super User
+2
dpin.de
+2
hydrogenaud.io
+2

c1=c0 and c2=c1: Maps left and right input channels directly to left and right output channels.​

c3=0.5*c0+0.5*c1 and c4=0.5*c0+0.5*c1: Creates the surround channels by mixing left and right channels.​

c5=0: Sets the LFE (Low-Frequency Effects) channel to silence.​
Super User

This method ensures that no audio information is discarded.​
ayosec.github.io
+6
VideoHelp Forum
+6
Stack Overflow
+6

2. Reordering 5.1 Channels from (L R C LFE Ls Rs) to (C L R Ls Rs LFE):

To reorder the channels, use the channelmap filter:​
ayosec.github.io

bash
Copy
Edit
ffmpeg -i input_5.1.wav -filter_complex \
"channelmap=0|1|2|3|4|5:5.1" \
-c:a pcm_s24le output_reordered_5.1.wav
Explanation:

channelmap=0|1|2|3|4|5:5.1: Maps input channels to the specified output order. Ensure that the mapping corresponds to your desired layout.​
ayosec.github.io

3. Downmixing 5.1 (L R C LFE Ls Rs) to Stereo (L R):

To downmix a 5.1 file to stereo, you can use the pan filter:​
Flaeri's tech talk
+1
Super User
+1

bash
Copy
Edit
ffmpeg -i input_5.1.wav -filter_complex \
"pan=stereo|c0=0.5*c0+0.707*c2+0.707*c4+0.5*c3|c1=0.5*c1+0.707*c2+0.707*c5+0.5*c3" \
-ac 2 -c:a pcm_s16le output_stereo.wav
Explanation:

pan=stereo: Specifies the output will have 2 channels.​

c0=0.5*c0+0.707*c2+0.707*c4+0.5*c3: Creates the left channel by mixing front left, center, left surround, and LFE channels.​

c1=0.5*c1+0.707*c2+0.707*c5+0.5*c3: Creates the right channel by mixing front right, center, right surround, and LFE channels.​
FFmpeg Trac

This approach maintains as much audio information as possible during the downmix.​

Resampling:

If you need to resample audio to match specific sample rates or formats, FFmpeg's aresample filter can be used:​

bash
Copy
Edit
ffmpeg -i input.wav -af "aresample=48000" -c:a pcm_s16le output_resampled.wav
Explanation:

aresample=48000: Resamples the audio to 48 kHz.​

-c:a pcm_s16le: Sets the audio codec to PCM 16-bit little-endian.​

Additional Tips:

Channel Layouts: FFmpeg's default channel layouts might differ from your expectations. It's crucial to specify and verify channel mappings explicitly.​

Verification: After processing, use tools like ffmpeg -i output.wav or mediainfo output.wav to verify the channel layout and ensure correctness.​

Documentation: Refer to FFmpeg's official documentation and community forums for more detailed explanations and examples.​

By carefully mapping and verifying each step, you can achieve the desired audio configurations without losing information.​

I hope this helps clarify the process.​

[tebasuna51]

ffmpeg -i stereo_input.wav -filter_complex \
"pan=5.1|c0=0.5*c0+0.5*c1|c1=c0|c2=c1|c3=0.5*c0+0.5*c1|c4=0.5*c0+0.5*c1|c5=0" \
-ac 6 -c:a pcm_s24le output_5.1.wav​

Please read my post, that upmix don't have any sense talking of wav files.