Last try : more than one language in one multichannel stream

[ChristianHJW]

Hi there,

i am demotivated :( .. seems as if i was investing a lot of time into something thats not supposed to work ?

Here a record of my recent conversation with Michael Smith ( vakor ) from the Vorbis dev team ( thread is here : http://www.powerdivx.com/phpbb2/viewtopic.php?t=382 ) :

<ChrisHJW> I am still trying to prove it should be possible to encode different language movie tracks into one single audio stream with very low extra bitrate
<ChrisHJW> if channel coupling can be used, assuming there is much common sound infomation between the individual languages
<ChrisHJW> as there is no channel coupling for multichannel Vorbis for now
<Bob_> but probably one of the most wanted features
<ChrisHJW> i was suggesting to do a 2ch encoding using f.e the left channel of the english and the left channel of the german soundtrack of a movie
<Bob_> in some variation or another at least
<ChrisHJW> and use Cooledit to make a 2 ch WAV from it
<ChrisHJW> now here is my question :
<ChrisHJW> what can i do to find out what extra bitrate ( compared to mono ? ) Vorbis encoder will need for the 2nd channel with same quality ?
<ChrisHJW> My test ( planned ) :
<ChrisHJW> Encode L channel of english alone ( mono )
<ChrisHJW> 2. add L channel german to that to make it 2 ch and encode with OggDrop with lossy channel coupling ?
<ChrisHJW> Any objections ? am i wrong in my assumption this could help to find out how many common information there is between the individual langauges in one movie ?
<ChrisHJW> I wanted to do this for a couple of movies to get a good overall idea if it was feasible at all
<Vakor> ChrisHJW: that's a terribly ugly hacky way of doing multi-language encoding, given that ogg is designed to do that sort of thing nicely by giving you multiplexed streams. That would suck.
<Vakor> Whilst your test will be roughly accurate (there's some overhead from the headers and framing data, but it's small enough you can probably ignore it), it won't sound good. You need to do lossless coupling, which requires some minor tweaks to the encoder mode setup
<ChrisHJW> vakor : could you please explain 'multiplexed streams' ?
<ChrisHJW> Its standard to put one stream for each language with your movie
<Vakor> yes
<ChrisHJW> but its always double the bitrate you need
<Vakor> multiple logical streams simultaneously in a single physical bitstream --> multiplexed streams
<ChrisHJW> my idea is aiming towards being capable of adding a 2nd language into a 128 kbps stream
<Vakor> yes, that's the price you pay for having two independent soundtracks.
<ChrisHJW> compared to a let say 96 kbps for one language
<ChrisHJW> thanks to channel coupling
<ChrisHJW> every ripper has a 'main' language, in most cases his mother tongue
<Vakor> You'll either save very little or _kill_ quality, and you'll only be able to do it at all by adding horrible hacks - how do you distinguish between a stereo stream and two mono channels that are logically seperate streams?
<Vakor> Answer: without adding non-standard ugly hacks that will be very specific to your implementation, you can't. Don't do it.
<ChrisHJW> well, dont forget we want to define a channel mapping table for Vorbis
<ChrisHJW> so we could put it in now, no hacks !
<ChrisHJW> kill quality : if ever than only for the 2nd ( added ) language, main language will not be affected
<ChrisHJW> on playback the DSF would have to decode all channels but only give the desired language to the wave_out
<Vakor> no, because that has nothing to do with channel mappings.
<Vakor> And it WILL kill quality - it'll either degrade both, or make the secondary one completely unusable.
<Vakor> You could propose a channel mapping which allows that, but it wouldn't be accepted. It's not useful to have multiple languages if the second is going to be a completely horrible quality. It also WON'T save you nearly as much bitrate as you seem to think it will, because there's too little similarity.
<ChrisHJW> so if i use lossless channel coupling instead ?
<ChrisHJW> would quality be acceptable then ?
<ChrisHJW> let me try it first before we drop it completely
<ChrisHJW> is anything wrong with the way i want to try ?
<tangent3> it sounds like an ugly hack to me
<Vakor> ChrisHJW: quality would be substantially degraded (you kill preecho stuff), but much better. Of course, your bitrate won't be substantially reduced doing it that way.
<Vakor> ChrisHJW: it's ugly, will sound nasty, and is completely unacceptable for a public implementation. If you want to experiment, however, you can do so.
<ChrisHJW> Vakor : sorry if i come back to it, i got your message loud and clear ... you dont like the idea
<ChrisHJW> but the trial with two L channels is only ment to find out if it could save bits
<ChrisHJW> i dont want this in the specs
<ChrisHJW> lateron we should do this with Stereo only .. of course
<ChrisHJW> just wanted to clarify this
<ChrisHJW> thanks for your time and input

Guys, i want your input here !!

As this here is Doom9's audio forum i expect a bit more than just a comment. I'd like you all to get your DVDs out, rip/decode a few different language tracks to Stereo WAV's with BeSweet, edit them with Cooledit ( extract L or R channels from the WAVs ) and compare them by making Cooledit ditract one from the other ( like L english - L german, or R french - R english ) and post the graphs here ( if you want me to upload to my webspace mail me ).

Next try is to encode ( with OggDrop or similar )

1. L channel English ( mono )
2. a 2 ch WAV conatining L channel English and L channel German ( or French )
with same qualities ( like 2 ...3 ) and different channel coupling modes.


We are aiming to find out

- if there are any time delays between different language audio tracks, making the idea of channel coupling impossible
- if channel coupling ( only working for 2 ch audio for now, so dont test 4 ch !!! ) does work for different language tracks and what the extra bitrate is we need at different channel coupling modes, plus what the audio quality is like ( distortion, etc. )
- this may only be possible for 2 ch AC3's ?

We dont give up that quickly, do we :D !!!!

[MaTTeR]

*Bump to the top*

[TheHeap]

*bump*

is there any new statement from the developers concerning the requested feature..(me got currently no IRC access)

how it could be implemented??
(the devs didn't seem to like the "2*6ch in one stream" solution).


One IDEA:
So i suppose we could use 3*6ch in one stream with having one main set of the first 6 channels containing the similarities between the two language specific ac3streams(The AND result). The next two sets would contain the language specific audio (which wouldn`t take much bitrate cos hopefully most things are spoken on only one channel).

The changes for the vorbis format would be the introduction of a set header, which contains some pointers to the channel beginnings and a channel map with the correct channels to display for each language.

Second IDEA:
the problem of the above Solution would be that
- we would have to change the ogg specs ( not that critical cos its still under development)
- we would have a lot of unused information in the stream when just using one language. Especially when having more than two languages its senceless.
For that reasons i would suppose using a main sound stream (for explosions etc.) and two language specific streams (which hopefully would just consider one channel).

Correct me if i am wrong, but things like subtitles are stored in an own stream and overlayed over the movie. Same would be for the main audio stream and the specific streams.
So this won't be the problem, the only ""problem"" i see is in the part of the production of these streams.
the encoder would have to compare the ac3s and split the info onto the streams. (which would go very wrong if they aren't in sync).
But i think the HeadAC3 coders always like a new challenge *G*..


anyway.. just my .2 cents

gr33tz TheHeap

[DSPguru]

http://forum.doom9.org/showthread.php?s=&threadid=22092

[Antimon]

I do not belive this woudl work as you expect.

First we'll tackle the 2 chanal issue.

If you're going to do this, wouldn;t a more senceable way be to downmix the english crack form stereo to mono, and do the same with the say german track?

If you just take simply "left" or "right" chanals....you're losing half the audio information off the bat.....

Youcould do this, but i dont think you should use chanal coupling, and you wouldn't have to.....

Enstead of havign 2 chanals of the same track taking up say 96 kb/s now you'll have 2 chanals one from each stream, still taking up 96 kb/s you woudln;t have to do any special coupling aside from whatever you'd use normally for a 2 channal file. the vorbis people wouldn;t have to do anything but allow left or right playback only of the mono stream.

Some amps even suport this as this is how duel laungage vcd;s were created with 2 mono streams one left one right.

But really why do this?

Audio should be every bit as importnat as video, why be cheap with your bits? When peopel are trying to get 5.1 in reasonable sizes you're proposing mono, just to get 2 laungages into a single low bitrate audio track? Maybe for some movies...but it's nothing i'd want to listion to....mono should be killed :-)

Multi chanal oses even more problems as rarely is the audio in just the center chanal, it's ussual mirrored abiet lower volumes left and right, and all the echo's you'd have to track in the surounds, the chanal least affected is the lfe and even that gets affected y low voices depending on yoru cross over.

youcould never get a perfect split between the chanals you'd always have one laungage bleading into the other unless you then mixed in 12 chanals of audio and i doubt you'd save even half of one file with coupling .

The other thing to consdier is what if the laungages are differnt bitrates which often they are, the main is a 5.1 and the second audio tracks are pro logic so nwo you have ttoaly differntly encoded streams at differnt bitrates and you want todo chanall analyases on them?


Your overlay idea might work if you had say 4 chanals of audio 2 with just the stereo effect information and 2 mono audio only "center" chanals mixed in, but that eats up cpu time, and it would be dependant on no dialog going to any speaker other then center...

Then what do you do about dialog that pans from left to right as a person walks across screen lets say?

you either have to lsoe that pan information or it would morph from english to german, if it's coupled left/right to maintain all info.

As i said the onyl way this could work would be to make 2 mono streams from the original L/R sorce tracks and join them, but then at that point why not just multiplex 2 mono stream s enstead of 1 split audio stereo stream? unless it has very little dialog, you're not going to save much coupling. Voice is pretty full frequency range, and even similarities amongs that which predominate that part of the audio when talking wouldn;t match cus you have differnt peopel speaking the laungages, you'd only save in non dialog scenes any signifigant amount, and a very small amount during talking.

audio is only getting about 25% of the bitrate when it's really given full quality settings, i dotn think it's fair to start splitign that up amoung mulltiple streams now.

[outlyer]

@ChristianHJW:
maybe (well, SURE) I don't fully understant channel coupling, but I think that it wouldn't work that well, you're assuming that a lot of the sound is common, but that's really true, at least in my country (Spain) dubbed audio doesn't always sound as the original (I'm talking about when there's no speech, of course), almost always the voulme is different (but this is maybe covered by the channel coupling) and often one track is more "vivid" than the other (I say ONE because is not always the same), often too one track seems to been filtered in some way to compress its dynamic range or something similar.

Well, these are my thoughts on the matter, /ignore me if I'm only saying stupid things :rolleyes:

[Antimon]

actually outlyer raises a godo point that i had forgotten about, many times, they are redubed, even the sound effects and stuff often are completly differnt you can tell it was remixed and not using the music and effects stems from the original mix.

[dani82]

why extract the left and right audio, when you can move them to one side, so you won't lose anything.

1st, of course they are time delays between two different audio stream*, that's why people ask for help about a/v synch; but it's not impossibe, it's just takes time to fix them.

2nd, the audio quality should be as clear as the wav when you convert the ac3 file.

3rd, if you ever saw a chinese vcd, you would known that it is possible; that's why i do this, because it's possible.


*some more than other, some none.

[Antimon]

I mentioned the VCD market

My amp has a built in left/right spliter designed for duel laungage single streams

But as i said thats 2 mono tracks, and i'm not sure people would accept that, i know i wouldn't

thast why i said abotu combining the left right into left and the left right of stream 2 into right which is not what the first psot said simply takign left to left and left of 2 to right.

no matter hwo you do it, the tracks need to be identicle apart from the spoken laungage to be canadites for coupling, atleast if you expect any results.

with vcd, it's a FULL STEREO un coupled stream, so it works perfectly...ifyou start doing thsi with js mp3's or coupled vorbis i wodner hwo yoru stereo image would handal that. Things that would normally be coupled now can't be Like center chanal dialog enstead fo one shared center you have 2 full streams, so in reality yoru files woudl probably be larger cus you went from a large comon center to less comoninformation meanign you'll use more bits then you might used to.

The other issue becomes playback...can you play just a left or just a right stream of a coupled track, as they are as far as i understand it, dependant of each other. Though i guess thatd simply be a matter of processing the stream but nto outputing that chanal to teh playback device, simple enough.