Funtime is over ! Here is some serious listening is required !

Come on guys,

lets show the world what the best audio encoding forums in ripping community is !!

You certainly remember my thread here http://forum.doom9.org/showthread.php?s=&threadid=21718 and seen DSPgurus other thread about this very subject here http://forum.doom9.org/showthread.php?s=&threadid=22092 about using vocoders for that.

Our aim is to prove that it is possible to store different languages in one single multichannel audio stream instead of using more streams, one for each language, and to save a lot of space doing so as the encoder could use its 'channel coupling' capabilities to only encode those parts of the soundtrack that are really dfferent between the two ( or three ) languages.

As channel coupling in Vorbis is not existing for more than a Stereo ( 2 ch ) encoding i was using a trick to be able to find out if this is feasible at all :

1. I decoded the German and the English AC3 tracks of Matrix to WAV ( INT 16 Bit ) using Azid , downmix was 5.1 to 2 ( Stereo, no DPL )

2. I used Cooledit to cut out the L channel of both languages and merged them in a new Stereo WAV. So we have the L ch german and the L ch English in one Stereo file. If you play this file you can hear English from L speaker and German sound from R speaker.

3. I encoded this WAV using oggifier with the experimental oggenc.exe 's from Roberto's site ( downloaded today ), using oggencHL.exe ( highly lossy = max. channel coupling ) and oggencNL.exe ( No Lossy = no channel coupling ) with same q values.

4. To make things a bit more difficult for you i was also doing a 2nd encoding with HL and a higher q value, so that file sizes are the same for NL encoding with lower q value.

Here are the files ( each 2.5 MB ) :

Matrix1
Matrix2
Matrix3

Important Note : If any of you suckers think about molestating my 3 year old girl ( its her account ) just do it suckers, you will get to know my lawyer !!!!

This what you have to do now :

1. Download all 3 files
2. Put your speakers together, so they are like one mono speaker
3. Play all the files ( as often as you want ), but with the balance slider always pointing to one channel only ( German or English )
4. Rate the files ! Always bare in mind that for those tracks being encoded with highly lossy channel coupling one of the two may sound much worse than the other !!!

Reply to this thread here ( no emails !! ) and rate all three files. In one week from now i will tell you the settings for the three and what coding mode was used on what track ... !!

[Selur]

"2. Put your speakers together, so they are like one mono speaker"
Ehmm, would it be okay to use once Headseats instead?
(easier to move, without having to clear up the whole cave )

Cu Selur

[MaTTeR]

I'm using headphones as well shifted to one channel(English).

I honestly hear very little difference between file 2 and 3. However, file 1 sounded somewhat flat and not as rich compared to 1 and 2. Bass levels sounded the same in all 3 files but trebles sounded worse in 1, like the treble was muddy. This is being very critical though :-)

I should point out that in my experience, Heavy Lossy channel coupling does not provide optimal DPL sound. Light Lossy seems to be the sweet spot if downmixing to DPL.

So my ears rate files 2 & 3 a tie. File 1 would come last.

MaTTer,

i dont expect we will be able to use DPL surround downmix in the end when doing a multilingual encoding into one stream, i would expect that the phase shifts caused by DPL matrixing makes things much harder for the codec in terms of 'finding' common information being useful for channel coupling feature ..... i maybe wrong though ...

One more thing about the rating :

We will not give scores for sound quality, as this wouldnt be a sensible thing to do in this case. What we want to know is :

- is any of the channels distorted, means is the channel coupling of Vorbis messing things for any of the language channels .... so there is not much use in you listening to English track only !!! You dont know whether L, R or both channels are affected by coupling, so you have to do the same with German track again i fear ... but only one channel at a time of course !

- These are mono tracks, so it will be very hard to rate sound quality ... but of course any info on this is highly appreciated

- If you really want to support this test, dont look at the filesizes before doing the test !!! ... is it really by coincidence you were rating the smallest file the worst ... i tell you a secret ..... doesnt have to be this way !!

Stop doing the test guys ....

Here is a copy of my conversation with Monty and Michael Smith from today :

<ChrisHJW> vakor : Hi .. you had a chance to have a short listening test with the multilanguage samples ?
<ChrisHJW> link is here : http://forum.doom9.org/showthread.php?s=&threadid=22283
<ChrisHJW> including a short description how they were done
<ChrisHJW> sorry for 2nd language being German !
<Vakor> ChrisHJW: no. Don't really want to either, that's a horrid way of doing multilingual audio, and won't be supported by compliant decoders.
<Gumboot> Well there's positive feedback.
<Gumboot> What's the method?
<ChrisHJW> Vakor : your personal opinion ?
<Vakor> Fact: won't be supported by compliant decoders. Personal opinion: it's a horrid way of doing that.
<ChrisHJW> well .. for the decoder this is simply a normal audio stream containing more than 2 channels
<ChrisHJW> like 4, 6 , 8 , etc.
<ChrisHJW> you decode them all, but forward only the corresponding ones to the soundcard
<ChrisHJW> and drop the rest
<ChrisHJW> so whats the problem ?
<Vakor> By the way, we've basically decided that it's neccesary (because the format and decoders are frozen, other than bugfixes or optimisations in the decoder) to support channel mappings in libvorbis I/mapping 0 by hard-coding 1 channel=mono, 2 channels=stereo, 4 channels=quad, 5 channels = FL, FR, C, BL, BR, 6 channels = 5 channels plus sub.. Whilst a generic channel mapping would be nice, that'll have to wait for mapping 1 (vorbis II)
<Vakor> The PROBLEM is that a correct decoder can't then decode it correctly. And changing the decoder is not an option.
<xiphmont> Vakor: that's my current working theory that will be coded unless I'm convinced otherwise. Not written in stone yet. But tstill, multilingual should *definately* be multistream I agree.
<Vakor> xiphmont: yeah, ok. Sorry, I thought you were stating it before as "this is how it shall be".
<ChrisHJW> Vakor : i see your point here ... but for us its easy to avoid such probs as the DirectShowFilter will allow us to change channel mapping easily
<ChrisHJW> so .. as the test ( with RC3 ) resulted in a 25% saving on bitrate for german and english
<Vakor> ChrisHJW: That is NOT an excuse. If you want something to be done, you have to do it properly and generally - not just for a single implementation on a single platform.
<ChrisHJW> i see
<ChrisHJW> but .. is it really such a big problem to include this into Vorbis specs ?
<Vakor> xiphmont: Can't say I've ever read the Declaration of Independence. I'll just have to believe you :-)
<Vakor> ChrisHJW: yes, I already said that - the format has been frozen, the decoder is final.
<ChrisHJW> irrepsective of whether other people ( audio encoders, aiming for highest quality ) will ever use it ?
<ChrisHJW>
<ChrisHJW> :sniff:
<xiphmont> ChrisHJW: when something goes into a spec, everyone who wishes to comply witht he spec is mandated to implement it. Even if they do not care about the feature in question. They have to handle it.
<Vakor> ChrisHJW: I don't see the problem anyway. Using a single stream for multiple languages seems fundamentally to be the wrong way to go about it.
<xiphmont> And your savings are illusory. Your getting that because channel coupling is causing cross-bleeding.
<ChrisHJW> vakor : if space is not an issue you are of course right
<ChrisHJW> xiphmont : well, this is why i was trying to setup the listening test
<Gumboot> ChrisHJW: Is it some kind of weird foreign humour I don't understand?
<Vakor> ChrisHJW: When space is an issue, do people _really_ provide multilingual audio? I doubt it, personally.
<ChrisHJW> to find out
<Gumboot> Isn't the stereo channel coupling somewhat incorrect for that job?
<Vakor> Today is Job Application Day.
<Vakor> Gumboot: completely, rather than somewhat, but you're on the right track :-)
<Gumboot> Vakor: Well, ignoring a lot of factors, I mean.
<ChrisHJW> Vakor : could we agree that we first wait for the results of the listening test before completely dropping the possibiity ?
<Gumboot> Presumably some kind of coupling makes sense if the speech is mixed with other sound... but stereo coupling takes into account that it's stereo coupling.
<ChrisHJW> you know ... we rippers have different perception of quality as you guys have
<Vakor> ChrisHJW: listening test is irrelevent. That's the wrong way to do it, and that won't change. It won't be allowed. Monty agrees that it's the wrong way to do it, and Monty has Final Say On All Thing<ChrisHJW> ok ... thanks
<ChrisHJW> Bye for now
<Vakor> Gumboot: well, it could be configured to suck less, but crackmonkey appears to have abandoned it here.
<Gumboot> An orphaned bot? That is sad.
<Vakor> ChrisHJW: in short, please don't do it. It will be incorrect, unportable, and buggy. Bye.s Vorbis.


Conclusion :

Monty is convinced the savings i got are not real ... but more likely to be the product of encoder messing up channel separation ( could you hear any english when listening to your german track ? ) and Multilingual en/decoding will not make its way into Vorbis specs it seems .... sorry for all the hazzle ...

I still do believe it should be possible and with some tuning we even might be able to get a better saving than the 25% i got in first try ... but the Vorbis encoders are audio freaks aiming for highest possible quality, they dont understand that for us it would just be nice to be able to add another language track to our movies ( even with lower sound quality, allthough i honestly could find any sound degradation ) if it doesnt cost too much extra bitrate. But what can you do ...

[DSPguru]

i fully understand why the developers wouldn't want to change the decoding spec. but i heard lots of talkings about adding wavelet transforms to RC4.
won't this chanGe the decoding spec ?!

[TobiasWaldvogel]

I agree that it is really difficult to organize multiple languages in one stream because then the language selection must be done in the decoder rather than the splitter. It will be quite difficult and confusing organizing something like that. I prefer the following approach:

- Define one stream as the "master stream"
- Compare the encoded vorbis packets and drop identical packets

For the playback:

- If there is a packet take it from the selected stream otherwise from the master stream.

I will do some tests and publish the results.

[DSPguru]

Originally posted by TobiasWaldvogel
- Compare the encoded vorbis packets and drop identical packets

i'm sceptic. even if the audio is similar, the packets could be different.
there's no way to implement this without doing signal-processing directly on the pcm samples.

Dg.

DG, Tobias,

thanks for your replies !

@DG : well, Vorbis decoder spec wasnt closed/finished until recently, seems that my tests were a bit too late ... anyway, if Vorbis dev team had a better feeling about the whole issue there would be a way to bring it into the specs for sure ... but i guess the two worlds ( audio encoding and ripping i mean ) are just too different ...

About how to do the decoding : There should be a very easy way to implement this into the specs even without leaving all existant decoders redundant :

What we have here is one standard multichannel audio stream, which is in normal Vorbis specs anyway. Only difference here is that we dont want to listen to all of the channels simultaneously ( could be quite a mess with three languages ), but only to two of them, being the one language ( Stereo ) we want. For playback this means we have to
- decode all channels same time
- forward the two corresponding channels to the soundcard ( WAVE-OUT )
- drop the rest
I am not sure here but it should be possible to tag the Vorbis stream such that every old decoder can at least read the first 2 channels ( = language 1 = main language ) and doesnt see the other channel/languages at all. Newer versions of the decoder should allow the user to select the language he wants, or maybe the DirectShowFilter could do also. All players with 'advanced Vorbis support' like PowerDivX or BSPlayer should be able to select languages from the DSF API then.

@Tobias : interesting idea ! Is this compatible with Vorbis specs ? I am not sure here but i really dont feel we should be doing anything without being approved by Vorbis-dev team ... we dont want any more hacks for ripping, we really do have too many already IMHO !!

[DSPguru]

Originally posted by ChristianHJW
DG, Tobias,

thanks for your replies !

@DG : well, Vorbis decoder spec wasnt closed/finished until recently, seems that my tests were a bit too late ... anyway, if Vorbis dev team had a better feeling about the whole issue there would be a way to bring it into the specs for sure ...

does the spec includes HWT ? and what about Speex ?
i'm sorry, but i have a strong feeling that (unforuntly) the spec will be re-openned.

but i guess the two worlds ( audio encoding and ripping i mean ) are just too different ...

imho, my other suggestion would complicate both encoders and decoders, but will keep high quality with high compression-gain.

About how to do the decoding : There should be a very easy way to implement this into the specs even without leaving all existant decoders redundant...

interesting idea, Chris. few comments :
in the irc log , it been said : only static mapping will be offered. meaning a 4ch track is considered as L+R,SL+SR.

now, i can only *guess* that, but there will be a channel coupling in the following form :
L coupled with R, SL coupled with SR.
L+R would have higher bitrate/quality, than SL+SR. (a guess, again).

so,
let's say we have Le,Re (english, stereo), and Li,Re (italian, stereo).

we would have to tweak the channels mapping as follows :
L=Le
R=Li
SL=Re
SR=Ri

this will give us coupling between the different tracks, BUT, the right channel will suffer from low-quality, comparing to the left channel. yuck !

and what happens when we want to have 3 languages ? a problem !
no more coupling relations. we only have independet LFE & Center, not to mention that LFE has a narrow bandwidth and "suffers" from really low bitrates.

@Tobias
great picture, dude ! reminds somewhat of James LaBrie from Dream Theater .

[TobiasWaldvogel]

Something I forgot:

<Vakor> By the way, we've basically decided that it's neccesary (because the format and decoders are frozen, other than bugfixes or optimisations in the decoder) to support channel mappings in libvorbis I/mapping 0 by hard-coding 1 channel=mono, 2 channels=stereo, 4 channels=quad, 5 channels = FL, FR, C, BL, BR, 6 channels = 5 channels plus sub.. Whilst a generic channel mapping would be nice, that'll have to wait for mapping 1 (vorbis II)

At least it seems that we have an "official" statement how to do channel mapping for up to 6 channels until further notice.

[DSPguru]

Originally posted by TobiasWaldvogel
At least it seems that we have an "official" statement how to do channel mapping for up to 6 channels until further notice.

yeap.

btw, please read my other posts to you.

Dg.

[DarkAvenger]

Guys, if Vorbis decoder is final, who cares As DG states, Wavelets are coming into Vorbis II, so these specs are open. (Or am I mistaken?)
I guess the channel mapping mode of Vorbis sucks, as 4.1 is impossible, isn't it?

I agree DA ... it wasnt very well implemented i guess ... it seems that Vorbis-dev focus is not so much on DVD conversion or video encoding in general ... but lets hope for next specs as DG pointed out ...

Just for you info, as this issue was raised again :

Sample 1 : Highly Lossy coupling, q = 1
Sample 2 : Highly Lossy coupling, q = 2.5
Sample 3 : Lossless Coupling, q = 1

Using highly lossy coupling i could use q of 2.5 compared to 1 to get same file size.

Just out of curiosity : anybody had results to tell us about ?

[Antimon]

Um...you have 2 left chanals...pro logic is impossible, especially since you're only utilizing a single chanal at a time...

So am i to udnerstand thsi is just a proof of concept test not the final way fodoing it, but long term you are talking multi chanal with each stream coupled like this?

thast just my intial post, downloading now

[Antimon]

Well first thing i noticed is the right chanal is lower in volume then the left

Thsi is pretty common actually in stereo files for some reason, so i guess you'd have to boost the right pre encoding so ti matches up

There is no chanal bleeding that i can hear....


Matrix 2 sounds better it's richer but thats expected because of the higher bitrate but ya knwo what you need to do?

Make an english only track now with 2 lefts using the same settings and compare fiel sizes of the duel english to the multi laung.

Somthign else i didn;t think of initially the speaking doesn;t even occor at the same time, :-) thats really going to screw with compresability cus it narrows the simileraties between tracks even further.

Were these streams the same? both 5.1 or both pro logic, or is it one of each?

So, you've sold me on the quality viability of your idea

Now you need to sell me on the file size savings viability.

Right now you have a stereo file with 2 mono laungages

this is what i sugest

it's a lot of leg work


Make the following:

a stereo l/r english only oog at each of the 3 settings you used for this and a stereo german at each of the same settings, add up the fiel siz of the english and german that share the same paramiters

add the 3 tests you did and repeat them with dual right chanals
Now add the dual left and the duel right of matching settings.

How do they compare to teh identicle settigns of the 2 true stereo 2 streams.

They need to be smaller to get the saved bits you;re thinking you'll get, if we're to maintain stereo audio and do your pass select tracks to decoder

Optioanlly i'd also like to see a test like i said before with duel left english usign the 3 settings compared to the duel left multi laug.

the duel english SHOULD be smaller then the duel laung which should be smaller then the l/r english

thats a good test of how much coupling is actually going on.

feel like doing all that?

Originally posted by Antimon feel like doing all that?

... simply answer : no ! I honestly dont have the time to do it now, i am in a very chaotic period talking about my private life ( 2nd baby, moving to new flat, bussiness going well, etc. ) so i hardly see my PC at home. And i cant do it here at work, for abvious reasons.

If somebody here is willingly to invest some time with Cooledit or any other proggie preparing some new test tracks with 2 english L channels to find out how good channel coupling is in very best case, PM me here and i will tell you how to setup the highly lossy channel coupling feature. You have to use special experimental oggenc.exe for this to be found on Roberto Amorims site : http://www.inf.ufpr.br/~rja00/ogg.html . Download the oggenc. package including the experimentals, you find oggencHL.exe with it. Now you can either go CLI or rename it to oggenc.exe ( as i have done ) and use the 'Oggifier' GUI with it.

BTW : the matrix 5.1 AC3 sources were downmixed and decoded to WAV with latest Azid to a normal 2 CH Stereo filewith DRC 'normal', so no DPL surround info in there.

@antimon, @all : anybody found the time to look this up a bit further ?