nuked
12th July 2003, 00:23
Ok, as many of you have seen I'm trying to understand
and create a consensus on fundamentally what's possible
with regards to compression and tricks to store surround
information. It seems like old analog ideas and
especially compatibility requirements for standalone
equipment is holding things back. We need a better
parameterization of frequency, phase and spatial
information along with compatible compression
algorthims. Speaker based representations(or simple
transformations of them) will never be ideal.
Fundamentally there is no question that 5 channels is
more than 2. It may be possible to distort the 2
channel signal in some way which is nearly imperceptible
to the ear to store 5 channels of info in some sense.
This is basically just taking advantages of
ineficeincies in the compression scheme though, since
the cpmression scheme should have removed the
pshcoacoustially unuseless data parameters in the first
place.
Three examples( these are long and skippable):
1)
The simplest such shecme is to store the extra
channeles in the really high inaudible frequency range
with a direct one to one frequency map. This is no more
than a silly labelling scheme in the data storage though
and you could just compressed out the high frequencies
in the first place and should and generally do, and then
if you want another channel just write another channel
and takes the same amount of space. the above scheme
could possibly be useful for storing extra info on
backward compatible CD's though, but again, thats the
type of vice that should be phased out.
2)
dolby crudely tries to use our ears inability to notice
polarity info. Unfortunately a) our ear can notice it
somewhat, and b) it's not linearly-independent from
volume info, so info from back channels leaks into
volume in front channels in ways it shouldn't (for
stereo playback) on ecncode, and more so, volume info
from front channels leaks into back channels on decode
and things get generally washed out a bit. The stereo
playback quality is comprimised some for a better
surround effect no matter what dolby says about
"compatibility". Pro logic 2 claims to steer and fix
this somewhat aftificially and is not without it's own
artifacts which I find even more irritating than DS.
3) it looks like encoding B-format into UHJ stereo tries
to use our inability to distinguish true phase info to
encode in each frequency using true 90 degree phase
offsets. This will probably enhance spatial volume
separation at the cost of spatial phase info which our
ears can detect a little, but much less than volume so
it's probably a good trade off but may also cost some in
waveform distortion and maybe thus even screw with
harmonics a little. If 90 degree phase info wasn't
relavent and could be played with at will with
"tolerable" distortion, it shouldn't have been stored in
the first place.
My ideas:
now I certainly don't understand ALL the details of all
these down/upmixing tricks entirely properly as they
indeed are complicated in practice, but I understand
what lineraly independant information is and there's no
getting around it. What's needed is a completely
lineraly independent parameterization of the data with
the parameters transformed so as to give a maximally
hierarchical physco-acoustic priority ordering to the
parameters. Such a parametarization is certain to have
it's spatial iformation linked to a raw coordinate
expansion and not to speakers as all the encoders now
are.
For the future,(not for your current mp3s obviously) now
that everything is digital, is seems like focus should
be put on compression schemes which fully remove
phsycoacoustically useless(yes there's middle ground
which is why there are bit-rates of course) including
any data that's truly redundant between channels. Once
this is done it will be essentially IMPOSSIBLE to do ANY
tricks of encoding extra "upmix"(we should even think in
terms of upmix becasue its a speaker representation and
not a spatial one) information without increase of
bit-rate or comprimising some quality at least at the
threshold qualitly level that was chosen on source
encode. This is still not pointless if the vice of
relying on stereo compatibility is lifted so that at
least artifacts are not created. to explain: The
upmixed surround info shoudl be encoded IN PLACE OF or
as well as the next least significant bits of
information acording to the established parameter
hierarchy, but it shoudl not be entangled with that data
to produce unentagleable artifacts. Of course an
apropriate decoder is needed and this philosphy would
have to draw a line in the sand and the rules would
still need to be bent to produce anything backward
compatible with hardware made before today. But
hardware makers are responding quickly these days to
software demands.
You then encode as much info as you want, it takes
however much space it takes, and you cannot do any
better than that at the given quality level. Indeed it
seems like a compression of true B-format encoded
material(which could probably be made from a 5.1) might
be close to this ideal if higher bit-rate priority is
given to W and then X and then Y and then Z (optionally)
and then higher order spatial harmonics to create shape
and size for the sound sources(all as a function of
frequency). Probably a valuable extension would be a
signal that could describe mixes of multiple source
locations for a give frequency. One would have to
subjectively decide the hierarchy of expansion cut-offs
for the various signals obviously and some freedom could
be left to the user. Any encode could then be played on
any speaker arrangement with various degrees of spatial
separation in diferent axes depending on the encoding.
It could be VERY NICE to comrpess y and z data almost
equally or better, in chosen amounts. That way I could
include a LITTLE extra surround info with my stereo mix
on the side if I want in a way that's better than
producing separate poor quality rear channel streams or
mixing the info ireversibly with the front channels.
There's no point in trying to store the Y result
entangled is some trickery with teh X and W streams and
add distortions. (Y is front back right, or do I have
it backwards... anyway) If you'll accept less W quality
to get more Y, then compress the W more, but store Y
independantly in the Y stream and make compatibilitly
software only to produce stuff for old standalone
machines..
Actually it seems like AC3 is a little closer to this
model than I realized but still with too much focus on
speaker channels and encoders are still weak(or jsut
getting there?), and from what I've heard (which is very
little) results with existing consumer encoders are
still worse than mp3 size for size, but I could be wrong
on that as I haven't tried ac3 encodes enough recently.
AC3 though has no way to chose how much Y and Z
repsectively are kept without using old fashiond
pro-logic to mix that data into the audible stream.
You can only chose how many speakers to keep which is
certainly not the best basis for the information.
Internally it seems to be reparameterized some but it's
still basically a speaker oriented basis as oposed to a
spatially oriented one, and I think most of the encoders
are.
So it seems like this kinda exists in B format, the
thing that's needed are maybe better transcoders,
compressors and on the fly decompression software for
it.
Any thouhgts? Does something like this exist and I just
have missed it entirely. Am I just dreaming. Yes yes,
I know.. then code it yourself....ok ok, no I can't or at
least wont, but it's just an idea.
nukeD
and create a consensus on fundamentally what's possible
with regards to compression and tricks to store surround
information. It seems like old analog ideas and
especially compatibility requirements for standalone
equipment is holding things back. We need a better
parameterization of frequency, phase and spatial
information along with compatible compression
algorthims. Speaker based representations(or simple
transformations of them) will never be ideal.
Fundamentally there is no question that 5 channels is
more than 2. It may be possible to distort the 2
channel signal in some way which is nearly imperceptible
to the ear to store 5 channels of info in some sense.
This is basically just taking advantages of
ineficeincies in the compression scheme though, since
the cpmression scheme should have removed the
pshcoacoustially unuseless data parameters in the first
place.
Three examples( these are long and skippable):
1)
The simplest such shecme is to store the extra
channeles in the really high inaudible frequency range
with a direct one to one frequency map. This is no more
than a silly labelling scheme in the data storage though
and you could just compressed out the high frequencies
in the first place and should and generally do, and then
if you want another channel just write another channel
and takes the same amount of space. the above scheme
could possibly be useful for storing extra info on
backward compatible CD's though, but again, thats the
type of vice that should be phased out.
2)
dolby crudely tries to use our ears inability to notice
polarity info. Unfortunately a) our ear can notice it
somewhat, and b) it's not linearly-independent from
volume info, so info from back channels leaks into
volume in front channels in ways it shouldn't (for
stereo playback) on ecncode, and more so, volume info
from front channels leaks into back channels on decode
and things get generally washed out a bit. The stereo
playback quality is comprimised some for a better
surround effect no matter what dolby says about
"compatibility". Pro logic 2 claims to steer and fix
this somewhat aftificially and is not without it's own
artifacts which I find even more irritating than DS.
3) it looks like encoding B-format into UHJ stereo tries
to use our inability to distinguish true phase info to
encode in each frequency using true 90 degree phase
offsets. This will probably enhance spatial volume
separation at the cost of spatial phase info which our
ears can detect a little, but much less than volume so
it's probably a good trade off but may also cost some in
waveform distortion and maybe thus even screw with
harmonics a little. If 90 degree phase info wasn't
relavent and could be played with at will with
"tolerable" distortion, it shouldn't have been stored in
the first place.
My ideas:
now I certainly don't understand ALL the details of all
these down/upmixing tricks entirely properly as they
indeed are complicated in practice, but I understand
what lineraly independant information is and there's no
getting around it. What's needed is a completely
lineraly independent parameterization of the data with
the parameters transformed so as to give a maximally
hierarchical physco-acoustic priority ordering to the
parameters. Such a parametarization is certain to have
it's spatial iformation linked to a raw coordinate
expansion and not to speakers as all the encoders now
are.
For the future,(not for your current mp3s obviously) now
that everything is digital, is seems like focus should
be put on compression schemes which fully remove
phsycoacoustically useless(yes there's middle ground
which is why there are bit-rates of course) including
any data that's truly redundant between channels. Once
this is done it will be essentially IMPOSSIBLE to do ANY
tricks of encoding extra "upmix"(we should even think in
terms of upmix becasue its a speaker representation and
not a spatial one) information without increase of
bit-rate or comprimising some quality at least at the
threshold qualitly level that was chosen on source
encode. This is still not pointless if the vice of
relying on stereo compatibility is lifted so that at
least artifacts are not created. to explain: The
upmixed surround info shoudl be encoded IN PLACE OF or
as well as the next least significant bits of
information acording to the established parameter
hierarchy, but it shoudl not be entangled with that data
to produce unentagleable artifacts. Of course an
apropriate decoder is needed and this philosphy would
have to draw a line in the sand and the rules would
still need to be bent to produce anything backward
compatible with hardware made before today. But
hardware makers are responding quickly these days to
software demands.
You then encode as much info as you want, it takes
however much space it takes, and you cannot do any
better than that at the given quality level. Indeed it
seems like a compression of true B-format encoded
material(which could probably be made from a 5.1) might
be close to this ideal if higher bit-rate priority is
given to W and then X and then Y and then Z (optionally)
and then higher order spatial harmonics to create shape
and size for the sound sources(all as a function of
frequency). Probably a valuable extension would be a
signal that could describe mixes of multiple source
locations for a give frequency. One would have to
subjectively decide the hierarchy of expansion cut-offs
for the various signals obviously and some freedom could
be left to the user. Any encode could then be played on
any speaker arrangement with various degrees of spatial
separation in diferent axes depending on the encoding.
It could be VERY NICE to comrpess y and z data almost
equally or better, in chosen amounts. That way I could
include a LITTLE extra surround info with my stereo mix
on the side if I want in a way that's better than
producing separate poor quality rear channel streams or
mixing the info ireversibly with the front channels.
There's no point in trying to store the Y result
entangled is some trickery with teh X and W streams and
add distortions. (Y is front back right, or do I have
it backwards... anyway) If you'll accept less W quality
to get more Y, then compress the W more, but store Y
independantly in the Y stream and make compatibilitly
software only to produce stuff for old standalone
machines..
Actually it seems like AC3 is a little closer to this
model than I realized but still with too much focus on
speaker channels and encoders are still weak(or jsut
getting there?), and from what I've heard (which is very
little) results with existing consumer encoders are
still worse than mp3 size for size, but I could be wrong
on that as I haven't tried ac3 encodes enough recently.
AC3 though has no way to chose how much Y and Z
repsectively are kept without using old fashiond
pro-logic to mix that data into the audible stream.
You can only chose how many speakers to keep which is
certainly not the best basis for the information.
Internally it seems to be reparameterized some but it's
still basically a speaker oriented basis as oposed to a
spatially oriented one, and I think most of the encoders
are.
So it seems like this kinda exists in B format, the
thing that's needed are maybe better transcoders,
compressors and on the fly decompression software for
it.
Any thouhgts? Does something like this exist and I just
have missed it entirely. Am I just dreaming. Yes yes,
I know.. then code it yourself....ok ok, no I can't or at
least wont, but it's just an idea.
nukeD