MPEG-4 on XCD: already available Error Recovery Mechanisms

[bond]

the mpeg-4 standard (ie Simple- or Advanced-Simple-Profile) already supports tools which let you recover broken mpeg-4 bitstreams (audio and video) for example from scratched xcds

these tools are known under the headline error resilience. error resilience means that special types of bitstream writing are used, so that errors can be completely corrected or can be made as unannoying as possible:

for video there are three options available:
- Resync Markers/Slices, which split a frame into several independent video packets/slices. when a packet is lost, you can partially decode the frame based on the other video packets, the video packet size defines the size between two resync markers (it has to be activated to use the following options too)
- Reversible VLC permits to read the bitstream from 2 sides (fowards/backwards) to spot the error
- Data Partitioning separates the macroblock info into 2 parts: texture and motion vectors. it gathers data from several MB together (ie N x MB headers, then motion vectors of N macroblocks, and finally texture for these MBs)

for aac audio there are also three options available:
- Reversible VLC permits to read the bitstream from 2 sides (fowards/backwards) to spot the error
- Huffman Codeword Reordering reorders the codewords so that always a big part of the spectrum can be decoded if there is an error
- Virtual CodeBook

note that these tools will require more bits (make the files bigger), i dont know how much space each tool will require (the used intensity can also be individually adjustable). we will need to test that


also important is that you need encoders and (!) decoders which can handle these tools (recovery data is written by the encoder, the error correction is done by the decoder), decoders which dont understand error resilience cant decode such streams:

- resync markers:
encoding: ffmpeg/libavcodec (ffvfw?), nero (not in the consumer version), divx5 (until 5.0.2)
decoding: 3ivx, nero, divx5, enviviotv, ffmpeg/libavcodec (ffdshow), perhaps xvid
- reversible vlc:
encoding: nero (not in the consumer version)
decoding: nero, divx5, ffmpeg/libavcodec (ffdshow)
- data partitioning:
encoding: ffmpeg/libavcodec (ffvfw), nero (not in the consumer version), divx5 (until 5.0.2)
decoding: nero, divx5, ffmpeg/libavcodec (ffdshow)

for aac currently there doesnt exist any freely available encoder which supports error resilience (Digital Radio Mondial compliant encoders should handle this for example) maybe someone will implement it in faac too if people ask for it? :(
faad2 (CoreAAC) already supports decoding these

as you can see this mainly is a codec, not a container issue, but as it is mainly usefull for situations in which the risk of loss of data is high, like with xcds, and these issues were discussed mainly in this forum i posted it here
lets see how the discussion evolves, were to finally place it

[bond]

ok some findings on the overhead:

1) audio aac:
as digital radio mondial will use the error resilience tools, they also provided infos on the expected overhead (well they want to keep overhead as low as possible too ;) )
In the DRM system, all AAC bitstreams shall use the HCR tool, since this tool reduces the error sensitivity of the bitstream significantly with a minimum of overhead. The VCB11 tool shall be used, since for low bit rates, the VCB11 overhead is less than 1%. The RVLC tool shall not be used, since it introduces a significant bit rate overhead that would be a major drawback for the low bit rates used by DRM.2) video:
bobololo from nero/ateme kindely did some testing:
in his test he used all video ER tools and they involved 7% overhead (compared to the filesize without ER) on the bitstream with 2048 bytes video packets
if you take into account that RVLC seems to produce much overhead (and isnt supported by most encoders anyway) you can expect an relatively lower overhead when not using RVLC...

[bond]

ok some updates on the support of these features:

1) software
- the XviD guys already have plans to add these features to the codec, but will only start implementing it after XviD 1.0 final is released

- ffvfw also seems to already support these features (of course, as it is based on ffmpeg) but i am not sure if it works correctly:
ffvfw only offers the option "data partitioning" although resync markers (called error resilience in ffmpeg) is mandatory to be enabled before data-part can be used (i dont know if they are enabled by default?)

now i tested it and got the following results:
i got exactly the same filesize with data-part enabled or disable. well, perhaps data-part doesnt create any overhead, but i dont think so!?
when i tried to play these files with decoders, which dont understand data-part (ie 3ivx) the picture only showed some blocks, which means it isnt a "normal" mpeg-4 file

ok, perhaps i am just too dumb and i dont see that files encoded with quant 2 (as i did) will always have the same filesize no matter if error resilience is used or not :D


2) hardware
the elta 8883 is already able to play these error resilience tools (exactly: resync markers and data partitioning)

as it uses the mediatek 1389 chipset, this chip seems to be damn good!
it also supports XCD decoding, which means you can already use error resilience with XCD and play it on standalones :)

[mikeX]

great job there bond :)

i have a question though (probably a bit stupid),
now that all of this error recovery overhead will be hardcoded into the original stream what will happen with the containers? i mean do they have to be hacked to support it or do they just read 'mpeg specs' and don't really mind
+
what about all the error correction plans of the newer containers (mkv, ogm etc)? are they gonna go to waste?

[bond]

Originally posted by mikeX
now that all of this error recovery overhead will be hardcoded into the original stream what will happen with the containers? i mean do they have to be hacked to support it or do they just read 'mpeg specs' and don't really mindi dont think so as the error recovery information is stored in the bitstreams themselves not on the container level

what about all the error correction plans of the newer containers (mkv, ogm etc)? are they gonna go to waste?well currently there is no error correction possible on container level so...
if it were so you wouldnt of course had to use it twice in bitstream and container

imo a container level error recovery would mainly make sense with audio/video streams that dont offer these superior features that mpeg-4 offers, ie rv9, vp6 or wm9...

[Phobos]

Originally posted by bond
2) hardware
the elta 8883 is already able to play these error resilience tools (exactly: resync markers and data partitioning)

as it uses the mediatek 1389 chipset, this chip seems to be damn good!
it also supports XCD decoding, which means you can already use error resilience with XCD and play it on standalones :)

omg!!!! thnx bond, im also an mp4 fan and this makes me an XCD fan from now on :cool: 800mb cds all the way!!!

[bond]

mpeg-4 rules http://www.kurts-smilies.de/rock.gif :D

[Hyper Shinchan]

I think that maybe you already knew about it, but it's time to make a little update to the list of complaint encodere and decoder ^^.
QuickTime supports resync markers in both encoding and decoding (in the encoding tab is the only video option avaible when you are using MPEG-4 in non-ISMA MP4. In the ISMA MP4 you can't use the resync markers.
I tried data partitioning but it doesn't work in QT (and I don't have any sample with reversible vlc to make a test).
Quite impressive the old Philips Platform4 Player supports both resync markers and data partitioning!
An other player that supports both of them is the MP4IP project player, but it's pretty obvious, if i'm right it uses an ffmpeg engine.
Maybe I'll test some more players, if I remember an other one that I could test.... I'm a sort of addicted of error protection, but to be honest, splitting each video frame in several pieces and separeting the MBs into two parts won't hurt the quality???
Anyway using esync markers and data partitioning I was able to detect a 3% of overhead, with FFMPEG.