"High Error Recovery" in MKV Container?

[BluDChyLD]

Hi,
I've been using ogg vorbis with OGM container for my encodes for about 1 1/2 years with no probs, but since Matroska has matured a lot I thought i'd check it out :) For me, it's most appealing feature over OGM is "high error recovery" (stated on matroska.org) can anyone elaborate on this - does this mean the container can handle scratched cds more effectively than OGM/avi?
Thanks for the help!

[bond]

well the matroska page claims a few things practically not existing, like menus

afaik there is no error correction possible atm in matroska demuxers (dunno if it can be set while muxing?)

[BluDChyLD]

Cheers for the info bond, I think i'll stick to OGM for now then!

[Didée]

Well ... a "just-muxed" or "just-muxed-and-burned" OGM has no special or extended error resilience either. Unless you add it manually. Do you do? ...

BTW, with the huge pile of [OGM + Mode2]-CDs I made in the past, I'm starting to get big problems with reading.

[Koepi]

Those problems i get with "normal" CDs from back then, too. Until now I can see nothing much different between XCD and usual CD burns. Well, ok, some readers can read the "mode1-cds" more easily, but bottom line, with good, recent dvd readers, I have no real problems accessing the residual data. I have to add that this happens only with bad, cheap media where the top reflective layer starts to dissolve(!).

Cheers
Koepi

[BluDChyLD]

Originally posted by Didée
Well ... a "just-muxed" or "just-muxed-and-burned" OGM has no special or extended error resilience either. Unless you add it manually. Do you do? ...


Nope, just mux and burn for me :) I'm strictly Mode1 burning so there's some basic error protection there... Plus I only use Ritek discs, so their reliable most of the time :)

P.S You mentioned manually adding "extended error resilience" to cd? If this is possible, i'd be interested to find out how!

[Didée]

If you burn mode-1 anyways, then the best prevention of data loss is ... having a 2nd copy.

Error correction has always been a topic for mode-2 CDs, since there you trade ~100 MB more capacity for less reliable error recovery. On mode-1 CDs, these MBs are automatically used for error protection ( speaking very loosely).

Latest effort about error protection for mode-2 CDs can be found here (http://forum.doom9.org/showthread.php?s=&threadid=93048&).

Bottomline is, despite all the talk that has been done about the topic, up to know there is (was) not much more than "it's possible" ...

So, your best friends still are good media, and dark'n cool storage.
(And a disc that is kaput, is just kaput.)

[Sharktooth]

http://www.webalice.it/f.corriga/misc/kaput.jpg

[Koepi]

Originally posted by Sharktooth
Master of the short answers... [/B]

So very true. ;)

[Elic]

@ Didée
> If you burn mode-1 anyways, then the best prevention of data loss is ... having a 2nd copy.
...but often, two (or more) discs from one spindle have defects in same place :(

> up to know there is (was) not much more than "it's possible" ...
Sorry, my English is not quite good (although I am training in English almost every day :) ) and I seem to be misunderstanding you about above sentence. Did you mean that my RR package is still quite raw and unfinished? Yes, it is. But it still works.

Apropos, I appreciate any messages about errors and strange behavior of my programs, especially on processors made by AMD - unfortunately I have not access to AMD-based machines and cannot test some vital parts (partycularly, polynom multiplications) thoroughly on Athlons and Durons. So, if my program don't work, tell me please! And I'll try to correct error and upload fresh version as soon as possible.

[iapir]

Matroska has no error resilience, but much better recovery than AVI, maybe even than ogg because it has syncwords and CRC checks. So it's >= AVI/OGM.

[Didée]

Elic, I didn't intend to depreciate your work in any way. In contrary, it is to consider very valuable, and hopefully you're encouraged to develop it further.

However nowadays, DVD burning has become pretty popular ... we should have had easy-to-use error correction for mode2-CDs a year or two earlier. Even the best possible implementation you could come up with does not help with data that has already become lost by now.

One a sidenote, I would not feel all that comfortable with a protection scheme that does not allow on-the-fly playback. Imagine I just want to quickly show a friend that particular one scene from ... wait ... was it this movie, or the other one? "Half an hour" of extracting first movie to HD, browsing through ... no, it was the other one. "Half an hour" of extracting the second movie ... a little cumbersome, isn't it.

[Atamido]

Originally posted by bond
well the matroska page claims a few things practically not existing, like menus

afaik there is no error correction possible atm in matroska demuxers (dunno if it can be set while muxing?) Menus are currently in beta, but that's another topic.

The "High error recovery" refers to it's ability to maintain playback when it encounters an error, not (for instance) in it's ability to reconstruct damaged video data. The ability to reconstruct damaged video data is in the specs, but there has been no real desire by anyone to implement it. Like Elic's mode-2 ECC, it would be a slow process and not something that would likely be done on-the-fly.

Matroska files can contain CRCs to detect if information is corrupted, and extra copies of vital information for use should one be corrupted. It also handles random file corruption rather well. For instance, if there is data missing from the middle of a file (where the total file size is smaller than it should be) the file should still playback fine. The same error in an MP4 file could potentially cause total sync loss after the missing data (as per Stux).

There are many different forms and types of data corruption, and Matroska tends to handle most at least as well as, or better than, other containers.

[bond]

i didnt think that "recovery" could refer to "maintaining playback", cause for me "recovery" clearly means reconstructing something thats broken in that context

[iapir]

AFAIK you say "recover from something". In this case, it's recovering from an error. So that you can still read the file even though it's corrupted. Like a scratched CD.

[Elic]

Originally posted by iapir
you can still read the file even though it's corrupted.Sorry, but isn't it "error resilience"? IMHO "recovery" means "revert to original contents" and "resilience" means "continue playback even if part of contents is lost permanently", is it right?

[Elic]

@ Didée
> I would not feel all that comfortable with a protection scheme that does not allow on-the-fly playback.
Me too. :) But RR is to resolve a different task - precisely, it must protect wide variety of data on CD, not only XCD.

IMHO there is much better mechanism (although more expensive in sense of disk space) for on-the-fly recovering movies and for improved error resilience: Koepi's XCDBackupCreator (and XCD specs at CVS) (see Koepi's post here (http://forum.doom9.org/showthread.php?s=&postid=657188&highlight=XCDBackupCreator#post657188)).

IMHO "improved error resilience" scheme will be useful in situation when "quickly show a friend that particular one scene from ... wait ... was it this movie," and full-valued recovery (by my RR or by "conventional" means) - for making new copy if time is no matter.

> "Half an hour" of extracting first movie to HD
Err... Things are not so gloomy at all; in my program, time of computation during recover is nearly proportional to number of damaged sectors and is tens of seconds of CPU when there are tens of damaged sectors - this is my "know how" :) (indeed, plus few minutes for copying entire CD onto HD).

Apropos, when damaged sector is occurred on CD, then system itself (or CD drive, I don't know exactly) spends tens of seconds trying to re-read each damage, instead of returning error code at once; so, when CD is damaged, no "recover-on-the-fly" solution is possible at all... :(