Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th April 2002, 21:59   #1  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
MCF-CD maker tool

I've build up this small piece of software... it intends to be a MCF-CD maker (Ingo's version).

It currently takes any file and writes it as M2F2, with an ISO track referencing it. It's like VCD but without the VCD stuff

Of course you can burn a MP4 file, too... the only problem is, you won't be able to read it with any software (yet), since you must parse the RIFF/CDXA stuff..

Since it's based on vcdtools from Rainer Johanni, it currently only creates TOC files (to be burned with cdrdao). And unfortunately seems that cdrdao doesn't works under WinXP/2000 (only Win9x and Linux). But don't worry, I'll add BIN/CUE support later.

Here you have a link with the tool (source included). It's GPL of course.

http://webs.ono.com/de_xt/mcf.html

Remember, for reading it properly you need a DS filter that is able to parse RIFF/CDXA, Ingo is working on that. Currently you can test it with a small tool I've included (dat2file), it can parse the RIFF/CDXA stuff and write the original file again (with some trailing 0's if it wasn't multiple of 2324).

Hope you find it useful. At least it's a start...

DeXT

[Edit]

Oh I forgot, althought the included binaries are compiled for Win32, you should be able to compile it under Linux, too.

Last edited by DeXT; 17th April 2002 at 23:27.
DeXT is offline   Reply With Quote
Old 18th April 2002, 00:02   #2  |  Link
ReferenceDivx
Registered User
 
ReferenceDivx's Avatar
 
Join Date: Oct 2001
Location: Washington State University
Posts: 110
Good work guys!

I just tested it. It works in Daemon tools and it also made a valid disc in Cd Mate.

You can download a demo here:
http://www.cd-mate.com/downloads/CDM_Enu_Latest.exe

the settings to use are
1. open the *.bin file
2. Under write method make sure you have Disk At Once selected
3. Just burn it and enjoy

I was able to easily copy the file in windows Xp to a drive.

I need to pull out my parser program i made this weekend and see if it works.

I'll post it if it works.
ReferenceDivx is offline   Reply With Quote
Old 18th April 2002, 01:46   #3  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
Hey that's great! A very smart idea...

What you're doing is writing it as a single track... in fact a mixed M2F1/M2F2 track. We concluded this was possible as long as you linked to the files separately... and it really works!

I did my tests a while ago, and I was able to burn it successfully with Nero. Steps:

1. Choose File/Burn image
2. Click on All files (*.*) and load mcf_image.bin file
3. Select Mode2
4. Burn!

It works nicely... althought this is not The Real Thing(TM) it works fine. I guess you can use any recording software that can deal with raw BIN images.

Oh a quick note, you don't need extra CRC checking, Windows already checks the EDC data, so you won't get unnoticed read errors. I really think this is THE way for MCF-CD, since it simplifies many things.
DeXT is offline   Reply With Quote
Old 18th April 2002, 01:55   #4  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,455
Hmmm...
Since OGG is quite error robust (well, and some other strange reasons I don't want to tell ) can you add OGG-CD support as well please?

*EG*
Koepi
Koepi is offline   Reply With Quote
Old 18th April 2002, 02:05   #5  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
Hi there Koepi. It's great talking to a Real(TM) programmer... I'm afraid I'm just crap(TM).

Well in fact you can NOW burn OGG files with this tool... you can burn watever you want! MP4? Sure! So you can make your OGG-CD, already. This baby is not MCF-dependant (yet). (well the only MCF thing in there is the volume name, but this can be changed easily).

The only problem is, the OGG guys should add RIFF/CDXA parsing to OGG filter for it to work. Otherwise than that, it's done!

(of course any other suggestion is welcomed)

Thank you!

DeXT
DeXT is offline   Reply With Quote
Old 18th April 2002, 02:36   #6  |  Link
Neo Neko
Registered User
 
Neo Neko's Avatar
 
Join Date: Mar 2002
Location: Kansas City, Missouri
Posts: 1,812
These tools will be a great benefit to the encoding community! And best of all it is opensource. Things are getting very exciting!
Neo Neko is offline   Reply With Quote
Old 18th April 2002, 18:30   #7  |  Link
ChristianHJW
Guest
 
Posts: n/a
... and im sitting in Geneva without internet access ... be back Friday night folks ... keep up the excellent work folks !!
  Reply With Quote
Old 18th April 2002, 22:40   #8  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,948
i'm really excited about this 800M stuff

@DeXT:
excellent stuff

i have one question though: from what i understand, the headers part is the most sensitive one, and if it can't be read properly, all the error resilliance in the world won't make the clip playable, so is there any special considerations for that crucial parts of the file?

@container developers (ogg/mcf): since there are crucial parts in most containers, parts which should be read absolutely correctly, and this causes a headache to make the burning/imaging app separate the file to crucial and uncrucial parts, how about including error CORRECTION codes for these parts (simplest one would be just duplicating them several times on the disc, more advanced ones could be the same as for mode 1 CDs). but the point is: implement the error correction in the container itself. this way image creation would be much easier, and not propriatary for each container. this will also enable to write a single riff/other stripper for all containers, and the condainer decoder will handle any errors from there.

looking forward to playing my 1st 800M clip right of the CD

great work
avi

ps.
how does current streamable formats handle this issue? is mpeg 2 stream ( i.e. via sattelite link) playable from arbitrary point? what if the 'medium' is noisy, will the clip be playable at all? what about mpeg4??

Last edited by avih; 18th April 2002 at 22:44.
avih is offline   Reply With Quote
Old 18th April 2002, 23:26   #9  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
Thank you very much for your nice words

Well if you feel EDC is not enough for you (and it probably isn't, with severely scratched media), yes you could implement a propietary ECC data in some parts of the stream itself. You can also make several copies of the header to have it available in any case (MCF team do this AFAIK). But there is also a neat thing: you can write an "extra" header inside the ISO track with M2F1 (which includes ECC). I think this last one is great since it's the CD unit itself which would make the ECC, not the software, making it all in a transparent manner.

I don't think you can mix M2F1/M2F2 in a single file, but if that works (I haven't tried it), this could just be the answer, because you could then write the header and sensitive parts in M2F1 which is error-resilient. Just leave me some time and I'll do a quick test.

Oh and of course, I'd LOVE seeing some support for RIFF parsing from any DS filter/player developer. It's SO simple, that it wouldn't take more that a while to implement, I guess. People MOVE on!
DeXT is offline   Reply With Quote
Old 18th April 2002, 23:34   #10  |  Link
oddball
Registered User
 
Join Date: Jan 2002
Posts: 1,246
Call me stupid. I don't program. But if you can create the dat2file prog to read the data back into it's original file on the hard disk (with trailing zeors perhaps as yu say in the readme) then why not on the fly? Or is it just that you don't know much about DirectShow?

Great work though!

Do we have any update on this filter?
oddball is offline   Reply With Quote
Old 18th April 2002, 23:45   #11  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,948
@DeXT:
using mixed mode 1/2 misses the point imho.

the point is that a streamable format should not rely on a 'safe' medium (not even for the headers). it should work as is on a 'noisy' medium. therefor, ECC should be implemented in the stream itself.

this would also have these potential advantages:
1. no need for a propriatry imaging application for each container (and new ones).
2. imaging application is simpler.
3. resiliance of imaging application to container versions.
4. standartization of image type.
5. the imaging app should only know that it's a streamable container. NOTHING more
6. standard definition of 'medium'

the point is to treat a 800M CD as a noisy medium and from there, the container should handle it.

[edit]
of course, all the above applies for web/other based streamming as well (where streamming server replaces the imaging app). i.e. in a web streamming, there's no need to mix tcp/udp differently for different containers. that will enable writing a single streamming server using udp, for all these containers and a single streamming client that should only choose the right decoder. no messing with the meduim standartization is the word!
[/edit]

of course, till the containers implement internal ECC, using mixed mode CD in container specific format will make the CD playable long enough.

avi

Last edited by avih; 19th April 2002 at 00:55.
avih is offline   Reply With Quote
Old 19th April 2002, 00:28   #12  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
@oddball: well dat2file is a very simple tool. A DShow filter is a *much* more difficult thing. And I have no idea at all about programming DShow, or even C++. Just plain C, and MinGW/GCC.

@avih: Ok I understand, so you prefer no hardware ECC, and even no EDC at all. All managed by the container itself, and able to read from a totally scratched CD. There is a solution for that: it's called CD-Audio

Ok so I currently will stick with the current implementation (CD-Bridge, M2F2) and let the stream container to do any ECC. For non-streaming formats, I still see the header backup in a separate M2F1 file as a good idea.

Tronic is working on a pure Mode2 ("CD-I") implementation which is more suitable for what you want, since it even doesn't include EDC, all is done by software. The problem is, special software needs to be developed to read this, and probably not all drives will be able to read it (I can't get mine working with this, it always reads it as M2F2, thus giving read errors). Pure Mode2 is rare these days.

Yes, Work-In-Progress is so nice...
DeXT is offline   Reply With Quote
Old 19th April 2002, 00:48   #13  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,948
don't get me wrong here i love your work and the idea of mixed mode/hardware ECC. and i'd really like to see it working, i'm eager to store a 800M clip safely on a cd, and since (well, i didn't check it enough though) the containers don't have ECC atm, your solution could be the 1st to hit the target of making a storable 800M clip, without sacrificing headers 'safety'.

my posts were more general, regarding streamming formats, potential collaboration between developers, standartization etc.

also, if there would be a gpl ECC code, all containers would be able to use it. again the less propriatry code is out there, the faster the technology/features will advance. and when one imaging/streamming, encoding, stripping/client apps are created, all containers will be able to utilize them.

it's just that this point in time gives us 2 different developing containers, and i think it would be a miss if each one of them will have to use different tools.

both (and users) will suffer from that.

don't let me stop you from testing new options
avi

[edit]
ps.
this gives me another idea:
if u implement this partially safe storage system, it might even be used to store avi files (headers/indexes are safe, data is not). the stripper will have to replace errors (that should be detected correctly by the hardware if i understand correctly) with 'empty' avi segments. i.e. ones that will cause a 'blank' decoded images or something similar, but without breaking the avi file structure.
so the player will get a 'correct' avi file but the image quality will depend on the noisiness of the medium.

indeed, WIP is fascinating
[/edit]

Last edited by avih; 19th April 2002 at 01:05.
avih is offline   Reply With Quote
Old 19th April 2002, 22:36   #14  |  Link
ChristianHJW
Guest
 
Posts: n/a
Quote:
Originally posted by avih your solution could be the 1st to hit the target of making a storable 800M clip, without sacrificing headers 'safety'.
DexT,

i will check your program tomorrow, cant do it now, but i am really excited i have to say.

About what Tronic wnats to do :

As i understand it ( i may be wrong ) he wants to store the MCF header(s) in mode 1 with full error correction, and only store the data as mode 2 .... but i have to talk to him again, we cant develop a solution that will not be supported by all CD drives ....

BTW, where is Tobias ??? No comments to thsi subject from the one and only Ogg format developer ??
  Reply With Quote
Old 19th April 2002, 22:53   #15  |  Link
Tronic
Registered User
 
Join Date: Mar 2002
Location: Turku, Finland
Posts: 22
We are trying to gather all the relevant MCF-CD stuff on one webpage (instead of 20 message-boards ...

http://mcf.sourceforge.net/mcf-cd.htm

Currently we have two versions, both named MCF-CD:
-My version
-Ingo's version

Mine has a header that allows using it for any file, allows protecting any parts of the file and is free of all ECC code. However, unlike DeXT said, it DOES protect headers: we just have backup-copies of sectors requiring protection. Because headers only need protection, the overhead caused by this "stupid" implementation is minimal (around 5 kilobytes per movie).

Ingo's version doesn't have a header yet, or any other specification yet (at least he hasn't published anything). It's practically a VCD with different data in it.

Well, you can find more info on the webpage I linked to.

I don't know if CD-ROM EDC/ECC are patent-free or not, but my implementation is free of those.

So far my implementation has worked in all CD-ROM drives I have tried it in, but I'll need to do more testing. In theory it should work everywhere since Mode2 tracks (XA or not) are always read as Mode2 and it's the software that does XA error-correction.
__________________
Matroska's father and companion: http://mcf.sf.net/Docs/MCF/
Tronic is offline   Reply With Quote
Old 19th April 2002, 23:26   #16  |  Link
avih
Capture, Deinterlace
 
avih's Avatar
 
Join Date: Feb 2002
Location: Right there
Posts: 1,948
tronic, by looking at your page i get the impression that u use specific disk formatting (i.e. using image format to support protected areas for the file).

can u pls comment on this suggestion?:

"streamable containers should not rely on a [even partially] safe medium by any means"

from what i understand, the imaging application/streamming server will have to handle this protections, on a container specific level.

how about just put this recovery information into the mcf file itself? and the imaging app will only have to burn a single long file/data in mode 2?

avi
avih is offline   Reply With Quote
Old 20th April 2002, 00:06   #17  |  Link
Tronic
Registered User
 
Join Date: Mar 2002
Location: Turku, Finland
Posts: 22
IMO checksums should be in storage media level because that allows easy extraction of broken sectors (ie. format-level wouldn't have checksums for 2332 octet pieces, which would correspond with CD sectors (one type of those).

Also, for additional security, there should be additional checksums in data, ordered so that the data is protected best (in MCF that means that all Clusters should start with video keyframes (a cluster is protected by a single checksum)).

Error-correction is normally not required as data links (TCP/IP) and storages (HDD) are pretty error-free. Even if the harddisk broke or the peer server in TCP/IP died, ECC data wouldn't help us.

Error-correction is even more dependant on the media than checksums are - a CD normally has broken data in 4 octet chunks (audio CD sample size) and have no misaligment (unless we are talking about CD-AUDIO, which always has offset of n*4 octets, I think).

In MCF-CD (Tronic's version) the trick is that the correction is format-independant. My extraction program can extract any file format from MCF-CD-images. Check out MCF-CD header specs for more info.

However, writing programs should be format-specific, so that those know which parts of the file should be protected. It is possible to write unknown fileformats too (even with my tool, with minor tweaking), but there you won't have your headers protected (unless you manually define positions to protect for each file).
__________________
Matroska's father and companion: http://mcf.sf.net/Docs/MCF/
Tronic is offline   Reply With Quote
Old 20th April 2002, 00:10   #18  |  Link
Koepi
Moderator
 
Koepi's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 4,455
How about being a _little_ more generic and allow OGG as well?

It seems to me more or less a kind of a "fight" is coming up here.

We have OGG, MCF is still to be born, ...
I don't mean that ogg isa superiour- i just think we want to have a choice.
Koepi is offline   Reply With Quote
Old 20th April 2002, 00:30   #19  |  Link
Tronic
Registered User
 
Join Date: Mar 2002
Location: Turku, Finland
Posts: 22
My version allows ANY file in it. That could be Ogg, EXE, BZ2, BMP, AVI - anything.

Also, in addition to file extension there is 4-char identifier. I also plan that a MIME-type should be added to the header.

It is possible to make CD filesystem drivers for Windows and Linux (and probably all other OSes too) - those would allow reading the file directly, thru filesystem. Like, copy movies to HDD directly from Explorer.

Windows' VCD-bridge driver isn't very good. It displays the data incorrectly (adds RIFF header and displays all sectors in 2352 octet mode). Thanks to that you can't copy the data directly to HDD, but you have to use special software for that.

I don't know how difficult it is to write such driver, but I have seen people doing it. Sources for some reference driver (apparently the normal one, used by Windows) are available in some M$ developer kit.
__________________
Matroska's father and companion: http://mcf.sf.net/Docs/MCF/
Tronic is offline   Reply With Quote
Old 20th April 2002, 00:43   #20  |  Link
DeXT
Registered User
 
Join Date: Mar 2002
Location: Spain
Posts: 307
@Christian: I asked Tobias about the support for RIFF/CDXA parsing, and he said he may add it in future... at least after he has completed 5.1 and chapters support in OGM.

@Tronic: well I didn't say it doesn't has protection, I said it doesn't has EDC, and that's true since pure more2 lacks EDC (as you already know). Anyways for a streaming content this is the best possible case because it relies all error management in software.

And about the Mode2 error correction, well it may not rely on the hardware itself but the OS driver (I don't know). Anyways on M2F1 and M2F2 you can't read a sector which has broken EDC data (at least I can't, it gives read errors -- I have an Acer 24x10x32, and WinXP).

@Koepi: my tool is fully generic. Even if I may add MCF-specific stuff in future (just anything Ingo or Tronic may suggest me), I will still maintain a generic version supporting any file. I think M2F2 writing is GOOD for the community and it's here to stay. I'd LOVE seeing it work with OGG.

Last edited by DeXT; 20th April 2002 at 00:45.
DeXT is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.