DV compressed by archiver?

[Terka]

Please correct me if i am wrong:
DV format (in avi) is an collection of jpg images.
jpg can be compressed: by +-25%
http://compression.ca/act/act-jpeg.html
Is it possible to develop an compressor for DV (lossless) archiving?

[JohnnyMalaria]

DV compression is similar to JPEG but it isn't the same.

JPEG allows the level of compression to be adjusted and that level of compression applies to the entire video frame.
DV compression is at a fixed level though it can vary across a video frame to optimize high detail regions vs. low detail regions.

The overall DV compression level is 3.5MBps/25Mbps which also includes the audio and other data.

So, to archive DV in a "lossless" way, you should just archive the DV as-is. Recompression to another format will change the video information (even if a small amount) and lossless encoders such as Huffy won't help because the DV compression has already made it lossy.

[smok3]

the more interesting question would be: is it possible to use dv tapes for lossless data backups? (apparently it is, since there are some apps to be found googling around...)

[nm]

Please correct me if i am wrong:
DV format (in avi) is an collection of jpg images.
jpg can be compressed: by +-25%
http://compression.ca/act/act-jpeg.html
Is it possible to develop an compressor for DV (lossless) archiving?
Yes, a similar approach (replacing Huffman coding with something more efficient) would work with DV too.

[JohnnyMalaria]

the more interesting question would be: is it possible to use dv tapes for lossless data backups? (apparently it is, since there are some apps to be found googling around...)

It's possible but not advisable.

DV recordings use a lot of error correction to minimize loss due to drop-outs etc on the tape but it isn't perfect. Consequently, the DV specification uses a few tricks to mask any problems - e.g., corrupt parts of the video can be replaced with parts from the previous frame. Audio has to be more robust and so more error correction is used. Actually, a significant portion of the data written to tape is the error correction stuff but it isn't enough to guarantee perfect recovery.

For data archiving, you need perfect error correction. One bit in error can render archived data useless. DV audio/video can survive that.

Crudely, at 13GB per $2.50 tape and with $250 1TB hard drives available (~77 tapes), tape-based archiving is less attractive.

[Inventive Software]

Hard drives these days are far more prone to failures than they were 5 years ago. Tapes are basically under the same rules they've been for 30 years, keep them in a cool dry place, ideally a vacuum, and they'll not degrade.

[slavickas]

Hard drives these days are far more prone to failures than they were 5 years ago. Tapes are basically under the same rules they've been for 30 years, keep them in a cool dry place, ideally a vacuum, and they'll not degrade.

and how hdd gonna fail if it would be keeped like tape you describe?

[foxyshadis]

Domain-specific compressors will always be able to do amazing feats unheard of in generic compressors. In this case for optimal compression you'd first need to decode the stream to the original DCT quants, removing all padding and error correction, then apply the algorithms of packjpg (explained in this paper (http://www.elektronik.htw-aalen.de/packjpg/_notes/PCS2007_PJPG_paper_final.pdf)) on it to spatially optimize it. You can temporally enhance it just by not resetting contexts at frame boundaries, like FFV1. There's your ~30% compression. Then you have to build the decoder and get it into ffmpeg (and then ffdshow) if you ever want to play it back, though. :p

[Terka]

I ment if for DV archiving on hdd or spare space before cutting the stuff in editor.

[JohnnyMalaria]

I ment if for DV archiving on hdd or spare space before cutting the stuff in editor.

Well, if you only want to archive the audio and video streams in the DV then there are other portions of data that you could remove. The audio could also be compressed in a lossless fashion. But you would only save about 1% of the space required.

You could also implement a hybrid MPEG scheme where interframe compression is performed but without any of the quantization etc of the motion blocks. It would reduce the size but would be a programming nightmare.

[Euph0ria]

I know this is a reply to an old post, but I'm sure there are a number of people who are still dealing with DV M-JPEG video out there. DV uses Baseline JPEG for video with is means it has a lot of room for compression. If you have some original DV video that you want to archive, visit http://www.freearc.org/Download.aspx and download v.666, download the installer and the Addons. Then visit http://www.freearc.org/Download-Alpha.aspx and download v0.67 Alpha and upgrade. FreeArc us opensource and supports the adding of additional archivers. Alternately there is nanozip, which also does a very good job but is closed source.
FreeArc also does an excellent job at archiving pretty much all data in general, and is generally considered the best ranked all around archiver to date, with NanoZip right next to it in 2nd place.