Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
1st March 2012, 15:18 | #1 | Link |
Registered User
Join Date: Sep 2008
Posts: 40
|
Lossless compression that works for DV?
Hi.
I haven't done any testing, but I will, eventually. Perhaps someone has already tried this... Are there any lossless compression algoritms that compress DV content to any degree? Since there are no temporal compression in DV I guess this should be possible, depending on the source. Has anyone tried? 7z? Zip? Rar? Edit: Did some quick testing on a random 25s VHS capture. WinRAR using compression "Best" shrinks it to 79% of the original size. The same clip compressed using setting "Fastest" renders a file that's 81% of the original. This leads me to believe that it's DV overhead that are being compressed, not the video content itself. The compression window might be far to small far that. (Or it's not very compressable.) Edit#2: Someone got 60% back in 2005 with 7z... http://forum.doom9.org/showthread.php?t=92590 Last edited by VonOben; 1st March 2012 at 15:57. Reason: Own testing |
2nd March 2012, 00:42 | #3 | Link |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
DV is already compressed. In a lossy way.
As far as I know, DV compression is pretty similar to Motion-JPEG. So putting DV files into a Zip archive is like putting JPEG files into a Zip. You won't get much additional compression out of that Actually ~75% compression with 7-Zip is pretty high for an already compressed video format! It indicates that the "entropy coder" used by DV is not very good! And of course the compression ration that can be achieved with 7-Zip can vary for different DV files. It also can vary with different 7-Zip settings - using a bigger dictionary size might help a bit. But I think you won't get much more compression in a lossless way. Converting from the lossy DV format to a lossless video format, such as HuffYUV or FFV1, will certainly result in a much bigger file. So the best you can do to "shrink" your file is re-encoding to another lossy format, which can retain the content of your DV file at significant smaller size with only minimal or negligible quality loss. The H.264/AVC format, encoded with x264 (in CRF mode, with a suitable CRF value) probably is the most promising candidate here
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 2nd March 2012 at 00:48. |
12th March 2012, 10:05 | #4 | Link | |
Registered User
Join Date: Sep 2008
Posts: 40
|
Quote:
I think, or thought, that there might be a way to implement a lossless compression that looks at DV-frames mainly in the temporal way, and it would be possible to compress it a bit. Then again, large dictionaries with 7z and the likes might do this, but I'm not sure how large their "working windows" are in kilobytes. Will they fit several frames inside RAM while trying to find patterns, or will the buffer not be large enought for that? Anyhow, as you say, 75% isn't to shabby. After all, on a 100Gbyte file that's quite a lot of gigabytes saved. (Written fast and will no spell checking. Now back to work.) Edit: One could say that DV are compressed in X and Y direction, lossy. But no compression, not lossy nor lossless, are applied in the Z direction. (X = Horizontal, Y = Vertical, Z = Time). At least this is how I think it works. Last edited by VonOben; 12th March 2012 at 10:11. |
|
18th March 2012, 01:19 | #5 | Link | ||||
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
Quote:
Quote:
The data stored in a DV file is not pixel data, it is compressed data that pretty much looks like "random" bits! So you can't do any "temporal" compression on that, unless you decompress the compressed data first - in order to retain the pixel data. But again: Decompressing the existing lossy DV file and then re-compressing it to a lossless format (e.g. HuffYUV or FFV1) will result in much WORSE compression ratio If you re-compress anyway, use a high-efficient lossy format, such as H.264. It will be able store your DV at much smaller size with only minimal quality loss... Quote:
That's because all compressed multimedia formats (lossless or not) use entropy compression. Anything that happens before that, e.g. in the "lossy" step, is just intended to improve the final entropy compression. JPEG uses Huffman Coding in the last step, for example. Arithmetic Coding is supported optionally, but rarely used in reality. After the entropy compression, you get data that look like "random" bits and therefore another entropy compression, like 7-Zip, won't be able to further compress that. Dictionary compression doesn't help either. Quote:
It only shows that the entropy coder used by DV is amazingly bad (Try the same with a H.264 and stream, for example, an you will get compression ratio of no better than 99%)
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 18th March 2012 at 01:34. |
||||
18th March 2012, 23:33 | #6 | Link | |
Avisynth Developer
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
|
Quote:
|
|
26th March 2012, 08:13 | #7 | Link | |
Registered User
Join Date: Sep 2008
Posts: 40
|
Quote:
However, imagine a completely black stream. Compressed DV frame #1, somewhere around 125kbyte large for PAL, will contain exactly the same data as compressed frame #2. As they both contains the same data uncompressed. That's compressable, temporally. (And since it's CBR also non-temporal-wize, but that's out-of-scope in this context.) Perhaps not a valid real life example, but it's a way to describe how I'm thinking. |
|
29th March 2012, 18:17 | #10 | Link | |
Software Developer
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
|
Quote:
This will cause the uncompressed data to be slightly different, at least. And I would assume that after the entropy coding that difference will be large enough to botch another layer of compression. Also: How often do you record in complete darkness to generate a sequence of "black" frames ???
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 29th March 2012 at 18:19. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|