Lossless video codec with best compression

[Xiaopang]

I'm looking for a lossless codec that currently provides the best compression available. Compression and decompression speed is not an issue as long as the decoder works reliably. I'm currently capturing video footage with huffyuv, but my HDD is filling up quickly, so I need to recompress the videos with something more efficient. Any suggestions?

[MatLz]

x264 :cool:

[Xiaopang]

I need it avi-compatible. Also, x264 isn't exactly edit-friendly.

[MatLz]

I need it avi-compatibleH.264 is avi-compatible. But it is not recommended.

x264 isn't exactly edit-friendly.If you open the resulting file in the right way with the appropriate tools, it is edit-frendly.

Edit:
IIRC, even xvid/divx was not "compatible" with avi because it uses B frames.
But a "hack" has been found to make it compatible.

Also note lossless files produced by x264 do only use I and P frames.

[Xiaopang]

I should give it a go then. Can you recommend a tool that allows for lossless h264-encoding and supports avi-containers? So far I have been using Megui, but it needs too much hdd-space due to separating the muxing process from encoding.

Edit: In the meantime I conducted my own test. VVF1 beats x264 in final size, encoding time, usability and space requirements so I'll go with it.

[Atak_Snajpera]

i use utvideo codec

[Xiaopang]

Thanks for the hint. Didn't know about that codec yet. I benchmarked it with my test file and it seems to compress slightly worse than Lagarith, but also being 50% slower despite using 6 threads. FFV1 still compressed 25% better within the same time, but by only utilizing a single thread.

I guess I'll turn this into a benchmark test for current lossless codecs since it was pretty interesting to see the huge differences among the codecs. Currently, I tested Huffyuv, Lagarith, Fastcodec, FFV1, MSU, Yufsoft, h264 and UT Video. If you know any major codecs I'm missing, please let me know.

[Dark Shikari]

x264's speed is going to depend heavily on what speed preset you use; most lossless encoders aren't very adjustable.

But for intra-only lossless encoding, x264 is awful, use something more suited for the task like ffv1/huffyuv.

[Xiaopang]

Thanks, did that. The whole point was to find the codec with the best compression. Huffyuv is good for speed, but its compression leaves a lot to be desired. FFV1 does an excellent job on both speed and compression and that's also what I ended up using. x264's speed held up pretty well too, thanks to it's excellent multithreading capability. It' just a joy to run it on a six core system :)

[Dark Shikari]

I think sliced threading support was recently added to ffv1, along with a few other improvements.

Also make sure to try out the arithmetic coding and large context model options.

[MiroLx]

You can also try MLC codec. New version was released recently...

[Xiaopang]

I think sliced threading support was recently added to ffv1, along with a few other improvements.

Also make sure to try out the arithmetic coding and large context model options.

AC and the large model managed to shave off a whopping 10% off the final size. I'm using ffdshow rev 4323 and it doesn't seem to have threading support for FFV1 yet.


You can also try MLC codec. New version was released recently...

I gave it a try yesterday, but was rather disappointed. My 10 minute testfile of camcorder footage was compressed to 5,998,637,828 bytes with Huffyuv. MLC 1.1. took two and a half hours to compress the file to 6,167,338,680 bytes despite using max compression and multithreading. In comparison, FFV1 managed to lower the size to 3,965,415,218 bytes in 14 minutes. Any explanation why MLCs compression results are so mediocre while being so slow?

[MiroLx]

Any explanation why MLCs compression results are so mediocre while being so slow?

Please disable motion estimation, the codec will be much faster. Still have no explanation for such bad results. Can you tell me which colorspace you used for input?

[Xiaopang]

The source video was captured using YUY2, but I just realized that VirtualDub was set to convert the output to RGB32. May be that's the problem?

[MiroLx]

but I just realized that VirtualDub was set to convert the output to RGB32

Don't know exactly what setting you mean, but if you wan't to compress losslessly YUY2 in VDub use 'Fast recompress' mode, in 'Full processing mode' VDub converts input
to RGB, than the compression is not so good.

Edited 13:40

[Xiaopang]

I'll give it a try. I meant the option "color depth" in the "video"-tab.

[SubOne]

Thanks for the hint. Didn't know about that codec yet. I benchmarked it with my test file and it seems to compress slightly worse than Lagarith, but also being 50% slower despite using 6 threads. FFV1 still compressed 25% better within the same time, but by only utilizing a single thread.

I guess I'll turn this into a benchmark test for current lossless codecs since it was pretty interesting to see the huge differences among the codecs. Currently, I tested Huffyuv, Lagarith, Fastcodec, FFV1, MSU, Yufsoft, h264 and UT Video. If you know any major codecs I'm missing, please let me know.

Yes, an updated benchmark test would be greatly appreciated!

Are any of these codecs 10-bit (apart from x264)? Or are all 8-bit?

[Xiaopang]

Good question, which I don't know how to answer :D

Where's the difference between them?

[Reimar]

Are any of these codecs 10-bit (apart from x264)? Or are all 8-bit?

FFV1 supports 9, 10 and 16 bit for YUV 4:2:0. For RGB and YUV 4:4:4 it only supports max. 8 bit per component though, at least currently.

[a451guy451]

Since Cineform go purchased by GoPro, their codec is free now. You just have to install the GoPro file converter tool, and it adds the full codec to directshow.

[Xiaopang]

I thought about using it, but wasn't able to find it quick enough. Also, is it truly lossless? It heavily advertises its "visually lossless" quality which is lossy after all.

[poisondeathray]

you are correct, cineform is not lossless

[Xiaopang]

Thanks for the hint. That rules that one out then :)

Does anyone know where to get Dirac as a compiled codec, preferably a VFW-version?

[gods_rifle]

When i "played around" with various lossless codecs, the lossless options of Dirac (Schroedinger?) were really bad in comparison to the well known others (ffv1, x264 lossless).
Good thing about Dirac could be the fact, that BBC used this codec internally (lossless mode) during the olympics (so maybe one could say: compression-wise not that good but should be stable).

When i recall my tests correctly, then MLC was in general the best regarding compression (really slightly above ffv1). In my case motion estimation worsened results.

I think none of the well-known codecs will compress as good as ffv1/mlc. For example even the "old" codecs from the MSU lossless codec test were worse while beeing really really slow (YULS). And most of the others (e.g. Lagarith / UT Video) are targeting efficiency instead of compression.

Important remark: i only tested these codecs with analog capture material of Hi8 tapes. There is a lot of noise! This noise characteristics will change the efficiency of all the codecs! (i assume ffv1/mlc without me will win every time with noisy sources; mlc with me and x264 will have a chance with non-noisy sources).

Something else: i think, that lossless video codecs could be much better (as always in lossless compression: we are talking about some %, not more; we can't beat entropy :-)), if there would be more people/experts working on theme. The ideas / algorithms which lead to high-compression seem to me most of the time to be straight-forward (MLC: context-based modelling).
If compression would be the only target, one could take a look to modern lossless image compressors, which will beat all the lossless codecs even if you compress each frame independently without keeping context information. Sadly these experimental compressors are most of the time closed source (high-compression while beeing fast in terms of single-frames!) or really really complex and more of a theoretical approach (high-compression while beeing insanely slow).
Maybe it's a good idea to mention, that ffv1 is more or less a an implementation of some lossless image compression standard (with keeping contexts between frames).

Last words: ffv1 is my favorite :-)

[smok3]

personally i'd need something that is

- 10bit and i422, i444, rgb (user choice) &
- plays in quicktime &
- opens in adobe apps directly & is lossless or slightly lossy (user choice).

and i'am not even close.

[richardpl]

FFV1 supports 9, 10 and 16 bit for YUV 4:2:0. For RGB and YUV 4:4:4 it only supports max. 8 bit per component though, at least currently.

Wrong!

FFV1 from latest git FFmpeg supports:
8bit: YUV 4:2:0, YUVA 4:2:0, YUV 4:4:4, YUVA 4:4:4, YUV 4:4:0, YUV 4:2:2, YUV 4:1:1, YUV 4:1:0, RGBA, GRAY
9bit: YUV 4:2:0,
10bit: YUV 4:2:0, YUV 4:2:2
16bit: YUV 4:2:0, YUV 4:2:2, YUV 4:4:4, GRAY

adding other colorspaces is straightforward.