Comparison of Lossless Realtime Codecs

[jmac698]

Want to know the "best" codec? Read on!

Tested with VirtualDub 1.9.10, XP Pro 32bit, Pinnacle PCTV 800i, Athlon XP 3200+ 2GHz (quite old).

UTVideo 4:2:2 cpu 41-66% Compression 1.9:1
UTVideo 4:2:0 cpu 41-63% Compression 2.5:1
Fastcodec 1.0b cpu 56-88% Compression 1.9:1 (lossless)
AlparySoft cpu 44-66% Compression 1.6:1
HuffyUV 2.1.1 cpu 33-53% 1.5:1 (best)
Lagarith 1.3.19 cpu 75-100% 1.9:1 (default)
mindvid 1.0b cpu 92-100% 1.6:1 (default)
msu 0.6 too slow
toponoky 1.0b didn't show in vdub, too slow
---
File only test, 1 min of noisy vhs cartoon, which compresses better?
utvideo vs fastcodec
UTVideo - 4:2:2 Best 628,689KB
UTVideo - 4:2:0 Best 449,833KB
FastCodec 689,627KB

I tried Toponoky and MSU, they would take more than 10 mins for a 1min clip! The compression looked quite good however, about 300k was the best estimate I saw.

For me, the best realtime codec is UTVideo. It had a very safe margin of speed and easily the best compression of others in it's class. I would use this also for archiving and file use because MSU is just too slow. And if you know the source is YV12, the compression is even better.
The only thing I'm worried about is does it have fast file seeking behavior.

[Dark Shikari]

Did you intentionally ignore three of the most commonly used ones (FFVhuff, FFV1, and x264) in favor of useless ones like Lagarith or proprietary ones (everything on your list except for Ut and Huff)?

[jmac698]

Update

The file test - same 1min VHS clip:
colorspace 422p, FFV1:
AC, small context 573,909KB - CPU 97-100% 2.0:1
AC, large context 538,060KB
VLC, small context 578,978KB
VLC, large context 541,541KB
all were too slow for realtime

FFVHUFF, adaptive huffman checked
Left 617,774KB - CPU 38-69% 1.9:1
Plane, 671,394KB
Median, 627,359KB - CPU 41-59% 1.9:1

Lossless JPG, YV12
501,341KB 88-100% 2.3:1

Conclusion: FFVHUFF is very slightly better compression by 2%, the CPU is also quite comparable. To me there is no compelling reason to say one is better, YMMV.
For YV12, UTVideo is better than lossless JPG; slightly better compression but a lot less CPU.

x264 lossless didn't show in my encode tab, maybe my version is too old?

[Didée]

FFV1:
AC, small context 573,909KB - CPU 97-100% 2.0:1
...
all were too slow for realtime

PAL 720x576 : FFV1(AC,large,12): >37 FPS @ 14~17% CPU (i7-860)

[jmac698]

I'm glad it works for you! You gain a bit better compression of 15% than I could. As I stated in my post,
Athlon XP 3200+ 2GHz (quite old)
For me, the best realtime codec is UTVideo.
Practically speaking, for a particular project I wanted to capture a VHS 3 times for a certain denoising technique (yes, it's really necessary and really works fantastic), but at 73GB per pass it doesn't change the fact that I need to use my entire spare 250GB drive for this.

Btw, xvid q=2 was too slow for me as well for realtime, plus, I need to encode again to mpeg2 at some point. I think putting 2 hours of even filtered VHS was too much for my mpeg2 encoder (QuEnc), because I did find it blocky (8.5GB DVD-DL).

[Dark Shikari]

x264 lossless didn't show in my encode tab, maybe my version is too old?"Encode tab"?

[mariush]

Virtualdub only works with Video For Windows codecs, x264 is command line encoder... there is a video for windows version but it's not "official"

Though you don't need Virtualdub to create a lossless version of the videos.

I don't see codecs using PNG to compress losslessly, jpeg 2000, lagarith normal vs lagarith multithreaded, huffyuv classic vs huffyuv modified to work with YV12 or other colorspaces (so that it's really lossless)... there was even a codec using the compression that's used in UPX I believe, lzo/lzop/nrv ...

[jmac698]

If you can point me to the downloads, I'll be happy to benchmark them on my machine.
Then I'll update everything into one message...

Do you know a capture program which can use DirectShow? Actually the one that came with my card only allows MPEG2, others require registration and/or payment for an upgrade -- duh!

[jmac698]

Note (2015): I had trouble going back and reading this again, Speed is a ratio to 51 seconds which is the baseline, so a lower number means faster. Ratio is compression ratio, a higher number is probably better. In this modern era, you can ignore some of the results of the Realtime column. FFV1 and UTVideo work well.

Note: This has been vastly updated from it's first posting. There's inconstencies in 10a-d (but still relatively comparable).

Summarized below are the results of testing over 10 lossless and nearly lossless codecs. Included in the results are the compression ratio and if it was real-time when capturing on a low-end machine. This can help you choose the best codec to save the most space without dropping frames in real-time. It could also be useful if you need to archive lossless video. I've focussed on YUY2 as this is the highest quality for capturing.


Codec Abbr Settings Size Recomp Time Ratio Speed Realtime
[1][a] Alpary realtime 791523 01:01:00 1.54 1.20 Yes
[1][b] Alpary max 668964 06:36:00 1.82 7.76 No
[2][a] Fastcodec 689627 01:20:00 1.76 1.57 Yes
[3][a] Huffy Predict 658471 00:56:00 1.85 1.10 Yes
[3][b] Huffy gradient 677685 01:00:00 1.79 1.18 Yes
[3][c] Huffy Left 660053 00:54:00 1.84 1.06 Yes
[4][a] Lagarith 623230 01:32:00 1.95 1.80 No
[5][a] Mindvid 651051 01:18:00 1.87 1.53 No
[6][a] MSU Speed 624649 07:38:00 1.95 8.98 No
[7][a] Toponoky Speed 981436 04:57:00 1.24 5.82 No
[8][a] Utvideo 422 628689 00:59:00 1.93 1.16 Yes
[8][b] Utvideo 420 449833 2.70 Yes *nearly lossless (YV12 only)
[9][a] Xvid L0 q=2 174589 03:36:00 6.96 4.24 No *Not lossless
[10][a] FFV1 AC-Lrg 529714 03:58:00 2.29 4.67 No
[10][b] FFV1 AC-Sm 573909 2.12 No
[10][c] FFV1 VLC-Lrg 541541 2.24 No
[10][d] FFV1 VLC-Sm 578978 2.10 No
[11][a] FFHuf Left 617828 01:00:00 1.97 1.18 Yes
[12][a] LJPG 504842 01:36:00 2.41 1.88 No *nearly lossless (YV12 only)
[13][a] Uncomp 1215669 00:51:00 1.00 1.00 Yes *estimate from 2/3 24bit avi
[14][a] MLC Speed 574596 00:01:51 2.12 2.12 No


Conclusions:
For this CPU, [8][a] Utvideo 422 and [11][a] FFHuf Left give the best realtime compression. [12][a] FFV1 gives one of the best overall compression, though some codecs are not tested (note: MSU Max is estimated to be 2.99:1 at 406937KB). If your CPU is fast enough, FFV1, MLC, or LJPG (YV12) seem good. I reached a different conclusion about Lagarith than other tests, it didn't compress well for me but was too slow also.

Choosing Settings
FFVHUFF compresses best with Left, then Median, Plane. FFV1 compresses best with large context vs small, and AC is better than VLC - for this material (VHS cartoon).

FAQ
Why did you mark some as nearly lossless?
I am focussing solely on YUY2 as this is the highest quality for capturing analog video (generally). Those codecs marked have no option for YUY2, and must use YV12, which causes some important losses, possibly destroyed chroma interlacing.

Why didn't you check interlaced encoding?
Good question. I made an assumption that progessive encoding would compress better for this material, which was a cartoon with largely static scenes.

Why didn't you test with patched/multithreaded version of X?
I only had a single core machine for testing. If another codec runs realtime for you, check the file test compression ratio to help make your decision.

Why didn't you include codec X?
If it's free, post the download link, and if I have time I can include it (this is a limited time offer, I may erase my test file some day).
Update: please see http://www.yuvsoft.com/download/lossless-codec/ which is a similar comparison. According to my interpretation of the results, LZO, CorePNG, and X264 still lose compared to FFV1 (which is just too slow for me). YULS seems to have the highest compression ratio though it's slow.

What did you consider real-time?
All codecs marked realtime were below 90% CPU time peak after 1 minute as indicated by VirtualDub. HuffyUV was the fastest at around 50%. I have used my Glitch Analyzer utility to verify no dropped frames (based on direct video analysis, not the dropped frames indication in VirtualDub which is sometimes unreliable, not Avery's fault I'm sure).

Why did you do this?
For an important project I needed to capture 2 hours of video multiple times. There would be extensive processing after this and disk space was a serious problem. I chose to use lossless capture, not because it would make VHS look any better, but because of the numerical errors introduced from processing. I kept testing different codecs and they were too slow and crashed my computer. I got mad and decided to beat this problem :) If I do something, I want to do it right because I will never do this again.

Settings Used:
[1] Alparysoft Lossless Video Codec version 2.0 build 957.040607 debug
Settings [a]:
encoding: enable prediction checked, (compression level) realtime
source video: no boxes checked
SSE acceleration: enable (auto detect) (this machine has SSE3)
Settings [b]:
encoding: enable prediction checked, (compression level) max compr.
source video: no boxes checked
SSE acceleration: enable (auto detect)

[2] FastCodec 1.0 beta from VideoSoft.org
Settings [a]:
RGB formats compression: Absolutely lossless
Logging: Enable not checked

[3] Huffyuv 2.1.1
Settings [a]:
YUY2 compression method: Predict (best)
RGB compression method: Convert to YUY2
Enable RGBA (RGB with alpha) compression not checked
Enable console-window logging (useful for debugging) not checked
Settings [b]:
YUY2 compression method: Predict gradient
RGB compression method: Convert to YUY2
Enable RGBA (RGB with alpha) compression not checked
Enable console-window logging (useful for debugging) not checked
Settings [c]:
YUY2 compression method: Predict left (fastest)
RGB compression method: Convert to YUY2
Enable RGBA (RGB with alpha) compression not checked
Enable console-window logging (useful for debugging) not checked

[4] Lagarith 1.3.19
Settings [a]:
Enable Null Frames not checked
Always Suggest RGB for Output not checked
Use Multithreading not checked (this machine is single core)
Mode: YUY2

[5] MindVid 1.0 Beta 1
Settings [a]:
Key frame delay: 300
Interlaced options: Manual interlace mode, Interlaced video enabled not checked
Logging: Logging level: No logging

[6] MSU Lossless Video Codec v0.6.0
Settings [a]:
Compression: Quality: Absolutely lossless Mode: Maximize speed
Format: Compressing interlaced movie not checked

[7] Toponoky Lossless Video Codec from VideoSoft.org 1.0 beta
Settings [a]:
Compression: Lossless, Maximize speed checked
Logging: Enable not checked

[8] Ut Video Codec YUV422 (ULY2) VCM x86 1.0 beta
Settings [a]:
Frame divide count=1
Same as # of logical processors not checked
Optimize for compression ratio (Predict median) Option selected
Assume interlace video not checked
Settings [b]:
YUV420
Frame divide count=1
Same as # of logical processors not checked
Optimize for compression ratio (Predict median) Option selected
Assume interlace video not checked

[9] Xvid 1.2.2
Settings [a]:
Simple @ L0, Single Pass, Target quantizer: 2.00

[10] ffdshow tryouts revision 3574 Sep 13 2010
Settings [a]:
FFV1
Colorspace: 422P
Coder type: AC
context Model: Large
keyframes distance: 10
Settings [b]:
FFV1
Colorspace: 422P
Coder type: AC
context Model: small
keyframes distance: 10
Settings [c]:
FFV1
Colorspace: 422P
Coder type: VLC
context Model: Large
keyframes distance: 10
Settings [d]:
FFV1
Colorspace: 422P
Coder type: VLC
context Model: Small
keyframes distance: 10

[11] ffdshow tryouts revision 3574 Sep 13 2010
Settings [a]:
HUffYUV
Colorspace: YUY2
Predictor: Left
Adaptive huffman tables checked

[12] ffdshow tryouts revision 3574 Sep 13 2010
Settings [a]:
Lossless JPEG
Colorspace: YV12

[13] Uncompressed, 24bit

[14] MLC Lossless Codec 0.9
Settings [a]:
Compression (predefined): Maximum speed


My System:
OS: Windows XP Pro 32bit
CPU: AMD Athon 64 3200+ (2GHz), supporting MMX(+), 3DNow!(+), SSE(1,2,3), x86-64
RAM: 2GB
Capture Card: Pinnacle PCTV 800i Driver 3.1.1.49

Software used:
VirtualDub 1.9.10
File Recompression settings:
Fast Recompress
Capture Settings:
Overlay, Test Video Capture

Methodology Notes:
File Tests: recompression from UTVideo 4:2:2, with some light multitasking. The video was 1:00 (1800 frames) of VHS cartoon material. Source and destination files were on separate hard drives.
I'm making the assumption that VirtualDub CPU usage is accurate in the realtime tests.
I produced an uncompressed file of 24bits, but estimated the size of a 16bit file, as I didn't figure out how to save this. I tested [3][c] 3 times and always got :54s, so the timing seems repeatable.

[Sharc]

Thanks for sharing your experience.
Xvid with q=2 is lossless or near lossless? I have my doubts.

[DmitryPopov]

FYI:
MSU lossless codec is abandoned and not developed nor supported anymore. It was never even close to realtime, heading for compression records, but later was beaten by a Yuvsoft codec which compressed even stronger but even slower. :)

[jmac698]

Sharc:
The following are lossy: [9]. This means XVID :) I've updated the table to make it clearer, thanks for prompting the suggestion, it makes me a better writer.
Dmitry: I didn't know anything about MSU, I just tested it anyway. I thought the GUI was the nicest one ;) Don't sound so disapointed. You learned a lot from doing it, and even though it's slow today, it could be ahead of it's time. Could it be implemented in GPGPU? Example, I have an old 'real-time' raytracer that the author never expected to run smoothly, but every few years I try it. Now it runs at a smooth framerate on modern computers.
Just for you I ran a new benchmark: with an estimated time to complete in 2:33, your filesize at max compression would be 397.4MB, 2.99:1 making you the lossless winner :)

[mp3dom]

UTVideo 4:2:0 is labelled as 'nearly lossless'. It should be lossless as long as the source is already in YV12 colorspace.

[jmac698]

mp3dom: I've updated the post to address this issue. Thank you for the feedback. And note that the MSU comparison recomends Lagarith; obviously it depends on the clip as it performed poorly for me.

[7ekno]

Why not compare a VfW version of x264 and lossless?

7ek

[jmac698]

The MSU comparison showed poor performance for x264, lossless version. However I'm sure it's improved since 2007 when the test was made. Can you point me to a place to download a vfw version of x264? Thanks.

Update: there is another comparison here
http://forum.doom9.org/showthread.php?t=158361

Don't get excited by the first post, keep reading... the basic summary was that you need to use -fastencode and reduce keyint to 3 if you want good seeking in x264 lossless, but this kills compression, and one person mentioned about 84%(?) of the size of UTVideo. Some people remark how fast UTVideo is, and it seems generally good for lossless editing purposes. For example one person said it's the only codec which is realtime on an Atom cpu, another said it's the only one that can playback 1080p lossless.

[MiroLx]

The MSU comparison showed poor performance for x264, lossless version. However I'm sure it's improved since 2007 when the test was made. Can you point me to a place to download a vfw version of x264? Thanks.

Update: there is another comparison here
http://forum.doom9.org/showthread.php?t=158361

Don't get excited by the first post, keep reading... the basic summary was that you need to use -fastencode and reduce keyint to 3 if you want good seeking in x264 lossless, but this kills compression, and one person mentioned about 84%(?) of the size of UTVideo. Some people remark how fast UTVideo is, and it seems generally good for lossless editing purposes. For example one person said it's the only codec which is realtime on an Atom cpu, another said it's the only one that can playback 1080p lossless.

It would be nice if you can add MLC codec to your comparison.
It supports YV12 natively, so there should be no problems with this; at least I hope.

[Usedocne]

Can you point me to a place to download a vfw version of x264?

SHAZAM! (http://sourceforge.net/projects/x264vfw/)

[jmac698]

MLC: Not realtime :( On Speed settings, it took 1:51 for vdub to recompress, final size was 574596. It's compression is about 1.3.
On max settings, it is estmated to take 37min and squeeze down to 501760KB.

Finally, x264:
Re-encode time: 1:40
Size: 498059
No realtime
better compression than MLC (capture shows 2.0:1, not comparable to the file test above)

x264 core 108 r1790bm 8eaf8a6
Encoding settings : cabac=0 / ref=1 / deblock=0:1:1 / analyse=0:0 / me=dia / subme=0 / psy=0 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=0 / threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=250 / keyint_min=25 / scenecut=0 / intra_refresh=0 / rc=cqp / mbtree=0 / qp=0

[JoeH]

I don't have specific numbers to post, but did run extensive tests about a year back with video editing in mind (not storage, where X264 might be the best), and can share my results in general terms. For video editing UT is by far the best lossless codec presently, as it decodes amazingly fast (which means easier video editing and faster render times), and in the RGB mode is lossless.

[easy2Bcheesy]

The omission of the AMV codec is a bit of a stand-out point for me. Yes you need to pay $10 for a license (watermarked without license), but it's even lighter on the CPU than UT and has both lossless and lossy support. Plus it can encode HD on an Intel Atom.

http://amamaman.web.infoseek.co.jp/english/top_e.html

[Chikuzen]

Because AMV codec(2MT/3) does not only intra compression but also simple inter compression, a good result is put out for a specific source.
(e.g. Game play with scene that halts and doesn't move often.)
However, it is not that for the video that always moves.
I think that the one that a steady result cannot be put out is not suitable for this comparison.

[Dark Shikari]

Yeah, comparing inter to intra formats is not really fair in most cases... as they're often not used for the same purposes either.

[jmac698]

Sorry, download link didn't work for me. Anyone know a mirror?
According to it's own test results, even in lossy mode, the compresion is worse than UTVideo.
There's is another codec that plays back HD in realtime on Atom, don't know about capture.

[kolak]

I tried amv on film HD source and it was not very good. It saved only about 30% space- very poor and speed was also not very good.