Lagarith lossless codec

[Zarxrax]

I'm announcing this for my friend since he's never joined this forum. He has made a very nice lossless codec called Lagarith. It supports a number of colorspaces (RGB24, RGB32, RGBA, YUY2, and YV12). It runs at about half the speed of huffyuv, so it can work well for capturing video on most systems. The compression ratio is slightly better than corepng (I didnt enable delta frames in corepng in my test).
It is an interframe codec (all keyframes) so it works very well for editing. It does have a null frames option though, which you can enable to make it encode null frames if there is no difference from one frame to the next. It's also open source.
A reduced resolution mode is currently being implemented, which will be a extremely nice feature for video editors.

I personally use this as my standard lossless codec now, replacing huffyuv and vble. From all the lossless codecs that I have tried, it offers the best mix of compression, speed, and stability, in my opinion.

For more information and to download it, go here:
http://lags.leetcode.net/codec.html

[Mug Funky]

looks good. you can never have too many lossless codecs :)

btw, love the avatar :)

[akupenguin]

Cool, a new codec is always good news.

Nitpick posted here because I don't see any method of feedback on the site:
from http://lags.leetcode.net/codec.html
Pixel values are first predicted using Paeth prediction (the same method used when "Predict Median" is selected in Huffyuv). Huffyuv's prediction (which Lagarith also uses) is "median(L,T,L+T-LT)". Paeth (used in e.g. PNG) is "whichever of L,T,LT is nearest to median(L,T,L+T-LT)".

[Soulhunter]

Wow, very nice codec... :)


Bye

[akupenguin]

Since the compression method sounds similar to ffv1, lets see how they compare.

My benchmarks on a random chapter of a random DVD I had on hand (Ghost in the Shell 2; mostly CG, cropped but not filtered)
YUY2 (because that's what all these codecs have in common), 704x384, 2000 frames
There were no duplicate frames in this clip, so that feature of lagarith wasn't tested. But note that MEncoder has a similar duplicate frame remover that can be applied to any codec, so ffv1, ffvhuff, and ljpeg would benefit the same amount.
This is just one source; YMMV.
Codec Bitrate Enc fps Dec fps
raw: 103705
huffyuv 2.1.1: 31524 93.41 71.61
ljpeg: 28965 44.85 45.63
huffyuv ccesp: 28170 96.48 72.54
ffvhuff: 24308 86.81 70.42
lagarith: 23242 41.08 29.88
ffv1(cavlc): 19756 27.99 26.31
msu: 17676 0.83 10.63
ffv1(range coder): 17221 23.23 25.77

[TEB]

good codec.. very stable..

but its about 20% worse at enc than ffv1 and around 3% slower at encoding.

Does anyone know how to force the encoder into 422 or 420 mode?

[Shinobu]

A multi thread implmentation will be cool, i can beta test it (dual athlon mp on tyan thunder k7).
Also a "compression level" will be realy cool (don't know if it's easily implementable) so every computer could use it for capture ^^.

++

[Atamido]

AFAIK, FFV1 only works in YV12, so if you're feeding it anything except that you are not encoding lossless. An AVISynth script could compare the original to decoded FFV1 to make sure.

Also, CorePNG with Auto-Delta frames at the maximum compression level would likely beat all of these for size, but unless you have an Athlon 30,000+, you aren't going to be capturing with it in real time.

[akupenguin]

FFV1 works in YUV of any subsampling ratio (4:4:4, 4:2:2, 4:2:0, 4:1:1, 4:1:0) and also RGB. Yes, I made sure all of the encodes in my test are lossless.

[akupenguin]

Lagarith doesn't seem to be affected by the "null frames" option when encoding YUV. Is it supposed to be working? (Yes, I'm sure the frames are perfectly identical.) I also can't get it to encode RGB (vdub crashes when I begin encoding).

edit: SirLagsalot just fixed RGB.


Another chapter of another random DVD ("Ghiblies 2", 672x432, 2500 frames, cropped but not filtered).
Yes, I made sure all of these are really lossless.

YV12:
Codec Bitrate Enc fps Dec fps
raw 83529
corepng(6i) 23903 7.31 47.49
ffvhuff 22264 156.46 86.09
lagarith 20975 44.79 32.89
corepng(9p) 18007 0.68 51.09
ffv1 15368 28.62 29.80
snow 13838 7.46 15.43

With duplicate frame removal:
ffvhuff 18440 163.06 104.61
lagarith 17346 50.89 39.72
snow 13685 8.50 16.76
ffv1 12737 33.93 36.09


RGB:
Codec Bitrate Enc fps Dec fps
raw 167048
hyffyuv ccesp 54583 48.79 59.29
corepng(6i) 52360 3.82 35.49
corepng(6p) 41752 2.29 28.24
corepng(9p) 40122 0.55 29.04
lagarith 39539 19.42 17.56
ffv1 29937 13.86 14.76

With duplicate frame removal:
huffyuv ccesp 45388 58.86 71.53
lagarith 32771 23.93 20.80
ffv1 24891 16.68 18.00

where (6i) means CorePNG 0.8.2 on defaults, and (9p) means best + auto-delta.

[CruNcher]

2605 1:44 Pal 25 Fps 720x576 (Real) (cleaned DVD)

flawed look below for correct results

[Zarxrax]

After seeing the good results of ffv1, i decided to try a comparison myself, and see if I could actually get the codec working this time. I've never been able to get either ff huffyuv or ffv1 working the couple times that I tried them in the past.


Video is YV12, animated source, 1:40 in length.

Codec/settings Encode Speed Size
vble 0:48 395MB
lagarith 1:15 330MB
ffv1 vcl Context 0 kf dist 1 1:47 281MB
ffv1 ac Context 0 kf dist 1 2:40 268MB
ff huffyuv (resulting files are corrupt)


Well, at least ffv1 works now.
My results basically show a pretty linear relation between the amount of compression verses the encoding speed. Lagarith seems to be a good intermediate range codec: good compression, and good speed.

[Atamido]

Originally posted by akupenguin
FFV1 works in YUV of any subsampling ratio (4:4:4, 4:2:2, 4:2:0, 4:1:1, 4:1:0) and also RGBA. Yes, I made sure all of the encodes in my test are lossless. Interesting, I wonder when they added that? But if you checked to make sure the output was bit identical, it's all good. :)

[akupenguin]

Originally posted by Zarxrax
ff huffyuv (resulting files are corrupt)


What version of libavcodec? There have been some recent fixes and improvements. Last week it was split into two codecs: "ffhuffyuv" is compatible with the the original, and "ffvhuff" contains our improvements (YV12, adaptive huffman tables).

For best results, use vcodec=ffvhuff:vstrict=-1:pred=2:context=1


And you've tried ffv1 with context=1 and keyint=10 or 20 and decided that the compression gain is not worth the speed or the addition of P-frames? (FFV1 isn't actually temporally predicted, it just uses P-frames to avoid resetting CABAC contexts.)

[Zarxrax]

Originally posted by akupenguin
What version of libavcodec? There have been some recent fixes and improvements. Last week it was split into two codecs: "ffhuffyuv" is compatible with the the original, and "ffvhuff" contains our improvements (YV12, adaptive huffman tables).
My ffdshow is dated Dec 1, 2004.

Originally posted by akupenguin
And you've tried ffv1 with context=1 and keyint=10 or 20 and decided that the compression gain is not worth the speed or the addition of P-frames? (FFV1 isn't actually temporally predicted, it just uses P-frames to avoid resetting CABAC contexts.)
I actually wasn't sure what the context setting did, so I left it at the defaults, which I assumed would result in the best compression. I didn't enable P-frames because the other codecs that I tested do not support this feature, thus it would not be a fair comparison. Also, P-frames are worthless for the applications that I typically use lossless files for, so it does not interest me.

[akupenguin]

Looks like ffvhuff hasn't been included in ffdshow yet.

Originally posted by Zarxrax
I actually wasn't sure what the context setting did, so I left it at the defaults, which I assumed would result in the best compression. In all of lavc's codecs, the defaults are near the fastest settings. I don't know if this is intentional, or if they just didn't want to change the default behavior when adding a new feature.
I didn't enable P-frames because the other codecs that I tested do not support this feature, thus it would not be a fair comparison. What kind of reason is that? Different codecs have different features. You don't turn off FFV1's Arithmetic coding just because Huffyuv doesn't support it.
Also, P-frames are worthless for the applications that I typically use lossless files for, so it does not interest me. OK, if you need frame-accurate cutting and seeking. It was only about a 5% improvement.

[RadicalEd]

I've managed to get intra-only FFV1 to encode and decode at the same rates and faster compared to Laga, with decent compression gain. I hate to be such a dick, but I've never really seen the point of lagarith... the algorithms are pretty mundane, and it doesn't really fill any niche.

I'll post my test results tomorrow or Monday when I'm home.

[CruNcher]

Sorry but the results i posted before were borked here are the correct ones

2605 1:44 Pal 25 Fps 720x576 (Real) (cleaned DVD)
http://cruncher.mufflastig.com/real.png


size enc dec
ffv1 (AC) 293 MB 2:46 2:28/19 Fps
ffv1 (VLC) 316 MB 2:06 2:00/22 Fps
lagarith 356 MB 1:58 2:05/22 Fps
vble-gr3 368 MB 2:57 1:33/28 Fps
vble-gr2 369 MB 7:01 1:48/24 Fps
milk 371 MB 3:16 3:05/15 Fps
vble2-gr2-lite 375 MB 2:57 2:00/22 Fps
loco 401 MB 1:55 2:11/21 Fps
vble 416 MB 1:20 3:16/13 Fps
huffyuv12 median 468 MB 1:22 2:22/20 Fps
ffdshow huffyuv12 median 689 MB 1:50 2:50/18 Fps


4545 3:09 Film 24 Fps 640x480 (CG)
http://cruncher.mufflastig.com/cg.png


size enc dec
ffv1 (AC) 297 MB 3:40 2:51/30 Fps
ffv1 (VLC) 325 MB 2:46 2:20/33 Fps
milk 370 MB 4:00 3:17/23 Fps
lagarith 390 MB 2:28 2:20/33 Fps
vble-gr2 390 MB 9:20 2:10/36 Fps
vble-gr3 393 MB 3:47 1:50/38 Fps
vble2-gr2-lite 407 MB 3:56 2:26/33 Fps
loco 427 MB 2:33 2:32/32 Fps
vble 520 MB 1:50 3:47/20 Fps
huffyuv12 median 556 MB 1:48 3:14/24 Fps
ffdshow huffyuv12 median 872 MB 1:50 2:40/28 Fps

[dragongodz]

Lagarith doesn't seem to be affected by the "null frames" option when encoding YUV. Is it supposed to be working?
did you use version 1.2.4(updated 12-04-04, same build that has the RGB fix)) to test that ?

I hate to be such a dick, but I've never really seen the point of lagarith... the algorithms are pretty mundane, and it doesn't really fill any niche.
whats that to you ? the guy wanted to write a lossless codec and did. he has made it available for anyone that wants to try it or use it etc. he even made it open source so if anyone wants to improve it they can do that aswell. its not up to you or anyone else to say its pointless because it has cost you nothing and you are not being forced to use it.

[RadicalEd]

Well, if you like it, use it. I still don't see any practical use, FFV1 pretty much covers all bases. I guess I'm just too supportive of FFV1 :P Laga and me go back a ways though. The story of my distaste for it is too long and political to go into without being OT, so I'll step off here.

[akupenguin]

Originally posted by dragongodz
did you use version 1.2.4(updated 12-04-04, same build that has the RGB fix)) to test that ?

I just redid that test with 1.2.4, and null frames still has no effect on YUV.
(When I reported the compression with duplicates removed, that was using MEncoder's decimate filter, not Lagarith's.)

[BlindWanderer]

*poke*
1.2.5 released.
"Bug fixes: Released version did not mantain a copy of the previous frame for YV12/YUY2, resulting in null frames not working for those color spaces."

[Atamido]

Originally posted by akupenguin
Yes, I made sure all of these are really lossless. Would you provide an AVISynth script for testing this? I want to run some tests, but I'm really don't feel like figuring out AVISynth yet again.

[akupenguin]

I did it using "mplayer -vo md5sum", but if you want to use avisynth, the script could be
compare( avisource("raw.avi"), avisource("ffv1.avi"), logfile="ffv1.log" )

[morsa]

Could be Lagarith be modified to accept Bayer mosaic video?
For example a 1920x1080 Bayer mosaic can be decomposed to 4 960x540 planes as follows:

1) Odd lines Green
2) Even lines Green
3) Red
4) Blue

[TEB]

Anyone know how to set Snow to lossless mode? I cannot seem to find anyplace to do that. FFDSHOW from 5.12.2004.

My conclusion is that FFV1 AC mode is almost twice as good compression as lagarith.. and almost the same speed at enc and dec... Will test som more...
FFV can infact be an alternative to mpeg2-50mbit I-frame only 4:2:2 storage for video archives.. I tested some educational films at work and they were 40% smaller in FFV lossless mode 4:2:2 than mpeg2 HP@HL 50mbit i-fram only !! in other words.... that ROX!!! keep up the good work.

[Tommy Carrot]

Originally posted by TEB
Anyone know how to set Snow to lossless mode? I cannot seem to find anyplace to do that. FFDSHOW from 5.12.2004.
Just set the quality to 100.

Speaking of Snow, it's already quite good, often better than ffv1, but it could reach even better compression ratio if it would have a good frame-type decision algorithm implemented, because in lossless mode using keyframes instead of predicted frames could often save a lot of bits, especially in fast motion scenes, so the fixed frametype-pattern seriously hinders the lossless performance (and the lossy too to a lesser extent).

[plonk420]

dunno about you guys but i was getting 40FPS encoding to FFV1 (and the output was garbled, too). OTOH, i'm getting 120FPS encodign to Logarith and a projected filesize 1/6 the size of the FFV1...

[PlazzTT]

Version 1.3.1 is out now: http://lags.leetcode.net/codec.html

I'm having trouble with the new version. The old version I was using (1.2.4 I think, there was no way of checking the version number of it) was working great.

I installed the new version, and the codec seems to encode okay. I'm using VirtualVCR to capture live TV. But I can't seem to decode, view or open the encoded AVI afterwards. The 1.3.1 decoder can decode 1.2.4 clips okay, so the problem must be with the encoding.

The 1.3.1 AVI I have crashes MPC and Vdubmod. Using AviSynth's AviSource to read it, I get "Evaluate: Unrecognized exception!"

I opened a working 1.2.4 in GSpot and it says the container is OpenDML (AVI v2.0). Fine.

But when I open the 1.3.1 clip in GSpot, it says:
Multipart OpenDML AVI
(5807 frames in first part, 3 frames follow) :confused:

Maybe the problem is with VirtualVCR? Either way, does anyone know how I can open this AVI I recorded. It's not a very important clip to me, but I've been recording with 1.2.4 ever since.

[TEB]

1.3.2 is out fixing some crashing bugs :)

[Sir_Lagsalot]

Theres now also a recovery version of the codec for corrupted video from 1.3.0/1.3.1, sounds like the error you're getting: http://lags.leetcode.net/Lagarith_131_recovery.zip

[PlazzTT]

Originally posted by Sir_Lagsalot
Theres now also a recovery version of the codec for corrupted video from 1.3.0/1.3.1, sounds like the error you're getting: http://lags.leetcode.net/Lagarith_131_recovery.zip

Excellent! It works. Thanks.

EDIT: At least it seems to work. How slowly is this fix meant to decode? It seems to stop doing anything after 2 or 3 frames (Vdub goes to 0.0fps for at least 5 minutes)

[Sir_Lagsalot]

It should still get a few fps... Try downloading it again (I tweaked it a little), make sure multithreading is disabled, and make sure you are encoding to a different codec. If it still gets stuck, try manually removing the frames it stops on in virtualdub so they don't get decoded.

[PlazzTT]

Yep, much better now! I'm getting 9-10fps encoding 720x576 to Huffyuv.

[Longinus]

Hi!
Thanks for this codec. It's really good.
And it works... something I can't say for FFV1 and MSU.

I'm encoding 2K material (2048x1556) in RGB. FFV1 just creates a unusable video (if I set it to YUV, it works... but RGB to YUV conversion is so lossy that my eye hurts :p). And MSU and Alparysoft crashes in the middle of the encode.

CorePNG works, but it's slower than your codec... and I'm having problem importing it in Premiere. And your codecs compress a tiny little better than corepng (3D cel-shade material).
Huffyuv also works, but compress a LOT worse than both codecs.

Again, thanks for a very nice codec!

[Longinus]

Oohhh, I have one request, If I may...

The reduced resolution function is useafull, but not as fast as I wish it was. Perhaps because of the resize method? That seams to be some kind of bilinear resize.
What about an option to make a faster decode, with Nearest Neighbor or something like that for the resize?

Thanks!

[Zarxrax]

Lagarith 1.3.4 is released.

Now decoding speed has increased, and the resize method for reduced resolution mode is also faster.

Also, version 1.3.3 added the ability to downsample the colorspace when encoding, to use less space.

I tested the decoding speedup myself. The old version decoded a clip at an average speed of 18.4 fps. The new version decoded the same clip at an average speed of 21.1 fps. About 15% faster decoding speed on my clip.

http://lags.leetcode.net/codec.html

[dragongodz]

i notice lagarith is up to version 1.3.8 now so thought it may be worth mentioning here.

change log since 1.3.4

Version 1.3.8 released on 03-11-2006
Fixed a bug that caused Lagarith to attempt to decode any file encoded with an old version as YV12 when asked to suggest the format (affected AviSynth). Thanks to Zarxrax for reporting this error.
Fixed a bug that could cause video corruption or crashes when encoding certain types of YV12 source video.
Changed threading calls and synchronization methods; this may improve performance slightly when multithreading is used.
Merged various changes from the AMD 64 build back into the base code; hopefully I will be able to merge the builds as the AMD 64 version matures.
Support for non-MMX processors has been ended completely.

Version 1.3.7 released on 11-12-2005
The codec now writes extra information in the format header to indicate which mode the codec used when encoding. This allows the codec to decode to the format used internally even when 'Report Original Colorspace' is used.
Several other minor code changes and small optimizations.

Version 1.3.6 released on 05-26-2005
Decoding RGB24 as RGB32/RGBA now sets the alpha channel to 255 instead of 0; previously, such video would show up as solid black in Vegas because it was perceived as fully transparent (thanks to Stephen Schlueter for reporting this error).
Fixed RGB24 encoding for systems that do not have an MMX capable processor.
Fixed a bug that prevented encoding RGB32 as RGB24 when the "Report original colorspace" option was checked.

Version 1.3.5 released on 05-28-2005
Fixed a bug that caused solid color frames to only decode partially when exporting RGBA.
Added exception handling when freeing memory.

[smok3]

can somebody nice post some command lines to use for RGB and RGBA material, using mencoder and its lossless codecs on win32?

(and the same for decoding)?

[akupenguin]

mencoder -ovc lavc -lavcopts vcodec=ffv1:vstrict=-2:coder=1:context=1:format=bgr32 in.avi -o out.avi

Note that despite the "bgr32", it doesn't store an alpha channel.

[smok3]

a. tnx a lot, so i guess for huff it is similar? like :
vcodec=huffyuv:format=bgr32 - will that store alpha as well?

---

b. another question, what version of ffdshow should one get (and where?) for proper win playback of ffv1?

---

c. and another thing concerning mplayer and avs, is it possible that mplayer forces avisynth to always return only yv12?
(i was trying to use '-vo md5sum' to find out that:
original != ffv1 when
huff = ffv1
p.s. when i compare using avisynth compare function, then:
original = huff
at least i cant seem to find any other explanation.... )

[akupenguin]

a. ffmpeg's huffyuv does rgb decoding, but can only encode yv12 or yuy2.
b. ffv1 hasn't changed in a while, so any build >= 2005 february should work. I can't recommend one in particular.
c. Yes, mplayer invokes ConvertToYV12() if the avs isn't already. There's no particular reason for this, just whoever is in charge of demux_avs.c never implemented other colorspaces.

[Ahmedbrave]

I been tested Logarith codec (ver. 1.3.9) wich I downloaded from site (http://lags.leetcode.net).

And found that it have the same issue as 1.3.8 ver.: a bug that allowed YUY2 and YV12 video to be decoded as RGB16.

So i looked at files in archive wich I downloaded more carefully (it`s name was Lagarith_139.zip): in properties of lagarith.dll i looked at version, it was 1.3.8!!!

May be author made a wrong version build?... (or just uploaded a wrong file?...)
stupidies marked by gray...
i`m sorry.. codec work fine.. mixed out some thing...

[wereHamster]

What does lagarith do differently then huffyuv? Prediction? huffman encoding? I'm using a modified huffyuv codec with the median prediction and a simplified huffman table (one table for all three YUV planes) and it's working fine, but maybe I could 'borrow' something from lagarith to make my codec faster?

[Ahmedbrave]

wereHamster
...don't really know, what different, anyway compression is grater, but encoding is slower.

[video_magic]

Version 1.3.11 released on 11-12-2006

http://lags.leetcode.net/Changes.html
http://lags.leetcode.net/codec.html

[akupenguin]

What does lagarith do differently then huffyuv? Arithmetic coding instead of huffman.

[TEB]

Hi. I did some very rudementary tests here on my DualCore Athlon 4200 in VDUB 1.6.17 DVCAM AVI file 4:2:2 -->

Lagarith 1.3.11 RGB24 SMP 42 Sec 328MB
Lagarith 1.3.11 RGB24 NON-SMP 53 Sec 328MB
Lagarith 1.3.11 YV12 SMP 37 Sec 165MB
Lagarith 1.3.11 YV12 NON-SMP 38 Sec 165MB
UNCOMPRESSED RGB32 36 Sec 2324MB
MSU 0.6 Balanced, yv12 4:01 167MB
FFV1 RGB32 VLC 2.07 301MB
FFV1 RGB32 AC 2.05 297MB
FFV1 YV12 VLC 1:12 181MB
FFV1 YV12 AC 1:12 176MB
HUFFY YV12 : 35sec 522MB


Note:

The files from lagarith differ in size based on using SMP or not, just a 200-300bytes, but still worrying

Anyway, Lagarith in yv12 is VERY impressive imo. ITs just a 2 sec slower than huffy and 3times as efficient

This gotta be the worst comparison ever but i thought id share my results ;)

teb

[TEB]

btw, anyone know the parameters for running x264 in lossless mode?

[Inventive Software]

Constant Quality mode 0. Think the commandline's:


x264.exe -q0


But search for it, or do:

x264.exe --help