A new lossless video codec: MindVid + codec comparision UPDATED: 5-5-04

[moogie]

In my search for other lossless video codecs to compare to my lossless video codec i am making, i have come accross MindVid Lossless Codec.

http://www.geocities.com/dcristoloveanu/

There isnt any information on the workings of the codec. It does have a free version to use however the "nag" text is quite large.

It does seem to be very fast however.

Has anyone else heard anything else about it?

[moogie]

THIS COMPARISON IS OBSOLETE! please find the new comparision further down this thread

I have peformed a little comparision.

All codecs are set for their best compression.
All videos are 24 bit RGB
Speed was not recorded as i was only interested in the filesize.



Video Description LossLess Codec File Size (bytes)


Timelapse Bateria Growth MindBend 42,506,240
480x480 CorePNG 74,381,312
200 frames AlparySoft 112,723,968
30 fps JPEG-LS 122,777,600
Mainly red, very grainy RAW 139,399,168
Fixed Camera HuffYUV 190,937,088

Clock Pendulum CorePNG 5,502,976
240x240 MindBend 5,888,000
66 frames AlparySoft 6,373,376
30 fps HuffYUV 6,598,656
Normal Colour range,v grainy JPEG-LS 6,637,568
Fixed Camera avizlib 9,129,984
RAW 17,111,040

Tank in desert CGI AlparySoft 79,685,632
600x300 MindBend 91,645,952
348 frames CorePNG 95,256,576
24 fps HuffYUV 103,180,288
Limited Colour Palette JPEG-LS 116,082,688
Camera always rotating avizlib 143,597,568
RAW 206,223,360

Slinky Shadows CGI CorePNG 40,972,288
720x576 avizlib 42,182,656
250 frames AparySoft 64,114,688
25 fps MindBend 65,771,520
High colour contrasts JPEG-LS 91,367,424
Fixed camera, subject moving HuffYUV 152,276,992
RAW3 311,048,192

Earth Rotating in Space CorePNG 30,507,008
640x480 AparySoft 31,971,328
240 frames MindBend 35,244,032
29.97 fps avizlib 39,393,280
Colour gradients JPEG-LS 53,336,576
Fixed Camera, subject rotating HuffYUV 78,968,832
RAW 221,196,288


What is interesting for me was HuffYUV actually increased the file size for in the Bacteria video. Contrasting that was the MindBend codec winning by quite a large margin.

Core PNG does seem to be winning however in each case it does win, it only manages it by a few percent.

[Neo Neko]

Wow I had not looked at corePNG lately. Looks like they have done some work on it!

[d'Oursse]

@moogie: would it be possible that you can add ffv1, vble and milk test in your comparison ?

[moogie]

Originally posted by d'Oursse
@moogie: would it be possible that you can add ffv1, vble and milk test in your comparison ?

I do not believe that it is possible as they do not have RGB24 colourspace options. And last time i used MILK it did not work in vfw programs such as virtual dub.

If I am incorrect please to tell me where i can obtain versions that will work in my problem domain and i will add them to the comparision.

[d'Oursse]

yeah vble and milk are only yuv. I don't know for ffv1. and vfw is buggy in milk.

[RadicalEd]

ffv1 supports 4:4:4 YUV, which for all intents and purposes is the same as RGB. However, last I checked it was b0rking color :|

[Zarxrax]

This MindVid codec really did a number on my system :|

First of all I tried to uninstall it normally. It appeared to work, but it actually didnt do anything. I then tried to delete it through the codecs thing in Device Manager. It would keep coming back. Finally I deleted the DLL file manually. Somewhere between all of this, the codec apparently took the liberty of deleting "MSVCRTD.dll" which caused Virtualdubmod to display an error upon loading. Finally, I managed to get another copy of that dll, and managed to manually remove all traces of the codec out of my registry.

A big pain in the ass, it was :(

[moogie]

Originally posted by Zarxrax
This MindVid codec really did a number on my system :|

First of all I tried to uninstall it normally. It appeared to work, but it actually didnt do anything. I then tried to delete it through the codecs thing in Device Manager. It would keep coming back. Finally I deleted the DLL file manually. Somewhere between all of this, the codec apparently took the liberty of deleting "MSVCRTD.dll" which caused Virtualdubmod to display an error upon loading. Finally, I managed to get another copy of that dll, and managed to manually remove all traces of the codec out of my registry.

A big pain in the ass, it was :(

ouch!

I have contacted the author of the codec and was able to convince him to give me a version of the codec that does not have the nag text for benchmarking and comparision purposes.

It now has its own installer which seems to be fine, well it worked fine for me. You might want to tell the author what happened, he might give you the codec for free.

[moogie]

Originally posted by RadicalEd
ffv1 supports 4:4:4 YUV, which for all intents and purposes is the same as RGB. However, last I checked it was b0rking color :|

Do you know whether the conversion from 24 bit RGB to 24 bit YUV ever clamps the Y,U or V values into the 8bit range? if so then it is not lossless conversion...

[RadicalEd]

I'm sure it does, and of course it's not lossless thanks to roundoff. But the loss isn't nearly the same as, say, 4:2:2 or 4:2:0, where actual information is discarded and rounding isn't the only source of error.

[MfA]

It uses JPEG2000's reversible transform, it isnt really YUV (also he uses modular arithmetic to decrease dynamic range of the final encoded differences ... for reasons I cannot fathom). If it isnt lossless then it is buggy.

[moogie]

Originally posted by MfA
It uses JPEG2000's reversible transform, it isnt really YUV (also he uses modular arithmetic to decrease dynamic range of the final encoded differences ... for reasons I cannot fathom). If it isnt lossless then it is buggy.

hmm, I think i will try it again... could you point me to a binary win32 version of ffv1? I am at home at the moment and do not have a compiler.

[Tommy Carrot]

Originally posted by moogie
hmm, I think i will try it again... could you point me to a binary win32 version of ffv1? I am at home at the moment and do not have a compiler.

You can find it here (http://athos.leffe.dnsalias.com) in the latest ffdshow build.

BTW, that reversible transform in ffv1 is most likely buggy, cuz it visibly alters the colors.

[moogie]

Originally posted by Tommy Carrot
You can find it here (http://athos.leffe.dnsalias.com) in the latest ffdshow build.

BTW, that reversible transform in ffv1 is most likely buggy, cuz it visibly alters the colors.

Thanks.

I use virtual dub so the ffdshow will not work with it. I did download the ffvfw at that url and it did have ffv1, I could compress it, however i could not view the resulant avi... it refused to play in WMP and crashed Vdub each time i tried loading it :(

[Leak]

Originally posted by moogie
Thanks.

I use virtual dub so the ffdshow will not work with it. I did download the ffvfw at that url and it did have ffv1, I could compress it, however i could not view the resulant avi... it refused to play in WMP and crashed Vdub each time i tried loading it :(

If you downloaded ffvfw then you most probably got quite an old version. ffvfw has been merged into ffdshow for quite some time now; it's an install option when installing the newer ffdshow builds from athos.

np: Basic Channel - Radiance (Edit) (Basic Channel CD)

[moogie]

Originally posted by Leak
If you downloaded ffvfw then you most probably got quite an old version. ffvfw has been merged into ffdshow for quite some time now; it's an install option when installing the newer ffdshow builds from athos.

np: Basic Channel - Radiance (Edit) (Basic Channel CD)

Thanks for that, i took your advice and downloaded the latest athos build. The ffv1 now encodes and decodes correctly... however the 444P option is not equivilent to RGB24. It does perform impressive compression, however it is not lossless in the 24bit RGB colour space and thus i cannot use it in my comparision...

[MfA]

FFV1 supports RGB32 (it ignores alpha) not RGB24.

[RadicalEd]

Ah, very nice. I suppose Milan's ffdshow settings are a bit outdated.

[moogie]

Originally posted by MfA
FFV1 supports RGB32 (it ignores alpha) not RGB24.

How do i activate this option, or tell ffv1 to use RGB32?

the only colourspace options i have are:

YV12
444P
422P
411P
410P

I assumed 444P was equivilent to RGB24... but i guess i was incorrect.

I compared the ffv1 compressed video with the raw video and they are not identical... i.e. There is non-infinite PSNR.

[Tommy Carrot]

RGB32 colorspace is now available in the FFV1. I cannot check right now if it's really lossless, but i hope so.

[moogie]

I can confirm that the RGB32 option is indeed lossless.

Good work!

[moogie]

Description Codec FileSize (bytes) Compression Ratio Bits per Pixel

Timelapse Bateria Growth Hdiff 34,659,172 0.25 6.02
480x480 MindBend 42,506,240 0.30 7.38
200 frames CorePNG 74,381,312 0.53 12.91
30 fps FFv1 90,146,816 0.65 15.65
Mainly red, very grainy AlparySoft 112,723,968 0.81 19.57
Fixed Camera JPEG-LS 122,777,600 0.88 21.32
RAW 139,399,168 1.00 24.20
HuffYUV 190,937,088 1.37 33.15

Clock Pendulum FFv1 4,972,544 0.29 10.46
240x240 CorePNG 5,502,976 0.32 11.58
66 frames Hdiff 5,841,708 0.34 12.29
30 fps MindBend 5,888,000 0.34 12.39
Normal Colour range,v grainy AlparySoft 6,373,376 0.37 13.41
Fixed Camera HuffYUV 6,598,656 0.39 13.89
JPEG-LS 6,637,568 0.39 13.97
avizlib 9,129,984 0.53 19.21
RAW 17,111,040 1.00 36.01

Tank in desert CGI FFv1 62,410,752 0.30 7.97
600x300 AlparySoft 79,685,632 0.39 10.18
348 frames Hdiff 81,764,242 0.40 10.44
24 fps MindBend 91,645,952 0.44 11.70
Limited Colour Palette CorePNG 95,256,576 0.46 12.17
Camera always rotating HuffYUV 103,180,288 0.50 13.18
JPEG-LS 116,082,688 0.56 14.83
avizlib 143,597,568 0.70 18.34
RAW 206,223,360 1.00 26.34

Slinky Shadows CGI Hdiff 40,499,519 0.13 3.12
720x576 FFv1 40,658,944 0.13 3.14
250 frames CorePNG 40,972,288 0.13 3.16
25 fps avizlib 42,182,656 0.14 3.25
High colour contrasts AparySoft 64,114,688 0.21 4.95
Fixed camera, subject moving MindBend 65,771,520 0.21 5.07
JPEG-LS 91,367,424 0.29 7.05
HuffYUV 152,276,992 0.49 11.75
RAW 311,048,192 1.00 24.00

Earth Rotating in Space FFv1 22,484,992 0.10 2.44
640x480 CorePNG 30,507,008 0.14 3.31
240 frames AparySoft 31,971,328 0.14 3.47
29.97 fps Hdiff 34,056,169 0.15 3.70
Colour gradients MindBend 35,244,032 0.16 3.82
Fixed Camera, subject rotatingavizlib 39,393,280 0.18 4.27
JPEG-LS 53,336,576 0.24 5.79
HuffYUV 78,968,832 0.36 8.57
RAW 221,196,288 1.00 24.00

The Cat Returns MindBend 60,399,616 0.21 5.14
640x336 CorePNG 65,800,192 0.23 5.60
437 frames FFv1 69,056,512 0.24 5.88
23.977 fps Hdiff 72,822,720 0.26 6.20
Multiple scene Changes AlaprySoft 92,069,888 0.33 7.84
Source from MPEG-4 JPEG-LS 117,014,528 0.42 9.96
Animation HuffYUV 126,054,400 0.45 10.73
avizlib 154,066,944 0.55 13.12
RAW 281,935,872 1.00 24.00

Appartment Walk-through Hdiff 57,337,948 0.10 2.44
640x480 FFv1 68,798,464 0.12 2.93
612 frames MindBend 89,802,752 0.16 3.82
24 fps CorePNG 95,692,800 0.17 4.07
CGI appartment JPEG-LS 120,266,752 0.21 5.12
Many colour gradients avizlib 121,169,920 0.21 5.16
Simple objects AlparySoft 137,172,992 0.24 5.84
HuffYUV 181,549,056 0.32 7.73
RAW 564,041,728 1.00 24.00



Best Codec by Total - File Size

Hdiff 326,981,478
FFv1 358,529,024
MindBend 391,258,112
CorePNG 408,113,152
AlparySoft 524,111,872
avizlib 622,264,320
JPEG-LS 627,483,136
HuffYUV 839,565,312
RAW 1,740,955,648

Best Codec by Avg bpp

Hdiff 6.32
FFv1 6.92
MindBend 7.05
CorePNG 7.54
AlparySoft 9.32
avizlib 9.66
JPEG-LS 11.15
HuffYUV 14.14
RAW 26.08

Best Codec by Ranking 1st = 5 points, 2nd = 4 points, 3rd = 3 points, 4th = 2 points, 5th = 1 point

FFv1 30
Hdiff 28
MindBend 22
CorePNG 21
AlparySoft 11
avizlib 2
JPEG-LS 2
HuffYUV 0
RAW 0


Hdiff is my experimental codec i am in the process of making. I am quite surprised that even in its preliminary stage it is performing very well!

My test video sequences are mostly CGI/animation as that is the domain i am making my codec for.

[Zarxrax]

Wow, hdiff looks impressive! But how does it compare in speed regarding the others?

[moogie]

Originally posted by Zarxrax
Wow, hdiff looks impressive! But how does it compare in speed regarding the others?

lol, u dont want to know ;)

There is currently no optimizations in my code as i am writing it as an honours thesis and as such i am wanting the code to be readily readable.

Once i have *finished* the codec I (or maybe others) to optimize it. (I am quite i high-level programmer and so i dont know many optimization techniques)

It is asymetric compression so even if the compression takes a while decomression of the frames should be fast. My target is for decompression at 640x480 at 30fps.

[Ark]

Originally posted by moogie
[code]
Best Codec by Total - File Size

Hdiff 326,981,478
AlparySoft 333,003,564
FFv1 358,529,024
MindBend 391,258,112
CorePNG 408,113,152
avizlib 622,264,320
JPEG-LS 627,483,136
HuffYUV 839,565,312
RAW 1,740,955,648


The total filesize for Alparysoft is 524,111,872 bytes, in fact it's always behind FFv1 in all clips.
Just wanted to point that out, but I haven't tested the other values for now, so I don't know if there're other errors.
Anyway, very good work moogie! :)

[moogie]

Originally posted by Ark
The total filesize for Alparysoft is 524,111,872 bytes, in fact it's always behind FFv1 in all clips.
Just wanted to point that out, but I haven't tested the other values for now, so I don't know if there're other errors.
Anyway, very good work moogie! :)

Quite right! i must of missed one or used the file size from another codec by mistake.

corrected now.

Cheers

[stephanV]

first of all: nice work :)

have you found an explanation for why huffyuv actually adds bits in the first video (is it reproducable?) it could be a bug in the codec...

[digitalman]

The speed of the newest build of the Alparysoft lossless codec is amazing. On my 3.06 HT PC, I get 70-80 FPS. I also like how it supports YV12!!! Speed and great compression makes this the top lossless codec for me. The only issue I had was, when using the resulting AVI in CCE, it would not load. I had to use AviSource() in Avisynth and frameserve it.

[ernstblaauw]

Is it possible to add a sort of indication of the speed?
I want to use a lossless codec for capturing programs, and I want the best quality. So first I capture lossless, and then I convert it to Xvid. But the lossless codec has to be fast enough to capture realtime.
BTW, I have an AMD Athlon 2500+.

[digitalman]

Check out the Alparysoft codec. They optimized it exactly for your purpose, captures.
http://www.alparysoft.com/prod/compression/index.php
"Modifications in version 1.6
The algorithm allowing encoding and decoding video in real time is added to new version. That makes it useful for encoding video when capturing from TV-tuners, analog video inputs of capture cards and another capture sources."

[Atamido]

Originally posted by Zarxrax
Wow, hdiff looks impressive! But how does it compare in speed regarding the others? Surely it is faster than CorePNG, right?

BTW, what settings did you use for the codecs and what versions? I am curious if you used Best/Auto Delta-Frames for CorePNG because I thought for sure it would clean up on the Earth rotating clip.

[moogie]

Originally posted by ernstblaauw
Is it possible to add a sort of indication of the speed?
I want to use a lossless codec for capturing programs, and I want the best quality. So first I capture lossless, and then I convert it to Xvid. But the lossless codec has to be fast enough to capture realtime.
BTW, I have an AMD Athlon 2500+.

I have not recorded the time it took to compress, sorry. At the moment i am not interested in compression time as my codec is very preliminary at the moment and any comparision in speed is not valid as the codecs i am comparing mine against are released/optimized where as mine is still in the developmental stage.

[zettai]

I'm really excited about this codec, moogie - what colorspaces are you hoping to support?

If you need any alpha/beta testing you know where to ask :)

[moogie]

Originally posted by Pamel
Surely it is faster than CorePNG, right?

BTW, what settings did you use for the codecs and what versions? I am curious if you used Best/Auto Delta-Frames for CorePNG because I thought for sure it would clean up on the Earth rotating clip.

At the moment i would say that my codec is running at about 3/4 the speed of CorePNG. i.e. is 1/4 slower per frame for best compression. however i can speed it up quite dramatically by adding a threashold ratio. (depending on the ratio and scene changes, the increase in file size can increase a tiny amount to a large amount)

However as i said any speed comparision is void due to lack of optimizations and that i am currently developing it in java.

I am not currently at my test computer so i cannot give you version numbers, however for:

ffv1 - RGB32, AC
huffyuv - Best RGB setting
Alparysoft - prediction on, best compression
CorePNG - all filters, 999 delta frames, auto delta
Hdiff - Threashold 0, auto delta, auto keyframe

[moogie]

Originally posted by zettai
I'm really excited about this codec, moogie - what colorspaces are you hoping to support?

If you need any alpha/beta testing you know where to ask :)

Thanks!

At the moment it is in it's developmental stage and so it only currently accepts RGB24/32 (no alpha)

however it should be realatively straight forward to incorporate other colourspaces.

Eventually i plan for it to use YUV2 and YV12 as well as the RGB24


My ultimate aim for my codec is to be able to fit a 512x384 animated episode of 25min run time in YV12 losslessly into 700meg.

Probably a pipedream... but i can dream. The only way for this to be remotely possible is for the source to be raw, uncompressed. It is Extreamly difficult to find animated episodes that have not been compressed.


Once i get to the stage of needing beta testers i will probably post on here asking for testers. However i feel that it is a long time in coming :(

[Zarxrax]

Originally posted by digitalman
The speed of the newest build of the Alparysoft lossless codec is amazing. On my 3.06 HT PC, I get 70-80 FPS. I also like how it supports YV12!!! Speed and great compression makes this the top lossless codec for me. The only issue I had was, when using the resulting AVI in CCE, it would not load. I had to use AviSource() in Avisynth and frameserve it.

This is interesting, I can't get the codec to encode to YV12 :|
It converts it to YUY2 instead. And I only encode at about 22fps (compared to about 60fps with vble). And when it encodes to yuy2, it cant be played back through vdub or directshow, avisynth is the only thing that can read it :|

[LiFe]

Be interesting to see how Huffy performs on those clips in YUV mode, because it gets considerably better compression than RGB mode.

Pasted everything in an excel (http://www.zip.com.au/~qcvp/dom/codecs.htm) so I could read easier.

Let me know if you can make the avi files available if you don't want to do huffy YUV testing because I would really like to know the differences.

LiFe.

[moogie]

That is to be expected as it is using less bits to represent the colour (16 bits for YUV2 i believe). All the codecs should therefore be about 2/3s the size of RGB24.

At the moment i have no inclination to do a YUV comparision in the near future. I will be when i expand my codec to use other colour spaces.

You are more than free to perform such a comparision, however i cannot host the test files (the test files come to ~250meg when compressed using the best codecs (apart from Hdiff) for each video. If you can host the files, or an ftp peferably, I can send them to you. However it will be coming off of my quotaed uni internet account so i can only do it once. PM me for if you do want the files. (or if you live in Australia i can post them on a CD. (probably the easiest for me))


The Test files are mostly cgi/animation as this is the domain i am developing my codec for.

[Bourla]

Some news regarding Hdiff ?

Could we have a comparison of decoding speeds ?

[moogie]

I have added more features to Hdiff at a cost of compression time but decreased the file size. They mostly little effect on decompression speed.

- the ability to choose between no stream compression, gzip stream compression (default) or bzip2 stream compression. (no stream compression makes decompression very fast, while gzip compression gives a nice trade off in time/space and bzip2 compression normally gives the best stream compression

- JPEG2000 lossless compression (can be used for more 'real-life' images)
- normal PNG compression (for some frames png does produce the smaller frames)
- paletted PNG compression (for low colour count frames paletted png may produce the smallest frame size)
- optimize huffman codes (performed after the frame has been compressed but before writing the the frame, usually gives 2-3% smaller frame size)


I am now working on using three different delta frame types:

- Huffman Encodec Temporal Differences (my current Delta frame type. uses the same techniques i use for key frame compression)

- XOR difference frame (creates an XOR image between the current frame and the last and then compress this XOR image as i would a key frame)

- Block Matching (use a simple block matching technique)


I am on a little hiatus from Hdiff... but will be getting back into it soonish.

As i have said, since this codec is purely for research at the moment i have not made any considerations to make it fast. however that said, it should be able to be decompress/compress fast as there are no advanced and timeconsuming modelling being performed.

I will most likely make my source code freely available so that people with the right skill set can possibly turn my slow java code into something fast and usable.

but that will be after my research thesis is complete (in approx 5 months)

[madness]

I just want to know which lossless codec is best to store the anime in YV12 colorspace without loosing so much of the details because I tends to clean the video before encoding it final. I don't care much about what the filesize would turn out for a 20 minutes video, but I want to keep as much details on the video as much as I can. At the moment I'm using Huffyuv v2.2.0 so I'm looking for something to replace that.

Thanks

[lamer_de]

All lossless codecs keep the same amount of detail, that's why they're lossless.However, unless you use the Huffyuff YV12 version that comes with ffdshow, huffyuff will be yuv2 (or was it yuy2, i'm too lazy too look it up o_O). I use VBLE as a in my opinion good trade-off between speed and filesize.

CU,
lamer_de

[Longinus]

MindVid is dead now?
I tried it with some CG matterial, and the result was just mind-blowing..

CorePNG gave me 300mb, FFV1 256mb (though I can't open it)... and MindDiv 60mb!!! All in RGB24.

But I can't use it, because of that wattermark.. Why don't the author open it? Beucase it seams that he lost interest in it..

[CQ]

Originally posted by Longinus
Why don't the author open it? Beucase it seams that he lost interest in it..

I`ll try to convince him :) .

[Longinus]

Originally posted by CQ
I`ll try to convince him :) .

It would be really cool!!
:D