Any 2021 lossless video comparison?

[PCU]

Any unbiased 2021 lossless video comparison?
Maximum compression is the most important thing for me.

[richardpl]

lagarith

[PCU]

lagarith

Even better than YULS?

[Zarxrax]

I have also been looking for recent comparisons but not found anything.
Purely from a size standpoint (for example, to archive footage), I have found x264 to be significantly smaller than any of the typical lossless codecs. I have not tried an HEVC encoder, but it could potentially be even smaller.
x264 is also hard to beat from a speed standpoint. I only saw about 12% size difference between veryfast and veryslow presets.
If you are willing to accept some minor loss (still visually lossless) you can set the crf to around 10 and cut the size to 1/3 of the lossless version.

[PCU]

I have also been looking for recent comparisons but not found anything.
Purely from a size standpoint (for example, to archive footage), I have found x264 to be significantly smaller than any of the typical lossless codecs. I have not tried an HEVC encoder, but it could potentially be even smaller.
x264 is also hard to beat from a speed standpoint. I only saw about 12% size difference between veryfast and veryslow presets.
If you are willing to accept some minor loss (still visually lossless) you can set the crf to around 10 and cut the size to 1/3 of the lossless version.

Have you tried AV1 lossless so far?

[Zarxrax]

Have you tried AV1 lossless so far?

No but it might be a bit smaller. For me speed is at least semi-important so I was good with x264. If you really want small size, I think its worth considering making just a very high quality encode rather than insisting on purely lossless content, as ultimately no one will be able to tell the difference as long as quality is sufficiently high.

[PCU]

Active lossless video formats:
FFV1
MagicYUV
x264 (not recommended)
x265
AV1
JPEG XS
Motion JPEG 2000 (MainConcept)
VC-2
UT Video Codec Suite

Why does no one fork CorePNG? (better rename to MPNG (Motion PNG)
CorePNG is a very attractive project.

[PCU]

No but it might be a bit smaller. For me speed is at least semi-important so I was good with x264. If you really want small size, I think its worth considering making just a very high quality encode rather than insisting on purely lossless content, as ultimately no one will be able to tell the difference as long as quality is sufficiently high.

I prefer lossless, but the size is the most important thing for me:
source: 35 MB, lossless: MagicYUV/PCM 7.5 GB!
Length: 2.30 min!

[Zarxrax]

Active lossless video formats:

Why does no one fork CorePNG? (better rename to MPNG (Motion PNG)
CorePNG is a very attractive project.

PNG is a very old technology. Modern formats like AV1 should easily outperform it.

Also, if size is the biggest concern, you need a codec that makes heavy use of temporal compression. X264, X265, or AV1. Codecs like MagicYUV, UTvideo, Corepng, etc are primarily designed with every frame being a keyframe, and this will be SIGNIFICANTLY larger in filesize.

[PCU]

PNG is a very old technology. Modern formats like AV1 should easily outperform it.

Also, if size is the biggest concern, you need a codec that makes heavy use of temporal compression. X264, X265, or AV1. Codecs like MagicYUV, UTvideo, Corepng, etc are primarily designed with every frame being a keyframe, and this will be SIGNIFICANTLY larger in filesize.

My question is that apart from the color space being lost in the conversion, is the x265 really like FLAC?

https://forum.videohelp.com/threads/382367-X265-Lossless-not-really-lossless-and-I-can-prove-it

So here's the deal, currently have a GTX960 and decided to buy mini1050 for $115 when payday comes around. In preparation of this change I have been doing a bunch of tests of the GTX960's nvenc as a baseline to compare how the Pascal powered 1050 performs quality wise, namely I wanted to see if the 1050 offered better quality because it executes the same settings better, is it because it offers more settings than the 960 or both.

So using a static build of ffmpeg and Ubuntu 16.04 LTS and the latest NVIDIA drivers I transcoded a slew of files and used ffmpeg to calculate baseline PSNR and SSIM values. As a reference point I decided to include some x264 ultrafast and medium encodes and some x265 ultrafast encodes, it was when I decided to do a bunch of lossless encodes that I realized something was wrong.

Using a 2.3gb 1080p source file I transcoded it to nvenc_h264 lossless (the 960 doesn't support H265 lossless) and the resulting files size was 30.9gb.

The x264 ultra fast lossless was 28.6gb, with the medium preset it drops down to about 22gb. Then I ran into something really odd, the x265 ultra fast encode was only 14gb, less than half the size of the other 2.

Using ffmpeg to caluclate PSNR and SSIM for all the files i got the following values:

NVENC H264 Lossless
[Parsed_ssim_0 @ 0x27956a0] SSIM Y:1.000000 (inf) U:1.000000 (inf) V:1.000000 (inf) All:1.000000 (inf)

[Parsed_psnr_1 @ 0x2795d40] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

X264 Lossless Ultra Fast
[Parsed_ssim_0 @ 0x37958a0] SSIM Y:1.000000 (97.759916) U:1.000000 (inf) V:1.000000 (inf) All:1.000000 (99.656556)

[Parsed_psnr_1 @ 0x3796040] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

X265 Lossless Ultra Fast
[Parsed_ssim_0 @ 0x4133a40] SSIM Y:0.996310 (24.329467) U:0.997240 (25.590147) V:0.997184 (25.503329) All:0.996610 (24.698549)
[Parsed_psnr_1 @ 0x41340e0] PSNR y:53.354600 u:55.613567 v:55.443195 average:53.965527 min:51.749530 max:72.329853

The only values that make sense are the NVENC Lossless values, SSIM has a range between -1 and 1, with -1 being completely different and 1 being exactly the same and the decibel value for both PSNR and SSIM should be infinite if the encode is truly lossless.

Even x264 isn't really lossless but it's close enough that we can let it slide but the x265 values really surprise me.

Anyone want to run similar test with x265 on test samples they have and see if they see similar results.

For the record, lossless encoding for x264 and x265 where done with CRF 0.

[Zarxrax]

My question is that apart from the color space being lost in the conversion, is the x265 really like FLAC?

Per the thread you linked, people responded that the wrong settings were used.

I just did my own test in x265 and it did turn out completely mathematically lossless.

[PCU]

Per the thread you linked, people responded that the wrong settings were used.

I just did my own test in x265 and it did turn out completely mathematically lossless.

:thanks:

What about this comparison, says x264 is not that good in compression, correct me if I'm wrong:

https://compression.ru/video/codec_comparison/pdf/msu_lossless_codecs_comparison_2007_eng.pdf

[Zarxrax]

:thanks:

What about this comparison, says x264 is not that good in compression, correct me if I'm wrong:

https://compression.ru/video/codec_comparison/pdf/msu_lossless_codecs_comparison_2007_eng.pdf

That was a very old version of x264 now. There were numerous improvements to lossless encoding after that, including revision 994 which was said to improve lossless encoding by 4-25%.

[PCU]

That was a very old version of x264 now. There were numerous improvements to lossless encoding after that, including revision 994 which was said to improve lossless encoding by 4-25%.

:thanks:

That's why we need new unbiased comparison!

[PCU]

Per the thread you linked, people responded that the wrong settings were used.

I just did my own test in x265 and it did turn out completely mathematically lossless.

What settings do you use for maximum compression?

[Zarxrax]

What settings do you use for maximum compression?

Dunno, I haven't played with it much aside from just checking if I got lossless output. I would guess the speed setting is the most important thing after the --lossless option.

[PCU]

Dunno, I haven't played with it much aside from just checking if I got lossless output. I would guess the speed setting is the most important thing after the --lossless option.

What software do you use?
I'm a pro user of MainConcept codecs, but noob user of x264/x265, because IDK anything about it beside main settings!

[poisondeathray]

Even x264 isn't really lossless but it's close enough that we can let it slide but the x265 values really surprise me.


Then there is some user error somewhere. Such as timestamps off or jitter, or not using --lossless switch for x265 (crf 0 is not lossless for x265)


What pixel format ?
What type of content ?

For 8bit 4:2:0, it will be x264 or ffv1 using long gop

Long GOP will almost always produce better results, so that rules out things like lagarith. The exception will be null frames. Content that has lots of identical frames. Lagarith "null frames" option will put it over the top

[PCU]

Then there is some user error somewhere. Such as timestamps off or jitter, or not using --lossless switch for x265 (crf 0 is not lossless for x265)


What pixel format ?
What type of content ?

For 8bit 4:2:0, it will be x264 or ffv1 using long gop

Long GOP will almost always produce better results, so that rules out things like lagarith. The exception will be null frames. Content that has lots of identical frames. Lagarith "null frames" option will put it over the top

I just want max compression for lossless, --lossless is enough?

[poisondeathray]

I just want max compression for lossless, --lossless is enough?

For x265 --lossless enables lossless mode. Anything else is not lossless .

Slower settings do not always reduce filesze, it depends on content. eg. You can demonstrate cases where "very slow" produces larger file than "very fast". There is no "max compression" setting for everything

But x265 always produces worse compression ratio than x264 for 8bit 4:2:0 content lossless encoding - there is not a single test in thousands that show otherwise (when using similar settings, not something like intra. vs long gop) . There is no reason to use x265 lossless - slower to encode/decode, larger filesizes

Your PSNR/SSIM tests look not done correctly (likely timestamp issue, or container timebase differences if it doesn't show "inf")

[PCU]

For x265 --lossless enables lossless mode. Anything else is not lossless .

Slower settings do not always reduce filesze, it depends on content. eg. You can demonstrate cases where "very slow" produces larger file than "very fast". There is no "max compression" setting for everything

But x265 always produces worse compression ratio than x264 for 8bit 4:2:0 content - there is not a single test in thousands that show otherwise. There is no reason to use x265 lossless - slower to encode/decode, larger filesizes

Your PSNR/SSIM tests look not done correctly (likely timestamp issue, or container timebase differences if it doesn't show "inf")

Thank you, very strange.
But for HDR files, the only option is x265 or AV1.
I'm using Adobe Media Encoder + Voukoder + Autokroma Influx plugins, first for encoding and second for decoding. the Voukoder plugin has all x265 command line option in a GUI. what commands do you use?

[poisondeathray]

HDR is usually going to be at least 10bit 4:2:0 . Not as many comparison tests for lossless compression HDR scenario

--lossless is the only required switch for x265. The other options can make it slightly better or worse in terms of compression ratio (it varies) . In general a longer GOP has diminishing returns, just don't use --keyint 1 if the goal is compression. HDR settings would be the same as the input file if you want to retain the metadata

[PCU]

HDR is usually going to be at least 10bit 4:2:0 . Not as many comparison tests for lossless compression HDR scenario

--lossless is the only required switch for x265. The other options can make it slightly better or worse in terms of compression ratio (it varies) . In general a longer GOP has diminishing returns, just don't use --keyint 1 if the goal is compression. HDR settings would be the same as the input file if you want to retain the metadata

--lossless preset=placebo is it ok?

[poisondeathray]

--lossless preset=placebo is it ok?

You can try it , but slower presets are sometimes worse for lossless encoding. Don't ask me why, they just are sometimes. There is no way to predict it's source dependent . +/- 0.1% filesize is not going give you a bad day

[excellentswordfight]

Did a quick test a year or two ago with a 1min sample from an 1080p24 bluray:

Original: 179 MiB
x264 (preset veryslow, crf 0): 733 MiB
x264 (preset fast, crf 0): 765 MiB
x265 (preset veryslow, lossless): 808 MiB
FFV1: 831 MiB
JPEG2000: 932 MiB
x265 (preset fast, lossless): 934 MiB
Ut Video: 1.30 GiB
HUFFYUV: 1.87 GiB
Uncompressed: 4.17GiB

[PCU]

Did a quick test a year or two ago with a 1min sample from an 1080p24 bluray:

Original: 179 MiB
x264 (preset veryslow, crf 0): 733 MiB
x264 (preset fast, crf 0): 765 MiB
x265 (preset veryslow, lossless): 808 MiB
FFV1: 831 MiB
JPEG2000: 932 MiB
x265 (preset fast, lossless): 934 MiB
Ut Video: 1.30 GiB
HUFFYUV: 1.87 GiB
Uncompressed: 4.17GiB

What about YULS and Lagarith?

[Zarxrax]

What about YULS and Lagarith?

The website for YULS states that it has a maximum resolution of 1024 x 768. If this is true, I can't see that codec being very useful these days, as most people are going to have footage larger than that.

Lagarith ought to fall somewhere between UTvideo and FFV1.

[PCU]

The website for YULS states that it has a maximum resolution of 1024 x 768. If this is true, I can't see that codec being very useful these days, as most people are going to have footage larger than that.

Lagarith ought to fall somewhere between UTvideo and FFV1.

Thanks.
What about AV1?
For HDR, the only option is x265 for now.

[excellentswordfight]

What about YULS and Lagarith?
Dunno about YULS, I only tested ffmpeg bundled codecs thats why lagarith wasnt inlcuded, as Zarxrax mentioned, it should be somewere between UT Video and FFV1.

Thanks.
What about AV1?
For HDR, the only option is x265 for now.
I tried AV1 lossless, but its so slow that I didnt even bother letting it finish. Someone that has a faster CPU/has more interested in tuning it for some speed maybe can give an comparison on how good the compression is in lossless mode. But I'm pretty sure that it will keep being to slow offer any real life value when in comes to lossless compression for quite some time.

Why is x265 the only option for HDR? As long as the codec supports the pixel format it should be fine for HDR content, no? Are we discussing real world applications here? Is there a specific workflow need that you have for these lossless questions or are we just talking compression in general?

[benwaggoner]

HDR is usually going to be at least 10bit 4:2:0 . Not as many comparison tests for lossless compression HDR scenario

--lossless is the only required switch for x265. The other options can make it slightly better or worse in terms of compression ratio (it varies) . In general a longer GOP has diminishing returns, just don't use --keyint 1 if the goal is compression. HDR settings would be the same as the input file if you want to retain the metadata
I've done some lossless HDR encoding. A few notes

Unlike with non-lossless, the higher bit depth increases bitrate (typcially >25% due to more noise in the least significant bits).)
With x265, --preset placebo can be >10% smaller than --preset veryslow
Improvement of HEVC versus AVC lossless is <<2x
Using all IDR isn't that much bigger than long GOP in lossless, and allows for more threading. Even --keyint 10 will capture most of the value from interframe encoding.

[benwaggoner]

--lossless preset=placebo is it ok?
It would be --lossless --preset placebo

[benwaggoner]

Thanks.
What about AV1?

Do AV1 encoders generally enable the lossless option? x265 via command line is the only HEVC encoder that does off the top of my head.

AV1 encode is generally quite slow and decode is as well. And in many/most cases a lossless AV1 would be going through software decode to due level requirements, so even slower to use in practice.

Off the top of my head I can't think of any reason AV1 would have a material file size advantage over HEVC for the lossless case.

[`Orum]

I have several TB of lossless footage so I've spent a lot of time researching them. To summarize what I've found:


For best ratio, x264 placebo is actually quite good, assuming you don't need alpha. There are some tricks to getting better compression ratios with it but they come with consequences. I've also yet to see where a slower preset for it produced a larger file than a faster one.
In situations you need alpha, I recommend FFV1. It tends to not be as good compression-wise as x264 (especially on footage with identical areas over multiple frames).
There is no reason to use x265 or any AV1 encoder that I have found (except for maybe HDR?). It's slower or worse ratio-wise compared to x264 for any given compression ratio/speed.
For high-speed lossless scenarios (e.g. capture), nvenc H.265 is what I'd recommend. It gets a better lossless ratio than nvenc's H.264 encoder.
If you don't have a Nvidia card for nvenc, but still need high-speed encoding, consider the faster speeds with x264 or ffvhuff.

[PCU]

Thanks guys.
It is true that the conversion speed is very important, but more importantly, the volume of the compression is much more important, because the space occupied on the disk is very important.
Placebo preset was terribly slow using the latest Voukoder (Adobe Media Encoder).

[PCU]

I'm confused: Does FFV1 support HDR and 4K? I searched but did not find any answer.

[FranceBB]

I'm confused: Does FFV1 support HDR and 4K? I searched but did not find any answer.

4K resolution, definitely.
BT2020 with HLG or PQ transfer characteristics yes, but not in the .avi container. I think .mkv and .mov both support it, but I can't test right now as I'm on my mobile.
The only thing that it doesn't support is dynamically changing metadata, so no Dolby Vision nor HDR10+, but you can totally have HDR10 or even different kind of HDR with 12, 14 or 16bit planar precision.

[PCU]

4K resolution, definitely.
BT2020 with HLG or PQ transfer characteristics yes, but not in the .avi container. I think .mkv and .mov both support it, but I can't test right now as I'm on my mobile.
The only thing that it doesn't support is dynamically changing metadata, so no Dolby Vision nor HDR10+, but you can totally have HDR10 or even different kind of HDR with 12, 14 or 16bit planar precision.

Replies from the official dev of FFV1:
HDR10+ might be be possible with ffv1 in Matroska with no changes to the specification but i think no implementation supports this ATM.
ive send the developer who implemented HDR10+ with vp9 in matroska in ffmpeg a mail asking why he didnt implemented it more generically so it would work with any codec in matroska

[FranceBB]

Replies from the official dev of FFV1:
HDR10+ might be be possible with ffv1 in Matroska with no changes to the specification but i think no implementation supports this ATM.
ive send the developer who implemented HDR10+ with vp9 in matroska in ffmpeg a mail asking why he didnt implemented it more generically so it would work with any codec in matroska

Got it.
Keep me posted 'cause I'm interested too, by the way.

[benwaggoner]

I'm surprised to hear that x265 isn't reliably producing smaller mathematically lossless files than x264.

The preset/efficiency ratio can be pretty non-constant. I've seen content where going from veryslow to placebo offers a bigger size reduction than going from medium to veryslow.

[kolak]

I'm confused: Does FFV1 support HDR and 4K? I searched but did not find any answer.

FFv1 same as most lossless codecs doesn't really care about resolution. Max frame size will be down to particular implementation limits, but it will be more like 16K or 32K.

Same with HDR. HDR is nothing more than 10 or 12bit video. Rest is flagging and metadata, but this can be stored in container (so codec itself doesn't need to support HDR flags). MOV, MXF or MKV will work (forget about AVI- old and limited).

I would not bother with x265 for lossless. It's terribly slow for no real gain. If you want good speed use ffvhuff. If you don't care about speed ffv1 is safe. If your source is RGB use codec which supports it natively at your desired bit depth.
As always- there is no simple answer what is the best. It all depends on your specific case.

Some limited numbers for UHD (10bit 4:2:2) source made of BM RAW, Canon RAW and Arri RAW source:
x265 lossless (preset=medium, GOP=1) 1740Mbit= 2.44 compression ratio
ffv1 lossless (ffmpeg) 1471Mbit=2.89 compression ratio
ffvhuff lossless (ffmpeg) 2021Mbit=2.10 compression ratio (10x faster than other 2)

High quality source with noise/grain etc. is typically 2-3x (even less sometimes). Anything above 3x will be rather specific case related to "easy" source (CGI, something lossy compressed many times already, etc.). Maybe long GOP is different.

x265 lossless (preset=medium, GOP=25) 1724Mbit= 2.46 compression ratio - not much gain over I frame only

[FranceBB]

I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed. :)

[PCU]

I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed. :)

Michael Niedermayer:

you mean HDR10+ or something else ? about HDR10+ i did send a mail to the guy who added this for libvpx in ffmpeg with matroska and the same code should work with ffv1 if it was just not intentionally vpx specific i think

if i hear nothing back from that guy (plausible) and i dont forget (i tend to forget these things) then ill move it over so it can be used with any codec

that should unlock HDR10+ for ffv1 but no way to be 100% sure if thats enough before implementing it

i didnt look into dolby vision yet so i cannot comment but i suspect its not hugely different

[kolak]

I'm really surprised to see ffv1 outperforming x265 lossless, even with long GOP. About huffyuv, that's what I generally use as intermediate format through the ffvhuff implementation and it's fast enough for my tastes given that the files I produce are really temporary and used in between before getting to the compressed version, but I always thought that x265 would have outperformed ffv1, huffyuv and lagarith by a margin and I didn't expect ffv1 to actually be better than x265. Thanks for your comparison. Next time I would suggest adding UTVideo to the test just to see how it performs in terms of compression and speed. :)

ffv1 was written from the ground as lossless codec. h264/5 are not about lossless mode at all. Lossless compression is very different than lossy and I doubt h264/5 lossless modes are polished.

[Balling]

ffv1 was written from the ground as lossless codec. h264/5 are not about lossless mode at all. Lossless compression is very different than lossy and I doubt h264/5 lossless modes are polished.

It is not very different it uses YCgCo-R presentation internally (without tagging stuff in matrix in VUI), so is rather effecient, 9 bit YCgCo is enough for RGB 8 bit. Complexity of ffv1 is not comparable with H.26X series. Both hevc_nvenc and h264_nvenc support mathematically lossless!

x265 encoder library lossless mode is not very optimised, so it is known, but that is just because it does not yet support SCC, just CABAC. Read this, as you can see reference encoder lossy and lossless are so much better with scc https://www.cambridge.org/core/services/aop-cambridge-core/content/view/S2048770315000116

Not yet even ffmpeg native hevc decoder support it yet, but there are patches from Intel https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=2878

While hevc with Ice Lake of Intel is supporting SCC already.

JPEG 2000 and AV1 also have lossless modes, last one can outperform (because it has SCC-like already).

Grain generation for HEVC can increase it even further (H.274)

[Balling]

Do AV1 encoders generally enable the lossless option? x265 via command line is the only HEVC encoder that does off the top of my head.

AV1 encode is generally quite slow and decode is as well. And in many/most cases a lossless AV1 would be going through software decode to due level requirements, so even slower to use in practice.

Off the top of my head I can't think of any reason AV1 would have a material file size advantage over HEVC for the lossless case.

AV1 had a bug in lossless encoding in ffmpeg for a long time but it was fixed. Now crf 0 works losslessly, I mean I participated in fixing this bug. All encoders, VP9, AVC, HEVC, AV1 supports lossless.

[nevcairiel]

I mean I participated in fixing this bug..

You are banned on the ffmpeg development mailing list. You participated in nothing constructive.

[tormento]

I'm really surprised to see ffv1 outperforming x265 lossless
The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used. :)

[FranceBB]

The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used. :)

Yeah but it's not supported by the Blackmagic Decklink capture cards, so... nope for my use case. :(

[excellentswordfight]

The huge difference is that you can have x265 lossless by a gpu card, with almost no CPU resorces used. :)
Since when did x265 support hw-encoding? Or are actually meaning HEVC in general, e.g. using nvenc?

Yeah but it's not supported by the Blackmagic Decklink capture cards, so... nope for my use case. :(
Cant you pipe the input to nvenc? FFMPEG can be built with decklink support.

edit.
Just tried the lossless mode in nvenc for hevc, actually surprisingly good, outperformed x265 preset fast (x265 really is bad for lossless), still both beaten by x264 (at 1080p) at preset fast (which isnt that much slower then nvenc on a modern CPU). x264 lossless mode is actually rather good, but a bit limited in that it doesnt support 10bit (afaik).

[PCU]

Lossless mode with HEVC? how did you do that!?