Best lossless codec for archiving

[PCU]

What's the best lossless codec for archiving?

[Kurtnoise]

Probably FFV1...

[FranceBB]

Tough choice.
I'm gonna go for x265 --lossless just because it's generally more supported by NLE etc than the alternatives like HuffYUV, Lagarith and UTVideo and it performs well...

[kolak]

h265 is slow. Also- is playback really that good for anything above 4:2:0 10bit? For archiving you want at least 4:2:2.
FFV1 is used by many institutions as archiving format.

[SeeMoreDigital]

For me...

Saving to x264 lossless in mkv has always produced reliable results when generating encodes with VirtualDub2.

[Jamaika]

What kind of movies are we talking about? If new HDR then you can use 10bit HDR. Which codec? That's the problem. Cineform is out. MJPEG2000 is out. The JPEG XS was not seen by anyone. X265 at high bitrate is best in 4K.
Old movies only in RGB. Don't record on Bluray.
Will the AV1, VVC, EVC codecs satisfy the recipient in 20 years? No, then it will be 16K, and we will be able to throw 2K films in the trash.

[benwaggoner]

Originally Posted by Jamaika

What kind of movies are we talking about? If new HDR then you can use 10bit HDR. Which codec? That's the problem. Cineform is out. MJPEG2000 is out. The JPEG XS was not seen by anyone. X265 at high bitrate is best in 4K.

Yep, x265 has the best combination of compression efficiency, content type flexibility, and long-term decodability.

Old movies only in RGB. Don't record on Bluray.
Will the AV1, VVC, EVC codecs satisfy the recipient in 20 years? No, then it will be 16K, and we will be able to throw 2K films in the trash.

I doubt even 8K will be standard in 20 years for consumer viewing. The industry has yet to identify moving image content that customers can discriminate 4K from 8K even in optimal viewing conditions. And as theatrical movies have long been the primary driver of content tech innovation, the lack of any plan (or business model) for 8K projectors will blunt that.

My 2041 vision has HDR 4K with dynamic metadata as the default, probably in H.268.

[jpsdr]

UTVideo and MagicYUV are good, i'm using them all the time but for intermediate production files. They are fast, but for archiving, there is probably slower codecs with better compression, like x264 or x265 lossless.

[benwaggoner]

Originally Posted by jpsdr

UTVideo and MagicYUV are good, i'm using them all the time but for intermediate production files. They are fast, but for archiving, there is probably slower codecs with better compression, like x264 or x265 lossless.

x265 lossless can be sped up a lot by doing IDR-only (--keyint 1) and using a faster --preset. You can really crank up -F in that case, as each frame is encoded independently. And better yet, you'll get HW decode on modern GPUs, which can give you performance back.

[kolak]

How much are you going to gain on typical film source (with all grain/noise etc.) using h265 lossless compared to other "fast" codecs?
I never found real world lossless scenario which would give more than 2-2.5x compression (regardless fo the codec).

How difficult would be to do hybrid mode which hits chosen ratio (eg. 3:1), but when it can switches to lossless mode? HDCAM-SR worked this way no?

[Thundik81]

In most cases, just keep original format!

[kolak]

I agree, but at some point it's a difference between spending 1MLN$ or 500K$ for archive and then lossless has its argument

[PCU]

x264/265 is not 100% lossless.

[benwaggoner]

Originally Posted by kolak

How much are you going to gain on typical film source (with all grain/noise etc.) using h265 lossless compared to other "fast" codecs?
I never found real world lossless scenario which would give more than 2-2.5x compression (regardless fo the codec).

How difficult would be to do hybrid mode which hits chosen ratio (eg. 3:1), but when it can switches to lossless mode? HDCAM-SR worked this way no?

It wouldn't be difficult at all, and allows for VBV compliance for profile @ level. Just use --cu-lossless, and coding units that can be efficiently encoded as lossless get encoded as lossless. Lots of fun info about lossless mode here: https://x265.readthedocs.io/en/master/lossless.html.

And yes, about 2.5:1 for content with grain or other random noise is about what you can expect. Totally clean content in Long GOP can go way, way lower. That's less than you'll get with ProRes and other intermediate codecs. Actual mathematically lossless is rarely needed, and should only be used when it really is. Like delivering VFX with alpha for compositing.

[benwaggoner]

Originally Posted by PCU

x264/265 is not 100% lossless.

They are in their lossless modes.

[kolak]

Originally Posted by benwaggoner

It wouldn't be difficult at all, and allows for VBV compliance for profile @ level. Just use --cu-lossless, and coding units that can be efficiently encoded as lossless get encoded as lossless. Lots of fun info about lossless mode here: https://x265.readthedocs.io/en/master/lossless.html.

And yes, about 2.5:1 for content with grain or other random noise is about what you can expect. Totally clean content in Long GOP can go way, way lower. That's less than you'll get with ProRes and other intermediate codecs. Actual mathematically lossless is rarely needed, and should only be used when it really is. Like delivering VFX with alpha for compositing.

3:1 is all what you really need.

[GMJCZP]

Originally Posted by benwaggoner

x265 lossless can be sped up a lot by doing IDR-only (--keyint 1) and using a faster --preset. You can really crank up -F in that case, as each frame is encoded independently. And better yet, you'll get HW decode on modern GPUs, which can give you performance back.

In Simple X264 I try to code with X265 lossless and I can only pass in custom parameters --keyint 1 --lossless. I can't pass a preset, what am I doing wrong?

Edit: I am using x265 version 3.2, is that why?

[rwill]

Originally Posted by benwaggoner

x265 lossless can be sped up a lot by doing IDR-only (--keyint 1) and using a faster --preset. You can really crank up -F in that case, as each frame is encoded independently. And better yet, you'll get HW decode on modern GPUs, which can give you performance back.

HEVC higher levels @High Tier have a minimum compression ratio base requirement of 4 so good luck with that HW decode.

[Asmodian]

My favorite lossless codec is MagicYUV. Very fast.

[benwaggoner]

Originally Posted by rwill

HEVC higher levels @High Tier have a minimum compression ratio base requirement of 4 so good luck with that HW decode.

Yeah, it comes out as High Tier Level 8.something. So far, the HW decoders I've tried seem to work okay. Not having to deal with interframe prediction saves a whole lot of work.