Re-encoding for higher compression for archive

[BadEdit]

Hello,

I have a large amount of Raw (meaning unedited) footage from a project I did a while ago.

It consists of daily shoot footage pre-editing.

It was shot on a variety of DSLRs.

Some footage was shot on an 80d at HD with a bit rate of 89386kbps. Audio @ 1532kbps in a .mov h.264 format.

So a 10min video was around 6gb


Other footage was shot on a C100 as MTS files in an Mp4 format.
These were shot in HD @23989kbps and audio @ 1536kbps

So a 10min video was about 1.72GB

With the project now complete.....I have a HDD full of large video files that can be archived for storage. More than 500gb in total.

Obviously these current formats hog a lot of HDD space.

So, for archiving is there a preferential method for re-encoding these into a similar format for storage with a better quality/size ratio.

I have tried a few tests with handbrake compressing to h.265 using quicksync hardware encoding to speed up the process. The results visually look pretty good and I have achieved a much higher compression ratio.

But, it is not something I have done before.

Is Handbrake a good method?

So, I was wondering if there is a common method? any suggestions on setting? A good baseline kb/s to aim for?

Many Thanks for any advice!

[Sharc]

My recommendation for "archiving" would be to copy your 500 GB of video to an external USB HDD.
Re-encoding in order to reduce the filesize will introduce losses which cannot be recovered should you some time in the future be disappointed by the quality.
You can still re-encode and keep a compressed version of your videos on the local HDD at a quality level which you are happy with for occasional viewing.

[benwaggoner]

You can also store data in AWS Glacier for $4/month per TB if it's really important.

It's pretty challenging to reencode sources to a lower bitrate while preserving all the quality needed to use it as a source in the future.

Using HEVC likely would allow some more savings, and x265 now has the ability to reuse information in a source H.264 bitstream to speed up a reencode. I suspect that this process would help maintain accuracy to the source as well, as encoding would be refining from how it was encoded as well as the actual output pixels. But I've not tested this archival case myself.

It's a fun question, but the effort to verify you have a process that preserves the needed detail is going to be way more than the cost of just storing what you've got, even if you just pay yourself minimum wage.

[benwaggoner]

Oh, actually Glacier Deep Archive is only $0.99/TB per month. That's a no brainer if this content is valuable to you.

[Blue_MiSfit]

^^ Just store it.

If that's totally untenable you can probably get a 2x reduction with HEVC using a very low QP number with x265, but why bother?

[jpsdr]

There is severall lossless codec. I don't know all, but there is classical UT Video or Magic YUV. There are intended to be fast, so compression factor is not the most fucussed.
It also exist other lossless codecs, created more in archival purpose, but wich are slow (there was a russian one, i don't remember the name for now). As far as i know, there is in H264 spec a lossless possibility, and i think x264 implement it.
If there is also a such thing in H265, and x265 implement it, you should also try and compare both results (my thoughts are that in theory, if there is such thing in both, x265 should produce better result).

The russian one is MSU codec...! But i just checked the site, it's old, and never been updated since a very long time, but it's still worth trying.

[Blue_MiSfit]

Lossless will always be bigger than lossy. That's not what OP wants.

[benwaggoner]

Lossless will always be bigger than lossy. That's not what OP wants.
Yeah, for natural image content, lossless will almost always be less than 3:1 compression.

Although I've seen 8:1 with a couple 8K sources; probably because there wasn't that much high frequency detail to code.