Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th January 2020, 21:31   #1  |  Link
BadEdit
Registered User
 
Join Date: Dec 2019
Posts: 1
Re-encoding for higher compression for archive

Hello,

I have a large amount of Raw (meaning unedited) footage from a project I did a while ago.

It consists of daily shoot footage pre-editing.

It was shot on a variety of DSLRs.

Some footage was shot on an 80d at HD with a bit rate of 89386kbps. Audio @ 1532kbps in a .mov h.264 format.

So a 10min video was around 6gb


Other footage was shot on a C100 as MTS files in an Mp4 format.
These were shot in HD @23989kbps and audio @ 1536kbps

So a 10min video was about 1.72GB

With the project now complete.....I have a HDD full of large video files that can be archived for storage. More than 500gb in total.

Obviously these current formats hog a lot of HDD space.

So, for archiving is there a preferential method for re-encoding these into a similar format for storage with a better quality/size ratio.

I have tried a few tests with handbrake compressing to h.265 using quicksync hardware encoding to speed up the process. The results visually look pretty good and I have achieved a much higher compression ratio.

But, it is not something I have done before.

Is Handbrake a good method?

So, I was wondering if there is a common method? any suggestions on setting? A good baseline kb/s to aim for?

Many Thanks for any advice!

Last edited by BadEdit; 8th January 2020 at 22:32.
BadEdit is offline   Reply With Quote
Old 8th January 2020, 23:27   #2  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,525
My recommendation for "archiving" would be to copy your 500 GB of video to an external USB HDD.
Re-encoding in order to reduce the filesize will introduce losses which cannot be recovered should you some time in the future be disappointed by the quality.
You can still re-encode and keep a compressed version of your videos on the local HDD at a quality level which you are happy with for occasional viewing.
Sharc is offline   Reply With Quote
Old 9th January 2020, 20:34   #3  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,048
You can also store data in AWS Glacier for $4/month per TB if it's really important.

It's pretty challenging to reencode sources to a lower bitrate while preserving all the quality needed to use it as a source in the future.

Using HEVC likely would allow some more savings, and x265 now has the ability to reuse information in a source H.264 bitstream to speed up a reencode. I suspect that this process would help maintain accuracy to the source as well, as encoding would be refining from how it was encoded as well as the actual output pixels. But I've not tested this archival case myself.

It's a fun question, but the effort to verify you have a process that preserves the needed detail is going to be way more than the cost of just storing what you've got, even if you just pay yourself minimum wage.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th January 2020, 20:41   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,048
Oh, actually Glacier Deep Archive is only $0.99/TB per month. That's a no brainer if this content is valuable to you.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th January 2020, 03:52   #5  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,587
^^ Just store it.

If that's totally untenable you can probably get a 2x reduction with HEVC using a very low QP number with x265, but why bother?
Blue_MiSfit is offline   Reply With Quote
Old 11th January 2020, 14:55   #6  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,775
There is severall lossless codec. I don't know all, but there is classical UT Video or Magic YUV. There are intended to be fast, so compression factor is not the most fucussed.
It also exist other lossless codecs, created more in archival purpose, but wich are slow (there was a russian one, i don't remember the name for now). As far as i know, there is in H264 spec a lossless possibility, and i think x264 implement it.
If there is also a such thing in H265, and x265 implement it, you should also try and compare both results (my thoughts are that in theory, if there is such thing in both, x265 should produce better result).

The russian one is MSU codec...! But i just checked the site, it's old, and never been updated since a very long time, but it's still worth trying.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 12th January 2020, 06:52   #7  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,587
Lossless will always be bigger than lossy. That's not what OP wants.
Blue_MiSfit is offline   Reply With Quote
Old 13th January 2020, 20:44   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,048
Quote:
Originally Posted by Blue_MiSfit View Post
Lossless will always be bigger than lossy. That's not what OP wants.
Yeah, for natural image content, lossless will almost always be less than 3:1 compression.

Although I've seen 8:1 with a couple 8K sources; probably because there wasn't that much high frequency detail to code.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Tags
archive, compression, encoding, storage

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.