Archive DVDs in "acceptable" quality

[rawyolks]

Hi.
I have a collection of tv show, 22 dvds.
It's old, and doesn't look good even untouched.
It's 1987 Married with Children.
I was thinking to passthrough audio and code video in DVDFab 2-pass.
I tried 500kbps, 1000kbps, 1500kbs, honestly the difference isn't much.
What do you think?
I don't want to keep it in MPEG2 - 1 Gigabyte per episode, it's a waste of hard drive, may as well buy a dvd player.
So... how'd you do it?

Here's sample of my source files, remuxed
https://drive.google.com/drive/folders/1Qoh-GTQ8WfYbbB8CoEuPjSA1AAhMsw2o?usp=sharing
It's worth checking both first and last, there's 11 years of difference in technology

[ShortKatz]

The thing is, transcoding in general is a bad idea. Transcoding to a lossy codec, you will add another generation of loss. Storage is getting cheaper, so I would recommend to keep your native video files. If it wasn't an issue for 11 years, why should it now?

[rawyolks]

Okay, okay. Is it a bad idea to choose x265 over x264 for old dvds?

[Stereodude]

Okay, okay. Is it a bad idea to choose x265 over x264 for old dvds?
Maybe, maybe not. It's unclear if much effort has been put into optimizing and tuning x265 for low resolutions. Try it and see.

[benwaggoner]

22 dual-layer DVDs is still < 200 GB. You can buy a 256 GB thumb drive for $14 on Amazon and store bit-exact copies of them all.

For progressive DVDs, x265 is fine at those resolutions, but I don't know why it'd be worth the effort. If they are true interlaced, x265 isn't all that much better than x264.

[Forteen88]

x265 at CRF 18 (with --preset slower) is a good starting-point.

During the encode (maybe encode ~10 minutes when looking at the projected encode-filesize), if the projected filesize is going to be equal or bigger than the source-video, you can abort the encoding and set a +1 higher CRF so that the filesize for the encode gets lower.

EDIT: The video-source looks a bit dirty though, so using a denoiser, like STpresso/Spresso, is a good idea if you're going to re-encode it with x265.
EDIT2: As mentioned, a deblocker is probably more important to use on this video-source.

[benwaggoner]

And if there are visible MPEG-2 artifacts, using a deblocking decoder can help.

[ReinerSchweinlin]

x265 in default settings lost a lot of detail in my test on DVD resolutions. Turning on the --grain parameter helped a lot, but filesizes went up (up to the point where a transcoe almost was pointless)
I turned to x264 after a while, it seemed to be suited better for "transparent" DVD backups at a given transcode time.
In recent tests, I have been playing around with some neural network scalers and some of them do a quite good job at denoising - and I had promising results throwing DVD footage at them to "clean up" instead of traditional denoisers or degrain filters.. Helped a lot with the transcoding - but of course then the next problem arrises - why not upscale then :)

[benwaggoner]

x265 in default settings lost a lot of detail in my test on DVD resolutions. Turning on the --grain parameter helped a lot, but filesizes went up (up to the point where a transcoe almost was pointless)
I turned to x264 after a while, it seemed to be suited better for "transparent" DVD backups at a given transcode time.
In recent tests, I have been playing around with some neural network scalers and some of them do a quite good job at denoising - and I had promising results throwing DVD footage at them to "clean up" instead of traditional denoisers or degrain filters.. Helped a lot with the transcoding - but of course then the next problem arrises - why not upscale then :)
If you just want to watch the nicest possible video from your sources, sure, upscale! I would always keep the original files though, as upscaling and noise reduction technologies get better every year.

We're not to far from preprocessing becoming "auto-rotoscoping."

[foxyshadis]

x265 in default settings lost a lot of detail in my test on DVD resolutions. Turning on the --grain parameter helped a lot, but filesizes went up (up to the point where a transcoe almost was pointless)
I turned to x264 after a while, it seemed to be suited better for "transparent" DVD backups at a given transcode time.
In recent tests, I have been playing around with some neural network scalers and some of them do a quite good job at denoising - and I had promising results throwing DVD footage at them to "clean up" instead of traditional denoisers or degrain filters.. Helped a lot with the transcoding - but of course then the next problem arrises - why not upscale then :)

x264 is probably the last codec that will ever be heavily tuned for SD, if you have to transcode; x265 requires a fair amount of shutting off things that are too strong for the low resolution. That said, x264 and x265 are so fast on modern machines that you could encode SD into 500, 1000, and 1500 kbps overnight and see what point they look least objectionable.

If you're going to degrain and keep an upscaled copy for ease of playback (I understand, setting that up and getting it consistent can be infuriating compared to spending a few more bits to get it done up front), x265 is a better bet at reasonable bitrates -- 500 certainly doesn't qualify. But if you half the DVD rate to, say, 3000kpbs, that's still a big net win, especially since you can use AAC or Opus for the audio instead of AC3. Even with x264 though, you likely wouldn't notice a big difference on upscaled 1080p at that rate. To really push it down you'll need to spend a little more of your life than Peggy Bundy might be worth to optimize every single thing.

[benwaggoner]

Rest assured, I've spent a LOT of time tuning x265 for quality down to 400x244 50 Kbps. Parameters need to be tweaked, but it can reliably deliver substantially better quality than x264 at a given bitrate all along the ladder. And HEVC degrades more gracefully than H.264 when there are just not enough bits for the content; more detail loss than artifacts being introduced.

Better graceful degradation via various in-loop deblocking and deringing filters has been a key aspect of the progression of codecs from MPEG-1/2 to H.264 to HEVC to VVC and AV1.

[Blue_MiSfit]

HEVC degrades more gracefully than H.264 when there are just not enough bits for the content; more detail loss than artifacts being introduced.


Yep. Contrary to the suggestion of others to limit CTU sizes for SD content, I've found leaving at the default CTU 64 is helpful as it reduces distracting blocking more. Yes you get less detail if you compare frame by frame, but the viewing experience is better / less distracting. People preferred this look in almost every subjective test I did.

[benwaggoner]

Yep. Contrary to the suggestion of others to limit CTU sizes for SD content, I've found leaving at the default CTU 64 is helpful as it reduces distracting blocking more. Yes you get less detail if you compare frame by frame, but the viewing experience is better / less distracting. People preferred this look in almost every subjective test I did.
Yeah. I've done bitrate ladders that used CTU 64 at the very top and bottom, and 32 in the middle. 64 is important at 4K but the smallest CU sizes aren't as needed, so going 64-16 is a good speed/quality tradeoff. Middle resolutions are better off with 32-8. And the lowest, which also have the lowest bits-per-pixel, trying the full 64-8 can help squeeze out some more quality, and having a high per-pixel CPU cost isn't that big a deal when there are so few pixels.

Insisting that only quality tests done at full fps playback count has been very helpful. Sweating how a single frame looks is irrelevant if viewers will never stare at a single frame.

Still, for this project I'd still just put all the .VOB files on a $20 USB drive!

[RanmaCanada]

I have to agree with Mr Waggoner. The time required to encode these down would be better spent on doing something else. Just buy a USB drive and call it a day.

[ReinerSchweinlin]

If you just want to watch the nicest possible video from your sources, sure, upscale! I would always keep the original files though, as upscaling and noise reduction technologies get better every year.

We're not to far from preprocessing becoming "auto-rotoscoping."
Yes, if the original source is an important one, I always keep it - who knows what will be possible in the future :)
In the times I´ve dealt with "AI", I started to develop "an eye" for effects caused by this.. Spotted some cleanups by Topaz Software in some older NASA Footage and it looks like amazon also is doing some processing with neural networks.

[ReinerSchweinlin]

Even with x264 though, you likely wouldn't notice a big difference on upscaled 1080p at that rate. To really push it down you'll need to spend a little more of your life than Peggy Bundy might be worth to optimize every single thing.
References to Katey Sagal might become more difficult to catch for the younger generation among us, but I enjoyed it and tuned into some Futurama instantly :)

Maybe it´s important to stress one point: The OP asked about "archiving DVDs in acceptabel Quality"... Well, in most cases the DVD simply is available in a better source (BD..) - if not and its important to me, I´d keep it as the original. But IF I have a case where SD Content needs to be converted to h265 - I mostyl decide to transform it somehow (denoise, etc..) to tune it a little... Try to convert Eyes Wide Shut from DVD to HEVC - you are better of leaving it as it is (or buy the BD) - Kubrik has so many "natural low light shots" in his Videos, that "taming the noise" has to be your hobby to make sense :)

[benwaggoner]

Yes, if the original source is an important one, I always keep it - who knows what will be possible in the future :)
In the times I´ve dealt with "AI", I started to develop "an eye" for effects caused by this.. Spotted some cleanups by Topaz Software in some older NASA Footage and it looks like amazon also is doing some processing with neural networks.
Note that streaming services license content from a variety of partners. Remastering/upscaling is almost always done by the content owner, not the streaming service, as it can impact creative intent.

The goal of streaming services is to deliver the original pixels as accurately as possible, not to change or "enhance" them.

If you're seeing upscaling artifacts via a streaming service, it's almost always going to be from remastering by the content owner, or potentially by the upscaling in the TV. Mainstream TVs are getting increasingly advanced ML-based upscaling built in.

[Forteen88]

... Just buy a USB drive and call it a day.For me, my three USB-flashmemorys have gotten faulty even though I barely used them. I rather recommend saving files on a SSD-drive, since they seem to be more secure. Even SD-cards have been holding up much better than my three USB-flashmemorys.

[RanmaCanada]

For me, my three USB-flashmemorys have gotten faulty even though I barely used them. I rather recommend saving files on a SSD-drive, since they seem to be more secure. Even SD-cards have been holding up much better than my three USB-flashmemorys.

Well USB drives can be more than just thumb drives :) OP can get a USB SSD and use that, use a regular HDD with a usb connection, etc. Even a 256 gig SSD is cheap enough, at least in Canada (though you do have to power it up from time to time to ensure it does not erase itself).

[ReinerSchweinlin]

Note that streaming services license content from a variety of partners. Remastering/upscaling is almost always done by the content owner, not the streaming service, as it can impact creative intent.

The goal of streaming services is to deliver the original pixels as accurately as possible, not to change or "enhance" them.

If you're seeing upscaling artifacts via a streaming service, it's almost always going to be from remastering by the content owner, or potentially by the upscaling in the TV. Mainstream TVs are getting increasingly advanced ML-based upscaling built in.

Thanx for the info.
I know that APPLE used to take the PCM masters of the studios and re-encode them with theyr own Encoders and market them as "high res masters". So my thought was that maybe amazon does simething similar - they might take a more or less uncompressed master from the Studio and encode it to whatever thy need it to be. Netflix seems to do this, youtube does re-encoding. An since Amazon needs different bitrates for different bandwiths, I thought they do this inhouse (whenever I see a drop in the bandwith of our DSL, I recognize the videostream switching to lower resolutions). This lead me to the impression that amazon and others do theyr own processing/encoding. Learned something new :)
The "AI Treatment" then seems to be done by someone else. It´s clearly visible in more and more footage.

[benwaggoner]

Thanx for the info.
I know that APPLE used to take the PCM masters of the studios and re-encode them with theyr own Encoders and market them as "high res masters". So my thought was that maybe amazon does simething similar - they might take a more or less uncompressed master from the Studio and encode it to whatever thy need it to be. Netflix seems to do this, youtube does re-encoding. An since Amazon needs different bitrates for different bandwiths, I thought they do this inhouse (whenever I see a drop in the bandwith of our DSL, I recognize the videostream switching to lower resolutions). This lead me to the impression that amazon and others do theyr own processing/encoding. Learned something new :)
The "AI Treatment" then seems to be done by someone else. It´s clearly visible in more and more footage.
All the premium media streaming services receive high bitrate high quality "Mezzanine" files from content provides/creators which are used as source for encoding different codecs and bitrate.

These are typically in at least 10-bit 4:2:2 and with a visually lossless intraframe-only codec like ProRes or J2K, with PCM audio.

Third-party content creators can and do create remastered, restored, and upsampled versions of older titles as a mezzanine source. Most "4K" movies have most or all elements originally created in 2K (2048x, so more than 1080p). Cinema HDR is normally graded to 100 nits, while home video HDR typically is mastered for 1000 or 4000 nits.

These are all content creation workflows performed by the content copyright holder, and for high-profile content, with the supervision and/or approval of the original filmmakers.

The goal of a streaming service is to preserve the creative intent embodied in a source mezzanine file as accurately as possible. Making good creative and technical decisions in the first place is the responsibility of the content provider.

[ReinerSchweinlin]

Thanx for the detailed information :) So in case of a movie with a lot of grain - would there be an adjustment for that in order not to compromise details due to block-artefacts ? Or is bitrate always "enough" nowadays?

[rwill]

Thanx for the detailed information :) So in case of a movie with a lot of grain - would there be an adjustment for that in order not to compromise details due to block-artefacts ? Or is bitrate always "enough" nowadays?

In my experience, if you cannot reproduce the grain or just cant encode it - it wont pass the quality control and the movie most likely will not be made available to end consumers.

[benwaggoner]

In my experience, if you cannot reproduce the grain or just cant encode it - it wont pass the quality control and the movie most likely will not be made available to end consumers.
Grain is a challenging case, particularly in UHD since the visual detail "signal" rarely goes up proportionally but grain detail "noise" normally does. There was a lot of work in the first years of x265 to be able to handle the "mostly grain" case in a way that reasonably preserved detail.

There are certainly cases where grain will not be precisely preserved at the maximum top bitrate. The goal there is to let it soften somewhat in ways that don't introduce obvious block patterns and keep the feel generally intact. UHD HDR remasters of Super35 movies from the 80's-90's can be particularly challenging, although the general quality has improved over time.

There was this "restoration means preserving all the grain for creative intent" bias, but that often resulted in way more grain than the original creatives ever would have seen on final print they approved. Relatively dim projectors projecting onto a perf screen can lose a lot of detail that can be just downright painful on a big home 4K TV, even before it gets encoded.

[ReinerSchweinlin]

Thanx Ben for that interesting insight :)
Sorry for the late replies, the mail notification is not working.

[burnix]

I arrived a bit to late, but just inspire you from the benwaggoner encoding challenge.

From at least 1 year i use 2 pass encoding with ffmpeg in hevc and i can compress a lot without any perceptible loss in quality (i accept to see sometime some artefact on hi-speed scene).

If your format is dvd, you can extract dvd simply with mplayer to mpeg2 file and the convert with ffmpeg.

For dvd format i think you can compress in 500k video and 128k or lees audio (in stereo conversion).

But read the "encoding challenge" topic it's a good source of inspiration

[Ghitulescu]

Interesting insight, Benwaggoner

I always thought the streamer services care only about bitrate and framesize (which are set by the billing system they use: premium, normal, free etc) and leave the quality decision to the signal processors (including upscaling if needed, yes I have heard stories).

[benwaggoner]

Interesting insight, Benwaggoner

I always thought the streamer services care only about bitrate and framesize (which are set by the billing system they use: premium, normal, free etc) and leave the quality decision to the signal processors (including upscaling if needed, yes I have heard stories).
Well, services vary quite a bit in how much they focus on imaging and human visual perception science versus blind belief in one objective metric or another. And there's a huge divide between paid premium services and free user-generated content services with very different economics and goals.

The rise of studio vertical services is a big new wrinkle, in that they have the rights to upscale etcetera their content. But that doesn't mean they do it well! Check out any old 480i animation on HBO Max (Check out Metalocalypse for some very--remarkable preprocessing).

I'm relatively unique in the premium video space in coming from a neuropsychology background, and my opinions, while universally correct :sly:, often come from very different perspectives than others in the field who approach encoding from more of a software development perspective where there's less room for subjective analysis.