Good lossless compression for VHS -> DV cam -> PC?

[bmartinson]

Hi everyone; first post here. I hope I'm posting in the right forum. I have read the forum rules, skimmed the Guide, and read the relevant portions of the FAQ. I've also read lots of capture information on lots of websites, and I still have a burning question about storage format. I don't mind learning to use new tools (even command-line tools), but have not gone down that road yet.

My Goal

Low-cost lossless (or nearly lossless) archival of VHS for later editing and authoring to DVD. For now, I want to burn to DVD, but as raw data (not as rendered videos). My first priority is to get all the quality I can off of the tapes, and — if possible — I want to do it on the first try (rather than have to play these old tapes repeatedly as I learn from many mistakes!).

My Setup 10-to-15-year-old VHS home movies
Brand-new Sony SLV-D370P DVD-VCR Player, $96 at Best Buy
Not-so-new Sony DV Handicam
Firewire port
Windows Movie Maker
Cyberlink PowerProducer

Related Questions

(1) I've run some tests with Windows Movie Maker and Power Producer, and had acceptable results with both when capturing from DV as well as VHS media. When capturing from DV, I like how WMM provides me with the date/time stamp, but I like how PowerProducer splits the files by scene. Is there a good tool that will do both of these for me?

(2) When capturing from VHS via the DV cam, I know I'm not going to have any datestamps to read, and I would guess that there also will be no way to autodetect cut points between different taping sessions. So what tool would be good to split up the resulting capture by apparent scene change and/or file size?

The Big Question

(3) In what format(s) should I investigate capturing the video? The camera is going to convert analog to DV for me, but saving raw DV files will probably take 6 to 8 DVDs per video (ouch), and it seems like I'd never be able to render back out at that quality anyway, given the source. I'm willing to go through the pain of burning multiple DVDs per VHS tape if that's what it takes to avoid a noticeable loss in A/V information, but it would be nice if I can trim this down by not saving at a higher quality than VHS deserves.

Thanks in advance for any advice, criticism, experiential anecdotes, and/or free pizza

-Bill

[jggimi]

Hello, and welcome to Doom9's forum.

I'm moving this to Doom9's DV forum, it seems more appropriate to your situation, per http://forum.doom9.org/announcement.php?f=10

[bmartinson]

Thanks... I considered posting it here, but since my question was about VHS image quality more than DV capture issues, I made a guess that the capturing forum would be best. Sorry about that.

[communist]

Since you want to edit it later you would need a format that can be edited easily. Your proposed workflow should work - its just very space hungry. I would recommend editing it first then directly authoring DVDs if possible.

Other really lossless codecs will ask for even more space (DV is lossy). Alternatives would be MJPEG (would give you 4:2:2 and adjustable frame sizes / compression amount) and XviD in combination with very high bitrate matrix. I cant seem to find that thread/website with screenshot comparison about Sharktooth's UHR/EHR which provide 'near lossless' quality.

[bmartinson]

I would recommend editing it first then directly authoring DVDs if possible.
Thanks for your reply. What you suggest would be ideal, but since I will need to draw from multiple tapes to author a single DVD, space-wise it's not practical for me to capture multiple tapes to hard drive, edit them, and then render. Also, I'm thinking this would require me to play these old tapes multiple times, which would make me worry about degradation.

Other really lossless codecs will ask for even more space (DV is lossy).
How lossy is DV? (I knew it couldn't actually improve my video, but I didn't expect a substantial loss in quality.)

Also, to capture in a format other than DV would pretty much require a capture card, right? What would be a reasonable price range for a decent card that would give me a significant quality increase over DV?

[bb]

DV is lossy, yes - compressed 5:1, which is not much. The quality is excellent for consumer purposes, more than you need for your VHS source. Keeping the workflow in DV format is a good idea, because most video editors are designed for DV in the first place.

I like Ulead MediaStudio Pro; you can capture the video, then you can split by scene (DV timestamp or optical) without actually splitting the AVI file.

There are several threads about the archiving topic. I think in your case storage on miniDV tapes would be a good solution.

bb

[communist]

Uups should have worded that properly - edit the captured footage not the tapes.. editing the tapes on VHS decks would indeed be a very bad idea ;)
In other words I wouldn archive DV material on DVDs (you will need a lot of them) and as far as reliability is concerned - (miniDV) tapes will last longer than DVDRs.

[2Bdecided]

Have you seen the price of hard disk drives decently?

~$100 buys you 400GB, which is ~36 hours of DV. How much do you need "on line" at once?

The other option is to burn ~8Mbps MPEG-2 DVD-Rs first (i.e. normal DVDs) - or just get a stand-alone DVD recorder and record all your tapes onto DVD discs. These DVD-Rs are then your unedited archive. Rip them into your PC to edit and re-author onto DVD for the finished product. This introduces an extra initial MPEG-2 encoding step, but it's probably OK for VHS - especially if you stay at max bitrate, which gives you an hour per single layer DVD.

Other people hate this approach, but I like its simplicity. You can losslessly crash-edit with something like VideoReDo and keep all quality. Or you can go to town and edit properly with fades and restoration, but that means going MPEG-2>edit format (DV)>MPEG-2. It's not that much worse than DV>edit format (DV)>MPEG-2.

Hope this helps, or at least gives you some ideas.

Cheers,
David.

[bmartinson]

Thanks for your comments everyone; some very good suggestions and useful information here ... which of course leads to new questions! :)
DV is lossy, yes - compressed 5:1, which is not much. The quality is excellent for consumer purposes, more than you need for your VHS source.
So if it's lossy but more than I need, it sounds like my setup is sub-optimal on both ends: I'm losing picture information as the analog goes through the camcorder, then I'm taking up more storage space than necessary when archiving because DV isn't highly compressed.

Am I getting that right?

If so, then it seems I need a better capture method that's less lossy (VHS is already bad enough) and more compressed. As it turns out, I've also been looking at TV tuner/analog capture cards. I bought an ATI All-In-Wonder 2006 (http://www.circuitcity.com/ssm/ATI-All-In-Wonder-2006-Edition-Video-Card-100-714145CCS/sem/rpsm/oid/127966/catOid/-13011/rpem/ccd/productDetail.do) for a great price (under $100), but it requires an AGP slot and I just discovered that my ThinkCentre S50 doesn't have one. :( I'm now looking at the $60 ATI TV Wonder Pro TV Tuner/DVR/Video Card (http://www.circuitcity.com/ssm/ATI-TV-Wonder-Pro-TV-Tuner-DVR-Video-Card-100703138/sem/rpsm/oid/154211/catOid/-13011/rpem/ccd/productDetail.do) as another inexpensive solution.

Since this board captures in MPEG-2 format, would this be less lossy overall than capturing DV via my camcorder?

... as far as reliability is concerned - (miniDV) tapes will last longer than DVDRs.
Hmm, I may have unintentionally misled you in my first post. My Sony Handycam is Digital8, not miniDV. And, since I can't get Digital8 tapes any more, I have to use Hi8 tapes, which the sales clerks tell me are "just as good" but shorter in duration (60 min. instead of 90).

Does your comparison still apply? In other words, will Hi8 tapes with Digital8 content really outlast DVD+-Rs?

The other option is to burn ~8Mbps MPEG-2 DVD-Rs first (i.e. normal DVDs) ... These DVD-Rs are then your unedited archive ... an hour per single layer DVD.
I was wondering about that. Two DVD+Rs per 120-minute tape seems very reasonable and workable to me.

Notice I'm using DVD+R (got a lot of 'em) rather than DVD-R. My burner will handle both, and the same drive that burns 'em would be ripping 'em back in. Any reason I should switch to DVD-Rs?

Other people hate this approach, but I like its simplicity. You can losslessly crash-edit with something like VideoReDo and keep all quality. Or you can go to town and edit properly with fades and restoration, but that means going MPEG-2>edit format (DV)>MPEG-2. It's not that much worse than DV>edit format (DV)>MPEG-2.

Why do other people hate the approach?
When you say "losslessly crash-edit", I assume you mean ripping my "raw" MPEG-2 back in so I can edit and author back out as finished DVDs. No problem there. But how lossy is this new proposed initial capture method?
When you say "not much worse" are you talking about effort or quality?

And a final summary question that may help me make some of these other decisions:

Ignoring cost and effort, would I get a better-quality archive by connecting VCR to Handycam (composite cable w/audio joiner adapter) and recording on Hi8 media, or by connecting VCR to capture board (perhaps w/ component video cables) and saving MPEG-2s on the hard drive?

Thanks again!

[bb]

[...]So if it's lossy but more than I need, it sounds like my setup is sub-optimal on both ends: I'm losing picture information as the analog goes through the camcorder, then I'm taking up more storage space than necessary when archiving because DV isn't highly compressed.

Am I getting that right?

I'm afraid not :)
VHS is really a crappy format, and unfortunately the source decides about the resulting quality in the first place. The amount of quality you lose when converting to DV is negligible. You'd lose more when archiving to higher compressed formats like MPEG-2 or MPEG-4. The main plus for DV is that it is a format dedicated for editing, which the MPEG variants are not. That means, DV is a keyframes-only format, whereas MPEG is GOP-based (GOP = group of pictures). Due to the delta frames in MPEG GOPs editing is a hassle, and more frames must be re-encoded at each point where you have edited the video. Re-compression of highly compressed formats quickly leads to visible artefacts, whereas DV is stable across multiple generations of edits.

If so, then it seems I need a better capture method that's less lossy (VHS is already bad enough) and more compressed. As it turns out, I've also been looking at TV tuner/analog capture cards. I bought an ATI All-In-Wonder 2006 (http://www.circuitcity.com/ssm/ATI-All-In-Wonder-2006-Edition-Video-Card-100-714145CCS/sem/rpsm/oid/127966/catOid/-13011/rpem/ccd/productDetail.do) for a great price (under $100), but it requires an AGP slot and I just discovered that my ThinkCentre S50 doesn't have one. :( I'm now looking at the $60 ATI TV Wonder Pro TV Tuner/DVR/Video Card (http://www.circuitcity.com/ssm/ATI-TV-Wonder-Pro-TV-Tuner-DVR-Video-Card-100703138/sem/rpsm/oid/154211/catOid/-13011/rpem/ccd/productDetail.do) as another inexpensive solution.

Since this board captures in MPEG-2 format, would this be less lossy overall than capturing DV via my camcorder?

You may achieve better quality than DV when capturing to lossless formats (e.g. using the Huffyuv codec) or less-lossy formats (e.g. MJPEG at high quality settings). In that case you need even more hdd space. If you capture to MPEG-2, either hardware- or software-based, you'll end up in worse quality, and editing is more painful (see above). That may still be a good compromise for you, but you may consider to buy an hdd dvd recorder as well, which would save you from all the analog capturing adventure.

[...]My Sony Handycam is Digital8, not miniDV. And, since I can't get Digital8 tapes any more, I have to use Hi8 tapes, which the sales clerks tell me are "just as good" but shorter in duration (60 min. instead of 90).

Does your comparison still apply? In other words, will Hi8 tapes with Digital8 content really outlast DVD+-Rs?

I assume that Hi8 tapes with DV on it are just as good as miniDV tapes, probably even better (simply because they are larger).

[...] Why do other people hate the approach?
Other people probably hate this approach because of the re-encoding...
[...]And a final summary question that may help me make some of these other decisions:

Ignoring cost and effort, would I get a better-quality archive by connecting VCR to Handycam (composite cable w/audio joiner adapter) and recording on Hi8 media, or by connecting VCR to capture board (perhaps w/ component video cables) and saving MPEG-2s on the hard drive?

Thanks again!
The DV way you will get the better-quality archive.

bb

[bmartinson]

The amount of quality you lose when converting to DV is negligible.
Okay, that's what I needed to hear. I knew that editing from MPEG-2 would be a bear, and I have been considering it an end-result format only. (As a programmer, I've been thinking of DV as my source code and MPEG-2 as the compiled executable. [Better, perhaps: VHS is original source in an obsolete language; DV is secondary source after machine translation to a current language.] You can sometimes patch or decompile an executable if you really need to, but it's orders of magnitude better to retain the source code!)

I had just been starting to worry that the VHS-to-DV conversion might somehow be even worse, or might be so comparable in image degradation that the extra time and storage requirements might not have been gaining me much. But it sounds like I should stick close to the original approach.

You may achieve better quality than DV when capturing to... Huffyuv... or MJPEG.... In that case you need even more hdd space.
After some quick research, it looks like most of the PCI boards I've been considering will capture only to MPEG-* formats, and I don't see any of them advertising support for external codecs. On top of that, one of the helpdesk guys here at work just told me that, in his experience, going through a capture card always produced somewhat lower quality than going through the DV camcorder.

In fact, he recommended using the camcorder as a pass-through converter and saving the .AVI on the PC, then back-recording onto the Digital8 in a separate step. I would have assumed that the camcorder would do a better job of real-time capture than the PC. On the other hand, if I park my data all on the HDD then I have a little freedom to choose where my breakpoint will be (1 VHS T120 = 2 Hi8 tapes). IIRC, Windows Movie Maker will easily record my .AVIs back out to the camera.

Is there anything else I should consider when choosing a 1- or 2-step VHS-to-Digital8 dub?

So I still want a TV tuner card in order to have "poor man's Tivo", but it sounds like I should focus on picture quality & DVR features and not worry too much about how well it captures from VHS. It sounds like the device with the most opportunity to wreck my quality will be the VCR itself (or maybe the cables). Ooh, that reminds me that I forgot to ask my other n00b question:

Camcorder has one audio input, VCR has two audio outputs. I had been running the L & R cables through my little "merge" adapter, but since the Handycam can only record mono, and the VHS source is only mono, couldn't I just bag the adapter and run the white audio cable straight into the Handycam, thus eliminating one of the connections and perhaps getting slightly better audio?

[bb]

[...]
After some quick research, it looks like most of the PCI boards I've been considering will capture only to MPEG-* formats, and I don't see any of them advertising support for external codecs. On top of that, one of the helpdesk guys here at work just told me that, in his experience, going through a capture card always produced somewhat lower quality than going through the DV camcorder.
There are MPEG-2 capture cards, and DV boxes as well, e.g. the Canopus ADVC models. Their quality differs not only regarding the codecs and software additions, but also whether they have a TBC (= time base corrector) or not, which is important for crappy sources like VHS. Capture cards / TV cards without hardware codecs use software codecs together with their capture software; often you can choose from your installed codecs. You can use VirtualDub or VirtualVCR as an alternative to the capture software provided.
DV camcorders often have better electronics inside than cheap PCI capture / TV cards, so I second your helpdesk guy's opinion.
In fact, he recommended using the camcorder as a pass-through converter and saving the .AVI on the PC, then back-recording onto the Digital8 in a separate step. I would have assumed that the camcorder would do a better job of real-time capture than the PC. On the other hand, if I park my data all on the HDD then I have a little freedom to choose where my breakpoint will be (1 VHS T120 = 2 Hi8 tapes). IIRC, Windows Movie Maker will easily record my .AVIs back out to the camera.
There should be absolutely no difference in quality whether you record to tape or pass-through to the PC. Of course the tape is more limited in terms of playtime.

Is there anything else I should consider when choosing a 1- or 2-step VHS-to-Digital8 dub?

Make sure your camcorder has DV-in, so that you can record back to tape once you have your footage on the PC. Not every camcorder has that option, e.g. because of stupid European customs duties...

Camcorder has one audio input, VCR has two audio outputs. I had been running the L & R cables through my little "merge" adapter, but since the Handycam can only record mono, and the VHS source is only mono, couldn't I just bag the adapter and run the white audio cable straight into the Handycam, thus eliminating one of the connections and perhaps getting slightly better audio?

Yes, because your source is monaural, there should be the same signal on both stereo plugs. You can skip your "merge" adapter and connect one plug only (no matter which one you choose, red or white). For stereo sources, you should superimpose the two channels, though, else you'd lose audio information.

bb

[Blue_MiSfit]

I've heard that one way to really improve quality with repeatable analog captures is to capture multiple times, then sync up the captures so they are exactly the same frames. finally ,use mergeluma and mergechroma to average out the information. I've never done it, just hear its helpful.

~MiSfit

[2Bdecided]

bmartinson,

I have one very good piece of advice that I'm sure you'll follow: make sure you try it for yourself.

Unless you drop seriously below 8Mbps, I doubt any of the problems you see will be down to the codec.

What you will find is...

1. some capture cards don't work very well with VHS (no TBC, as bb has said)
2. whatever you do, the VHS isn't going to become DVD quality
3. (bizarrely) if your source improved over the years (new+better camcorder?) this is visible even via VHS
4. If you get into "restoring" and "improving" the picture, it will take you weeks to do anything!

Cheers,
David.

[bmartinson]

Again, thanks everyone for the excellent advice. Here's what I've settled upon for archival:

VCR --(Composite)--> Handicam --(FireWire)--> PC
Convert/capture VHS T120 as DV (.avi)
Save entire T120 onto hard drive in a single operation
Choose good breakpoint close to 60:00; split into two DV .avi's and record back out to 2 Hi8 tapes; flip their write-protect sliders
Possibly consider also burning raw data to 6-8 DVDs per T120 (unless I can keep my OCD under control!)

I'm first going to try Ulead as my capturer to see if it can still save different scenes as individual files. I don't know whether it uses digital or visual cues for its detection; I'm assuming the former, but it's worth a try.

If Ulead can't do it, what other applications (peri- or post-capture, free or cheap!) should I look at that could detect scene jumps and/or fade-to-blacks and write out separate .avi files for me?

As for the TV Tuner, I'm liking the ATI TV Wonder Elite ($78 at Amazon) because of its comb filter, advanced noise reduction, remote, and excellent customer reviews regarding picture quality. Some customers reported getting better quality than the original TV broadcast (at the cost of slightly more "rounded" images). Not that this has anything to do with my VHS capture project, but after all the research I really want a TV tuner now. ;)

Thanks again.

[bmartinson]

I've heard that one way to really improve quality with repeatable analog captures is to capture multiple times, then sync up the captures so they are exactly the same frames. finally ,use mergeluma and mergechroma to average out the information.
Intriguing. When converting from an interlaced source, is it possible that some captures would get the even frames and others would get the odds? Or does the process merge even/odd pairs into a single frame such that each capture gets the exact same frames as the other captures, just with different variations of artifacting (and probably different starting frames, depending on exactly when I hit the PLAY and CAPTURE buttons)?

My guess is that this process would, overall, "dumb down" or "round out" or "soften" the image, but also reduce noticeable artifacting, which would seem like a good trade. Not unlike filtering a camera to give the actress smoother skin. And probably not unlike the effect achieved by the ATI Elite board I mentioned earlier, I suppose.

[Blue_MiSfit]

If you're just going to store to DV (or Digital 8) tapes, then why bother putting it in the computer in the first place? If your camera can 'play through' digitized video to firewire (dv), then it should be able to record that dv stream to tape. Should be the same quality than the procedure you list here, but a lot shorter :)

[jmac698]

I see that you are a bit new to this, I have gone the same route in the last few years.
You can look at
http://www.doom9.org/index.html?/capture/start.html
which may be overwhelming at the moment. However, I can say that I prefer TV tuner approach because of increased flexibility. 1) You can choose which resolution to capture in 2) you can use any codec you like to store your video 3) you can calibrate your card precisely 4) You can make full use of advanced features of the TV card (the bt8x8 tweaker option in virtualdub, dscaler) 5) You get to watch TV on your PC now, with Dscaler :)

Even though it gives best quality, it may be too much work for a beginner.

Multiple captures: you can use the average filter
http://forum.doom9.org/showthread.php?t=100626&highlight=multiple+captures

If you have dropouts or particular noise in one of your captures, you can automatically replace that spot with a good spot from another capture:
http://forum.doom9.org/showthread.php?t=78187&highlight=multiple+vhs+captures

The original somewhat famous thread about recording the same vhs tape many times to improve quality (short version, he claims it works)
http://forum.doom9.org/showthread.php?s=&threadid=42733

As for using a TBC, this can add to your problems actually, over a long time period the TBC may have to drop a field because the VHS is too far out of sync. When this happens you have more trouble lining up two copies of the same tape.
There is also a software TBC.

So in summary if you want super quality you can get pretty deep into it, but really I think your fears of quality are overrated. Let me say that, the differences between cards or codecs (at the quality preset settings) are very little compared to a) what you can improve later with some avisynth filters b) the relative quality of the VHS.
I felt the same draw to ultimate quality and fear of this factor due to ignorance. In the end I don't think it's worth it, I only try to improve quality as a fun puzzle, no more.
I think it's only a psychological effect to worry about it and to seek reassurance, you cannot judge how little it matters until you know better :)

The thing to remember is that averaging by itself won't remove all the noise from a VHS source. As stated by many, it can only really reduces the noise introduced by the capture chain and gets you closer to the recording from which you're capturing.

But consider this. I now regard averaging as an essential part of the process. It serves to reduce the noise threshold, so that the conventional noise filters can be used with much lighter settings than you would otherwise be able to, thus enabling you to retain much more detail. In other words, the noise filters still do the main work, but the averaging makes it much easier for them.

[mikewillnot]

stale thread, perhaps, but my experiments in this area turned up one sticky problem I haven't found a reasonable solution to: the original VHS tapes being macrovision encrypted. My camcorder has macrovision blocking built-in, and won't convert such a source to DV output. The capture card solutions I played with, back when I was working on this kind of problem, didn't work well either, and were a whole lot of trouble. Any recent developments I'm not aware of would be of great interest. I'm not interested in stealing their bloody work, after all; just preserving my initial purchase.

[jmac698]

Could you post a clip ? I've recorded macrovision, actually it wasn't any type of movie at all but a bug in a device which left it turned on when I was only looking at a menu screen. It can be easily removed, there were only yellow/purple tints on each pair of lines.

[bb]

The easiest way to remove VHS Macrovision is to use a hardware box, a so-called "video enhancer" or "video stabiliser". You'll find lots of them by googleing for "macrovision remover" or "macrovision removing device". Whether they are legal or not depends on the country you live in.

bb

[mikewillnot]

I don't have any clips handy (though I still have lots of protected tapes, on tape); and when I said "reasonable" I meant that given my situation at the time, wouldn 't cost money for hardware. The best HW solutions I could locate were in the $100-200 range.

[jmac698]

Have you seen
http://forum.doom9.org/showthread.php?t=110124

?

[mikewillnot]

No. Interesting.

I found it less trouble to just get a netflix account, rent DVD's of the tapes I already owned, and make "movie only" backup copies. I personally see this as appropriate given the fact that I already own and paid for the copyrighted movie, and in many cases have already rented the tape prior to buying it to boot, and probably saw the movie in the theater as well, and given that I wouldn't have to do it this way if they didn't use that stupid macrovision thing -- I could and would just make a backup copy of my tape if it wasn't such a pain in the neck.

I also think it's really dumb that the industry hasn't instituted some kind of "upgrade" or trade-in path, so that if I already own a commercial tape of "toy story," for example, I could buy the DVD at a healthy discount. Technology changes are a constant, and the step up in quality would be worth something -- just not the full-boat retail price all over again. They've shot themselves in the foot, IMHO.

Sorry for the rant.

[GrofLuigi]

I also think it's really dumb that the industry hasn't instituted some kind of "upgrade" or trade-in path, so that if I already own a commercial tape of "toy story," for example, I could buy the DVD at a healthy discount.
They wouldn't be an "industry" if they do that, would they?
They know what they do, they're after your money. :)

GL