General Opinion on Upconverting to HD?

[Danux]

So I've done a bit of googling (& aimless clicking) for the last few hours, and haven't really found a lot of opinion on upconversion from "lesser" resolutions (VHS tape, DVD-5/-9) to, say, 720p/1080p.
I thought I'd take a stab at upsizing a few flicks, but am kinda wondering ATM if I should bother. Are most people just waiting for their favourite movie to be released in high-def?
Any links to upconversion discussions or guides would also be appreciated.

[Rockas]

Are most people just waiting for their favourite movie to be released in high-def?
I guess so :)
Everytime you do a resize (up/down) you'll need to re-encode... there is no re-encode without quality loss... this means that, answering your question, making it bigger won't make it better :D

[siddharthagandhi]

You could probably upscale, just increase the bitrate to make up for this. Its not true HD, but if you need to do it then do it.

[Rockas]

You could probably upscale, just increase the bitrate to make up for this. Its not true HD, but if you need to do it then do it.
... and it won't look any better (probably a lot worst) :D

[foxyshadis]

The only way it'd look better is if you spent some time to properly upsize and sharpen, and work harder to bring out the tiny details in the source.

But c'mon, faking HD with VHS? What. You barely have enough detail in that to make a DVD, let alone HD.

[Danux]

The only way it'd look better is if you spent some time to properly upsize and sharpen, and work harder to bring out the tiny details in the source.

But c'mon, faking HD with VHS? What. You barely have enough detail in that to make a DVD, let alone HD.

<grin> This is, more-or-less, what I had concluded as well.
For faking HD from VHS, my notion is that it would be better to throw a whole bunch of compute cycles at the problem and bring it up to HD rez through software, rather than have the monitor/player do a realtime hardware upconvert.
In a more general application of the process, purity of the original image would not be a concern for me, as long as the perception of a high-fidelity image could be displayed.
For instance, let's say a film has a poppy a the end of a green field, which is noticable in its 70mm glory at the theatre. In the DVD transfer, this poppy is lost because there isn't enough resolution to render it, but in highdef it *can* be displayed. This is a detail that retains the purity of the original, but is it a necessary detail?
If, like you said, through various software filters the important elements of the DVD image can be re-rendered to an upsized image, is the viewing experience really going to be *that* different? More pointedly, if, through these various filters, new pixels are added to the image, which don't add to the purity of the image, but still add to the perception of highdef, then it seems to me it might be worth it? If the director re-shot the scene with a camera filter which caused the image to be rendered in the manner your software filters rendered it, it would qualify as "real" high def.
I suppose I started down this road because I have recently pulled up a few fave VHS tapes that never made the cut to DVD, and I kinda anticipate the same will happen in the transition from DVD to HD.

[foxyshadis]

You'll need specialized software to make the leap from vhs to HD, without it looking obviously resized. So specialized that I only know of one program in the world that can do it: SAR (http://www.general-cathexis.com/). Its backprojected DDL and deconvolution methods are superior to anything else, but so slow. And unfortunately, it has no command-line interface, or I'd write an avisynth wrapper for anyone else who wanted to pay for SAR.

In the meantime, EEDI2 is avisynth's most powerful resize, though it tends to artifact up a bit after several iterations, and there are several command-line implementations of high-end resizers between eedi and SAR. (All of them but SAR have a bit of an oil-painting look at 4x, though it's nicer than smudgy and ringy of 4x lanczos; if your 70's vhs already looks oily thanks to age and film stock, the effective resolution might be half or less its actual resolution.) You can throw a little fake detail in, not quite noise and not quite texture, to fool the eye once it's larger too.

Can you tell this is one of those areas I'm very interested in? >.>

[Danux]

Can you tell this is one of those areas I'm very interested in? >.>

<nod>, I suspect you've already been around this block a few times. Thanks for the link to the image interpolation, that'll keep me occupied for a while. There are a couple of nice algorithms in there, the Jensen- / Zhoa-Xin-Li interpolations looked particularly good, especially when you pull them down to about 2X. I wonder how they'd look "in motion", as part of a movie stream.
4X upsize is a tough pull, I ran a few tests against the processes I'm experimenting with, they don't do well at 4X. But then, I'm aiming a tad lower; looking to pull a width of 720 to 1280, say 1.8X magnification, 480p to 720p. 1.8X seems like a qualitatively attainable value.
SAR is out of the question for me. My experiments are pretty much locked into Imagemagick or The GIMP right now; let's me distribute the work across multiple Linux boxes at the command line. I couldn't image trying to do this kind of work on one computer.
Up to now, I've been pulling the image up to a large resolution using, say, Hanning/Blackman interpolation, applying a/some filter(s) at that resolution, then pulling back down to 720p using Blackman. Not rocket science, but it keeps my computers busy <grin>. My best image has come by sharpening an image about 40 different ways, then averaging all 40 results into a single final image. 10+ minutes per frame though. Ouch.

[foxyshadis]

Jensen and Xin-Li are available elsewhere, fortunately, if more slowly, so you could use an avisynth plugin (forget what it's called, you're probably already using it) to shell out to a script to somehow load-balance it. eedi2 has a lot of xin-li method incorporated, though not in an optimal way for general resizing. I'd like to see the results of your hurculean sharpener sometime.

It is too bad that all the best image processing filters aren't nearly as optimized as many avisynth plugins!

[Danux]

I'd like to see the results of your hurculean sharpener sometime.

Sorry, I didn't save any of it, I haven't really been too scientific about my approach, y'know? Mess around, see what looks good as a still, run as a movie sequence, try again.
I ran the clown.png up 4X using my Averaging script, it doesn't really compare well against most of the images on that webpage you linked to previously, lots of jagged edges. Don't know what the upper limit on magnification is, but I'll bet it is below 2X.
I tried averaging image #7 Zhao-Xin-Li with #15 Lad Deconvolution, it looks pretty good, a good portion of the haloeing was eliminated from #7, a bit more detail was added than in #15. I think there's room to manouvre with averaging, just a question of finding a good mix of algorithms to weigh against each other. I may have to fire up the GIMP again, perhaps try a bit of weighted averaging (which I don't think I can do in Imagemagick yet).

[Hellworm]

I don't know much about upconverting, but isn't there a way to use the temporal aspect of video to get more information which can be used for magnifying?
If there are multiple pictures of a moving surface, there will be the same pixel raster in different sub-pixel positions of the original surface. If its possible to recognice the movement with sub-pixel accuracy , it should be possible to extract a higher resolution Image.
Just an Idea.

[Danux]

I don't know much about upconverting, but isn't there a way to use the temporal aspect of video to get more information which can be used for magnifying?
If there are multiple pictures of a moving surface, there will be the same pixel raster in different sub-pixel positions of the original surface. If its possible to recognice the movement with sub-pixel accuracy , it should be possible to extract a higher resolution Image.
Just an Idea.

Yeah, that does make sense.
From the technical side of things, I personally wouldn't know where to begin. I suppose you could take the data from the first pass of an 2-pass encode, use that to identify the scene changes. You could probably even use the XviD motion detection algorithm somehow, to identify consistent portions of the screen, and then use that to feed the upsize algorithm. That would only cover portions of the image though, and probably not the important parts (character's face as s/he speaks & nods, for instance).
Ultimately, I think it would be a lot of manual work, and really, that kind of detail is not such a big deal (to me). My biggest concern is that the final sequence has the appearance of clean high-def image. Purity of the film image (not the DVD/VHS image) is not such a big deal.
Interesting idea though, if someone was looking to "restore" a movie up to highdef.

[niknik]

If there are multiple pictures of a moving surface, there will be the same pixel raster in different sub-pixel positions of the original surface. If its possible to recognice the movement with sub-pixel accuracy , it should be possible to extract a higher resolution Image.
Just an Idea.

Not just an idea, it's already being done.
On the "low end", there are programs for smartphones that take higher resolution pictures by taking a several shots and relying on you to move slightly the phone for each frame.

(The only site I could find now was a tech paper: http://www.actapress.com/PaperInfo.aspx?PaperID=22853)

I remembered seing a demo video (think it was from Intel - showing up their new CPUs) some time ago. Where they showed a shaky video of a warning sign you couldn't read (low res). And then by applying those algorithms, you got a clearly readable sign. At the time, each "frame" took a couple of minutes to render - they hope someday it will be fast enough for you to get "HDTV" content out of lower resolution sources in real time.

If that's applied to a 1920x1080p source, you could get a real nice quality. :) (By that time, I hope a 4k projector is affordable ehehe :)

[Terka]

Danux, can you post your results?

[CWR03]

After almost a year I doubt he's still tracking this thread.

[Doobie]

Danux, can you post your results?

Don't bother. It's insane to encode at higher resolutions than the source. Such video enhancement should be done during playback.

[zambelli]

I think the compression part of upconversion should not be understimated. All that sharpening and bringing out of details will be wasted if you don't encode it well. Keep in mind that professional HD-DVD/BluRay movies typically require multiple encoding passes and go through a lot of tedious quality checking. Unless you're willing to do the same, I think you should stick with Doobie's advice and perform any upcoversion enhancement dynamically on playback. A modern dual-core machine can easily do at least LimitedSharpenFaster w/ resizing on a 720x480 source.

[shevegen]

"Are most people just waiting for their favourite movie to be released in high-def?"

Well not me. I got them on DVD, i found out the hard way that DVDs arent so good as praised, and I wont be fooled again into buying ANYTHING from high-def. No really, I dont need it too. And I am sure I wont need it in the future. I'll be waiting for the new generation without the DRM lockin :)