An informal resizer competition

[*.mp4 guy]

Source image (http://img177.imageshack.us/img177/1215/sourcekj7.png) (I will supply the original high resolution image later)

The source image was a 5K*7K jpeg, downsized, then converted to yv12, then compressed to png.

Rules:
-no processing of the image prior to resizing is allowed (unless pre processing is part of the resizing algorithm).
-1440*2048 (a 2X increase in each image dimension) PNG is the only accepted image format.
-if a resizer introduces a (sub)pixel shift, it must be corrected.
-The used interpolator must be specified
-If an interpolator has parameters, the parameters you used must be listed.
-in the case that additional steps (ie 3X upsize, then downsize to 2X) were used, they must be specified.
-There are no restrictions on which interpolators or programs should be used, beyond those which may result form the previous guidelines.

This is an informal competition, just to see what people are using, and what interesting algorithms are out there, so everyone will be their own judge as to which interpolations are best.

Unfortunately I don't have acces to any HD content at the moment, so I could not find suitably high quality sources that would allow for temporal super resolution to be used.

[Dark Shikari]

What resizer is the best will depend on what algorithm you used to downsize it. :p

[*.mp4 guy]

Not really, all good downsizers are essentially almost exactly the same, bicubic, spline, lanczos, and even bilinear would give very similar (essentially identicle for spline, lanczos, and a myriad of others) results. You are, however correct that the source image has a large impact on what resizing algorithms do well in a given comparison, In this case I choose an image that I beleive will be very dificult to resize well with any algorithm, due to its combination of subtle detail, and high contrast edges.

[niknik]

I can send you some HD samples if you want.

Though for now, what I'd like to see is some way to use that "temporal upscaling" to improve regular DVD resolution into something closer to HD resolution.

Are there any freely available programs out there that can do that?
(The only thing I hear/saw about it was a tech demo a long time ago, and it took something like 1 minute per frame or so - but with amazing results! Unreadable text would become perfectly legible and so on)

[Dark Shikari]

I can send you some HD samples if you want.

Though for now, what I'd like to see is some way to use that "temporal upscaling" to improve regular DVD resolution into something closer to HD resolution.

Are there any freely available programs out there that can do that?
(The only thing I hear/saw about it was a tech demo a long time ago, and it took something like 1 minute per frame or so - but with amazing results! Unreadable text would become perfectly legible and so on)
Sounds like something that used motion-compensated temporal superresolution.

[niknik]

Sounds like something that used motion-compensated temporal superresolution.

Yes, but is there any way to achieve that using freely available tools?

I wouldn't mind waiting a couple of days, to create HiDef versions of my DVDs.
(though, I just made some quick calculations: if it took 1 minute per "frame", a movie could take over 100 days to upscale! - Guess that's why Intel is pushing quad cores and experimental 80 core CPU's. :)

Best of all would be having a hardware temporal super-resolution chip in a upscaling DVD player. Just imagine instantly having all your DVDs in "HD"!

[Dark Shikari]

Yes, but is there any way to achieve that using freely available tools?

I wouldn't mind waiting a couple of days, to create HiDef versions of my DVDs.
(though, I just made some quick calculations: if it took 1 minute per "frame", a movie could take over 100 days to upscale! - Guess that's why Intel is pushing quad cores and experimental 80 core CPU's. :)

Best of all would be having a hardware temporal super-resolution chip in a upscaling DVD player. Just imagine instantly having all your DVDs in "HD"!
Yes, you could do it with available tools.

There are some free superresolution filters, so here's what I'd do.

For frame N of the video:
1. Use MVTools and motion-compensate all nearby frames of N (maybe the 12 nearest frames, 6 in each direction) to N.
2. Compare these 12 frames to N. Throw out all that differ enough as to be useless. One could also compare these frames on a block-by-block basis and pick only the blocks that are very similar (to assemble a smaller number of more accurate frames from the large number of frames); this would be very effective.
3. Run the SuperResolution filter on the final group of frames. This will yield you your upscaled frame.

[niknik]

Yes, you could do it with available tools.

There are some free superresolution filters, so here's what I'd do.

For frame N of the video:
1. Use MVTools and motion-compensate all nearby frames of N (maybe the 12 nearest frames, 6 in each direction) to N.
2. Compare these 12 frames to N. Throw out all that differ enough as to be useless. One could also compare these frames on a block-by-block basis and pick only the blocks that are very similar (to assemble a smaller number of more accurate frames from the large number of frames); this would be very effective.
3. Run the SuperResolution filter on the final group of frames. This will yield you your upscaled frame.

That sounds too advanced for me! :confused: :)
In any case, thanks, I'll google for superRes filters and try to figure out how to test it.

[Isochroma]

Qimage produces the highest quality upscales of any software I've yet tested.

[Isochroma]

Here's my scaling gallery comparing Qimage 2007.114 to AVISynth's lanczos4resize() at very large upscale values (like 500x373 to 1600x1194).

http://isochroma.8tt.org/Scale.htm

( Use IE 6.0 ONLY, FireFox doesn't work with image rollovers )

Beyond the actual comparisons, the pages are handwritten very simply and contain an easy-to-use layer flip code with precise pixel compensation.

It means when a comparison page loads, you can click the image as it loads and it flips between the two images. Also, when it does so the scroll position is not reset. It allows very precise partial or full image comparisons and took tons of time & testing to perfect.

Anyone is welcome to use the HTML code to make their own image comparison pages.

mp4guy & others: link me your starting images, and i'll make Qimage versions at the specified size (big upscales really highlight the differences in scalers, so they are preferred) and whatever other scalers you might prefer or have already, then put them in the gallery.

Finally, I have tested Qimage versions 2007.114 and 2007.141. 141 has blocky line upscaling using Hybrid mode (which I use), 114 doesn't. So I stick with 114, which was used to make the gallery images. There are even newer versions; I haven't tested them for this problem yet.

[MfA]

The VQEG ftp server (ftp://vqeg.its.bldrdoc.gov/HDTV/SVT_MultiFormat/) has a couple 16 bit per component 3840x2160p50 test sequences.

PS. the downscaling does matter, if you know it was a box filter (which is a poor unrealistic way of downscaling) even a couple of iterations of a simple forward/backward scheme can make a big difference.

[foxyshadis]

I'm totally unconvinced that anything can do better than refinements of EDI for anime-style images (♥ Belldandy), and Qimage is a pretty good EDI but not the best I've seen. Although past 2x more often I've been loading things into Illustrator's Live Trace (which I found better than the free software vector programs). It's not magic, but at least it's infinitely sharp and adjustable. I might have to try that on the timex later, though I expect interesting but not terribly useful results.

Iso, no wonder it won't work in firefox or anything else, what a strange abuse of z-order layers. For much simpler and much more compatible, try stripping both image divs out and replacing it with:
<script>var img = 1;</script>
<div align="center"><img src="Scale-04_files/04-lz4.jpg" onclick="this.src= (img>0) ? 'Scale-04_files/04-x.jpg' : 'Scale-04_files/04-lz4.jpg'; img=1-img;" border="0" height="1200" width="1600"></div>

I have no idea how absolute positioning without actually specifying a position is even supposed to work. Magic and browser gremlins, I guess.

Mfa, awesome. I don't think my ISP would be happy to download 25 gigs for 500 frames, but I've been looking for something like that to test. =D

[wonkey_monkey]

I have no idea how absolute positioning without actually specifying a position is even supposed to work. Magic and browser gremlins, I guess.

OT: It allows you to place elements inline with preceeding elements, but not affecting the positioning of following elements. I think it's handy for doing drop-down menus and the like, which should follow their links around, but not disturb the rest of the layout.

David

[Isochroma]

Believe me, I tried everything else but it was only that code which worked properly. I'm not a theorist, but a pragmatist.

As for the resizers, let's get some actual comparisons up with known source images.

[*.mp4 guy]

@ MFA it wasn't a box filter, box filters produce artificially hard to interpolate images through improper frequency limiting, which biases the comparison twards interpolators with heavy handed anti-aliasing, like step interpolation. I'm not going to specify the exact downscaling method, because some backprojection techniques depend on apriori knowledge of the downcaler used, which is imo, also unrealistic.

[MfA]

Unless the original was unsharped already I would guess you used something with at least two lobes just from looking at the image. Using a sharpening linear downscaler isn't very realistic either though. Unfortunately with small ratio downscaling it's probably the best you can do (with a larger ratio you could simply use gaussian filtering).

[buletti]

@ Isochroma: Nice comparison site. QImage seems to work much better than Lanczos4Resizer for upscalings with large factors. But would you mind to add a 3rd testing branch using Avisynth's brand new spline resizer implementation, too? Spline64Resize is available since Avisynth 2.5.8 alpha. Would be nice to have a comparison between both inbuild high quality scaling algorithms of avisynth and QImage.

[*.mp4 guy]

All of the test images Isochroma used had a lot of aliasing, I don't know why. Its well understood that NEDI has strong anti-aliasing properties, so it is't surprising it did so well on those images compared to lanczos, on a less aliassed source, lanczos and nedi would perform more similarly. Spline64 looks almost exactly the same as spline36 to me, and spline36 is only very slightly better then lanczos3.