Super-resolution

[4evrplan]

Has anyone written a script or a plugin that does super-resolution by combining details in neighboring frames, similar to the way video enhancer works? I figure it'd go something like:

1) double the width and height using a high quality interpolator (i.e. lanczos3 or nnedi2).

2) do motion compensation using - oh IDK - maybe mvtools - or whatever the latest & greatest is

3) combine details from neighboring frames to increase quality - this is the part that completely eludes me, but It'd sorta be the opposite of Degrain - anyone know how to do it?

4) repeat 1-3 as necessary

5) final downsampling to target size (bicubic?)

I'm pretty sure I can figure out everything but step 3. Anyone out there know the math behind this or an intuitive answer of how it could be done?

Also, while I'm at it, what would be the best (least lossy) colorspace for all of this?

[Didée]

If you search the forum on the term superresolution, you'll find lots of opinions that on real-world sources it's mostly a no-go.

In 99% of all practical cases, spline36resize (or pretty much any other resizer) is almost as good as video enhancer (since in fact it doesn't do very much, except for munching CPU time) ... and nnedi2_rpow2 most probably is ahead of video enhancer.

For more of a "w00t"-effect, you can first do nnedi2_rpow2, then use something from here (http://forum.doom9.org/showthread.php?t=153170) on that upscale.

[4evrplan]

Awesome, thanks D. I may code this up just for fun, even if I never use it for real. Any idea how this would compare to the algorithms used by hardware-based upscalers?

[Didée]

Any idea how this would compare to the algorithms used by hardware-based upscalers?
You provide the used algorithm - or even better, you provide a stream that has been processed by such hardware, along with the original input - and we will tell how it compares.

The crowd being impressed means nothing. If hardware would provide a switch [x] "magical image improvement", the crowd would get OOOH and AAAH, the image gets depth, and richness, and sharpness, and vibrance, and whatnotelse .... and internally the hardware just does "set_contrast=1.1". This is not a joke, but a realistic scenario.

Or, give the crowd two pairs of HiFi stereo boxes to compare. Use two absolutely identical pairs, but set one pair a tiny little bit louder than the other. The crowd will find that one pair sounds definetly superior, and the other pair is definetly worse.
(A test similar to this actually has been done, with the audiance being HiFi enthusiasts and experts, and the result was as described.)

Without something that can be examined closely, it's nothing but blablah and hot air. The crowd being impressed means nothing.

[4evrplan]

^ I don't doubt it, but I've got to say, it seems like a bit of a sore spot. ;)

I was really more curious about the algorithms themselves versus the result. White box versus black box, but I'm guessing anyone in the industry who actually knows isn't talking.

EDIT: Changed "approach" to "result". I meant "result", so I don't know why I said "approach".

[Didée]

In all those years, I've not come across one single example of what "hardware" could do. Not even ONE. Sure it's not the most easy thing to grab the result of a hardware processor - but it isn't impossible either. Given that someone has the hardware, and given that s/he has the interest, it is not-at-all an impossible task.

Since not even ONE example has come up in all those years, the possibilities are

a) no-one has such hardware

b) no-one really cares

or (my favorite)

c) there is fear that closer inspection might reveal that the mighty hardware infact isn't that mighty. Mysteries are only interesting if there's something mysterious in it. Mystery revealed, fun gone.

[4evrplan]

Does anyone on this forum work in the video hardware industry (i.e. systems software engineer for a DVD player manufacturer), and if so, can you give us any clues as to what the hardware is actually doing, or is it a trade secret? For example, is the upscaling done w. iNEDI or some other advanced algorithm, or does it use a much simpler/less CPU intensive interpolation technique? What other filtering, if any?

And for you hackers, have any of you ever dumped the firmware from one of these devices and picked it apart?

None of this makes a hoot of real world difference to me in any way, but I sure am curious.

[Guest]

None of this makes a hoot of real world difference to me in any way, but I sure am curious. Perhaps but it's now OT for this thread, so closing.