Upscale and "frequency doubler" codec/filter?

[niknik]

Hi,

I've always been annoyed with the "low" fps of movies, specially when doing something like horizontal panning.

Wouldn't it be possible to do in software something similar to what Philips and other TV makers do: special processing when doubling to 100Hz, to smooth the motion?

I guess that wouldn't be too hard in a PC.

Even better, it would be to use the motion to do a much better "upscaling". IT could theoretically allow normal def video to approach HD quality (in scenes with movement).
I think intel (or microsoft) already showed up something similar, using a low res camera to shoot a video, and showing the final processed version with amazing results - though it was said it couldn't be done in real time... yet. :)

(This opens huge possibilities, just imagine playing your regular DVD collection, and have a PC or hardware player do that sort of processing and upscaling... It would be fantastic. I don't know why no one has thought of it before)

Anyone knows if there's any codec or mpc filter that would do any of this already?

Thanks.

[foxyshadis]

This thread discusses avisynth methods:
http://forum.doom9.org/showthread.php?t=113256

All are much too slow for realtime, but near-realtime methods make their way into the discussion at times. It's still mostly the realm of specialized ASICs/FPGAs though.

[niknik]

This thread discusses avisynth methods:
http://forum.doom9.org/showthread.php?t=113256

All are much too slow for realtime, but near-realtime methods make their way into the discussion at times. It's still mostly the realm of specialized ASICs/FPGAs though.

Thanks for the thread, I'll be sure to check it.

Well, yes, hardware would be the best way to do it.
However, at least for the frequency doubling interpolation, I think any regular "medium-to-high end" PC should be capable of doing that (at least for standard def/DVD video).

As most codec are based on motion vectors and such, I guess it shouldn't be that hard to interpolate a few more frames at double the frame rate.

I just remembered of bringing this subject up here, because just a couple of days ago, as I was strolling around a store with dozens of different LCD sets showing the same footage, it was impressive that some "flickered" a lot, while a few were able to provide a fluid motion.

[foxyshadis]

I think the fastest you can get through avisynth - which actually does run realtime for me in ffdshow - is something like this:

SetMTMode(2)
mp=FindMotion(iterate = 2, reset = 50, initialise = 60)
m=MotionFPS(framerate*2,warpfast,Motion = mp)
interleave(last,m.selectodd())

Or you can replace warpfast with move to get even faster but blockier results. Either way tends to get really messy when things get exciting, though. If you have a seriously overclocked core 2 machine, then maybe you could run motionprotectedfps and get much nicer looking results.

There's a player out there that claims to do something similar, but when I looked at it, it wasn't particularly pleasant to watch.

[niknik]

Thanks, I'll try that out and check the result.

[foxyshadis]

Oh, I forgot something I was going to mention - using motion vectors from the original encoding would speed it up significantly, but it's been tried before and doesn't work at all. The vectors required to minimize the file size often don't necessarily correspond to the actual motion, especially when things get busy, but even on something as simple as a panning scene. You can see for yourself if you turn on motion vector visualizations in ffdshow - they're all over the place, often overlapping in crazy ways. If they weren't, it'd be a much easier proposition. ^^;