Hey SubJunk,
what kind of motion estimation technique are you currently using? Have you thought about using Optical Flow + Warp, if you're not using that yet? Optical Flow algorithms seem to have improved quite nicely. They're still rather slow (at least the good ones), but I'd guess that they should allow you to improve quality quite a bit. Here are some links to open source Optical Flow algorithms:
http://people.csail.mit.edu/celiu/OpticalFlow/
http://www.onera.fr/dtim-en/gpu-for-image/folkigpu.php
http://sourceforge.net/projects/open...ptical%20Flow/
http://nghiaho.com/?page_id=189
http://plyer.fr/aurelien/?p=124
http://varflow.sourceforge.net/
http://perception.inrialpes.fr/~chari/myweb/Software/
Here's the "official" Optical Flow benchmark site. It seems that there are rather large differences between different Optical Flow algorithms:
http://vision.middlebury.edu/flow/ev...results-i1.php
Interesting approaches would also be to compare motion vectors between consecutive frames and only use those which are comparable. That should nicely sort out erratic motion vectors. Finally, how are you filling the occluded space? If you check motion vectors to the next "future" frame, too, you could eventually fill the occluded space from the future frame. That might help removing many artifacts.
Just shooting from my hip here, haven't really done any motion estimation programming myself yet. Just some brainstorming from what I've read...