Image stabilization artefacts correctable?

[markanini]

Here's an example:https://www.youtube.com/watch?v=QG-z-EyG9WM
Notice the jello-like artefacts

[wonkey_monkey]

These problems can be reduced, in theory. The linked video seems to suffer from problems which are a combination of a rolling shutter (which can be partially corrected with, for example, Deshaker (http://www.guthspot.se/video/deshaker.htm)) and what I think is the more pronounced problem which occurs because a lot of stabilisation algorithms only use 2d rotations/translations - this results in a jelly effect which is more pronounced in the corners. I did some very rough tests a few years ago which showed that by using 3d rotations, this second effect could also be reduced.

Although as the video has already been stabilised, fixing this particular one is probably now impossible...

David

[johnmeyer]

As David notes, the problem is caused by the rolling shutter in the camera which took the video. All modern CMOS cameras have a rolling shutter and potentially can have the problem, but "real" video cameras (as opposed to those in cell phones) are designed in ways which minimize the problem.

There have been several threads in this forum about how to eliminate or reduce these artifacts, but I don't think anyone ever came up with a real solution. David mentioned Deshaker, and it does indeed have a rolling shutter compensation algorithm which does provide some relief, although to get maximum effect you have to enter a value that is specific to the camera that took the video (Gunnar Thalin, the Deshaker author, provides information on how to measure these values).

If you have some $$$, you can purchase the ProDad Mercalli stabilizer. I own this, and it does a better job than Deshaker at dealing with the "jello" artifacts from rolling shutter cameras. If you have the original footage from the example you posted, and you can upload that, I can put it through Mercalli and post the result. Or, if you want, you can download their demo:

Mercalli Demo (http://www.prodad.com/home/products/videostabilizing/300391667/trial,l-us.xhtml)

and do your own tests. Just select one of the presets, put a "check" in the "rolling shutter" box, and then evaluate the clip and play the result.

[Mug Funky]

it might be worth noting for Canon users that the latest magiclantern firmware gives a little control over shutter roll, at the expense of disabling sound.

typically i shoot 720p60 or 50 (depending on whether there's artificial light around) with a fast shutter because you can stabilise more easily and your jellyrolls are automatically much less than they'd be at 24fps (because the scanning interval limits it to a max of 1/60 or 1/50). with the new timing tweaks, i can just shoot at 24fps and sacrifice the ability to stabilize without fuzz on jolts, but i get the ability to triple the bitrate because i'm writing 60p to a card at only 24 intervals per second.

[martin53]

David,

the more pronounced problem which occurs because a lot of stabilisation algorithms only use 2d rotations/translations

absolutely.

Not being an expert, I'd say that the effect depends on the focal width of the camera, but maybe two cameras can still be different at same focal width. The lateral shift in fact comes from the angle the camera turns, and even a 2D object rotates by that angle.

As an initial thought:

With MVTools MMask(kind=5), x and y movement of frames can be measured; and potentially a correlation can be found between the absolute shift e.g. in x direction, and the y stretch/shrink that accompanies it at the right/left side (same thing for the x effect at top/bottom when y shift occurs).

RT_Stats allows a less complicated evaluation of the MMask clip in small tiles, to get e.g. bottom x zoom (x movement of right hand bottom tile minus x movement of left hand bottom tile, etc).

With quad(), the clip can be warped, so that these effects are neutralized, before then the frames are processed with DeShaker.

This is an example how I implement the described method for a different purpose:#make a helper clip that shows motion of frame areas as U,V
mc = c.MMask(vF, kind=5, thSCD1=640, thSCD2=255, ysc=255).crop(32,32,-32,-32)

#divide helper clip into tiles
w2 = 2*round(0.25*mc.width)
h2 = 2*round(0.25*mc.height)
w4 = 0.25*mc.width
h4 = 0.25*mc.height
mc00 = mc.crop(0,0,-w2,-h2) #cropping could be avoided if RT_Stats' AverageChroma functions were used below
mc01 = mc.crop(w2,0,0,-h2)
mc10 = mc.crop(0,h2,-w2,0)
mc11 = mc.crop(w2,h2,0,0)

#calculate quad target coordinates by using motion of corresponding tiles
xt1 = 0.25*(128-mc00.AverageChromaU)+w4
yt1 = 0.25*(128-mc00.AverageChromaV)+h4
xt2 = 0.25*(128-mc01.AverageChromaU)+c.width-w4
yt2 = 0.25*(128-mc01.AverageChromaV)+h4
xt3 = 0.25*(128-mc11.AverageChromaU)+c.width-w4
yt3 = 0.25*(128-mc11.AverageChromaV)+c.height-h4
xt4 = 0.25*(128-mc10.AverageChromaU)+w4
yt4 = 0.25*(128-mc10.AverageChromaV)+c.height-h4

#do transform
f = f.quad(xt1,yt1, xt2,yt2, xt3,yt3, xt4,yt4, w4,h4, c.width-w4,h4, c.width-w4,c.height-h4, w4,c.height-h4)