rolling shutter

[Terka]

is it possible to fix rolling shutter (d)effect?
some camcorder have it.
http://cpressvideo.superhosting.cz/streams/2008/09/camileo3.mp4

[morsa]

It is and it is not.Better ask manufacturers to waste a couple more pennies on sensors and choose to have Global shutter on CMOS... :D

(or at least a so fast readout that RS becomes almost invisible)

[Terka]

yes, i know. ;( i meant avisynth way.

[2Bdecided]

VirtualDub deshaker claims to try - it doesn't help with my camcorder though.

Cheers,
David.

[Undead Sega]

There is actually a recent plugin for Nuke by The Foundry that now corrects video with Rolling Shutter issues:

http://www.thefoundry.co.uk/pkg_overview.aspx?ui=47C4AB50-4636-4326-87D1-FB380B2119EF

but it is bloody expensive, if only someone took the concept of this and made it as a avisynth filter, can it be done? and wat would they need? a .dll of the sfotware or something? :D

[scharfis_brain]

have just tried to search for "rolling shutter" in the forums?

actually there seems to be a solution:
http://forum.doom9.org/showthread.php?p=1374487&highlight=rolling+shutter#post1374487

[Undead Sega]

that seems interesting, but the user claims "it works. Sometimes." Im assuming that it would only fix the typical panning effects of rolling shutter, abit like from Deshaker, if you know wat i mean. Also, i dont seem to bee is anyone has tried it out?

[um3k]

It works pretty well, actually, except that it's a memory hog, somewhat unstable, extremely slow, etc. But it's a proof-of-concept, and gives acceptable results on small videos (and crashes on large ones). The good part is that it corrects all rolling shutter motion artifacts, both global and local, much like the Foundry plugin. Someone just needs to implement it as a subfunction of MVTools with optimizations, something which I am currently not capable of. Here is the latest version, though I have not messed with it in months:

# RollAway filter by um3k/Justin Phillips
# Reduces rolling shutter artifacts on video from CMOS-chipped camcorders/DSLRS
# "Amount" is the same as that used for DeShaker. The formula can be found here:
# http://www.guthspot.se/video/deshaker.htm#rolling shutter setting
# "Segh" is the height of the segments the image is split into. It should be an even number, and a dividend of the clip height.
# This script is compatible with SetMTMode.
# Use SetMemoryMax in your script to increase speed. Set it to a fairly high number that is reasonable for you computer's capabilities.
#
# Alpha 2

function RollAway_alpha2(clip c, float amount, int segh, bool "test", bool "flip", int "pel", int "blk", int "ovr", int "thSAD")
{
Assert(c.height%segh == 0, "'segh' must be a dividend of clip height")
ctime = amount/2.0
segs = c.height/segh
den = segs-1
num = den
test = Default(test, false)
flip = Default(flip, false)
pel = Default(pel, 1)
blk = Default(blk, 16)
ovr = Default(ovr, 12)
thSAD = Default(thSAD, 100)
super = c.MSuper(pel=pel)
fv = MAnalyse(super, blksize=blk, isb = false, overlap=ovr)
bv = MAnalyse(super, blksize=blk, isb = true, overlap=ovr)

DeRoll(c, super, fv, bv, ctime, den, segh, t=test, flip=flip)
}

function DeRoll(clip c, clip super, clip fv, clip bv, float ctime, int den, int segh, int "segn", clip "stack", bool "t", bool "flip")
{
stackh = Defined(stack) ? stack.height
\ : 0
segn = Defined(segn) ? segn+1
\ : 0
num = abs(den-segn*2)
fclp = (flip==true) ? c.MFlow(super, bv, time=ctime*float(num)/float(den)).Crop(0, segn*segh, -0, segh).SelectEvery(1, -1)
\ : c.MFlow(super, fv, time=ctime*float(num)/float(den)).Crop(0, segn*segh, -0, segh).SelectEvery(1, 1)
bclp = (flip==true) ? c.MFlow(super, fv, time=ctime*float(num)/float(den)).Crop(0, segn*segh, -0, segh).SelectEvery(1, 1)
\ : c.MFlow(super, bv, time=ctime*float(num)/float(den)).Crop(0, segn*segh, -0, segh).SelectEvery(1, -1)
nseg = stackh > c.height/2 ? bclp
\ : fclp
nseg = (t==true) ? nseg.Subtitle("num=" + String(num) + " den=" + String(den) + " segn=" + String(segn) + " ctime=" + String(ctime*float(num)/float(den)))
\ : nseg
stack = Defined(stack) ? StackVertical(stack, nseg)
\ : nseg
segn = segn return stack.height == c.height ? stack
\ : DeRoll(c, super, fv, bv, ctime, den, segh, segn, stack, t, flip)
}

It basically motion compensates the frame numerous times, cuts up the frames, and recombines them into a new one. Needless to say, it is quite slow. Try it on a video downscaled to SD resolution, HD is practically guaranteed to crash.

I'd planned to put together a demo video, but, well, haven't.

Also, as a side note, it does not fully correct very fast motion, due to a limitation in MVTools. There's not really anything I can do about that.

[Undead Sega]

i would pay good money to make or see this happen (and also as an independant program based on the Avisynth filters) :D

[Undead Sega]

okay, i see no responses to that, hahaha, what i would like to mention is that many of the cameras sold today are unfortunately implemented with a CMOS chip, which will result giving the rolling shutter artifact, my problem is i want to shoot stuff using my HD CMOS camera and i would want to correct it as well for various good reasons (like tracking and adding visuals etc.), but after from what you're saying, it doesnt seem to be practical whatsoever?

[Undead Sega]

ummm, anyone at all may i ask?

any samples or clips showing this in action?

i would also like to mention, with the foundry's rollingshutter plugin, apprantly not only it fixes the skews in the frame but it can also only align those that are skewed keeping those that werent in motion unaffected.

[Mug Funky]

you could always shoot on the D20 or D21...

failing that, shooting around the defect is pretty easy - many a feature film have been shot with rolling shutter. the trick is not to move the camera too fast, and avoid flashing lights as they'll show up halfway down a frame and give you computer style "screen tearing".

i'd love to see the foundry plug in action, but i reckon it probably does the same as the avisynth script above - foundry have a large number of plugs based on their mo-comping engine.

[Undead Sega]

Well, for many many others like myself, we shoot on a Canon HV camera which is a great camercorder but at the same time it is unfortunate that is does have a CMOS chip implemented, and if we're honest there will be sojme or many times where fast movement of the camera has to come in, and this relates to what kind fo style the person is wanting to shoot, even steady movements of the camera is still affected because u still have movements happening anyways which all gets affected by rolling shutter anyways.

I think there should be a dedicated thread for this rolling shutter filter, just like TGMC and NNEDI etc. therfore people might be able to try it out?

[PitifulInsect]

CAVEAT: Until a couple of days ago, I had never played with this, so I act all knowledgable but if I'm spouting any BS, please pull me up on it.
---
Hey there rollers!

Two presentations came to my attention in the last two days about how to fix rolling shutter issues - they're being presented at a conference this week so it's up-to-date stuff. I got interested, so googled and found this thread. I had a play with um3k/Justin Phillips' script. Examples and an annotated version in a moment, but a couple of comments:

There are two distinct classes of problem: When the motion is slower/smoother than the framerate, and when it's faster/shakier.
If the motion between frames is smooth, then temporally you don't really have motion aliasing, and Deshaker or a script like um3k's can be enough.
If the motion between frames is not smooth (and especially if it's periodic) then your video is temporally aliased, and no simple approach can handle it. (Think of the shake frequencies being above the Nyquist sampling of the frame-rate if that means anything to you)
So smooth panning can be cleaned up, but fast shaking, especially the "jellycam" examples where the camera is attached to a mechanically-vibrating platform like a car, are unrecoverable (by any means available to mortals).


The first paper, "Rectifying Rolling Shutter Video from Hand-held Devices", (Forssen and Ringaby, Linkoping University, Sweden, http://www.cvl.isy.liu.se/research/rs-dataset/0382.pdf), shows the ultimate way to solve the simple cases (smooth motion) efficiently, by tracking interest points (not using optical flow) and doing fancy stuff for the regularisation - read the paper if you're interested. Requires a static scene (no moving objects).

The second paper "Removing Rolling Shutter Wobble", (Baker, Bennett, Kang, and Szeliski, Microsoft Corporation, http://research.microsoft.com/pubs/121490/0198.pdf), actually uses temporal super-resolution in order to do fix the faster-than-framerate wobble sequences! Fancy stuff. The "simple" version is robust but requires a static scene (no moving objects), and there's a more-fancy version but it's not robust and it's pretty computationally expensive.

I learned:
Results can be very good, but they still aren't perfect with complex and expensive state-of-the-art techniques from the top experts in the field!
I also learned more about the problems involved, about super-resolution generally, etc, but the take-home message is that if you camera is shaking fast or vibrating, you're screwed. Don't let your camera shake or vibrate - it's never good anyway. If it's just panning - use deshaker, you won't immediately find better. The script suffers issues (so would deshaker I think - I haven't used it), but it can help.

So here's an annotated version of the script and results on the original video:

#
# The MVTools2 plugin is used by this script. You'll find it at:
# http://avisynth.org.ru/mvtools/mvtools2.html
#
LoadPlugin("mvtools2.dll")


#
# Pixels begone! Doing this at high-res is too annoying
# (I'm working off an 800-pixel-high laptop screen here!)
#
DirectShowSource("camileo3.mp4").ReduceBy2()


# RollAway filter by um3k/Justin Phillips
# Reduces rolling shutter artifacts on video from CMOS-chipped camcorders/DSLRS
# "Amount" is the same as that used for DeShaker. The formula can be found here:
# http://www.guthspot.se/video/deshaker.htm#rolling shutter setting
# "Segh" is the height of the segments the image is split into. It should be an even number, and a dividend of the clip height.
# This script is compatible with SetMTMode.
# Use SetMemoryMax in your script to increase speed. Set it to a fairly high number that is reasonable for you computer's capabilities.
#
# Alpha 2
function RollAway_alpha2(clip c, float amount, int segh, bool "test", bool "flip", int "pel", int "blk", int "ovr")
{
#
# Just syntactic housekeeping: Define default values and set up some paramters.
#
Assert(c.height%segh == 0, "'segh' must be a dividend of clip height")
ctime = amount/2.0
segs = c.height/segh
den = segs-1
num = den
test = Default(test, false)
flip = Default(flip, false)
pel = Default(pel, 1)
blk = Default(blk, 16)
ovr = Default(ovr, 12)

#
# Just uses MVTools - See http://avisynth.org.ru/mvtools/mvtools2.html
# MSuper heirarchically decomposes the clip, while MAnalyse prepares
# the motion vectors.
#
super = c.MSuper(pel=pel)
fv = MAnalyse(super, blksize=blk, overlap=ovr, truemotion=true, searchparam=8, isb = false)
bv = MAnalyse(super, blksize=blk, overlap=ovr, truemotion=true, searchparam=8, isb = true )

#
# Call the method below
#
DeRoll(c, super, fv, bv, ctime, den, segh, t=test, flip=flip)
}

#
# This function calls itself recursively to cut up the clip into vertical
# segments, then assemble them together. Very clever!
# Each segment is created using MFlow
#
function DeRoll(clip c, clip super, clip fv, clip bv, float ctime, int den, int segh, int "segn", clip "stack", bool "t", bool "flip")
{
#
# More variable preparation - note especially that this is where the segment number 'segn' is increased.
#
stackh = Defined(stack) ? stack.height : 0
segn = Defined(segn) ? segn + 1 : 0
num = abs(den-segn*2)

#
# "flip" allows you to switch the forward and backward clips - I can't
# figure out why exactly. I think that it's for field parity for
# interlaced videos.
#
time=ctime * float(num) / float(den)
fclp = (flip==true) ? \
c.MFlow(super, bv, time=time).Crop(0, segn*segh, -0, segh).SelectEvery(1, -1) : \
c.MFlow(super, fv, time=time).Crop(0, segn*segh, -0, segh).SelectEvery(1, 1)
bclp = (flip==true) ? \
c.MFlow(super, fv, time=time).Crop(0, segn*segh, -0, segh).SelectEvery(1, 1) : \
c.MFlow(super, bv, time=time).Crop(0, segn*segh, -0, segh).SelectEvery(1, -1)
#
# Figure out which segment of the stack we're at (past halfway?), and
# whether we should be using a segment forward-interpolated from the last
# frame, or back-interpolated from the next frame.
#
nseg = stackh > c.height/2 ? bclp : fclp

#
# Just add subtitles to explain what's going on for debugging
#
nseg = (t==true) ? \
nseg.Subtitle("segn=" + String(segn) + "(num=" + String(num) + " den=" + String(den) + ") time=" + String(time)) : \
nseg

#
# Build the stack - either begin it, or add the current segment to it.
#
stack = Defined(stack) ? StackVertical(stack, nseg) : nseg

#
# Is the stack complete? return the stacked image. No? Call recursively
# to add the next segment.
#
return stack.height == c.height ? \
stack : \
DeRoll(c, super, fv, bv, ctime, den, segh, segn, stack, t, flip)
}

#
# Now just call the functions
#
StackHorizontal( \
last.Subtitle("Original", x=Width()/2, align=8), \
RollAway_alpha2(86.0, 2, test=false, flip=false, pel=2, blk=8, ovr=4).Subtitle("De-rolled", x=Width()/2, align=8) \
)


I couldn't find a single static frame to demonstrate the fact (it's more of a temporal matter), but for the wobble, the script just doesn't make a difference.
Worse, you sometimes get little artefacts from imperfect optical flow.
Worse still, is that small-artefacts aside, fast motion will sometimes kill the optical flow entirely, and then you get sadness all-around.
http://forum.doom9.org/attachment.php?attachmentid=11174&stc=1&d=1276761160


For smooth motion, you get strange effects during acceleration, but during the motion it more-or-less works.
http://forum.doom9.org/attachment.php?attachmentid=11175&stc=1&d=1276761169

So there you are - a whole bunch of info in one go. I hope that it's about right because as I said - I'm new to this so someone might know more. If you're reading and you know more - post an answer!

[cretindesalpes]

I tried to improve the speed and memory usage of the script. I ended up modifying the MVTools2 to let MFlow() accept the motion compensation time parameter as a clip, the time being defined for each pixel instead of a single scalar for the whole clip.

The visual quality hasn't changed but it resulted in a x10 speed up improvement over the original script (tested on terka's example camileo3.mp4 with segh=4 and pel=4). And now that the memory usage has become fair, it is possible to SetMTMode() it on HD material, giving an impressive x40 speed boost on a quad core CPU!

I also tried the same thing on MFlowInter() but the results were disappointing. It generates much more artifacts than a simple MFlow(). However I kept it in the script so you can try to play with it and find better parameters.

You can find the modified MVTools2 in this package (http://ldesoras.free.fr/src/dither-avsi-1.2.zip). And the updated script below:

# RollAway filter by um3k/Justin Phillips/Firesledge
# Reduces rolling shutter artifacts on video from CMOS-chipped camcorders/DSLRS
# "Amount" is the same as that used for DeShaker. The formula can be found here:
# http://www.guthspot.se/video/deshaker.htm#rolling shutter setting
# This script is compatible with SetMTMode.
#
# Alpha 2 mod
function RollAway_alpha2mod (clip c, float amount, bool "flip", int "pel", int "blk", int "ovr", bool "inter")
{
#
# Just syntactic housekeeping: Define default values and set up some paramters.
#
flip = Default(flip, false)
inter = Default(inter, false)
pel = Default(pel, 1)
blk = Default(blk, 16)
ovr = Default(ovr, 12)

#
# We'll split the picture at a height multiple of 4, and this split
# line is used as the reference time (no motion interpolation here).
# Therefore we need that the time function equals 0 at this line :
# t = amount * abs (ofs - y), with ofs close to 0.5 and y in
# the range 0 to 1.
# MFlowInter() timeclip formula is a bit different.
#
w = c.Width ()
h = c.Height ()
sy = 4 * Round (h / 8) # Split line
ofs = Float (sy) / Float (h)

OFS = String (ofs)
AMT = String (amount)
expr_norm = OFS + " y - abs " + AMT + " * 2.56 *"
expr_inter = OFS + " y - " + AMT + " * 2.56 * y " + OFS + " >= 256 0 ? +"
expr = (inter) ? expr_inter : expr_norm

#
# Just uses MVTools - See http://avisynth.org.ru/mvtools/mvtools2.html
# MSuper heirarchically decomposes the clip, while MAnalyse prepares
# the motion vectors.
#
super = c.MSuper(pel=pel)
fv = MAnalyse(super, blksize=blk, overlap=ovr, truemotion=true, searchparam=8, isb = false)
bv = MAnalyse(super, blksize=blk, overlap=ovr, truemotion=true, searchparam=8, isb = true )

#
# "flip" allows you to switch the forward and backward clips - I can't
# figure out why exactly. I think that it's for field parity for
# interlaced videos.
#
timeclip = c.mt_lutspa (relative=true, expr=expr, y=3, u=3, v=3)
fclp =
\ (inter) ? c.MFlowInter (super, bv, fv, tclip=timeclip)
\ : (flip ) ? c.MFlow(super, bv, tclip=timeclip).SelectEvery(1, -1)
\ : c.MFlow(super, fv, tclip=timeclip).SelectEvery(1, 1)
bclp =
\ (inter) ? fclp.SelectEvery(1, -1)
\ : (flip ) ? c.MFlow(super, fv, tclip=timeclip).SelectEvery(1, 1)
\ : c.MFlow(super, bv, tclip=timeclip).SelectEvery(1, -1)

#
# Figure out which segment of the stack we're at (past halfway?), and
# whether we should be using a segment forward-interpolated from the last
# frame, or back-interpolated from the next frame.
#
top = fclp.Crop (0, 0, w, sy)
bot = bclp.Crop (0, sy, w, h - sy)
stack = StackVertical (top, bot)

return (stack)
}

[um3k]

PitifulInsect: Good job, thanks. It's been a while since I worked on any of my Avisynth functions, it kind of comes in bursts. The "flip" parameter was created to deal with an oddball video from, I believe, an iPhone. The rolling shutter effect seemed to be reversed and required special treatment.

cretindesalpes: Wow! Awesome! This is exactly the sort of thing I was hoping someone would do, as I lack the programming ability to modify MVTools. I'm gonna give it a try and see how it does.

[um3k]

Ok, cretindesalpes, I gave it a try. It works pretty well, but there is one major downfall, and that is the 8-bit limited color depth of Avisynth. This results in stair-stepping the time clip, which translates to discontinuous motion in the interpolated image. I think the next step in the developing of this filter would be to port it completely into a new MVTools function, which would allow the use of a smooth gradient and optimized speed and memory usage. The problem, of course, is my lack of programming skills. If someone would take up this task, they would earn the thanks of a great many camcorder users.

[Terka]

have an idea how deal with RS:
1.calibrate a camera for different movements speeds.
for every speed got frames: good, distorted. calculate motion vectors for their comparsion.
2.estimate the speed using global motion compensation (depan).
3.use mcompensate (using vectors from 1)

[Undead Sega]

have an idea how deal with RS:
1.calibrate a camera for different movements speeds.
for every speed got frames: good, distorted. calculate motion vectors for their comparsion.
2.estimate the speed using global motion compensation (depan).
3.use mcompensate (using vectors from 1)

Possible if you can elaborate on this for those who are not totally knowlegable to this language (I understand abit but sometimes I do have trouble :D)

Almost sounds like a solution to Rolling Shutter :D

[PitifulInsect]

have an idea how deal with RS:[...]

No, Terka, that would not help at all.

Rolling shutter causes a very simple problem, but it is very difficult to remove it because you can't know enough about the scene to undo it. The Microsoft-paper approach is maybe the closest approximation possible.

Lets say that you had a very simple scene with a vertical line in it:
+-----------+
| | |
| | |
| | |
+-----------+

If the camera is moving at a constant speed, then you can "simply" interpolate back and forth in time. Um3k and cretindesalpes have the script which can solve this perfectly:
+-----------+ +-----------+ +-----------+
| / | | / | | | |
| / | | / | -> | | |
| / | | / | | | |
+-----------+ +-----------+ +-----------+
frame 01 frame 02 solution is
difference of positions

But when the camera moves in any non-linear way (which is all the time, for example acceleration before and after even the simple motion as above), the problem is hard. Um3k and cretindesalpes' script will not solve the problem because the interpolation is only linear (though it will help a little, so it is probably still the best option available):
+-----------+ +-----------+ +-----------+
| | | | | | | | |
| | | | / | -> | / |
| | | | / | | / |
+-----------+ +-----------+ +-----------+
frame 01 frame 02 difference of positions
(imagine a curling doesn't work
line like a 'j')

You can imagine that the problem is even worse when the camera is moving up and down also.

To solve the problem, you need to know the camera position at every instant in time to know where/when to compensate the data from. If you have the position only at every frame, and you try to use that, then your solution is even worse than above:
+-----------+ +-----------+ +-----------+
| | | | | | | \ |
| | | | / | -> | | |
| | | | / | | / |
+-----------+ +-----------+ +-----------+
frame 01 frame 02 global per-frame
(imagine a curling motion is not
line like a 'j' as enough
camera accelerates)

If the camera shakes a lot, the problem is worse and you get "jellycam" (search Youtube):
+-----------+ +-----------+ +-----------+
| | | | \ | | |
| | | | / | -> | ? |
| | | | \ | | |
+-----------+ +-----------+ +-----------+
frame 01 frame 02
(camera shakes)
And of course this problem is the worst because then the video is motion-blurred, and then knowing the position to compensate to is not possible (I don't mean hard, I mean that it is mathematically not possible at all).

If one day we have optical flow methods good enough to measure position at every *line* of a video, then perhaps a per-line compensated approach would be possible, but even then, only if the video has enough detail! Such optical-flow technology does not exist though, and there would not be enough detail in most videos anyway to make this possible. The Microsoft approach measures what information is available, and uses temporal super-resolution methods to discover the rest, which is very clever, but again: They must be able to measure motion, so their method breaks (as does every other known method) when the scene has moving objects which pollute the flow-field.

So if your camera has a rolling shutter, don't shake it about!

[Terka]

nice summary :)
i understand its a big problem, im not speaking about totally removing, but at least minimizing.
if camera is moving there and back..., there must exist a place where it stops. such frames could be used for compensation.
(assume that the filming object are not warping, only translating, maybee zooming)

[PitifulInsect]

nice summary :)
Thank you.

i understand its a big problem, im not speaking about totally removing, but at least minimizing.
Well the existing script does a reasonable job of that, but it's important to understand that it's very very difficult to get rid of the problem entirely, so that you should take care to avoid letting it happen in the first place. Pan your camera slowly, don't let it vibrate. The rolling shutter effect is not very visible on moving objects, so it's fine as long as you keep your camera still or pan only slowly.

if camera is moving there and back..., there must exist a place where it stops. such frames could be used for compensation.

Sadly, no, there is no such place. Remember that the whole problem of rolling shutter is that the camera cannot be moving
*or accelerating*. I get the feeling that you are a school student Terka (you ask many questions), if I am right then what you need will be covered in year 11 science class. If I am wrong, or for those who did not have science or physics at school: If the camera is changing direction, it is accelerating, it is not "stopped" at any time ("stopped" here meaning that it is neither moving nor accelerating).

Incidentally at 30fps and for an 86% roll, the camera would have to be still for at least 29ms (given perfect luck, 58ms to get "at least" one frame). That's actually not that short a time by video standards.

If you had a reference scene appearance though, I could see that it could be possible to warp each frame locally to approximate the correct appearance of the frames (removing any global offset, removing bad-match blocks due to moving objects and perspective transforms, allowing that columns are caught simultaneously but rows are offset in time).
That is not a bad idea, but it would be difficult to accomplish. It would only work for panning, since appearance changes in other motions and you couldn't make such a map. You would also need a reference bigger than any one frame, which may be difficult if you are panning a camera, so it could only help in specific situations. If you only want to try to keep the appearance constant between frames - well that's what the script does more-or-less.
My challenge to you Terka is to find such a scenario, and try such a solution, it would be interesting to everyone :-)

(assume that the filming object are not warping, only translating, maybe zooming)
Actually, zooming is one of the worst problems for image matching or optical flow, even rotation is not as bad as zooming. The change in scale makes matching difficult! I would recommend keeping away from zooming if you can, it isn't impossible to deal with, but it makes results difficult.

[smok3]

anybody had to actually deal with this with canon 7D? how bad is it with this specific camera? is there any difference with changing the shutter speed? (with about 600 clips taken so far i could find exactly one shot that looks problematic and it was a tilt (squash effect).)

i'am thinking of getting this specific DSLR.

[Terka]

i think the shorter shutter, the smaller 'rolling effect'

[2Bdecided]

i think the shorter shutter, the smaller 'rolling effect'I don't know about the 7D, but generally this isn't the case at all. The rolling shutter is due to the sequential readout of the sensor - it has nothing to do with shutter speed. A faster shutter speed means each pixel on the sensor is exposed for a shorter time, but the delay between the top pixel being exposed and the bottom pixel exposed remains the same, irrespective of shutter speed.

Shorter shutter speed = less blur = rolling shutter artefacts might be slightly more obvious, but that's a secondary effect.

Cheers,
David.

[Terka]

are you sure?

[2Bdecided]

are you sure?Yes!

I own an HV20(!).

If a shorter shutter speed meant less rolling shutter, then we could avoid rolling shutter outside in daylight just by making the shutter faster (at the expense of strobing). But this doesn't work at all.

Cheers,
David.

[Terka]

I own an HV20 too. :p

[Soulhunter]

anybody had to actually deal with this with canon 7D? how bad is it with this specific camera? is there any difference with changing the shutter speed? (with about 600 clips taken so far i could find exactly one shot that looks problematic and it was a tilt (squash effect).)

i'am thinking of getting this specific DSLR.
I got the 7d a half year ago, awesome piece of technology... :)
If you don't move the camera "spasticity-style", you will hardly get visible rolling-shutter!

[2Bdecided]

I got the 7d a half year ago, awesome piece of technology... :)
If you don't move the camera "spasticity-style", you will hardly get visible rolling-shutter!Sometimes camera movement is inevitable, or wanted. Can you imagine Saving Private Ryan shot with a rolling shutter! The other day I filmed on a roller coaster - I used by cheap (CCD) point-and-shoot camera in video mode, because I knew the HV20's output would be unusable. The point-and-shoot has no optical image stabiliser, but that was useful for creating realistic effect. VirtualDub deshaker worked very well on the video, but it didn't feel so much like a roller coaster then!

btw, rolling shutter can be measured - there's no need to guess or give subjective impressions.

e.g. see the suggested method in the deshaker guide...
http://www.guthspot.se/video/deshaker.htm
(search for rolling shutter)


I own an HV20 too. :pThen you must know that a shorter shutter speed doesn't help at all? (where's the "confused" smiley? ;) )

Cheers,
David.

[Terka]

im quite new with the hv20, and didnt have some 'real issue' with its RS, i also dint change the shutter yet. ;(

[*.mp4 guy]

Theoretically, if you use a shutter speed of 1/framerate you don't lose any temporal information (it just gets mixed in with spatial info in a non-separable fashion). It will probably be a lot easier to reduce rolling shutter artifacts if you stay close to this limit. Although this will make high motion stuff difficult to deal with as well, the degradation introduced should look more normal.

[2Bdecided]

Theoretically, if you use a shutter speed of 1/framerate you don't lose any temporal information (it just gets mixed in with spatial info in a non-separable fashion).If it's mixed and non-separable, it sounds lost to me! I get what you're trying to say, but I'm not convinced.

It will probably be a lot easier to reduce rolling shutter artifacts if you stay close to this limit. Although this will make high motion stuff difficult to deal with as well, the degradation introduced should look more normal.Not really. If you're panning past a vertical thing (e.g. a pole, a window, whatever) a slower shutter just makes that "thing" blur, i.e. it looks wider. Whereas the rolling shutter means it's not vertical! It's the same amount "not vertical" (i.e. it's the same angle) whatever the shutter speed.

On less obvious examples, the blur might help to hide the effects, but it's marginal really.

Cheers,
David.

[*.mp4 guy]

If it's mixed and non-separable, it sounds lost to me! I get what you're trying to say, but I'm not convinced. temporal and spatial data are already highly interdependent and non separable, I was just saying that the change is non-ideally reversible.

Not really. If you're panning past a vertical thing (e.g. a pole, a window, whatever) a slower shutter just makes that "thing" blur, i.e. it looks wider. Whereas the rolling shutter means it's not vertical! It's the same amount "not vertical" (i.e. it's the same angle) whatever the shutter speed.
The primary reason a slower shutter should help is that it should allow motion compensation to work more optimally. slower shutter speeds should also greatly reduce the severity of "tearing" caused by flashing lights etc. Basically the slower shutter speed should ensure that just about everything left over should be almost entirely recoverable.

[johnmeyer]

Many thanks cretindesalpes for the modified MVTools2 and the script. What you were able to do is exactly what I suggested here:

MVTools2 discussion of Rolling Shutter (http://forum.doom9.org/showthread.php?p=1339685#post1339685)

when this idea was first brought up. I expect that it will probably help fix the problems associated with left/right camera panning. I'm not sure what happens with vertical camera movement or with camera rotation.

[Soulhunter]

Sometimes camera movement is inevitable, or wanted. Can you imagine Saving Private Ryan shot with a rolling shutter! The other day I filmed on a roller coaster - I used by cheap (CCD) point-and-shoot camera in video mode, because I knew the HV20's output would be unusable. The point-and-shoot has no optical image stabiliser, but that was useful for creating realistic effect. VirtualDub deshaker worked very well on the video, but it didn't feel so much like a roller coaster then!

Yeah, ultra shaky run-n-gun and filming from a roller coaster or a motorbike (without one of this glide or gyro stabilizers) is what i meant with "spasticity-style". Okay, sometimes its wanted for realness effect, but in 99% of situations its not the case, and that's why I wrote you will hardly get visible rolling-shutter! ;)

[smok3]

shot some stuff with 50mm 1.4 primes (handheld), everything set to manual, including ISO (100) and i got gazillion problems with rolling shutter.
Could there be any direct connection between ISO and the defect? (that would mean using some slower lens or some sort of ND would be nice for outside stuff, with an ISO setting of... =?)

[Undead Sega]

No, Rolling Shutter is caused by the method of capturing in the Image Sensor. I own a HV30 :D

[Undead Sega]

Hi everyone, sorry for bringing something that hasnt been talked for awhile, but once again Rolling Shutter has become an issue with me, one of the reasons was that I intend to buy a new Lumix GH2 that was rumoured to be 'Global Shutter' in their LiveMOS image sensoprs, unfortunately that development was haulted and focus was shifted to developing the 3D lens, however the image scanning is faster this time but the rolling sbutter artifacts will still remain, nevertheless the DSLR is fantastic and still will get it.

Because of this, i have came back here and I believe it can be fixed but we users would have to know our own equipment at the same time. I dont know exactly how this current script deals with the Rolling Shutter artifact, but I would like to add if somehow we were able to Temporally analyze the previous and future frames bearing in mind that vertical edges (that was straight is straight no-more due to the lag) will be straight again, this will be incredibly useful for those who are doing point & shoot and run n' gun filming. Probably a preprocessing mask to detect vertical edges could be something of use.

Also how does the RollAway script perform? I hear its slow as hell and would crash on HD footage, has there been any development or optimizations afterwards or recently? Also does it bear its own artifacts like the Foundry plugin by any chance?