Deshaker by Gunnar Thalin

[wonkey_monkey]

Hey Gunnar, have you thought (more? I may have suggested this before) about allowing for perspective in the resampling stage? If you take a look at the videos on the blog page hiviking linked to, you can really see the wobbling that's introduced at the edges. I did some tests a long time ago, and taking account of perspective improves things a lot on wide angle videos.

David

[MaxForce]

The software is amazing! The default settings work perfect on every video I have ever tried. It might even work too well and the videos look unrealisticly smooth. My only tip is, right at the end, to limit the Max correction for horizontal and vertical panning to 5%. It makes videos look smooth but not weird.

Thank you so much for such a wonderful product!

[guth]

MaxForce, I'm glad you like it!

David, I missed your latest post. I'm afraid there's just too much work for me to want to deal with perspective. (Determining FOV (which can change within a clip), filling borders from past/future frames, ideally also take perspective into account during matching in pass 1, and all this combined with rolling shutter distortion etc...)

[CruNcher]

Is there any compare between AMDs 1st and 2nd Generation Hardware implementation and Deshaker ?

[guth]

Don't know.

[DESuser]

I am not quite sure is such a way of using the deshaker IS in accord with this topic, i have to mention this :
The _author_ intented it not only for a simulated camera motion within short stabilizitions like usual setting, hi did mention that filter is capable of producing a panoramas, where missing data is filled by blur or frames. Going further with a low resolution videos like 3gp sources or lowq phone videos being used to make a dvd movie there pops up a limitation of the filter. First one, which the author may not seen is when original camera motion IS slow eg from left to right and past-future data become unusable. Solution is to make two passes with odd and even frames, and then combine these two deshakes. Nasty thing. Next, the fillrate - every user in the world is DISsatisfied with the fillrate of that tricky blurry colour. Just disabling it brings speed improve, AND new passes of doing. I had a 9K+ frames video to be done within 29 hours on E2160 1800. Extremely slow fillrate of that. Restoration i did is to use surrounding frames of course
, and these are not the usual 30+-, good resullts for a 480p to 576p are 300+-, better ones are 450+-, and for 144p to 576p are 800-900+- and the 1000 i used too. It is just not enough for a usual scene duration of 2000-3000 frames.
It seems to me that algorithm used by author is not done neatly, it immediately take up to 2g ram and still may produce out of memory on some scene changes.
AND these is a so slow processing with no camera motion parts of any video. The movie ive done is all to last frame was restored andor deshaked via it.
LOTS and surely tons of videos are to be unfolded to greater resolutions depending on original camera motion, and those amateur shaky videos are the first to it, they really contain MORE data. And the final solution of such deshaking is a simulated panned zoom to avoid bad parts, there are currently no good ones ive seen for vd1911.
Does anyone have any ideas ?
Maybe the author may have some words ?

[guth]

Yes, Deshaker *is* extremely slow when it comes to creating panoramas, but that's mainly because Deshaker is a stabilizer and not a panorama maker.

I know the "extrapolate colors" feature is extremely slow, but for normal use it's rarely used, I think. (When I made that feature, I prioritized quality over speed, but to be honest it got a lot slower than I thought it would be. You're the first person to complain about this, though.)

If you use previous and future frames to fill in borders, Deshaker saves all those frames in memory, so if you use a lot of them, it will use a lot of memory. And since Deshaker, for now, is only 32-bit, it's limited to 2GB memory (including memory used by VirtualDub). You can patch the 32-bit VirtualDub to make it use up to 4GB memory, though, by running a "4GB patch". (Google it if you're interested.)

[DESuser]

Yes, thats great, i will try it. Thank you Guth, this yours piece of work is quite usable, there are no same level software, and you are the One. There no substitutes, and msu deshaker visually being alike is not to be available. Many people just do not give a try with the way i described, simply not being familiar with the configuration. Though there is a solution for such this limit, by inserting already calculated restores into video like simulated pan and thus the far-frames data may survive the limitation. And this should be called not a pan but a full scene view, surely depending on camera motion within a certain rectangle. Even HD video may be toss-n-played this way.
Despite that you do not want to progress this, may be there any workaround to recompile it to 64 bit and just increase the number bounds ? And yep, it seems no sources on page. Anyway, it is ok.
I think that deshaker may be used for deinterlacing and result to be completed with DGbob or ELA-2 or else, i just had not time, with the ideas still in mind.
And, is that you point is like a /LARGEADDRESSAWARE patch ive seen somewhere ?

[DESuser]

And yes the to-complete-a-dream bounds might be like this - alpha blended stream support, unlimited frame bound, gui for manual correction with 1.5 pass and 2.5 pass for a pan-zoom graphs timeline, revised data handling by using a 16x16 image blocking to save ram and speed, gpu rendered output with vignettes at least, built in-pre-pipeline strong sharpening and levels - and that be kiiler-stabilize open source application. Thats it. Within years to come none be able to supercede? while it already is so.

[guth]

Originally Posted by DESuser

And, is that you point is like a /LARGEADDRESSAWARE patch ive seen somewhere ?

Yes.

I don't have a 64-bit OS yet, but I can almost promise there will be a 64-bit version of Deshaker when I get one (not too long after Windows 8 is released).
As for other new features, I'm not sure. I'm afraid I don't have anything new planned at all...

[DESuser]

It works, takes up to 3.9 Gb within vd process, nice, deshaker takes four cores to work, much better now. Usual speed of 0.2 to 0.06 fps now greatly superceded. It is within W7x64U 8G/2500K@4.1G.
Thank you for reply, Guth.

[isidroco]

Hi guth, thanks for your excellent software. I found out that reducing previous/future frames to 24 (maybe 25) stopped "Out of memory" situation in FullHD videos under 32 bits windows (at least if using h264 crf). So you should recomend that setting for those cases (unless you use 4gb patch in 64bit os).
I noticed some slight back and forth zoom waving in some still videos I deshaked (camara was on a tripod), is there any way to avoid that? Thanks!

[guth]

Originally Posted by isidroco

I found out that reducing previous/future frames to 24 (maybe 25) stopped "Out of memory" situation in FullHD videos under 32 bits windows (at least if using h264 crf). So you should recomend that setting for those cases (unless you use 4gb patch in 64bit os).

Nice to know.
I can use more than 30 previous/future frames for full-hd video. But if you encode with h264, I guess it probably uses a lot of memory.

Also, the 4GB patch works on 32-bit OS too. You won't get the full 4GB memory, but I can use more than 70 previous/future frames (i.e. 140 in total) for full-hd video on my 32-bit Vista if I do this.

Originally Posted by isidroco

I noticed some slight back and forth zoom waving in some still videos I deshaked (camara was on a tripod), is there any way to avoid that? Thanks!

Here's from my FAQ:

-----------
- I get unwanted zooming in the stabilized video.
Deshaker can add two types of zoom. One is for stabilizing zooming that it detects in the video. If this detected zooming isn't really camera zoom (for example, Deshaker usually detects zoom if the camera moves forward), or if you just don't want the camera zoom stabilized, you can turn it off by setting the zoom motion smoothness to 0. It's always a good idea to do this if there's no real zooming in the video.
The other kind of zoom that Deshaker can add is adaptive zoom. If you don't like it, choose another Edge compensation option.
-----------

Let me know if you still can't get rid of it.

[isidroco]

Memory limit was not only because of h264 vfw but also because of AVISYNTH used to provide MOVs to virtualdub. Using Mp4Cam2avi to convert MOVs to AVI without recompression solved the need of AVISYNTH. Tried without Avisynth +-45 frames with a small sample without problems (I will try to make more tests).

As for small waving in deshaked videos, using Scale Full (most precise) has improved A LOT. What I don't know if it's better using All pixels (veeery slow), every 4th (what I'm using now) or every 9th.

I made a batch file with virtualdub script for deshaking all avis in a folder without user intervention. Here's the post with it's source: http://forums.virtualdub.org/index.php?act=ST&f=5&t=17772&st=30#entry91160
Hope it helps others.

[guth]

Originally Posted by isidroco

As for small waving in deshaked videos, using Scale Full (most precise) has improved A LOT. What I don't know if it's better using All pixels (veeery slow), every 4th (what I'm using now) or every 9th.

If you set zoom motion smoothness to 0 and don't use an edge compensation option with the word "adaptive" in it, the zoom waving should be completely gone. But there can still be other types of waving.
I don't know the exact reason for your waving, but you might want to try this FAQ suggestion as well:

--------------
- I get a slight waving effect when objects move slightly, even when the camera is completely still.

Try decreasing the value for Discard motion of blocks that move more than X pixels in wrong direction.
--------------

Use as low value as you can without getting too many red vectors on the background areas. This should hopefully ignore objects that move even slightly. If the camera is somewhat stationary (but still panning/rotating or zooming), you can often use values as low as 1.0, or even lower.

"All pixels" is always better than "Every 4th" and "Every 9th", but the improvement is usually too small to see. If you can't see a difference, it's probably not worth using a slower setting.

[isidroco]

I will try your suggestions, but I do use zoom so I rather stabilize it too. A very useful option in pass2 would be: Ignore zoom corrections below: x% with a low default value of x=0.1% (0% would be current behaviour). Because much of the waving is very slight with zoom values very close to 1 in LOG, this would force to deshake only when zoom is significant.

[guth]

Originally Posted by isidroco

A very useful option in pass2 would be: Ignore zoom corrections below: x% with a low default value of x=0.1% (0% would be current behaviour). Because much of the waving is very slight with zoom values very close to 1 in LOG, this would force to deshake only when zoom is significant.

Do it yourself!
Seriously, though, you actually could do it yourself, by changing all zoom values in the logfile that are close to 1, to 1. Use Excel, for example.

[isidroco]

Originally Posted by guth

Do it yourself!
Seriously, though, you actually could do it yourself, by changing all zoom values in the logfile that are close to 1, to 1. Use Excel, for example.

Yes, I know that, but it's not practical, I should program it in pascal, because I need to do it in hundreds of LOGs (the whole point of the script I've done is to deshake automatically all my short videos). Still I think it could be an easy option to add in a future deshaker version.
Talking about that, LOGs numbers are in absolute pixel movement or as a multiplier?

Thanks for your answers

[guth]

Originally Posted by isidroco

I should program it in pascal, because I need to do it in hundreds of LOGs (the whole point of the script I've done is to deshake automatically all my short videos). Still I think it could be an easy option to add in a future deshaker version.

So, do it in Pascal then.
I don't really like the idea of such a setting. But it's also not the worst suggestion I've got.
We'll see...

Originally Posted by isidroco

Talking about that, LOGs numbers are in absolute pixel movement or as a multiplier?

If you're talking about the zoom values, it's a zoom factor (i.e. multiplier). There's a chapter "Log File Format" on my Deshaker page, btw.

[fbs]

any thoughts on how to deshake a VHS content with lots of noise and obviously low resolution?