Deshaker by Gunnar Thalin

[bxyhxyh]

How b and c of bicubicresize of this filter are set?

[guth]

How b and c of bicubicresize of this filter are set?
It uses a = -0.6, which I believe is the same as b = 0 and c = 0.6.
You can't change this value.

[sorin86]

I have some shots taken in the night, aiming at the moon. Zoom is big and so is the shaking (no tripod at that time). How can that be "de-shaken", since there is no real "background"? I only have the bright moon circle against the black background. I tried many settings for Pass 1, but none worked. Any recommendations, please?

[StainlessS]

Sorin86, search PlanetCrop.

http://forum.doom9.org/showthread.php?p=1606032#post1606032

http://forum.doom9.org/showthread.php?p=1606025#post1606025

Updated to the above posted script, avs in RT_Stats plug.

EDIT: Expects images rather than video.
EDIT: Oops, Avisynth plugin, not Vdub.

[raffriff42]

I tried many settings for Pass 1, but none worked. Any recommendations, please?See the screenshot below. It's partly guesswork.

Most importantly, tune the borders to the area the moon is. To get these numbers, use the "crop" function built into every VirtualDub filter. Copy Crop's "offset" numbers into Dehaker's "ignore border."

https://www.dropbox.com/s/1wgz13k4qf8nup7/deshaker-moon1.png?raw=1

EDIT whoops, max horizontal & vertical panning should both be about 20.

[sorin86]

See the screenshot below. It's partly guesswork.



Thank you for your try. Yet nothing works. It seems to be beyond the power of Deshaker. A variant ot the video is here https://www.youtube.com/watch?v=lqyCdM8JJBI

[bxyhxyh]

https://dl.dropboxusercontent.com/u/58215671/dddd.png
with this settings before deshaker, it detected edges.
Then you can play with deshaker settings.
But finding good values for deshaker wasn't easy task for me.

[bxyhxyh]

I have a question.
Is there any settings that chooses third, fourth or Nth frame as the first frame of the scene and stabilize previous frames compared to that frame?
Setting to add big black bars on those "falling" frames, and less black bars on other frames, instead of first frame is unaffected and the frames that shouldn't be shifted are shifted too much which current version works this way.
For some films I worked on, the first two or three frames were very off compared to the rest of the scene due to camera movement or something.

Sorry if there are some English difficulties.

[guth]

Is there any settings that chooses third, fourth or Nth frame as the first frame of the scene and stabilize previous frames compared to that frame?
Not really. But you can edit the logfile after Deshaker pass 1 to offset the first frame by a certain number of pixels. It's not as convenient, but you'll get the exact same effect.

Add a line to the logfile for the frame number of the first processed frame. For example, if you started running Deshaker pass 1 at the beginning of the clip, and then add the following line...

0 100 0 0 1

...the first frame will be shifted to the right by 100 pixels. The first 0 is the frame number, and then follows x-shift, y-shift, rotation and zoom factor.

[sorin86]

Thank you for your suggestion. It seemed to work for a lot of frames, but there were also some terrible jerks. As one can't spend hours and hours to detect the proper Deshaker settings (if there are any), the conclusion is that this software just isn't able to deal with such a situation which is not so uncommon.


https://dl.dropboxusercontent.com/u/58215671/dddd.png
with this settings before deshaker, it detected edges.
Then you can play with deshaker settings.
But finding good values for deshaker wasn't easy task for me.

[affter333]

thanks for the de-shaker, it's so useful !!

I'm still using it.

[dipje]

@sorin86: Programs like Mocha are for that kind of stuff. Plane and image-trackers, which track a certain object in the frame (instead of all the motion detected).
If you still have huge jerks there is little any kind of video-stabilizer can do with that. Huge jerk-movements are just that. You can even stabilize it 100% but you will end up with almost no frame left, or leave them in since they would ruin the overal 'cropfactor' of the stabilization. Cutting those huge jerks out before entering any kind of stabilization process helps a lot.

[StainlessS]

Has anyone ever seen DeShaker source code published anywhere ? (any version, but later better)

I have a requirement for spatial alignment/zooming, for sequence matching, I have a bit of an idea for rapid detection of spatial alignment and zoom,
but it may not work at all. Would like to see how someone who has some clue what he is doing, does it.

[guth]

It's not published (I hope). But if you describe your idea (privately if you prefer), I'll tell you what I think about it. I promise I won't steal your idea... :)

[StainlessS]

Hi Guth,

Was thinking that maybe use something like Zebra() Avisynth plugin:- https://forum.doom9.org/showthread.php?t=167663&highlight=zebra
(Arh damn, my images have gone from there too. [EDIT: Fixed images])

and scan on one or several of the (resized) bands(maybe minmaxdifference band), perhaps using something like RT_Stats RT_LumaCorrelation,
maybe in 2 parts vertical and 2 parts horizontal, to gauge shift and zoom (H and V separately).

EDIT: Then once rapid/rough estimate is done, and approx shift/zoom gotten then exhaustive subpixel shift/zoom
to get final crop/resize.
We dont actually have to do it at every frame, only per clip, but is complicated by fact that we have two different clips with
perhaps differing color/gamma.

[guth]

I don't know the exact problem you're trying to solve. "Sequence matching"... "two different clips"... you mean you have two clips of the same scene and you want to "time match" them? And you also want to match them spatially? (Does this have something to do with 3D stereo video?)

Anyway, things that I guess could be problematic with your idea is if objects move differently in the two clips, or if rotation is different. Also, if it's not a rectilinear lens (maybe you have 180 degree video with two cameras, for VR??) the "distorted" geometry might be a problem too.

And to find shift and zoom, are you going to guess values for these and see which set gets the best match? Guessing two unknowns at the same time can be time consuming, but maybe you'll do it hierarchically, at different scales?
You might want to look into something called "least squares matching" (not to be confused with "least squares fit") that can find a shift without guessing. I'm not using this in Deshaker, though, and I don't remember why.

In short, I believe your idea might work, but depending on the exact intended use it may not be very robust.

[StainlessS]

And you also want to match them spatially?
Yep, thats it, need to match spatially so can then match sequences, for eg replace bad quality sequence with similar good quality sequence of different clip.
Perhaps of different input clip size, up to about 32 pixels difference H or V. (3D, of no concern).
The two clips would be spatially aligned on a single in-sync frame, rotation does not come into it.

Was thinking, something like, split vertical into top and bottom, process top via shift/resize until find match (single vertical pixel column from zebra),
same again for bottom, calc shift/resize from two results. repeat for horizontal.
Then with rough estimates, go 2D subpixel to find and return best shift/resize, and intersection coords.

The Zebra thing is intended just to do away with the gazillions of 2D shift/resize/correlation ops, do it individually in two 1D passes
for primary estimate. [selecting best Zebra band via RT_YPlaneStdev() ].

I'll have a look for "least squares matching" , thanks very much for your time, I think I have enough for now. :thanks:

EDIT: For the particular immediate use, I have a good quality, good color clip with massive opaque logo, and a low res bad color clip
without logo and of different size and cropping. I want to match spatially, then do color matching with GamMatch (from GamMac thread)
and replace logo with resized, shifted, recolored, low quality area.
The recoloring part already works better than anticipated.
EDIT: We also have to spatially match with logo area chopped off and extrapolate crop and resize stuff.

it may not be very robust
Yep, I figured that :)

[guth]

I was wondering what kind of strange use this could be, but your edit explained it.

[StainlessS]

Hi Guth,

Just had a glance at your docs for DeShaker, the Rolling Shutter docs Motorola Defy as 848x480 widescreen: 52%,
Actual screen resolution is 854x480, but captures video highest rez at 640x480 4:3 (dont know if of any interest).


General
Complete name : C:\Z\2017-08-02_01-09-46_141.3gp
Format : MPEG-4
Format profile : 3GPP Media Release 4
Codec ID : 3gp4 (3gp4/mp41/3gp6)
File size : 5.10 MiB
Duration : 16 s 789 ms
Overall bit rate : 2 550 kb/s
Law rating : 0x00000000
Released date : 0
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10
Classification : 0x00000000

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Baseline@L3
Format settings, CABAC : No
Format settings, ReFrames : 1 frame
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 16 s 789 ms
Bit rate : 2 483 kb/s
Maximum bit rate : 2 000 kb/s
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Variable
Frame rate : 29.841 FPS
Minimum frame rate : 8.197 FPS
Maximum frame rate : 34.483 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.271
Stream size : 4.97 MiB (97%)
Language : English
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10

Audio
ID : 1
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 16 s 768 ms
Bit rate mode : Constant
Bit rate : 64.0 kb/s
Channel(s) : 1 channel
Channel positions : Front: C
Sampling rate : 16.0 kHz
Frame rate : 15.625 FPS (1024 spf)
Compression mode : Lossy
Stream size : 131 KiB (3%)
Language : English
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10


From Aida64

SYSTEM:

Manufacturer Motorola
Model MB525
Brand moto
Board jordan
Device umts_jordan
Platform omap3
Product MB525_RTGB
Installed RAM 512 MB

Device Features
android.hardware.camera
android.hardware.camera.autofocus
android.hardware.camera.flash
android.hardware.location
android.hardware.location.gps
android.hardware.location.network
android.hardware.sensor.accelerometer
android.hardware.sensor.compass
android.hardware.sensor.light
android.hardware.sensor.proximity
android.hardware.telephony
android.hardware.telephony.gsm
android.hardware.touchascreen
android.hardware.touchascreen.multitouch

CPU:

Core ARM Cortex-A8@800
Architecture MHz
Instruction Set 32-bit ARMv7
CPU Revision r3p2
CPU Cores 1
CPU Clock Range 300-800 MHz
Core 1 clock 300 MHz
CPU Utilization 7%
Scaling Governor ondemand
CPU ABI armeabi-v7a
AES Not Supported
NEON Supported

DISPLAY:

Screen Resolution 480 x 854
xdpi / ydpi 96 / 96 dpi
GPU Vendor Imagination Technologies
GPU Rednderer PowerVR SGX 530
Refresh Rate 68Hz
Default Orientation Portrait
OpenGL ES Version 2.0
GPU Version OpenGL ES-CM 1.1

ANDROID:
Android Version 2.1-update1 (Eclair)
API Level 7
Rooted Device Yes
Android ID xxxxxxx
Build ID JRDNEM_U3_2.21.0

DEVICES:

Camera
Resolution 5.0 MP (2592 x 1936)
Focus Modes auto, Macro, infinity, off
Flash Supported

USB Device - Motorola Inc Motorola Phone ()
Manufacturer Motoroloa Inc
Product Motoroloa Phone

SENSORS:
KXTF9 3-axis Accelerometer ????
AK8973 3-axis Magnetic field sensor ????
AK8973 Temperature Sensor 29.0 degrees C
ISL29030 Proximity sensor ????
Orientation Sensor Azimuth:???? / Pitch: ???? / Roll: ????
ISL29030 Ambient Light Sensor 2.0 lux


The Defy+ (with a software upgrade,) is supposed to support up to 720p, but I dont know if 16:9 or what, nor if update was released.
Only difference with ordinary Defy was the plus had a Bayer lens(red, some later Defy also had red), whereas plain Defy had SOC (green) lens.
Also, Defy+ had bigger battery and the (pretty much) same processor was clocked at 1GHz instead of 800MHz as in the plain Defy.

Plain Defy:- http://www.gsmarena.com/motorola_defy-3514.php
Defy+:- http://www.gsmarena.com/motorola_defy+-4098.php

Picked myself one up for £8.00 a couple of months back :) :- https://uk.webuy.com/search/index.php?sortOn=sellprice_asc&stext=phones+android+motorola+defy&section=
(Gonna try Jelly Bean on it when I get time, great little phone, me loves it).

[guth]

Do you mean that my rolling shutter listing for this mobile is misleading?
I just add the rolling shutter info that people send me. I think it's better if the end user uses that info as best as he can instead of me trying to correct it, without knowing what the guy who sent it to me might have done.

[StainlessS]

Dont know much about rolling shutter or whether affected by wrong screen size info (and cap size also different), just posted as I stumbled across your
doc (again), and had recently gotten a Defy.
Incidentally, in the last link above, there are a number of Defy's @ £5.00 and £8.00 and above, but many out of stock.
If you click on "By Availability - In Stock Only", you will see first on list at £8.00 is out of stock, but if you click
on it, they currently have one in Bromley and one "In-Stock On-Line".

My latest party trick is to play a movie then accidentally drop my defy in my beer :)
EDIT: It's an IP67 phone, water resistant to 1 Meter for 30 mins (with covers in place).

[trondmm]

Hi.

Is it possible to stabilize the image based on the position of a channel logo?

I have this video, that seems to have been the victim of a botched stabilization, and I want to reverse the damage as best as possible.

https://drive.google.com/open?id=0B1VlH492frJrQmhnWjl0RGFGLWc

In the first frame, the logo is in its proper position, and I'd like to "lock" the logo in this position, and move the video around to wherever the logo position takes it. I've tried to ignore the pixels outside the logo area, without success.

[guth]

I'm afraid I don't think you'll have much success in using Deshaker for this. And I actually doubt any tool can fix it in a good way. If it had been stabilized only vertically and horizontally it would have been easier, but this clip has rotation and zoom altered too, and I think the logo is too small to measure rotation and zoom accurately enough.

[trondmm]

Thank you for taking the time to look at this. I kinda expected the answer to be negative, but it was worth a shot.

[bxyhxyh]

Deshaker seems to ignore skipped n_scene when motion smoothness is set to -1.
Black bars don't get "resetted" after new scene n_scene changes.
Is it made that way or is it a bug?

[guth]

A bug. :o
To my defense, smoothness -1 is a pretty experimental feature.
But if you really, really want a fix, I can send you one.

[bxyhxyh]

But if you really, really want a fix, I can send you one.

Yes please, I would like to. Any other smoothness values don't work quite good on my source.

[guth]

(bxyhxyh, I sent it in a private message.)

[MysteryX]

This is the best deshaker out of the various options I tried. The downside is that it is only a VirtualDubFilter, and needs RGB32 instead of YV12 input -- although it can be used in Avisynth with VirtualDubFilter. Also the source code isn't available.

Would it be possible to have a version for Avisynth and VapourSynth? That would be very useful.

[guth]

"Best deshaker"? It should hopefully be the only deshaker! When I made Deshaker I googled "deshaker" and got one single hit; some guy with that name. It's funny that some people seem to be using it as a general term for video stabilization now. :)
Anyway, thanks.

I understand your wishes completely. I've considered making a native AviSynth plugin and I've looked into how much work it would be to support more color spaces. The problem is that I've become a bit lazy, or at least prefer doing other things on my free time. Also, I'm not quite ready to release the source code. Sorry... :(

[guth]

Btw, about color spaces. Maybe someone could write a simple plugin that converts YV24 into RGB32 by simply assigning R=Y, G=U and B=V, or similar. After that, you could deshake (or other things) RGB32 video that's really YUV, and then do the reverse "conversion". It's not an ideal solution, because the colors would look like crap in Deshaker, but I can't see why it shouldn't work. You would get no quality loss due to color space conversions that way.
Just a thought...

[shekh]

I could add this fake RGB in vd2, sounds funny. Would it actually be useful? :) Btw, black borders will turn green borders, and what about color mask detection.

[shekh]

Just noticed a bug:
add filter->deshaker
enable color mask, open dropdown
click cancel in filter, cancel in filters list -> crash
No input file needed, tested in 1.10.5

[guth]

Yes, you would have to select a different color for mask detection. As for black borders, maybe you could use a lossless conversion where black is still black?

And the crash... Yes, indeed, I can reproduce it. (But not when using a Deshaker built with a newer compiler version. Probably just luck, though.)
We'll see if I care enough to fix it. It's not a procedure you'd do often. Thanks for letting me know, though.

[StainlessS]

Guth, we appreciate everything that you have done for us, you are the best, please, let the new guys have a go at making it better,
we look forward to your decision, thanks ether way, we does love you. :)

[MysteryX]

Btw, about color spaces. Maybe someone could write a simple plugin that converts YV24 into RGB32 by simply assigning R=Y, G=U and B=V, or similar. After that, you could deshake (or other things) RGB32 video that's really YUV, and then do the reverse "conversion". It's not an ideal solution, because the colors would look like crap in Deshaker, but I can't see why it shouldn't work. You would get no quality loss due to color space conversions that way.
Just a thought...
The problem is that YV12 has U and V planes at half resolution, while RBC32 has 4 planes at full resolution. Even a fake conversion can't be done without rescaling U and V planes with rounding errors. And then you lose performance by processing more data than necessary.

Any specific reason why you're not comfortable releasing the source code?

btw I'm glad to see you're still active and responding to messages. The authors of lots of old plugins are long gone.

[guth]

The problem is that YV12 has U and V planes at half resolution, while RBC32 has 4 planes at full resolution. Even a fake conversion can't be done without rescaling U and V planes with rounding errors.
Yes, but I think those errors are negligible. Deshaker still has to resample all pixels, which destroys quality a lot more. If it's doing it in 4:4:4 instead of 4:2:0 mostly just means it's doing it in a higher resolution than needed.
Not being able to process YUV video at all could however result in banding problems, which I think could actually be seen in some cases. I haven't really examined this, though.

And then you lose performance by processing more data than necessary.
A little, yes.

Any specific reason why you're not comfortable releasing the source code?
Partly because I've sold Deshaker source code licenses to a few companies in the past, and that might be harder to do if the source was "more open". And partly I guess because it somehow feels better to have full control over it.

[isidroco]

I've perfectioned my batch which automatically [generates LOGs, deshakes] and compress A/V of all video files in folder with plenty of options. For anyone interested it's here: https://sourceforge.net/projects/virtualdub-batch-video-deshake

[MysteryX]

I'm also working on implementing it in the Yin Media Encoder -- but it's implementation is more complicated than other filters due to requiring a pre-scan, and it's not so simple to do full preview for only a portion of the clip, and the fact that the second pass (in x86) requires MP_Pipeline to split into 2 processes to avoid running out of memory; adding to that the RBG32 conversion and a couple of other technical details.

I was also considering whether I could switch to Vapoursynth to avoid all kinds of MT and memory issues, but there are filters that don't work in VapourSynth:
- Deshaker
- my own filters (AvisynthShader, FrameRateConverter)
- Audio filters

[MysteryX]

Converting YV12 to fake RBG32 is very easy.

1. Convert to YUVA444P8
2. Use dummy filter to change video format to RGBAP8 without changing content
3. Convert to RGB32

RGB32 is flipped up-side-down, but is RGBAP8 up-side-up? If so, even the flipping will be handled by this method. If it's inserted within a script that reverse it again after, then even the previews will look good.

As for VapourSynth, someone pointed out that VDubFilter source code isn't complicated at all, it would be very easy to port to VapourSynth for native support of VirtualDub filters.
https://github.com/pinterf/AviSynthPlus/blob/master/plugins/VDubFilter/VDubFilter.cpp

[redfordxx]

Hi.
Thank you for wonderful work, in a first place.

I am just curious:
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...

[nji]

Hi.
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...

That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.
It is as your movie shows only flat surfaces,
or as if you're using an extreme telephoto lens.
DeShaker does very good results for that.
Rotation on x and y axis (called panning and tilt)
is simply x and y shift then.

But the less the condition holds
(i.e. you have 3D objects or using a lens that's not telephoto)
the operation on the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).

So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there are at least two serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146
https://forum.doom9.org/showthread.php?p=1747268#post1747268

[redfordxx]

That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.

The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...

the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).

I don't know about that...
Especially if there would be option to limit x and y rotation.

So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there is at least one serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146

(https://forum.doom9.org/showthread.php?p=1747268#post1747268
just uses DeShaker's analysis data and does some lens correction.
So: no 3D at all.)
But they still use deshakers "2D" data.

[guth]

The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

To be honest, I'm not sure which movements you mean. If we talk movements of the video, basically only one kind of rotation exists. And if we talk movements of the camera, Deshaker handles rotation of the camera along all axes (+zoom), but nothing else. Moving the (center of the) camera forward, backward, right, left, up, down often works somewhat well too, though.

Anyway, nji is right. Deshaker both analyses and transforms the video in 2D only, which means wide angle video will look ugly when there is heavy shaking.

[nji]

The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...

I don't know about that...
Especially if there would be option to limit x and y rotation.

But they still use deshakers "2D" data.

First of all:
You are right, I did a mistake in my last post (#442):
"Deshaker3D" uses DeShaker pass1 data, too.
(So maybe both tools do some kind of "3D-magic"?
And then "Lens transform" will be probably more sophisticated
with alpha consideration and rolling shutter/ "skew"?)

Second:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.
It does only 2D-operations:
left/right, up/down, resize and (2D-) rotation.
These 2D-operations do ("by chance") match the effects that
the corresponding 3D movements would have on the 2D projection (= frame).

Thinking it over ... DeShaker very well possibly could
take into account the 3D rotation at x- and y-axis
(or even an arbitrary axis) too.
(Would be a kind of magnifying "above" and reducing "below"
the axis I guess).
Seems possible, but isn't done (and won't be ... believe me :))
But maybe/ probably the other mentioned tools will?

[nji]

I do hope it's not too much OT if I post a summary of my (small)
experiences of the filter?

DeShaker (VD)
Pros:
- excellent results on all but wide angle scenes (no 3D/FOV accounting)
- optional rolling shutter accounting
- optional progressive/ interlaced source
- accounting sophisticated border and logo handling (by neighbour frames)
- many options to handle difficult source/ get optimal results
- excellent description of all that
- multithreaded (? nevertheless takes the most times of all filters)
Cons:
- Bad results on wide angle scenes (even worse than the shaky sources)

Lens Transform (VD)
Pros:
- accounting of field of view (FOV)
- accounting of simple border handling
Cons:
- no accounting of logo handling (have to delogo before)
- no interlaces sources (have to convert before)
- Rolling shutter accounting ("skew") "not implemented completely" (although active check-box)
- Unsufficient description (at least for me)

Deshaker3D (AviSynth)
Pros:
- Accounting of FOV
Cons:
- no border/ logo handling
- no rolling shutter accounting
- no interlaces sources (have to convert before)
- needs format conversion (= quality loss)
- no multithreading (?)

Deshaking (Shotcut)
Does about the same as DeShaker but with no options and worse results.

Conclusion:
The best (for non-wide angle scenes) does DeShaker.
If results are unacceptable in DeShaker (due to wide angle)
the best choice is Lens Transform (if rolling shutter source: try Deshaker before with all smoothness = 0).

4 runners at the launch, each performing good on their
respective distance, but some targets aren't reached by only one.

[nji]

https://forum.doom9.org/showthread.php?p=1895181#post1895181

[redfordxx]

First of all:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.

No, I didn't.
I thought if there will be 3D added, there could be these limits added so that it will not lead to worse results ( which I still don't know what you refer to).

(Would be a kind of magnifying "above" and reducing "below"
the axis I guess)

Exactly.

Seems possible, but isn't done (and won't be ... believe me :))
But maybe/ probably the other mentioned tools will?
You mean even if I will find some time and make a proof of concept that it could work, there is no one who will continue the deshaker?
Because other projects will not help without better 1st pass.

[nji]

Deshaker3D and Lens Transform do something like that.

But of course you're free to try whatever you want to.

I'm just a user and not the one to answer about the availabilty of source code.

[redfordxx]

Maybe I test something in Avisynth.
But first, can someone tell me, how can I get rid of the black frames in the beginning of the video, when I
"Use previous and future frames to fill in borders"?
Only Avisynth does that, VDub is ok...