Panorama Filter for VirtualDub needed

[nji]

Hello.

I need a way to convert a movie from rectilinear to fisheye mapping
and vice versa. Like the "panorama tools" from Dersch.
http://www.panotools.org/dersch/
I searched but didn't find ... a filter for VirtualDub, or for
avisynth (although not familiar with that), or another way.
Any help appreciated.

[StainlessS]

Perhaps of interest.

Work in progress, a few posts before your new thread. :- http://forum.doom9.org/showthread.php?t=172886

Deshaker3D (avisynth, might be of interest):- http://forum.doom9.org/showthread.php?t=171051

DeFish:- http://forum.doom9.org/showthread.php?t=152860

Some more:- http://avisynth.nl/index.php/External_filters#Rotation.2FShear.2FSkew.2FPerspective

[nji]

Thank you very much.

I already ran upon this and that thread, but as I never used avisynth
I tried to find something for VirtualDub directly.
Now I see the "Lens Transformation Tool" seems to be capable of the
mapping to fisheye and remapping to rectilinear.

Actually I "need" this, as I recently observed shortcomings of DeShaker
in wide angle scenes (kind of ... wrangling, wobbling).
Deshaker's author confirms to that, but stated it would be very much
work to include field of view (FOV) consideration in DeShaker.

To me it seems it is impossible in principle, as for that one would
need complete 3D information, which we don't have.

But my intuition said to try remapping the original movie from its
rectilinear mapping "towards" something where the edge part is
enlarged (something like fisheye - "Equisolid angle (equal area)".
(https://en.wikipedia.org/wiki/Fisheye_lens#Mapping_function).
For the whole movie. Save lossless.
Do DeShaker. Save Lossless.
Back-map.
(and cry about the rounding errors ;-)

An intuition only ...

[shekh]

Hi,


To me it seems it is impossible in principle, as for that one would
need complete 3D information, which we don't have.


It is difficult to change view point. However it is possible to do everything else (rotation around fixed point). No objects need to reveal another side, it is the same rays coming to same projection center.


But my intuition said to try remapping the original movie from its
rectilinear mapping "towards" something where the edge part is
enlarged (something like fisheye - "Equisolid angle (equal area)"


You may get some partial improvement but for generic "perfect" result you need math that you can reason, not guess :)

Unfortunately I made a huge pause in my project. Now it is barely exciting, given there exists omni with full 360 capture and whistles.

[nji]

Hi.

Yes, of course I agree, at last there is math.
But with me there is first an idea/ imagination/ intuition ... then a try ...
If success then the math :)

To me it seems the problem with deshaking wide angles is about the mapping. If the object distance would be (near) infinite (FOV about 0) there would be no "wrangling" when camera moves. So my idea to re-map. But this seems to mean a change of view point. Which means revealing/ hidding objects = impossible. But why not try an improvement at last ...

Including perspective consideration into existing DeShaker would mean to write it anew I guess. But maybe the (literal) work around does ... as I described above?

"Barely exciting with all the 360 captures around"?
Hey, and what about all that millions of movies taken since that devices arose? Grandpa's birthday in the 70ies, in his small home, wide angled and shaky like hell (because of the drinks :)
And what about all that encouragements in the thread of your project?

I'll try that "Lens Transformation Tool" for my re-mapping-rounding-error-producing-thing until then. :)

[shekh]

Somehow vd2 ate all my brain.
Nevertheless, LensTransform project ought to continue one day or another, because I have tons of dusted shaky videos I recorded in all cool places :)
Here some early notes I made, maybe interesting for you :) http://shekh1.blogspot.com/

[nji]

Thx to you! :) :)

I'll post here when I have any kind of results.

[wonkey_monkey]

Thank you very much.

Actually I "need" this, as I recently observed shortcomings of DeShaker
in wide angle scenes (kind of ... wrangling, wobbling).
Deshaker's author confirms to that, but stated it would be very much
work to include field of view (FOV) consideration in DeShaker.


Deshaker3D was made to solve that problem, although it's not nearly as configurable as Deshaker itself. You don't need 3D information, you only need to make a 3D transformation of the image.

[nji]

I never used AviSynth since now. I would learn it if DeShaker3D would do what I'm looking for.
But still it doesn't, due to lack of rolling shutter consideration and filling of border areas.
The videos I have are both rolling shutter and global shutter, and with DeShaker it makes a different if I turn it on/off.
The "invented" borders:
I could crop them off, but then lots of other wanted frame info would be cropped too. That's why I use DeShaker's "invent border" feature.
So DeShaker is not so good with wide-angle lenses.
And DeShaker3D is not so good (?) with RS and border handling.
Sigh ...

[StainlessS]

Dont know what the 'invent borders' thing does but assume that it is like Avisynth Padding script function.

Perhaps try add padding before DeShaker3D(), I'm interested to know if it has any +ve effect.


# no requirements for Padding
function Padding(clip c, int left, int top, int right, int bottom) {
# Didee: http://forum.doom9.org/showthread.php?p=1596804#post1596804
# eg, Padding(32,32,0,0).Padding(0,0,32,32)
w = c.width()
h = c.height()
c.pointresize( w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom )
}

# eg

############

#AviSource("...\myfile.avi")
ColorBars().Killaudio # Quicktest

PAD=32 # whatever (multiple of 4 probably best)
Padding(PAD,PAD,PAD,PAD)

#
#... # Whatever filter
#

Return Last
############


It may not work at all [probably stop DS3D from working at all, just curious and hope that I'm wrong].
(Make padding only just big enough to swallow black borders).
(the 'invent borders' thing probably does as padding but AFTER skewing the images, instead of before [ie skewed padding],
Maybe David is listening :) ).

[nji]

I'm sorry to have drawn you to the wrong path with my light-minded wording "invent borders".

Actually DeShaker is much more sophisticated than just inventing borders, but it can take the content of previous and following frames to fill borders of the actual frame. So much more than just a padding. It's the best that can be done at all. Better wording would perhaps be "borrow borders".

[Edit: Still as long there is no high-quality borrowing as in DeShaker the padding will be an improvement I may think. If it works]

[poisondeathray]

I think you're referring to deshaker's border fill option, from previous and future frames

If you correct for a barrel / fish eye lens distortion , the problems are 1) image will be softer (both from the transformation, and usually because of zooming in) 2) loss of borders anyways - because they are areas that are not captured

The "math" is more correct for tracking if you correct for a lens distortion . But you usually have to zoom in and crop anyways because you will have ugly black curved borders - the areas that were not captured. If you use border fill on that - it will be filled with messed up black regions. So the "border fill" option in deshaker will not necessarily be improved and you will have loss of details, both from the zooming in and from the transformation (resizing softness) , loss of peripheral details