"Defish", barrel and pincushion distortion correction filter

[wonkey_monkey]

Updated 2010/02/21 - version 1.1

Download: defish (http://horman.net/defish.zip)

It's taken me several years since my first post on this subject, but I've finally written a usable fisheye distortion correction plugin:

http://horman.net/avisynth/defish.jpg

It uses spline16 interpolation.


defish (v1.1) for Avisynth 2.5
(c) David Horman, 2010

This filter can be used to correct (or induce) barrel and pincushion
distortion.

USAGE
-----

The filter takes the following parameters (in addition to the input clip):

[unnamed]:
Input clip. Must be RGB32 and have even width/height.

fov:
useful range: -180.0 - 180.0
default: 0
The horizontal field of view, in degrees. Positive values will
correct barrel distortion (induce pincushioning). Negative values
should correct pincushion distortion (induce barreling), but note
that this may currently be implemented incorrectly.

scale:
range: 0.0 - infinity
default: 1.0
An adjustment parameter which scales the image after correction.

aspect:
range: 0.0 - infinity
default: 1.0
The pixel aspect ratio of the input clip.

direction:
values: "x", "y"
This parameter restricts the correction to either horizontal or
vertical only.

scaling:
values: "fitx", "fity", "fitxy"
This parameter automatically scales the image to fill the image
horizontally, vertically, or both. The "scale" parameter above is
still effective.

The default behaviour when using fov is to scale the image such
that the central area stays at the original scale.

This parameter is not effective when using the polynomial values
below.
a,b,c:
sensible range: 0.0 - 1.0
Parameters for correcting distortion polynomially. These are only
effective when fov=0 (which is the default value).

a+b+c must be less than or equal to 1.0 when pin=true.

pin:
boolean
Set this to true to correct pincushion distortion, and false to
correct barrel distortion - this may seem the wrong way around but
it was chosen to match the behaviour of the similar parameter of
V.C. Mohan's DeBarrel filter.


David

[Lyris]

Wow! I think I have some footage somewhere that is begging for this. Looks like a fantastic tool!

[wonkey_monkey]

It looks there are still some bugs to pin down - certain extreme values of fov aren't behaving as I thought they were, and the filter can crash (possibly only when using the scaling parameter), but it should be fine for any "sensible" usage.

David

[StainlessS]

Looks very nice, think VcMohan has a filter called "DeBarrel" that does same type thing.
Never tried it.

[scharfis_brain]

defish is much faster than debarrel, but defish still is slower than the 'barrel distortion' filter from Emiliano Ferrari.

But I liked the different parameters for the distortion of debarrel.

[wonkey_monkey]

I could probably add those parameters pretty easily, though I find it more intuitive to work in degrees. As far as speed goes, there is probably still plenty of room for optimization.

David

[scharfis_brain]

this sound promising.
Also I like the spline interpolation since it is much smoother than linear or cubic distortion.

[um3k]

Would it be possible to add the option to correct for cameras with an offset sensor? I.E. one that isn't centered on the optical axis.

[wonkey_monkey]

Would it be possible to add the option to correct for cameras with an offset sensor? I.E. one that isn't centered on the optical axis.

Not really, because the filter only calculates remapping and spline16 coefficients for one quarter of the image, which is why it can only accept evenly dimensioned images.

You can addborders to put the center in the center, if you see what I mean, then crop afterwards.

David

[um3k]

Yeah, that works. I was just trying to avoid extra steps (for myself). :P

[wonkey_monkey]

First post updated - defish can now also take a,b,c parameters similar to DeBarrel (see updated readme). It should also be slightly faster (but consumes a little more RAM).

David

[Ashley_Pomeroy]

I'd just like to publicly thank David Horman for this plugin. It's esoteric, but I've been experimenting with defished fisheye images for a couple of years, and I've always wanted to try it with video; once I got a Canon 5D MkII I had a go. Until now I've been splitting the footage into individual frames, defishing them as a batch job, and then reassembling them into a movie again, but this plugin works a treat and is much faster.

In fact I've written a blog post about it, here, complete with a sample video of me on my bike:
http://women-and-dreams.blogspot.com/2010/04/defishing-video-2.html

Here's a shot of it in action:
http://2.bp.blogspot.com/_ecpW7iDmhG0/S8HQA2TvsxI/AAAAAAAABUs/LuzxGgL2gxY/s1600/Defishbeforecrop.jpg

EDIT: It'd be interesting to run it on the "HAL's eye view" sequences from 2001: A Space Odyssey, but I can't find my DVD. There must have been lots of trippy films in the early 1970s that are crying out to be defished. I'm reminded of the bloke who destretched the slit-scan sequence at the end of the film.

[wonkey_monkey]

Hey, great to see someone's found a use for defish :) Just read through that blog post...

The tricky thing is that it only works with AVISynth, a free, command-line video editor which can be run independently, or invoked as part of VirtualDubMod, an older version of VirtualDub with some novel features.

AviSynth works fine with new versions of VirtualDub, too.

I chose 800x400 because they are good whole numbers, and I like the colours. Eight is yellow and four is blue and zero is water.

Do you have synesthesia?

David

[Wilbert]

Hi David,

Look interesting. Would you be willing to release your plugin under an open source license?

[wonkey_monkey]

Don't see why not - not really sure how I should package it up or anything. I've probably taken a few too many shortcuts with my VC++ Project, like having it compile straight into my AviSynth plugins folder - should I just put the .cpp file somewhere, maybe?

David

[Wilbert]

Don't see why not - not really sure how I should package it up or anything. I've probably taken a few too many shortcuts with my VC++ Project, like having it compile straight into my AviSynth plugins folder - should I just put the .cpp file somewhere, maybe?
You could package the .cpp file, the documentation, the license file and the plugin itself. You can put something like this in the source file (at least if you release it under the GPL, like most plugins):


/*
"Defish", barrel and pincushion distortion correction filter for AviSynth.

Copyright (C) 2010 David Horman

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

The author can be contacted at:
...
*/


License file for GPL. (http://www.gnu.org/licenses/old-licenses/gpl-2.0.html)

If you want to release it in the public domain, you could package the .cpp file, the documentation and the plugin itself.

[wonkey_monkey]

I've updated the zip file linked above (and here (http://horman.net/defish.zip)).

I make no apologies for my coding style! ;)

David

[Zep]

In fact I've written a blog post about it, here, complete with a sample video of me on my bike:
http://women-and-dreams.blogspot.com/2010/04/defishing-video-2.html




Great video nice job! music and speed up are really cool. What you need though is a surprise ending for those that watch the whole way. Like say a buxom blonde that you ride by and get caught looking so you then turn your head forward again but too late you hit a bench and flip over the handle bars all caught on camera. You can fade it to black just before you hit the ground. THAT i would love to see! :D


(haha all kidding aside great video keep making them!)

[Efenstor]

Just made a multithreaded version of Defish v1.1. Enjoy!
http://www.efenstor.net/defish_mt.zip (dll+source)

[wonkey_monkey]

Just made a multithreaded version of Defish v1.1. Enjoy!
http://www.efenstor.net/defish_mt.zip (dll+source)

Nifty, thanks!

David

[Efenstor]

Just a little advice for those who are troubled searching the a,b,c parameters for their conversion lenses: you can use PTGui to find these automatically. Just make a panorama of 5-6 frames with huge overlaps (70-80%). Hugin also does the job but seemingly fails on converters wider than x0.5. In fact, PTGui also does it only with a trick: load the images, click "Align Images", switch to "Lens Settings", click "EXIF", enter the conversion factor, also turn off "Auto" and select "Rectilinear", then click "OK", switch back to "Project Assistant" and re-optimize the project. Then in "Lens Settings" you'll see the correct a,b and c. You shouldn't do this trick for narrower lenses.

[Efenstor]

And another little advice for newbies. For high-quality multithreaded defishing of interlaced sources I recommend the code similar to this:

MT("""Yadifmod(mode=1, edeint=nnedi3(-2))""", threads=6, overlap=16) # Set "threads" to your number of cores
ConvertToRGB32().Defish(a=0.022924525, b=-0.096146197, c=0.035261053).ConvertToYUY2() # Set a,b,c to the values for your lens
SeparateFields().SelectEvery(4,0,3).Weave()

Plugins needed: MT, Yadifmod, nnedi3.

UPDATED: thanks to Gavino for a smarter weaving method.

[Gavino]

For bottom-field-first footages edit WeaveBob: exchange 0 and 1 at the end of PointResizes and change b,a to a,b in the Interleave.
The body of WeaveBob() can be replaced by
s.SeparateFields().SelectEvery(4,0,3).Weave()
Then you just need to precede the call to it by AssumeTFF() or AssumeBFF() as appropriate.

[Efenstor]

Thanks Gavino! I totally forgot about that solution, remembered there was something like that, but couldn't find it again. Hence used the "brute force". :) Updated my post.

[scharfis_brain]

I also suggest using PC.601 colour conversions, because they wont shrink and expand 0..255 to 16.235 and vice versa.
Thus you'll avoid colour banding:

ConvertToRGB32(matrix="PC.601").Defish(a=0.022924525, b=-0.096146197, c=0.035261053).ConvertToYUY2(matrix="PC.601")

[smok3]

is there a way to go from this:
http://imagebin.org/154029
to this:
http://imagebin.org/154030
?

[Kisa_AG]

Hello!

I' m using GoPro 3BE, I decided to make a defish for it.
I found a, b, c coefficients with Hugin's calibrate_lens_gui.exe, it works good.
I tried different projections in Hugin and found that standard Rectilinear projection is not the best choice (at least for me). Coefficients looks fine - lines are straight inside the picture, but projection looks not natural enough.
I tried Hugins different projections and found some that works visually much better than Rectilinear, i.e. Merkator or Equirectangular projections.
Is there any way to make defish with Avisynth and with projection other than Rectilinear? Let's say, with these Merkator or Equirectangular projections?

[wonkey_monkey]

There's no way to do that with defish, but if you're feeling mathematical you could try working out the formulae for yourself and use xyremap (http://forum.doom9.org/showthread.php?t=166087). It won't be quick, though.

[Kisa_AG]

There's no way to do that with defish, but if you're feeling mathematical you could try working out the formulae for yourself and use xyremap (http://forum.doom9.org/showthread.php?t=166087). It won't be quick, though.

Ok, thanks! It's worth to try.
I think it should be quicker than to split clip into separate frames, save them as TIFF files, batch defish with Hugin batch converter and to join them back to clip...

[Kisa_AG]

There's no way to do that with defish, but if you're feeling mathematical you could try working out the formulae for yourself and use xyremap (http://forum.doom9.org/showthread.php?t=166087). It won't be quick, though.

Hello!

Ye, it was a long way for this function, but I did it!
I made a function which make defish with Lambert Cylindrical Equal Area Projection I've asked here a year ago.
It's based on xyremap, thanks davidhorman for such a powerful filter!


#############################Cylindrical_Equal_Area v.1.0 RC1 ##############################
function Cylindrical_Equal_Area(clip clp, float "zoom", float "FocalLenght", float "MatrixWidth", float "StandardLatitude_Fi_s", Int "VCropping")

Cylindrical_Equal_Area function v.1.0RC1 by Kisa_AG, (c)2015.

This function makes picture defishing with Cylindrical equal-area projection.

Necessary plugins:
1. xyremap v0.2 by davidhorman. http://forum.doom9.org/showthread.php?t=166087 , http://horman.net/xyremap.zip
2. ColorIt by V. C. Mohan. http://www.avisynth.nl/users/vcmohan/ColorIt/DrawLines.html

Typical usage:
ConvertToRGB32(matrix="Rec709")
Cylindrical_Equal_Area(FocalLenght=3.5, zoom=0.97, VCropping=54)
ConvertToYV12(matrix="Rec709")

Usage instruction:
1. Input color format must be RGB32
2. Set the desired StandardLatitude_Fi_s. For the first time it's better to leave it by default.
3. Set FocalLenght and MatrixWidth. Then you can fine tune it visually, if you want.
4. Set zoom=0, open script in VirtualDub and move forward to find the number of frame,
which have no left and right black areas.
5. Set zoom=frame_number_founded/100.
6. Set VCropping=-1, open script in VirtualDub and move forward to find the frame,
which have no top and bottom black areas between cropping lines.
Read the correct cropping value in upper left corner of picture.
7. Set VCropping=cropping value founded.


A Zoom of 1 has no effect. Passing a Zoom of 0 returns an animated version
where Zoom=current_frame/100 (so if frame 96 looks good, use Zoom=0.96).

VCropping crops vertical size of picture to remove black areas. So it changes the aspect ratio of picture.
A VCropping of 0 has no effect. Passing a VCropping of -1 returns an animated version
where VCropping=current_frame+1 (so if frame 95 looks good, use VCropping=96).
Cropping line width is 1 pixel, the cropped picture will be strictly between the upper and lower lines.
Pixels "behind the line" are NOT included into final picture.
For convinience the correct VCropping values for the final clip is shown in upper left corner of picture.

FocalLenght and MatrixWidth Default was set for GoPro 3 BE

StandardLatitude_Fi_s Default - Lambert cylindrical equal-area projection

StandardLatitude_Fi_s means Standard Latitude for Special cases
of cylindrical equal-area projections are summarized in the following table (Maling 1993):

0 degrees - Lambert cylindrical equal-area projection (this is my choise, the reason why I decided to make this function)
30 degrees - Behrmann cylindrical equal-area projection
37.383 degrees - Tristan Edwards projection
44.138 degrees - Peters projection
45 degrees - Gall orthographic projection
50 degrees - Balthasart projection


Here it is:

Function Cylindrical_Equal_Area_v.1RC1.avsi with readme: http://goo.gl/8LY6uN
Direct link: https://www.dropbox.com/s/ladvkp9rlwx28bg/Cylindrical_Equal_Area_v.1RC1.zip?dl=0

Full size test pictures (2.7К) - original and defished with Lambert cylindrical equal-area projection:
http://goo.gl/pB4Pez
Direct link: https://www.dropbox.com/s/6xwhiw8msog49u5/Test_Pics_Big_Theater_GP3BE_2.7K.zip?dl=0

Sample videos defished with Lambert cylindrical equal-area projection:
Small frame size 1280x612p25, 9MB - http://goo.gl/6foqUE
Full frame size 2.7K25p, 31MB - http://goo.gl/5aclCK

And here are four pictures which shows (I hope so) the advantage of Lambert cylindrical equal-area projection over the standard defishing.

The original picture from GoPro 3 BE is the lower left picture.
The upper left is a picture defished with Lambert cylindrical equal-area projection.

And on the right you can see pictures defished with defish_mt.dll.

PS: Lambert cylindrical equal-area projection is very suitable for defishing of 4:3 frames because the output frame is very close to 16:9 apect rate and it's not necessary to crop big parts of picture.

[wonkey_monkey]

The first two shortened URLs have been disabled by Google.

[Kisa_AG]

The first two shortened URLs have been disabled by Google.

Hmmm, strange. There is nothing there except .avsi, readme.txt and sample jpeg file with screenshot from my video (MY own, made by me!!!). I made a request to google about it.

I will put avsi here as text little bit later.

[Kisa_AG]

I added direct links to Dropbox to my post about my function.

[Kisa_AG]

bHello davidhorman!

I made a separate thread in Avisynth Usage about "Defishing" with different map projections: http://forum.doom9.org/showthread.php?p=1713391#post1713391. It's more logical, I think.
So you may delete my messages if you think it's not necessary here.

[StainlessS]

This will give you a different perspective (Explore This Area):- https://www.google.co.uk/maps/place/236+Earls+Court+Road+W.8,+Kensington,+London+SW5+9AA/@51.5286416,-0.1015987,11z/data=!4m2!3m1!1s0x48760f8b6515e8e5:0xc90aaef0d1664217?hl=en

(X marks the spot)
EDIT: I am the real Doctor, no really.

[tormento]

Sorry to resurrect a old thread. I have just received a GoPro Hero7 and I am playing a bit with its videos.

Anyone here has "GoPro 6/7 baked" parameters to have proper aligned videos? I would avoid trial and error, if possible.

[wonkey_monkey]

If I remember correctly - and I might not - the a, b, and c parameters follow the de facto convention as laid out here:

https://wiki.panotools.org/Lens_correction_model

So if you can find such parameters elsewhere for your camera, they should work for defish as well.

[tormento]

So if you can find such parameters elsewhere for your camera, they should work for defish as well.
Thanks but I can't find them.

I have found this (http://argus.web.unc.edu/camera-calibration-database/) but no idea how to transform them.

[wonkey_monkey]

S1, S2, S3 in the fisheye table may correspond to a, b, c. I'm not sure what the C and D parameters mean.

[Kisa_AG]

Anyone here has "GoPro 6/7 baked" parameters to have proper aligned videos? I would avoid trial and error, if possible.

There is an utility calibrate_lens_gui in Hugin - Panorama photo stitcher.
So you can create real "lens profile" for your particular camera at particular working mode.

[shekh]

afaik Hero7 has lens parameters factory calibrated and exposed in video metadata, if you could find way to get it and convert to appropriate equation. Just an idea.

[tormento]

afaik Hero7 has lens parameters factory calibrated and exposed in video metadata, if you could find way to get it and convert to appropriate equation. Just an idea.
I can extract GPS, acceleration etc but no idea about lens properties.

[tormento]

I have found the FOV table (https://gopro.com/help/articles/question_answer/hero7-field-of-view-fov-information) for my camera.

Any idea how to transform from angles to polynomial?

[an3k]

Hello!

Ye, it was a long way for this function, but I did it!
I made a function which make defish with Lambert Cylindrical Equal Area Projection I've asked here a year ago.
It's based on xyremap, thanks davidhorman for such a powerful filter!

Excuse my probably stupid question but can I use your script to "defish" into "Equirectangular" for eg. VR180 videos? What projection (degree) would I have to use? Thanks A LOT!

[vcmohan]

Today I happened to see this thread. I have some comments:
1. Barrel and Pincushion distortions can be removed by my DeBarrel plugin. It has several features specially developed for requirements of an academician of UK.
2. Still images of paintings or buildings will show the distortions clearly, as they have vertical and horizontal lines as frames etc;.. The amount of distortion may vary from frame to frame depending on the distance of the object and orientation. So for each image which needs correction a,b and c values need to be determined. For this my plugin has a test mode in which these values can be varied in a linear range automatically and the distorted lines which will be straightened are shown superposed on the image. Best fit values to be used. This may take some tens of minutes. The speed of the actual process is therefore has no significance.
3. Videos with Fish Eye are used to project on the inside of a spherical half dome as in a planetarium, Disney World. People stand in the center of the dome to view. As such the special projector need to be placed near the wall. All calculations / projections made assuming center as projection will no longer be valid. There are some confidential copyrighted files which calculate for every pixel what should be the new place and what luma correction is to be done to account for varying distance of screen from projector lens. I have a plugin specially made for such situations which were being used by some planetariums. So this is not released on this forum.

[Selur]

@wonkey_monkey: any change you could create a Vapoursynth port of defish?

[kedautinh12]

Any chance have x64 for avisynth too

[kedautinh12]

I findout x64 ver, thank theChaosCoder
https://github.com/avisynth-repository/Defish

[Reel.Deel]

I findout x64 ver, thank theChaosCoder
https://github.com/avisynth-repository/Defish

It was originally posted here:
Here (https://www.dropbox.com/s/x19w62kjzjqgomo/defish_mt.7z?dl=1) it is, I deleted it by mistake, sorry.

Followed by:
Now also on github https://github.com/avisynth-repository/Defish
(open for everyone, pm me if you want to be added aka write permissions)