Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VirtualDub, VDubMod & AviDemux

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th November 2015, 13:43   #1  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Lens Transform tool (work in progress)

This is plugin for VirtualDub.
It combines all of the following:
* Remove lens distortion using calibrated profile. Can use Adobe fisheye lens profile (used in Lightroom etc).
* Stabilize camera motion. For analysis I just use Deshaker log file.
* Apply arbitrary animated rotation (3d), zoom, barrel/pincushion.
* Crop/extend image size.
Rolling shutter correction would be nice, but atm it is not finished.

The image transformation may combine up to all these steps: convert from fisheye to rectilinear, remove skew, apply 3d rotation/zoom, apply barrell, convert from rectilinear back to fisheye. In all cases it is a single bilinear lookup directly from source (no intermediate bitmaps are rasterized inbetween). In other words it must be high quality.

There is no automatic path creation like found in Deshaker. Instead I make reasonable UI to do whatever path adjustments manually (animate rotation/zoom). Performance is quite good for interaction. Also there is manual shake correction for cases where Deshaker fails.

Right now I am satisfied with every aspect except skew correction, it is quite usable.
But before publishing I better polish something about lens profiles.
Anyone interested?

Example screenshot:


to be continued

Last edited by shekh; 25th November 2015 at 16:18.
shekh is offline   Reply With Quote
Old 25th November 2015, 13:39   #2  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,177
Sounds interesting. I have a couple of cameras that could use some lens corrections.

Can a mod please approve the attachment.
Reel.Deel is offline   Reply With Quote
Old 25th November 2015, 14:23   #3  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
The tough part is getting good profile for each camera. I have calibrated mine myself - quite challenging process to do at home, especially for water.
It is possible to pick a profile from the adobe library but this way you apply wrong correction for the center offset, as it varies per unit.
If you have some grid shots and profiles, throw it at me so I can test compatibility.
shekh is offline   Reply With Quote
Old 27th November 2015, 23:52   #4  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
What amount of skew distortion is tolerable (there should be animated gif):

Left picture is sequence shot while knocking the camera to get some rolling shutter skew effect, most noticeable on frame 3 (bottom part skewed left).
Right picture is the attempt to align frames as much as possible, eliminating both shake and skew. The analysis by Deshaker failed - I matched frames manually.



Btw the filter was setup to output result in original (distorted, fisheye) space, to ease comparison.

Not perfect but this result convinced me of two things:
1) it is very hard to ruin material while camera is handheld - the skew is quite soft over all frames I have inspected - I never got a frame where camera is changing direction (the shots are 48 fps).

2) the sufficient de-skew algorithm is very simple and fast (the part which does pixel projection, analysis is whole different story).
shekh is offline   Reply With Quote
Old 24th December 2015, 16:20   #5  |  Link
n01d
Registered User
 
Join Date: Dec 2015
Posts: 1
Very promising, it would be great to have avisynth's deshaker3d features inside vdub, can't wait to try it out!
n01d is offline   Reply With Quote
Old 7th January 2016, 01:53   #6  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Demo: canyon
Project source here includes source video (75Mb) and plugin setup.

How to get started:
1) download and unpack executable here
It includes VirtualDub, plugins, and sample lens profiles.

2) download and unpack canyon project.

3) launch VirtualDub, use File->Load project, point to canyon.vdproject

Suggestion for some fun:
open Lens Transform filter configuration, activate preview, click and drag big square directional button in the "Rotate xy" row.
If it does not react make sure current frame is 0 (it has keyframe in this project).
shekh is offline   Reply With Quote
Old 7th January 2016, 02:46   #7  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
The plugin actually has 3 filters:

Lens Transform is the main thing.

Lens Test is much simpler, made to inspect lens profiles whether they work as expected.
It can do just simple forward/reverse distortion without any rotation etc. This is style of transform useful for camera tracking and cg composition (but I am not doing this).

Repair is simple border filling, same purpose as Deshaker` extrapolate pixels.
Repair uses alpha channel to determine what to fill and Lens Transform in turn uses alpha channel to signal good/bad pixels.

About lens profiles
The most accessible way to get one AFAIK is to use Adobe Lens Profile Creator free utility.
To calibrate, just open checkerboard image on flat display, take about 9 shots at different angles, then put them in the profile creator and it will automatically determine distortion, fov, and lens center.
At the moment I only read "fisheye" model (there is also "rectilinear" model which I have ignored).
I made profile this way using ordinary 23" monitor. I also made water profiles, for this purpose monitor is not an option
shekh is offline   Reply With Quote
Old 11th January 2016, 13:10   #8  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Made some basic manual: https://sourceforge.net/p/vdfiltermo...LensTransform/
shekh is offline   Reply With Quote
Old 11th July 2016, 11:37   #9  |  Link
otalp
Registered User
 
Join Date: Feb 2016
Posts: 1
shekh!

This is the most amazing tool for stabilizing video clips I've stumbled upon. Great work I must say.

Thanks a thousand times for sharing!
otalp is offline   Reply With Quote
Old 11th July 2016, 12:16   #10  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Quote:
Originally Posted by otalp View Post
shekh!

This is the most amazing tool for stabilizing video clips I've stumbled upon. Great work I must say.

Thanks a thousand times for sharing!
Thanks, I had more progress with it but was lazy to post any update due to lack of feedback.
Have you any success at using it?
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 5th August 2016, 19:50   #11  |  Link
b2kguga
Registered User
 
Join Date: Jul 2016
Posts: 21
Excellent work. How you managed to make the preview without controls ? Im trying to grab the handle of the preview window without using EnumWindowsProc.
b2kguga is offline   Reply With Quote
Old 5th August 2016, 20:02   #12  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Thanks!

"How you managed to make the preview without controls"

It is without controls because I modified VirtualDub source to make this. Also see my reply in other thread
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 5th August 2016, 20:57   #13  |  Link
b2kguga
Registered User
 
Join Date: Jul 2016
Posts: 21
Ok...Btw...i simply loved the filter. have you considered using the output data from BCC or Mochapro insetad only relying on Deshaker?
b2kguga is offline   Reply With Quote
Old 5th August 2016, 21:04   #14  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
I have internal motion analyzer in progress (but I really need to get back to it). I dont even know what is BCC and Mochapro. What is useful out there?
It all started from playing with syntheyes demo, I use its lens profiles but not camera track (demo does not export it and I dont think I need it anyway).
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 5th August 2016, 21:22   #15  |  Link
b2kguga
Registered User
 
Join Date: Jul 2016
Posts: 21
Mochapro is the best professional tracking tool out there. It is used to remove objects from the scene, stabilize, lens correction, etc. http://www.imagineersystems.com/products/mocha-pro/

BCC - BCC Motion Key (Boris FX Motion key) is also a very good motion tracker. https://web.borisfx.com/helpdocs/?page_id=951

Both export their data on a regular txt file, from where you can use them on Sony Vegas, for example. Although the result of both are very accurated, the problem of those two apps is that they are very slow and the configuration is a complete headache. The good is that the resultant txt file is somewhat similar to deshaker, therefore, it maybe ported to get a better result.
b2kguga is offline   Reply With Quote
Old 5th August 2016, 21:57   #16  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
From my experience with Syntheyes (I assume all motion trackers are same or more difficult to use) there is huge gap between hollywood and consumer material.
I want to repair shots which are poor quality by professional standards, so I suppose no software is trained to help on them. Or difficult anyway. With syntheyes I had to eyeball misaligned trackers all the time. This is so much manual intervention that doing full manual tracking looks not bad too.
If Mocha etc. can export motion tracking data, this is maybe useful for someone in VFX, but I have doubts I can make with these data something better than Mocha itself?
Anyway my goal is to create easy workflow to do it all in one place, quick and happy.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 6th August 2016, 06:59   #17  |  Link
b2kguga
Registered User
 
Join Date: Jul 2016
Posts: 21
I totally agree with you. Professional apps are a hell to configure. About mocha tracks, in fact the tracker can be improved. Their results, i mean. For example, we can get the data from the tracker to completely remove a object without the needs of using mochapro itself. Removing a logo (or other static objects), for example, in Mocha is very hard.

Your idea of making a tracker be simple and fast is really awesome. This can be used not only for stabilization, but also for object removing as well (Static or moving object). One thing to be considered is maybe implement a scene detector in order to gain accuracy on the tracking/stabilization. On this way, whenever a scene is detected, another track data is performed for the next scene.

The 2 filters i saw for vdub that are suitable for object remover (i.e. tracking/stabilizing) are yours and stabbo. I could give a help, but im still learning about Vdub Api while building my own version of LogoAway. When i have enough knowledge on how exactly Vdub Apis can be used i can try helping you on stabilization and object remover techniques.

Last edited by b2kguga; 6th August 2016 at 07:01.
b2kguga is offline   Reply With Quote
Old 8th August 2016, 07:38   #18  |  Link
b2kguga
Registered User
 
Join Date: Jul 2016
Posts: 21
Shekh, i was trying your vdub version with the Lab plugin. The Lab plugin was made by Neiromaster, but If you have the source it can be improved a little bit more. For example, it can use a limitation of Hue (from CieLCH) to check for difference ranges. For example, whenever you set a image to use more blue or red etc, it will increase the amount of smooth "spots" of Blue/Yellow/Red/Green. The "spots" can be removed simply making check for a Hue difference range.
Like this: when you are increasing the Blueish tone (a and b factors from Lab) , whenever the pixel Hue varies in, lets say...10 degrees (user can define the range), the conversion from RGB to Lab stops at that pixel. Or...instead stopping the conversion, the Hue can be fixed inside that range and all that will change is the Luma or the a/b factors without changing the hue anymore.
This probably will remove the "spots" that appears when you increase too much the tone from yellow/blue/red/green used in Lab colorspaces.

Last edited by b2kguga; 8th August 2016 at 07:51.
b2kguga is offline   Reply With Quote
Old 8th August 2016, 08:47   #19  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
I dont understand most part of what are you saying
But if you want to try you can get source here: https://sourceforge.net/projects/vdf...r.rar/download

However by inspecting source I conclude it has nothing to do with CieLab.
I made "6-axis color" filter based on same colorspace as "lab" filter, it is basically some YCbCr.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 8th August 2016, 16:13   #20  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,372
shekh - any ideas on how to address the rs skew and wobble ? That is one area that is lacking in free/open source tools. That's a large part of what prevents deshaker or any stablilizer from working properly. Skew can sometimes be handled by commercial software/plugins fairly well, but the wobble from vibrations (multiple vector directions) isn't handled as effectively

Nice location in the canyon clip BTW, where is it ?
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.