Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VirtualDub, VDubMod & AviDemux

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th August 2016, 16:59   #21  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
This is göynük canyon, turkey.

For skew I have some ideas in todo list. It is quite easy to match frames to each other using 2 sets of parameters instead of 1 (same as deshaker).
Then it is quite easy to try full compensation matching all frames to single reference frame, this is suitable for shots without motion. This should certainly work, but I am not sure RS is an issue at all with such shots.
The trouble is to build a path when camera moves. I certainly cannot just accumulate differences stacking one frame after another: this works for no-skew mode but with skew included the growing error tears frame apart.
I tried deshaker approach without success (either I did it wrong or did not understand the result), also tried filtered average as immediate reference (same here).
Another idea is to collect frames with lowest skew and use them as references.
First I need better motion data. For this I started to make analyzer
Second problem is to understand what happens. This is hard with real footage. I dont see something simple like skewed pillar, instead the shot just looks "jelly" but I cant tell for each frame where it is wrong.
To some extent I train the math on synthetic RS-skewed shots. This way I can have perfect result to compare with. But I need to go deeper and prepare realistic synthetic shot with motion and shaking similar to reality, and this is damn long task to prepare

For wobble I have no ideas. I suppose it requires different level of enginery (structure from motion etc).
I chose as milestone some piece of footage which has really bad shake but still no wobble.

Btw maybe by the time I finish that, they come up with practical global shutter sensor?
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 25th March 2017, 16:50   #22  |  Link
djonline
Registered User
 
Join Date: Mar 2011
Posts: 40
shekh, b2kguga. I was used Synthyes too, it works fine on CCD camera (my example https://www.youtube.com/watch?v=wsrs28aUAKw and https://www.youtube.com/watch?v=bxQ8AkyK70k and https://www.youtube.com/watch?v=NTYdAVV71p8) , but it completely unusable on consumer CMOS camera due Rolling Shutter. There is no way to fully eleminate them, so better result for CMOS is to use Warp Stabilizer in Adobe Premiere.

I dont 'like shaked edges in this example of Canyone, it is not usable for me. But I still use deshaker for little shaked video to eleminate small shake, coz Warp is still very slow.
djonline is offline   Reply With Quote
Old 9th September 2019, 11:22   #23  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Hi shekh.

I posted I question to your plugin in another (wrong) thread

https://forum.doom9.org/showthread.p...79#post1884379

Greetings.
nji is offline   Reply With Quote
Old 10th September 2019, 12:59   #24  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Quote:
Originally Posted by nji View Post
Hi shekh.

I would like to ask if there is an up-to-date version of your plugin.
(Only the plugin and the needed files for ini etc. for use in VDub2)

I use DeShaker for a long time, but had to make the experience
that with large FOV it produces "wobbling" results (due to the
lack of taking perspective into account) that make the result
worse than the shaking version.

In Deshaker - as a first step - a simple option for FOV
(wide-angle, normal, tele) would be fantastic.
But ... sadly ... DeShaker isn't improved anymore.

As far as I understand the matter, BEFORE phase 1 in DeShaker
there would have to be the perspective correction (panotools etc.?)
and after that to calculate the motion vectors etc.

But maybe your plugin works well enough already?

Greetings! :-)
Hi,
actually the published version is the last one. Any further improvement is in awful "maybe resume later" state.

That said, the last version can do something, I believe, better than deshaker wrt perspective and distortion.
If you really want to use it as is but don't understand the GUI, I can guide you Post your sample.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 11th September 2019, 11:38   #25  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Hi shekh.

Thank you for your response and your offer for help.
I'll try on my own first.
And if nothing more helps ... I could read your posted instruction ;-)

Only one thing about the "published version" you mentioned.
It is a collection of a complete (and old) VD version and some other things.
Would it be possible for you to do only the needed files for your tool/ filter in one package/ zip, so that one can easily integrate it into an existing VD2?
That would be nice.

BTW: Strange that there is so few interest in that wonderful quality improvement of deshaking (And ever more the lens correction). As most shaky videos will be taken on a middle FOV, and not FOV=0 (= standard DeShaker without perspective accounting).
nji is offline   Reply With Quote
Old 14th September 2019, 10:39   #26  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by shekh View Post
[...]

Btw maybe by the time I finish that, they come up with practical global shutter sensor?
Please don't forget that most videos VD manipulates are from the past (at least for me it's the case). And all they are taken by "previous" optics/ sensor types.

Last edited by nji; 14th September 2019 at 11:01.
nji is offline   Reply With Quote
Old 14th September 2019, 17:55   #27  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,025
Quote:
Please don't forget that most videos VD manipulates are from the past
I'm curious to know who's processing videos from the future.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 8th January 2020, 11:54   #28  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by shekh View Post
Hi,
actually the published version is the last one. Any further improvement is in awful "maybe resume later" state.

That said, the last version can do something, I believe, better than deshaker wrt perspective and distortion.
If you really want to use it as is but don't understand the GUI, I can guide you Post your sample.
Hi,

I recently tried (had to ) your plugin on a wide angle movie.

I stripped the nescessary files to the minimum (see attachment).
Is that complete?

Does your alg do "3D calculation", or "simply" performs DeShaker's log?

Does the rolling shutter/ "skew" work?
Attached Files
File Type: zip lens_transform.zip (191.3 KB, 14 views)

Last edited by nji; 8th January 2020 at 11:57.
nji is offline   Reply With Quote
Old 8th January 2020, 19:50   #29  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Quote:
Originally Posted by nji View Post
Hi,

I recently tried (had to ) your plugin on a wide angle movie.

I stripped the nescessary files to the minimum (see attachment).
Is that complete?

Does your alg do "3D calculation", or "simply" performs DeShaker's log?

Does the rolling shutter/ "skew" work?
Please note that this project is somewhat buried.
Can't see attachment. What you planned to show?

I saw your posts about it in other places, so I try to summarize replies

The "transform" rebuilds true projection to different direction and with different lens (but from same focal point).
By true projection I mean, if you had actual real life shot of the same scene it would match exactly.
For this to work it needs accurate lens description with FOV and distortion curve.
In this way it does true 3d calculation. Did you mean something else?
(also the amount of rotation for all axes is expressed in units of rotation not pixels)

Deshaker log was used as a guide to derive motion but not as direct instruction to move pixels.

Rolling shutter: no, I don't think it worked
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 8th January 2020, 20:59   #30  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Hi shekh!

Actually I don't understand why you don't see
why nobody can see the attachment at the moment.
Being not logged-in it says (for everbody I guess):
"Attachments Pending Approval".

The case I'm interested in is that I don't have detailed
information about the lens the movie has been taken with.
(Except a vague value of FOV).
And even if I would know, first there would have to build
a description as you wrote.
If I understand it right (do I?) without that information
your plugin is useless for all its functionality?

Next: I do not understand how Deshaker's log could be
of any use for your plugin. As its analysis was done
without knowing the specific lens characteristics.
"... as a guide ..." ???

Rolling shutter ("skew") doesn't work?
But there is an active check-box for it ...

Last edited by nji; 8th January 2020 at 21:10.
nji is offline   Reply With Quote
Old 8th January 2020, 21:40   #31  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
The goal of this plugin was use with known camera which you can calibrate and test.
Some program may help you to extract lens params from movie. Don't know much on this.

The principle how deshaker log was used here: suppose deshaker recorded 100px move to the left. By comparing pixels and how they are projected from world you can translate this into radians.
Together this is not accurate enough but is 'move in right direction'.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 9th January 2020, 01:18   #32  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
So, is there a way to "feed" your plugin just with the FOV value for a movie?
With "neutral values" for the other distortion.
And if ... how?
nji is offline   Reply With Quote
Old 9th January 2020, 14:10   #33  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
try to add your new profile like below

Code:
[lens]
  name = "my test"
  hfov = 90
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 9th January 2020, 17:55   #34  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by shekh View Post
... suppose deshaker recorded 100px move to the left. By comparing pixels and how they are projected from world you can translate this into radians.
Together this is not accurate enough but is 'move in right direction'.
You calculate which kind of angle movement (wrt to FOV)
would cause a Deshaker's 2D movement.
Then you take that angle movement and transform it back to 2D.
Do I get it right?

How does this compare to Avisynth Deshaker3D approach?

And how does it compare to Deshaker solely?
Identical for FOV = 0; and better results than Deshaker with larger FOV?
nji is offline   Reply With Quote
Old 9th January 2020, 20:59   #35  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
nji

It is really boring to explain application of deshaker log because there is nothing to say, it is just basic trig and the method is also just a proof of concept (better log is needed).
Deshaker3D is bare minimum algorithm which is also included in lens transform (minimum math is so basic you can't avoid it).
Looks like I made this for you to help understand the place of FOV (old video): https://www.youtube.com/watch?v=JpKHxfC28mQ
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 9th January 2020, 22:15   #36  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Sorry having asked.

Last edited by nji; 10th January 2020 at 00:44.
nji is offline   Reply With Quote
Old 10th January 2020, 00:45   #37  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Is there a way to let not only the borders "repair" but also dubbing-ons (like logos etc.)?
nji is offline   Reply With Quote
Old 10th January 2020, 01:00   #38  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
"repair" just fills based on alpha channel, so you need some way to "erase" alpha where you need. Something like push logo to alpha channel. I don't have such filter but it is really simple. Do you have a use-case?
Btw, I believe delogo filters solve this better so what is your intent?
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 10th January 2020, 09:42   #39  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
My intent is simply to prevent a dubbed-on date jumping around
when the movie is deshaken.

In Deshaker you can mark arbitrary areas by a special color
which is recognized by Deshaker and repaired.
But - and this is important - Deshaker does much better than
just delogo. As it takes neighboured frames for the filling
of the areas.

EDIT:
I suspected that repair would do something similar.
But that was wrong thinking, as repair gets only the alpha from lens transform.
Conclusion:
Do delogo before lens transform.

Last edited by nji; 10th January 2020 at 11:22. Reason: See "EDIT"
nji is offline   Reply With Quote
Old 10th January 2020, 12:08   #40  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 756
Ok, I assumed such workflow but it is not ready.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.