Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VirtualDub, VDubMod & AviDemux

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th January 2020, 23:57   #441  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Hi.
Thank you for wonderful work, in a first place.

I am just curious:
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...
redfordxx is offline   Reply With Quote
Old 7th January 2020, 14:09   #442  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by redfordxx View Post
Hi.
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...
That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.
It is as your movie shows only flat surfaces,
or as if you're using an extreme telephoto lens.
DeShaker does very good results for that.
Rotation on x and y axis (called panning and tilt)
is simply x and y shift then.

But the less the condition holds
(i.e. you have 3D objects or using a lens that's not telephoto)
the operation on the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).

So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there are at least two serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146
https://forum.doom9.org/showthread.p...68#post1747268

Last edited by nji; 11th January 2020 at 11:56. Reason: Typo
nji is offline   Reply With Quote
Old 8th January 2020, 00:18   #443  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by nji View Post
That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.
The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...
Quote:
the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).
I don't know about that...
Especially if there would be option to limit x and y rotation.
Quote:
So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there is at least one serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146

(https://forum.doom9.org/showthread.p...68#post1747268
just uses DeShaker's analysis data and does some lens correction.
So: no 3D at all.)
But they still use deshakers "2D" data.
redfordxx is offline   Reply With Quote
Old 8th January 2020, 08:12   #444  |  Link
guth
Registered User
 
Join Date: Apr 2003
Location: Uppsala, Sweden
Posts: 152
Quote:
Originally Posted by redfordxx View Post
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?
To be honest, I'm not sure which movements you mean. If we talk movements of the video, basically only one kind of rotation exists. And if we talk movements of the camera, Deshaker handles rotation of the camera along all axes (+zoom), but nothing else. Moving the (center of the) camera forward, backward, right, left, up, down often works somewhat well too, though.

Anyway, nji is right. Deshaker both analyses and transforms the video in 2D only, which means wide angle video will look ugly when there is heavy shaking.
guth is offline   Reply With Quote
Old 8th January 2020, 11:40   #445  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by redfordxx View Post
The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...

I don't know about that...
Especially if there would be option to limit x and y rotation.

But they still use deshakers "2D" data.
First of all:
You are right, I did a mistake in my last post (#442):
"Deshaker3D" uses DeShaker pass1 data, too.
(So maybe both tools do some kind of "3D-magic"?
And then "Lens transform" will be probably more sophisticated
with alpha consideration and rolling shutter/ "skew"?)

Second:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.
It does only 2D-operations:
left/right, up/down, resize and (2D-) rotation.
These 2D-operations do ("by chance") match the effects that
the corresponding 3D movements would have on the 2D projection (= frame).

Thinking it over ... DeShaker very well possibly could
take into account the 3D rotation at x- and y-axis
(or even an arbitrary axis) too.
(Would be a kind of magnifying "above" and reducing "below"
the axis I guess).
Seems possible, but isn't done (and won't be ... believe me )
But maybe/ probably the other mentioned tools will?

Last edited by nji; 8th January 2020 at 12:19.
nji is offline   Reply With Quote
Old 8th January 2020, 13:38   #446  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
I do hope it's not too much OT if I post a summary of my (small)
experiences of the filter?

DeShaker (VD)
Pros:
- excellent results on all but wide angle scenes (no 3D/FOV accounting)
- optional rolling shutter accounting
- optional progressive/ interlaced source
- accounting sophisticated border and logo handling (by neighbour frames)
- many options to handle difficult source/ get optimal results
- excellent description of all that
- multithreaded (? nevertheless takes the most times of all filters)
Cons:
- Bad results on wide angle scenes (even worse than the shaky sources)

Lens Transform (VD)
Pros:
- accounting of field of view (FOV)
- accounting of simple border handling
Cons:
- no accounting of logo handling (have to delogo before)
- no interlaces sources (have to convert before)
- Rolling shutter accounting ("skew") "not implemented completely" (although active check-box)
- Unsufficient description (at least for me)

Deshaker3D (AviSynth)
Pros:
- Accounting of FOV
Cons:
- no border/ logo handling
- no rolling shutter accounting
- no interlaces sources (have to convert before)
- needs format conversion (= quality loss)
- no multithreading (?)

Deshaking (Shotcut)
Does about the same as DeShaker but with no options and worse results.

Conclusion:
The best (for non-wide angle scenes) does DeShaker.
If results are unacceptable in DeShaker (due to wide angle)
the best choice is Lens Transform (if rolling shutter source: try Deshaker before with all smoothness = 0).

4 runners at the launch, each performing good on their
respective distance, but some targets aren't reached by only one.

Last edited by nji; 10th January 2020 at 12:31. Reason: Changed Lens Transform and Conclusion
nji is offline   Reply With Quote
Old 8th January 2020, 21:01   #447  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
https://forum.doom9.org/showthread.p...81#post1895181

Last edited by nji; 10th January 2020 at 12:33.
nji is offline   Reply With Quote
Old 8th January 2020, 23:58   #448  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by nji View Post
First of all:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.
No, I didn't.
I thought if there will be 3D added, there could be these limits added so that it will not lead to worse results ( which I still don't know what you refer to).
Quote:
(Would be a kind of magnifying "above" and reducing "below"
the axis I guess)
Exactly.
Quote:
Seems possible, but isn't done (and won't be ... believe me )
But maybe/ probably the other mentioned tools will?
You mean even if I will find some time and make a proof of concept that it could work, there is no one who will continue the deshaker?
Because other projects will not help without better 1st pass.
redfordxx is offline   Reply With Quote
Old 9th January 2020, 00:54   #449  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Deshaker3D and Lens Transform do something like that.

But of course you're free to try whatever you want to.

I'm just a user and not the one to answer about the availabilty of source code.

Last edited by nji; 9th January 2020 at 18:10. Reason: Changed first sentence
nji is offline   Reply With Quote
Old 9th January 2020, 01:09   #450  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Maybe I test something in Avisynth.
But first, can someone tell me, how can I get rid of the black frames in the beginning of the video, when I
"Use previous and future frames to fill in borders"?
Only Avisynth does that, VDub is ok...
redfordxx is offline   Reply With Quote
Old 10th January 2020, 23:19   #451  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
I hope I understand optics roughly correctly.
I played a little with the idea of compensating x and y rotation.

I took one frame of my video and tried to find warp with best improvement of OK blocks.

Original motion estimation: 28.7%
After warp: 62.9%

See the pictures. I used avs code
Code:
points_f1="
0,0 0,0                	
640,0 1,0
640,480 15,8
0,480 0,3
"
warp(points_f1, interleaved = true, relative = true)
I know there is also partial z rotation and x and y shift but it does not matter to the demonstration.

Anyone thinks it has a potential?


BTW:
"19|1|20|4|1|0|1|0|640|480|0|1|1000|1000|1000|1000|4|1|0|2|8|30|350|4|deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|1|0|10|1000|0|88|1|1|20|5000|100|20|1|0|ff00ff"
Attached Images
  

Last edited by redfordxx; 10th January 2020 at 23:35.
redfordxx is offline   Reply With Quote
Old 10th January 2020, 23:41   #452  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,021
Do you get a better result with Quad in place of Warp?
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 11th January 2020, 01:04   #453  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
I would like to mention I've been told
that the equations to determine the 4 parameters
are kind of "ill-posed".
If you reduce the number of parameters the situation
gets better (That's the reason for being able to disable
calculate for rotation and zoom in Deshaker's GUI).
If you enter two more paramters the problem probably
will get way worse?

Second: I'm not sure if a better matching of the OK blocks
in general means better matching of the actual movements.
Don't forget there is no accounting of optical conditions (FOV etc.)
nji is offline   Reply With Quote
Old 11th January 2020, 09:47   #454  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by wonkey_monkey View Post
Do you get a better result with Quad in place of Warp?
I read somewhere warp is successor of quad.

Ok, I try again with quad...
redfordxx is offline   Reply With Quote
Old 11th January 2020, 10:21   #455  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Quote:
Originally Posted by nji View Post
I would like to mention I've been told
that the equations to determine the 4 parameters
are kind of "ill-posed".
Meaning complicated or what?
Quote:
Second: I'm not sure if a better matching of the OK blocks
in general means better matching of the actual movements.
But I think it is good estimate as long as I keep all the discard parameters pretty strict.
Quote:
Don't forget there is no accounting of optical conditions (FOV etc.)
Meaning what?
redfordxx is offline   Reply With Quote
Old 11th January 2020, 10:21   #456  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Quote:
Originally Posted by nji View Post
...Don't forget there is no accounting of optical conditions (FOV etc.)
To be more specific about what I mean by that.

To my "math intuition" analyzing the 2D data without taking into account
FOV (for larger values) is like doing the first step, very accurate,
more and more sophisticated ... but into the wrong direction.

That's why I was heading for doing something with FOV before analyzing
https://forum.doom9.org/showthread.php?p=1895012

Also ... if you have a look on the example video that the author of
Lens Transform did
https://forum.doom9.org/showthread.p...55#post1895255
and have a closer look on the difference between
Deshaker original and Lens Transform
you see that the "perspective distortion from FOV" in the original 2D data
are so massive that it doesn't seem to make sense to do sophisticated
analysis on them.

But ... as said ... just intuition from me, ... maybe completely nonsense.

EDIT:
Our posts where at the same time .... so I didn't answered on your last.
Concerning "ill-posed": Please see the def of it, can't explain better.

Last edited by nji; 11th January 2020 at 10:29. Reason: See EDIT
nji is offline   Reply With Quote
Old 11th January 2020, 10:29   #457  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
I tried Lens Transform and even created aan adobe profile. But probably did something wrong, since I did not observe any change on my video.
redfordxx is offline   Reply With Quote
Old 11th January 2020, 10:32   #458  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Did you import Deshaker.log? (Right click in the graphic)
Did you set the smooth value to say 50++ ?

EDIT
And it suffices to edit in the lens profiles something like:

Code:
[lens]
  name = "24 mm (mFt)"
  hfov = 40

Last edited by nji; 11th January 2020 at 10:38. Reason: see EDIT
nji is offline   Reply With Quote
Old 11th January 2020, 11:17   #459  |  Link
redfordxx
Registered User
 
Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 863
Loaded the lens profile. There was hero (extremely visible) and my profile not perceivable.
Could be maybe downloaded profiles for smartphones? I have Samsung A520F.
redfordxx is offline   Reply With Quote
Old 11th January 2020, 11:59   #460  |  Link
nji
Registered User
 
Join Date: Mar 2018
Location: Germany
Posts: 137
Did you checked all points from my last post?

BTW
Maybe specific discussion on Lens Transform
should be in
https://forum.doom9.org/showthread.php?p=1747268
as this thread is about Deshaker related.
nji is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.