Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
12th October 2018, 19:42 | #1 | Link |
Registered User
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
|
Completely remove camera motion
Hi
I remember doing this before, but I've lost my scripts and can't seem to figure out what functions to use. So I have a video recorded with a phone camera, that is supposed to be completely still, but of course there's some small movement. I'd like to make it completely still, not just remove shakiness but simulate a tripod. I remember that I first picked a reference frame, interleaved that with the whole clip, generated motion data so that every frame's motion is compared to this reference frame. But what to do next, I don't remember. What plugin/function will fully move/zoom/rotate a frame back to where its motion vectors are pointing from? |
12th October 2018, 20:52 | #2 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Would it be some variation of this from mvtools examples (?),
Code:
To use MDepan with Depan plugin for interlaced source (DepanStabilize function example): AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc AssumeTFF().SeparateFields() # set correct fields order vectors = MSuper().MAnalyse(isb = false) globalmotion = MDepan(vectors, pixaspect=1.094, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094) Weave() (I aint set up for using Avisynth at the moment) EDIT: Maybe something like Code:
AVISource("c:\test.avi") Interleave(Last,RefClip) vectors = MSuper().MAnalyse(isb = false) globalmotion = MDepan(vectors, pixaspect=1.094, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094) SelectEven
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 12th October 2018 at 22:30. |
12th October 2018, 23:16 | #3 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,695
|
Deshaker can do this. I wrote a long tutorial fourteen years ago that, unfortunately, seems to have disappeared from the web. There is a variable that you set to -1 and it will then lock the camera so that it appears to be on a tripod.
You can also do this with Mercalli, but it costs money. |
13th October 2018, 00:22 | #4 | Link | |||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Topic, Discussing Johnmeyer/Gunnar Thalin Guides, in French:- http://letransfert.soforums.com/t215...-de-la-DOC.htm Topic, Discussing Johnmeyer/Gunnar Thalin Guides, translated from French, into something sensible :- https://translate.google.com/transla...-de-la-DOC.htm John Meyer Pdf in French:- http://letransfertpellicule.free.fr/...ex.guideFr.pdf EDIT: And an actual translation for those that dont speak french (XLate from the PDF):- https://translate.google.com/transla...ex.guideFr.pdf EDIT: Keep a link to that John, the "missing guide" topic has come up before. EDIT: Maybe this (near end of pdf) Quote:
EDIT: "movement in the girl track", John, what should that have said, should we perhaps read something like "mate" (or "matched") instead of "girl" ? EDIT: From Tormento below, Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 14th October 2018 at 16:54. |
|||
13th October 2018, 12:28 | #5 | Link | |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,575
|
Quote:
From my little french knowledge, "fille" should be translated as "daughter", literally. Probably it was meant as "subsequent" or "derived" track.
__________________
@turment on Telegram |
|
13th October 2018, 15:33 | #6 | Link |
Registered User
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
|
@StainlessS
DepanStabilize can't do it. It will find some middle ground between the reference and the clip, pulling both closer to each other. And it's not always just in the middle either, it can sometimes actually pull both in the "wrong" direction, as long as the overall resulting movement is smooth (which is what it's designed to do I guess). @johnmeyer That seems to be a VirtualDub filter, which I haven't really got working before, so my old solution had to use something else. But if nothing else comes up, I'll have to figure out how to use it. |
13th October 2018, 15:42 | #7 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
What type of shot is it ? What are the subjects / any foreground movement ? Can you describe it or post a sample ?
A simulated locked off tripod shot should be stabilized based on known stationary objects like a wall, the ground, etc... because if you have foreground subjects, people etc... moving - it will contaminate the motion data and you typically won't get good results with something like mvtools2, deshaker, mercalli etc....For example, you don't want to include the motion vectors from someone nodding their head, or a car driving in the background etc....because that would skew everything and you actually introduce more compensatory camera movement, making it worse than before . If you can crop to a "clean" area, then do the calculations and apply back to original that might work. Or deshaker has the exclude border option. But those only help on avoiding peripheral objects Usually for those types of situations , different types of motion trackers are used (some are better for certain types of shots) . They are typically found in packages such as after effects, mocha, hitfilm, nuke, shake . Free ones like natron, blender, blackmagic fusion can do this type of thing too. The difference is you can define areas to include/exclude used in motion tracking so it's a lot more accurate, but the downside is it takes some user guidance to define those areas so it's slower, less automated |
13th October 2018, 18:08 | #8 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,695
|
StainlessS, thanks for finding that quote from the old guide which does indeed identify the setting that should be set to -1. I couldn't remember that off the top of my head.
Perhaps I should re-post my tutorial on my own web page, for those who don't speak those other languages. Then again, maybe it isn't worth it. While Mercalli costs a fair amount of money, and Deshaker is free, the algorithms in Mercalli are far better, especially when trying to achieve a "tripod look." Also, Mercalli also contains a really good algorithm for removing the sometimes grotesque "rolling shutter" artifacts that are so common in cell phone video. If you stabilize using a program that doesn't have this feature, the result is pretty awful. Last edited by johnmeyer; 13th October 2018 at 18:11. Reason: added last paragraph |
14th October 2018, 11:37 | #9 | Link |
Big Bit Savings Now !
Join Date: Feb 2007
Location: close to the wall
Posts: 1,545
|
BTW, depending on camera motion asking for full stabilisation might yield only a tiny part of the picture, if any...
So if camera moved completely out of the original frame, for these source frames one would get an empty frame in return. (Of course and out of curiosity I tried total stiff stabilisation on Mercalli. Seeing the resulting frame becoming tinier the more I restricted motion made it clear: I had ordered content that hadn't been shot, so got presented with the almost empty result. Went back to the possible, allowing some smoothened motion yielded more pleasing results.)
__________________
"To bypass shortcuts and find suffering...is called QUALity" (Die toten Augen von Friedrichshain) "Data reduction ? Yep, Sir. We're that issue working on. Synce invntoin uf lingöage..." |
14th October 2018, 11:47 | #10 | Link |
Registered User
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
|
@poisondeathray
There is a table and some walls, it's an indoor shot. Two dogs in the foreground, but they are contained in a small area on the image during the whole shot. I know their movements will affect the motion vectors, so I'll probably need some extra processing to get around that, but first I need to get the stabilization to actually work with the motion vectors I have. @Emulgator Of course it would restrict the picture. Or well, you could in theory have the full frame, but the video image would move around and not always show itself completely and maybe go outside. But I'd hope the person doing this knows this =). My shot has 90% of the scene always visible, and it's going to get cropped even further anyway. |
16th October 2018, 06:13 | #11 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
Quote:
Depanstabilize uses inertia method when set to 0 , but the 1st frame (frame 1) looks like it's solved as good as can be expected by mdepan vectors. After that, it won't work well because of averaging based on frame 0, you can see the values with info=true. So you if you interleaved it so each frame has a preceding reference frame , and chopped it into pairs, and only did the calcs on the pairs, then ditch the 1st frame in the pair, then reassembled it , it would probably give you as good as can be expected with that method. You can change the reference frame (e.g. a tilted one), an it still works - frame 1 becomes similarly tilted and transformed. Maybe a wrapper function could automate it, but it does work as expected But it's still a bit wobbly, not what I would call "rock stable"; and that's with "ideal" test material (you'd expect a lot worse with real material, especially rolling shutter cell phones, motion blur, compression issues). Depanstabilize info=true will overlay the x,y, rotation info, and you can compare with the known values. Even a slight rotation difference from the actual values like a delta of 0.01 will register as a wobble . You can play with the analyze parameters like blksize etc..., but rotation seems weak with this method, at least with the values I tried. I do not think method=1 will usable in any combination (even with various combinations interleaving+selecting, of because it averages the frames and the base frame increments (watch the behaviour with info=true) (Also I noticed slight green hue shift with depanstabilize , it's reduced if you use ConvertBits(16) beforehand , but still there) |
|
16th October 2018, 22:45 | #12 | Link |
Registered User
Join Date: Mar 2018
Posts: 447
|
That sounds something AvisynthOptimizer could do as well, since we can simply compare the stabilized video to the original still image. If you can share your still image, translated video and avs script I could give it a try.
|
17th October 2018, 00:53 | #13 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,374
|
Quote:
Sure, but it's just a simple 5 frame synthetic test - an exercise for me to figure out the behaviour for depanstabilize. But you wouldn't have the original, original reference image used to create the video in real life ; you'd only have the "shaky" video and you'd pick a reference frame to stabilize against. It was actually a larger image scaled down to 1920x1080 and then the transformations were applied. The reason it's done this way, is so the "BG canvas" is larger than the target field of view of 1280x720, so there are no black borders when producing the "shaky" video . Since we are trying to emulate a real case, you should use frame 0 as the reference. (If you want I can upload the other images, but I don't think they should be used; or it's easy enough to make your own "shaky" videos using known values) The video uses lagarith ~5MB https://www.mediafire.com/file/upiwb..._LAGS.avi/file Actual transform values used: Code:
Frame number, x,y,rotation,zoom% 0 1 +5,+5,+1,100% 2 -5,-5,-1,100% 3 +5,+5,+1,101% 4 +2,-2,-0.5,99% Code:
orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16) a=orig.ConvertToYV12(matrix="rec709").Trim(0,-1) b=orig.ConvertToYV12(matrix="rec709").Trim(1,-1) c=orig.ConvertToYV12(matrix="rec709").Trim(2,-1) d=orig.ConvertToYV12(matrix="rec709").Trim(3,-1) e=orig.ConvertToYV12(matrix="rec709").Trim(4,-1) a+b vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0) globalmotion = MDepan(vectors, range=1, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true) trim(1,-1) a1=last a+c vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0) globalmotion = MDepan(vectors, range=1, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true) trim(1,-1) a2=last a+d vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0) globalmotion = MDepan(vectors, range=1, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true) trim(1,-1) a3=last a+e vectors = MSuper().MAnalyse(search=4, blksize=8, isb = false, dct=0) globalmotion = MDepan(vectors, range=1, thSCD1=400) DepanStabilize(data=globalmotion, cutoff=1, dxmax=160,dymax=90, method=0, zoommax=1.1, info=true) trim(1,-1) a4=last a+a1+a2+a3+a4 #histogram("levels") FYI, a handful other stabilizers can solve it pretty much perfectly and automatically - but only because it's a simple test. But in real life, there are the other problems mentioned above, also things like moving objects (like dogs, people) . There are different types of stabilizers and trackers used in different situations. There are "ideal" tools for certain situations, and it depends on the specific shot characteristics |
|
17th October 2018, 13:23 | #14 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Dont know if of any use (as for PDR script above, untested, I'm not set up for Avisynth use)
Code:
Function Func(clip c, \ Int "Search",int "blkSize",Bool "isb",int "dct", \ Int "Range",Int "thSCD1", \ int "cutOff",int "dxMax",Int "dyMax",Int "method",Float "zoomMax",Bool "info" \ ) { n=current_Frame if(n != 0 ) { # Req Avs+ ### Search=Default(Search,4) blksize=Default(Blksize,8) isb=Default(isb,false) dct=Default(dct,0) range=Default(Range,1) thSCD1=Default(thSCD1,400) cutoff=Default(cutoff,1) dxmax=Default(dxmax,160) dymax=Default(dymax,90) method=Default(method,0) zoomMax=Default(zoomMax,1.1) info=Default(info,true) ### Last = Trim(0,-1) + Trim(n,-1) vectors = Last.MSuper.MAnalyse(search=search, blksize=blksize, isb=ISB, dct=DCT) globalmotion = Last.MDepan(vectors, range=range, thSCD1=thSCD1) Last.DepanStabilize(data=globalmotion, cutoff=cutoff, dxmax=dxmax,dymax=dymax, method=method, zoommax=zoommax, info=info) Last = Trim(1,-1) } return last } ARGS="Search,blkSize,isb,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info" SSS="Func(Last,"+ARGS+")" orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16) conv=orig.ConvertToYV12(matrix="rec709") Last=conv ### Set non default args # ... # EDIT: For defaults, set eg non defined Search, "Search=Undefined" ### Last.ScriptClip(SSS,args=ARGS,Local=True) # Req: Grunt for Local & Args Code:
SSS=""" n=current_Frame if(n != 0 ) { # Req Avs+ Last = Trim(0,-1) + Trim(n,-1) vectors = Last.MSuper.MAnalyse(search=search, blksize=blksize, isb=False, dct=DCT) globalmotion = Last.MDepan(vectors, range=range, thSCD1=thSCD1) Last.DepanStabilize(data=globalmotion, cutoff=cutoff, dxmax=dxmax,dymax=dymax, method=method, zoommax=zoommax, info=info) Last = Trim(1,-1) } return last """ #ARGS="Search,blkSize,isb,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info" ARGS="Search,blkSize,dct,Range,thSCD1,cutOff,dxMax,dyMax,method,zoomMax,info" orig=AVISource("stabilizetest_LAGS.avi")#convertbits(16) conv=orig.ConvertToYV12(matrix="rec709") Last=conv ### Set all args (where none allowed to default) Search = 4 blksize= 8 #isb = false dct = 0 range = 1 thSCD1 = 400 cutoff = 1 dxmax = 160 dymax = 90 method = 0 zoomMax= 1.1 info = true ### Last.ScriptClip(SSS,args=ARGS,Local=True) # Req: Grunt for Local & Args
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 17th October 2018 at 14:16. |
3rd December 2020, 15:49 | #15 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,695
|
Since this thread is linked to from other threads, here is a link to my Deshaker guide, as originally written (in English), without multiple translations:
John Meyer Deshaker Guide |
Thread Tools | Search this Thread |
Display Modes | |
|
|