GenMotion and DePan plugins for global motion compensation

[scharfis_brain]

there will be A/V async.
remove assumfps and it will be okay.

[oddball]

Hey guys I need some help. I really cannot understand why it's so difficult to fix this problem. I have a movie clip that moves up and down by a few pixels every few frames (Something like every 15 frames but not always). I just need to stablize the up down motion. I thought it would be simple but it appears not.

The depan command I tried was

i = ConvertToYV12()
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata, dxmax=0)

But the frame moves up and down by mad amounts every few frames.

In another thread someone suggested I use a cropped clip for motion estimation and apply that to the second clip or somesuch. Damned if I know what he means. The clip has a logo on it that I could use as a guide to help with the motion detection. Since that logo is the same all the way through the clip and it too moves up and down every few frames I thought it would be a matter of applying a filter that detects that area when it moves up and down.

I don't know very much about all these Avisynth commands apart from the basics. The Depan manual seems rather cryptic to someone who does not do much Avisynth work.

Can someone lead me to a solution?

Thanks.

EDIT: OK I tried something from some earlier postings. It seems to work a lot better. No wild motion with big frames appearing this time. Is this OK?

i=ConvertToYV12()
r=1
mdata=DePanEstimate(i,trust=4,range=r)
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
SelectEvery(r+r+1,r)
i=last
mdata=DePanEstimate(i,trust=4,range=r)
DePanStabilize(i,data=mdata,cutoff=1,prev=1,next=1,mirror=15,dxmax=0,dymax=0)

Awwww crap. Just realised it did nothing to stop the up and down motion. Back to square one.

Nope. I think I nailed it. It needs a few frames to figure out the motion then it stabilizes it rock solid. I removed what looks like a repeat line from that code too.

EDIT: Well it does not seem to stop the very slight up and down motion every few frames at all. Very frustrating.

[oddball]

I'm getting closer to solving my problem. I used this script to first analyze only the bottom border:

mpeg2source("test_720p.d2v")
Trim(0,400)
i = ConvertToYV12()
j = ConvertToYV12().crop(0,712,-0,-0)
mdata = DePanEstimate(j,info=false)
DePanStabilize(i, data=mdata, dymax=1, dxmax=0, info=true)

Now I can see the data being diplayed on the video. the "dy" is either between -0.15 and +0.10 or between +0.25 and +0.55 when the video jumps. Now I now the values, but how do I translate that into my DePanStabilize function? So I have the sweet spot at around 0.18 and I want the picture to shift one pixel up when going over that threshold. How do I do that?

Thanks for the help in advance!


Same problem. Fizik how do I compensate for a vertical dy = 0.01 to 0.04 pixels frame motion?

Here is a screenshot of DePanStabilize output.

http://img213.imageshack.us/img213/4462/motionpm8.png

[Fizick]

Please do not cross-post.
Are your source is progressive? If yes, here is the steps:

1. find appropriate crop values to remove almost all besides logo. Not crop logo at any frame. And use cropped width and height as power of 2 (i.e. 16, 32, 64, 128, 256)

mpeg2source(...)
c=crop(100,200,300,400)
return c

2. Try analyse the motion
mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dx=0)
depan(last,data=m, info=true)

(logo is very small to use directly depanestimate(c, info=true))

(You can also produce log file to analyze and process it somehow)

dy = 0.01 to 0.04 pixels is very very little values :)
Probably you will get around 3.0 to 10.0

Chack scenechange too.

4. Next step is stabilization.
mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dx=0)
depanstabilize(last,data=m, info=true,cutoff=2)

Try tune some parameters, for example cutoff

But I am not sure that DepanStabilize can give usable result for your case. (NO WARRANTY :) )

[oddball]

When I open up the Avisynth in Vdub I get an error.


Script error:depanestimate does not have a named argument "dx"

I am using 1.9.0. dx=0 does not work.

[Fizick]

NO WARRANTY ;)

may be dxmax ? (I did not try it)

[oddball]

NO WARRANTY ;)

may be dxmax ? (I did not try it)

Hey it's your plugin lol :D

Oh well looks like the warranty is void. Depan simply cannot deal with small vertical movements. Thanks anyhow.

[Fizick]

I did not try that script right now...
(ones upon a time i tested similar)

what dy shift have you got?

[oddball]

It shifts between 0.01 to 0.04 up and down. Using the logo was no good. Depan was not sensing any movement on it. If I use the previous posters line 712 which is pretty much where my bottom border is then it detects the small motion up and down. It detects it but DepanStabilizer cannot move the frame by such small increments up and down to correct the vertical motion.

[Fizick]

I am sure you still do get right settings for Depanestimate.
What is real shift?
0.04 pixels is invisible.
Post your real script like it:

mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dxmax=0,info=true)

And why you do not use v1.9.1 ? ;)

[oddball]

I initially tried

DirectShowSource(video.m2v)
c=crop(1058,570,76,80)
m=depanestimate(c, dxmax=0)
depanstabilize(last,data=m, info=true,cutoff=2)

But it does not detect any movement whatsoever where the logo is (It is correctly set as i can see the logo only when I add 'return c').

So I tried.

c=crop(0,712,0,0)
m=depanestimate(c, dxmax=0)
depanstabilize(last,data=m, info=true,cutoff=2)

Like a previous person used and it detects the movement and displays it in Depan's output but it does not stop the movement up and down.

I am using 1.9.1 now. No difference though.

[oddball]

Would it be helpful if I uploaded a clip from the movie for you to look at?

[Fizick]

Logo is not moving or depanestimate do not detect its movement?

upload short compressed clip.

[Fizick]

if movement between frames is always really so little, and it slow accumulates to large values, then some new specific algorithm must be implemented (for example, motion estimation across large interval). But I have no plan to do it right now.

[AVIL]

Hi,

A mad idea. If you suffle the frames, do depaning and reorder it ¿could result?.

As example:

a=clip.selectevery(10,0,5,1,6,2,7,3,8,4,9)
v=a.depanestimate()
d=a.depanstabilize(v)
d.selectevery(10,0,2,4,6,8,1,3,5,7)

[scharfis_brain]

I get some weird greenish tint when I try to use depanstabilize at default values.

It is best noticable with greyscale videos.

Is this issue known?

[odyssey]

I have trouble writing a log file. What's the correct syntax for it?

I'm trying to stabilize some of the very old disney shorts, which are badly transferred resulting in a very jumpy clip. Base frames seems to be detected quite well, but i'm not so happy with the correction itself. I've been tweaking some of the parameters with different results. My main concern is that when i specify dxmax/dymax in both DePanEstimate and DePanStabilize, there are still cases where it corrects with insane values.

I dislike that it focuses so much on the center, because it appearently often detects a moving character as a reference instead of the actual scene.

I don't really understand the way this works. I would expect it to take the base frame and use that frame as a reference for correcting other frames, but as I understand, it only corrects based on the range, that is the surrounding frames (is this correct?).

Can someone enlighten me? Also I don't understand the meaning of parameters cutoff, damping and trust.

[odyssey]

Bump. Could someone please clear up my confusion in the last post?

[DarkT]

Can't help you there, but I share your pain - I messed with the whole stabilization thingy for a couple days, and gave up on it - no matter the settings, sometimes it messed up big-big, detected stuff incorrectly, and so on... The only solution I got to was use really small values, and then record frames where it mis-detects, and then, theoretically, not apply it to those places... Which is all too much work then I'd care to do, so I dropped it...

Edit:

In case you wonder what I mean by small values, here's what I used:

i = ConvertToYV12()
mdata = DepanEstimate(i,dxmax=2,dymax=2,range=3,stab=1.0)
DePanStabilize(i, data=mdata)

[johnmeyer]

If you don't mind using Deshaker instead, it does a VERY good job of motion stabilization, if that is the only thing you need to do. I wrote a very detailed guide several years ago which you can find here:

http://www.sundancemediagroup.com/articles/deshaker_guide.htm

It was then ripped off and posted at dozens of other places around the Internet, so you can easily find it elsewhere (in case you have problems linking to the above site). Just Google my name and "Deshaker."

[DarkT]

Ah, thanks, I was put off by the author's original guide, he wrote there:

"Deshaker works best for video shot using high shutter speeds (at least 1/200 sec or so). Otherwise you might get motion blur without motion in the deshaked video. Also, the matching will work better."

I need it to stabilize ANIME - should I still give it a try?

[johnmeyer]

I need it to stabilize ANIME - should I still give it a try?
I do a LOT of work with film, both film that I transfer with workprinter as well as film that I inverse telecine back to 24p. Deshaker works wonders eliminating gate weave/judder (the slight motion introduce because the film doesn't come to rest exactly the same way from one frame to the next, either in the camera when it is being shot, or in the projector when it is being shown). In fact, the guy that wrote Deshaker even added a few settings specifically for a guy that wanted to get rid of this problem, but without affecting the global motion. In general, you want to use really "mild" settings so that only the small motion from gate weave is eliminated.

[odyssey]

Great. I'll look into it :D

Addendum:
Another thing, I've messed slightly around with the Virtualdub filter plugin and Deshaker in avisynth. Is it possible to set the nessesary parameters all inside a script and avoid the vdub step?

Depan should be able to read these log files (although I haven't had any success with it yet), would it make any difference, for Depan to do the 2nd pass instead of deshaker?

[johnmeyer]

I wrote a script for the editing program, Sony Vegas. That script communicates with Deshaker through the VirtualDub scripting language. If you want to take a look at what I did, you can download that script here:

http://www.lawrence-arnold.com/Vegas/DeshakerScript.htm

The .js portion is the script for Vegas which you can read in Notepad. If you go about 3/4 of the way down in that script you will see the call I make to VirtualDub and will see how I pass parameters into the VirtualDub script. It is a real kludge because the VirtualDub scripting was something that was added as an afterthought and has never really been enhanced or supported. However, the kludge works and I have a lot of happy Vegas users, especially since Deshaker actually works better than most of the commercial solutions.

[DarkT]

Thanks a bunch, I'll be trying it out as soon as I encode my next messy episode.

[johnmeyer]

After you run the "deshake.exe" file (which is just a zip file), this will put all the files in the c:\deshake directory. That is all it does (no registry changes or anything else). As soon as the files are unzipped, immediately open the readme.txt file. It will tell you (I think) everything you need to know for your purposes.

[odyssey]

Since this is the Depan-thread, I would really like to know how i'm able to load (and save) a log-file as described in on the web page.

It's interresting, if i'm able to scan the video with eigher Depan and Deshaker, but use Depan only for stabilizing the image, especially if i'm able to make corrections to the log-file.

[johnmeyer]

I would really like to know how i'm able to load (and save) a log-file as described in on the web page.
If you are talking about using the Deshaker log file for other plugins/filters, I have seen that discussed in many places, including the home page for each of the filters and here on the forum. Try doing a search for "Deshaker" AND the name of the filter that you want to have read the Deshaker log file.

[Fizick]

odyssey, syntax is simple as it can be:
DePanStabilize (source, data=source, ..., inputlog="deshaker.log")
So, you are able to read (at last old Deshaker 1.x versions - i did not test recent).
But log file does not contain "stabilized" data. It contains estimated motiion data only.
so, you cant change intrenal stabilization algo

[odyssey]

odyssey, syntax is simple as it can be:
DePanStabilize (source, data=source, ..., inputlog="deshaker.log")
So, you are able to read (at last old Deshaker 1.x versions - i did not test recent).
But log file does not contain "stabilized" data. It contains estimated motiion data only.
so, you cant change intrenal stabilization algo

Why is a data source needed, if an input file is specified? I thought you could get past the estimation part, and concentrate on stabilizing?

[Fizick]

its dummy data

[odyssey]

its dummy data

So it could be anything? How can I send dummy data without wasting CPU?

[Fizick]

any way :)

[odyssey]

Got it ;)

At which stage is the BASE frame detected? I have trouble with some scenes where it cannot find a BASE frame and just counts on, and other scenes where a BASE frame is picked too often.

It seems like the log file contains just cordinates, but they are not equal to the cordinates displayed with info=true.

It would be great if I were able to get a pretty good estimation logged and are able to edit out a few wrongs manually.

This is the intro of one of the old disney shorts i'm working on (http://www.megaupload.com/?d=D2T5QSGI), and I'd assume it's an easy task for any image stabilizer as no elements in the picture are not moving (just the film). Still I have problems with the duck-image and the title, where BASE frame is simply ignored and no correction made.

[anton_foy]

I have tried depan very little but here is ( IMHO ) a great way to convert 50i footage to look like it has been shot at 15 fps, "silent film"/super-8 -style.


input = input.nnedi()

vectors = depanestimate(input)
input_1_of_12 = input.duplicateframe(0)
input_2_of_12 = input.depan(vectors,offset= 1./12.,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset= 2./12.,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset= 3./12.,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset= 4./12.,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset= 5./12.,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset= 6./12.,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset= 7./12.,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset= 8./12.,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset= 9./12.,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10./12.,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11./12.,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,
\ input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)



ChangeFPS(15,linear=false)
assumefps(25,true)


I have not seen any artifacts when using this script, very cool!


Edit: mostly silent film was shot at 16 fps rather than 15 but it has occured.

[johnmeyer]

That's a great script. Now if only someone could post one that would do a good job of taking 30p video taken with the "movie" function on still cams, and turn it into decent 60i. I've tried various things with mvtools (which may or may not be a better tool for this than DePan), but so far have not gotten consisten results.

[odyssey]

Could you please try to describe the use of cutoff and damping values? I don't understand how to use these, but I think they have a significant impact on my trouble.

[Fizick]

do not touch damping.
try firstly play with cutoff.
this thread is 12 pages only :)

[odyssey]

do not touch damping.
try firstly play with cutoff.
this thread is 12 pages only :)

I did, but still don't really understand it. If I set it to 0.1, the intro is almost perfectly stable, but throughout the movie it only corrects few frames at a time.

Setting it to an insanely high value (say 100) makes the picture shake around randomly with insane correction values - Why is that?

Regardless of any value I put, the duck-image is not detected as a scene - Why not?

[Fizick]

I can not use megaupload in Russia.
My internet is not cheap (about 1 USD per 10 MB). :(
So, I do no know nothing about your "duck" :)
As I wrote above, DepaStabilize was designed not for movie, but for camcorder video.
Try apply DePanstimate on cropped border only of your films if it present and stabilize whole frame.
Try watch "trust" values and scenedetection.

[odyssey]

How do i watch trust values?

[Fizick]

info.

[odyssey]

info.

I see the trust value but have no idea how to read it. Also I don't understand how it relates to scene changes - I have a clip where even identical frames are recognized as "SCENE CHANGE!" with default trust=4. Setting trust below 1 gives a very steady picture, but it seems odd to use such low values.

[Fizick]

I see your message, but i have no idea how to read it :)
try live with this odd values

[odyssey]

I see your message, but i have no idea how to read it :)
try live with this odd values

Can you explain what the value represents?

[Fizick]

some measure of correlation.
I can not to add almost nothing to plugin doc and info collected in this (and some other?) thread at forum.

[Fizick]

But now I can release version 1.10 ;)
DepanStabilize: added new 'average' method,
small bug fixed in edge interpolation.

(But I am still not ready to rewrite many very old functions.)

[odyssey]

Would it be possible to implement a "invert mask" flag, so estimate will work on borders instead of central elements?

[Roco]

Is it possible with these plugins to achieve an almost absolute stabilization with less than a pixel resulted shaking if you define a static object in the scene?
The camera is shooting almost stable, but I want to change that to the degree that the shaking won't be detected visually. There are static elements on the background and also moving ones. The camera is actually on a tripod but due to strong wind or my hand holding a button, or a not so stable tripod etc, there is some shaking.
I have tried some scripts but I can't make it that stable (far from that) but maybe I'm missing something.
For example I tried the following where I defined a focus-window of a stable object:

AviSource("Final.avi")
source = ConvertToYV12()
focus = source.Crop(1108,916,512,162)
vectors = focus.MVAnalyse(isb = false, pel = 2)
globalmotion = source.MVDepan(vectors, zoom=false, rot=false, pixaspect=1.0, thSCD1=400)
DepanStabilize(source, data=globalmotion, cutoff=0.05, mirror=0, pixaspect=1.0)

[johnmeyer]

Have you tried Deshaker (VirtualDub plugin)?