GenMotion and DePan plugins for global motion compensation

[Fizick]

GenMotion - General Motion Compensation
C-Plugin for Avisynth 2.5
http://bag.hotmail.ru

This plugin makes general motion compensation (generates a motion),
using ready-made general motion estimation data.
As a such data, the filter uses a log file of Deshaker VirtualDub plugin (by Gunnar Thalin).
This plugin may be used for:
- simple partial motion stabilization (not as good as Deshaker);
- motion compensation for strong temporal denoising and clip (film) restoration.

P.S. Now I know: correct English word is GLOBAL, not GENERAL. :rolleyes:

P.P.S. The GenMotion plugin is now obsolete and not supported.

Now I release once more similar plugin! :devil:
It has got name DePan - tools for estimation and compensation of global motion (pan).
The DePan plugin replaces my slow experimental GenMotion C-plugin.
DePan is native Avisynth 2.5 plugin.
The DePan works without Deshaker, in one pass, has many options.
See posts at page 3 and below.
The download URL (see depan link)

http://bag.hotmail.ru

[morsa]

Fizick read this post


http://forum.doom9.org/showthread.php?s=&threadid=66682


I guess you are getting to something good!!

[Mango Madness]

though I haven't tried it out yet, the process seems a little cumbersome. Would it be possible to write a tool to automate some of the process?

[Fizick]

May be, it is possible to load Deshaker in avisynth.
But I dont know how.

[Mango Madness]

it's in the included avisynth documentation concerning how to import vdub filters.

[Fizick]

New version 0.5, 28 December 2003 - corrected formula for delta<0 (thanks to Gunnar Thalin), some bugs fixed

[Kintaro]

I never heard of deshaker before, and was pretty amazed when trying it for the first time. Unfortunaly the vdub filter works MUCH better compared to GenMotion, and so I tried importing it. Whenever Im importing it vdub crashes instantly, and Im not good enough to know whats wrong


function VD_DeShaker(clip "clip", int "xy", int "mode", int "block_size", int "differential_search_range",
\ float "source_pixel_ar", int float "source_pixel_ar_preset", float "destination_pixel_ar",
\ int "destination_pixel_ar_preset", int "dest_width", int "dest_heigth", int "scale", int "use_pixels",
\ int "color_mode", int "motionsmooth_horp", int "motionsmooth_verp", int "motionsmooth_rota",
\ int "motionsmooth_zoom", int "move_wrong_direction", int "video_output", int "edge_compensation",
\ int "resampling", int "skip_if_blocks_ok", int "initial_search_range", int "discard_match_value",
\ int "discard_2nd_best_match",
\ string "logfile", bool "append_log", bool "ignore_image_area_out", int "ignore_out_l", int "ignore_out_r",
\ int "ignore_out_t", int "ignore_out_b", bool "ignore_image_area_in", int "ignore_in_l", int "ignore_in_r",
\ int "ignore_in_t", int "ignore_in_b", int "video_type_a", int "video_type_b", float "extra_zoom",
\ int "maxcorrect_horp", int "maxcorrect_verp", int "maxcorrect_rota", int "maxcorrect_zoom",
\ bool "use_previous_to_fill", bool "use_future_to_fill", int "previous_frames", int "future_frames",
\ bool "let_area_follow_motion"){
LoadVirtualDubPlugin(VirtualDub_plugin_directory+"\Deshaker.vdf","_VD_DeShaker",1)
return clip._VD_DeShaker(default(xy,4), default(mode,1), default(block_size,30),
\ default(differential_search_range,4), default(source_pixel_ar,1), default(source_pixel_ar_preset,0),
\ default(destination_pixel_ar,1), default(destination_pixel_ar_preset,0), default(dest_width,640),
\ default(dest_heigth,480), default(scale,1), default(use_pixels,2), default(color_mode,1),
\ default(motionsmooth_horp,400), default(motionsmooth_verp,400), default(motionsmooth_rota,400),
\ default(motionsmooth_zoom,1500), default(move_wrong_direction,4), default(video_output,1),
\ default(edge_compensation,0), default(resampling,2), default(skip_if_blocks_ok,5),
\ default(initial_search_range,40), default(discard_match_value,300), default(discard_2nd_best_match,4),
\ default(logfile,"C:\deshaker.log"), default(append_log,false) ?1:0, default(ignore_image_area_out,false) ?1:0,
\ default(ignore_out_l,0), default(ignore_out_r,0), default(ignore_out_t,0), default(ignore_out_b,0),
\ default(ignore_image_area_in,false) ?1:0, default(ignore_in_l,0), default(ignore_in_r,0), default(ignore_in_t,0),
\ default(ignore_in_b,0), default(video_type_a,0), default(video_type_b,0), default(extra_zoom,1),
\ default(maxcorrect_horp,15), default(maxcorrect_verp,15), default(maxcorrect_rota,5),
\ default(maxcorrect_zoom,15), default(use_previous_to_fill,false) ?1:0, default(use_future_to_fill,false) ?1:0,
\ default(previous_frames,30), default(future_frames,30), default(let_area_follow_motion,false) ?1:0)
}


please help me out :)

[Kintaro]

after sleeping ovet it I just found this evil little thing
\ float "source_pixel_ar", int float "source_pixel_ar_preset",
now its not causing crashes anymore, but Im getting invalid arguments error. any ideas?


edit:
somehow seems to me like this filter is incompatible, even when Im just using the parameter from vdub it says invalid arguments :/

function VD_DeShaker(clip clip){
LoadVirtualDubPlugin(VirtualDub_plugin_directory+"\Deshaker.vdf","_VD_DeShaker")
return clip._VD_DeShaker(4,1,30,4,1,0,1,0,640,480,1,2,1,400,400,400,1500,4,1,1,2,5,40,300,4,
\ "C:\Deshaker.log",0,0,0,0,0,0,0,0,0,0,0,0,0,1,15,15,5,15,0,0,30,30,0)
}

hit me if Im doing something wrong :D

[Fizick]

New version Genmotion 0.6, 05 April 2004 - add bicubic interpolation (set as default), add grey parameter, remove deflick, correct first and last frame calculation, some reorganization, some speed optimization, use Avisynth C version 0.15, released with source under GNU GPL 2

[SoonUDie]

Fizick, is it possible to turn this into a one-pass plugin? It seems that the deshaker code is closed-source, so you'd have to figure out your own method of getting the motion vectors... but once you do, 1-pass would be relatively easy to do: calculate motion vectors for each cut, then apply the compensation, then move on to the next cut...

Also, I have an idea I've been toying around with for a while - a temporally-based resizer. Perharps you would be interested in coding it? You start out by motion-compensating a scene with sub-pixel accuracy, and output at 200 percent original size. Then, you give unique pixels (the ones obtained through motion compensation) priority over interpolated pixels, because they contain real detail. You can do this because "unique" pixels will not directly overlap from frame to frame; in the best-case scenario of a 1/2 pixel accuracy version, you would be overlaying an image like this:
1 i 2
i i i
3 i 4

Onto an image like this:
i 5 i
6 i 7
i 8 i

to get the final image
1 5 2
6 i 7
3 8 4

Where the numbered pixels are all non-interpolated.

It's not exactly that simple, but I'm sure you get the jist of it.

[SoonUDie]

The 0.6 zip file appears to be corrupted (after re-downloading 3 times...).

[Fizick]

to SoonUdie:
1. temporally-based resizer is some interesting (but not for me right now)
2. Now I try to made one-pass plugin. (I have alfa version).
3. My download quite fine. Anybody else ?
(I can send it by e-mail, if you want. My e-mail is on my WWW).

[SoonUDie]

It turned out that the file I kept downloading was an html file renamed as a zip file. I think it is a problem with the file host.

You may want to change the link to this:
http://www7.newmail.ru/messages/file.dhtml?file=bag.hotmail.ru/genmotion/genmotion06.zip

[kassandro]

Fizick,
as I can see from the source code of your plugin. The motion information retrieved from the deshaker log file is only 16 bytes per frame (or more precisely per field). Of course, to "deshake" a clip only global motion information is of interest. On the other xvid has a much more sophisticated motion compensation algorithm which is local and not global in nature. If you could transfer this part of xvid to your plugin it could perhaps be improved significantly and may be even faster.

[SoonUDie]

An intersting idea Kassandro, but what would you do on a more local level? Once the camera's been corrected for, what's left?

Perharps, after determining the global motion, the relative motion of macroblocks could be zeroed if under a certain value, potentially increasing compressibility.

... but in any case, I consider full-screen deshaking more important, as you can take care of most other temporal issues with a simple temporalSoften().

[kassandro]

Originally posted by SoonUDie
... but in any case, I consider full-screen deshaking more important, as you can take care of most other temporal issues with a simple temporalSoften().
You certainly did have this thought in mind, when you layed out your temporal resizer. For it you really want to exploit any motion information, to keep normal interpolation at a minimum.

[Fizick]

Kassanro,
I know about the one attempt to use the local motion algorithm from XVID.
Manao make a draft of such plugin.
http://atlas2.tgv.net/~media-video/forum2/viewtopic.php?t=3778&sid=03066b1d2f0d7f57e8f40bb956da64e2

(French forum)

I try it. The motion vector quality is not very good.

For my old 8 mm kino film, the most important is camera pan.
Of cource, it is not universal.

But good result may be by combining global motion plugin and local motion plugin (DUST).

[Manao]

I'm working on it : the motion estimation is now slightly better than Xvid's one, because I use more predictors than they do. It's much slower though.

I'm currently trying to make the code cleaner, and I don't know yet when I'll be able to release something usable.

[SoonUDie]

One suggestion for improvement: Can you add parameters for max variatians? I've found that Deshaker can be very bad at detecting rotation in motion scenes, so I'd rather just leave it out. I wrote a program to zero the information in the log files, and it worked out well - almost no visual change in low-motion scenes as compared to using rotation, and no problems with artificially introduced rotation. However, it would be much simpler if you could just turn off rotation in the filter.

[Fizick]

O.K., GenMotion version 0.7 is released :

Add parameters dxmax, dymax, rotmax, zoommax.

dxmax - maximum value of x shift in motion data, (default = 100),
dymax - maximum value of y shift in motion data, (default = 50),
rotmax - maximum value of rotation (degree) in motion data, (default = 5),
zoommax - maximum value of zoom in motion data, (default = 1.2),
(if any above max parameter is positive, the motion data value will be limited to it,
if negative, the motion data value will be reseted to null )

P.S. tested very little - try.

[morsa]

Well, somebody tell me I'm terribly wrong, please.
I guess, seeing that a good motion estimation is on its way, that in a relative near future we will have the possibility of making a good frame synthesizer.Is it OK?

[SoonUDie]

Originally posted by morsa
Well, somebody tell me I'm terribly wrong, please.
I guess, seeing that a good motion estimation is on its way, that in a relative near future we will have the possibility of making a good frame synthesizer.Is it OK?
If what you mean is that we'll be able to accurately interpolate any framerate, then yes... and no. I believe some very good solutions already exist... however, they're many hundreds of dollars. The key is to relatively accurately find motion vectors for every pixel in the image.

However, while we could interpolate something like a 24FPS movie to 60FPS, how much do we really gain? We're not really re-creating the motion here... it's like taking an antiderivative - you have a +c floating around, and we don't know what that is.

For example, take the motion of a really fast moving ball. If we capture it at 30fps, it'll be a blur. At 60fps, it will be more like a ball, but still blurry. At 120fps, it'll look even more like a ball. Now let's say we interpolate that 30fps video up to 120fps. It won't look like a ball - you'll still have the blury 30fps picture - but it'll move 4x smoother: it'll traverse only 1/4 the pixels per frame as it did before.

I'd still like to see it happen, though, and preferably in real time - we live in a world where 60p is finally becoming the standard, and I'm starting to get tired of 24fps...

[morsa]

Nice answer :)
One idea I have would be to use it to recover lost frames, may be 2 or three in a row, or to convert from 24 fps to 30, 25 to 30, or better, 50 fields to 60...Who knows, there are so many options.....

[mf]

Most interesting is to synthesize frames that are in an odd position between frames, e.g. convert 30fps to 24fps, without blending.

[Fizick]

1. MF answer is nice, but not quite correct. ;)
If we capture fast moving ball at 30fps, it'll NOT must be a blur.
Camera's (camcorder) frame shooting time (exposition) is dependent on record mode ("normal", "sport", "golf"). This time of course is not large than 1/fps (1/60 sec), but may be quite small for bright scenes (up to 1/2000 sec). The fast moving ball may be quite sharp, and framerate interpolation (with motion compensation) will be sharp too.

2. GenMotion may be used, for example, to change framerate with factor=1.5, from 16.6 fps (old 8 mm film) to 25 fps.

Script:

Avisource("kino.avi")
loadplugin("avisynth_c.dll")
converttoYV12()
loadCplugin("genmotion.dll")
i=ConvertToYV12()
f1_3=GenMotion(i,filename="c:\deshaker.log",delta=1./3)
b1_3=GenMotion(i,filename="c:\deshaker.log",delta=-1./3)
interleave(f1_3,i,b1_3)
selectevery(6,0,1,2)
Crop(48,32,720-48-32,576-32-32).AddBorders(48,32,32,32)
converttoYUY2()

[SoonUDie]

Originally posted by Fizick
If we capture fast moving ball at 30fps, it'll NOT must be a blur.
Yea, I was just using that as an example. Besides, any object of any speed will have varying motion blur depending on how close it is to the camera. For the sake of the example, please assume that it is fast enough / close enough that it is simply a blurred streak.

Even then, I believe what I said still holds - you're not actually recreating temporal detail; you're simply morphing objects into a spot that is inbetween the real frames.. If you wanted to more accurately recreate the detail, you'd need to perform some serious "un-motion-blurring" in addition to frame interpolation.

[morsa]

SoonUdie.
You are completely right, but I was just talking about a normal situation.Everybody knows a filter doesn't performs 100% OK with every situation, if this were the case what would you and I be doing here? :D
Anyway all of the really pro, really expensive software packages have the same limitations.That's why they ask the user to use high shutters.

[morsa]

I was wondering if I could use this approach to go from a 30p video clip to 60i to end with a 24p version of it.
I don't know how to set delta to get a double framerate from genmotion, so I could interleave the frames and obtain a 60i video.
Then I would use Scharfish's script to go from 60i to 24p.I'm sure this should work (although it won't be fast or easy) but I've got problems getting the correct params.My clip is from a JVC HD10U camcorder 1280x720.

[scharfis_brain]

@morsa. I've never used genmotion.
but why not going to 120fps and then doing a 1 out of 5 selection.
this would IMO be better than 60fps an then alternate blending (2x / 3x / 2x /3x ....)

[morsa]

Great idea!!
So now the question would be: How can I go from 30p to 120p with genmotion?

[scharfis_brain]

okay, after 10 minutes of testing, I've got running a framerate-doubler:

# script to create doubled frameratewith genmotion

loadplugin("C:\x\avisynth_c.dll")
loadCplugin("c:\x\genmotion.dll")
avisource("30p.avi")
i=converttoyv12()


f=GenMotion(i,filename="c:\deshaker.log",delta=0.5).trim(1,0)
b=GenMotion(i,filename="c:\deshaker.log",delta=-0.5)

# mix fwd. and bckwd. compensation
g=overlay(f,b,opacity=0.5)

# make a 50:50 blend of the input
g1=overlay(i,i.trim(1,0),opacity=0.5)

# mix the blended input into the compensated video to achieve some kind of stability
g2=overlay(g,g1,opacity=0.3)

#double the framerate
interleave(i,g2)

#if the output of the interpolated frames is to bluury, try to reduce the 2nd opacity=0.3 value, or use
#interleave(i,g)

I'll post a quadrupler, when I am coming back from work (about 12hrs.)

and I think, we can nail static items in the video, so they aren't compemnsated, thus resultingin jittery station logos etc.
this would require introducing motionmasking with some tricks.

[scharfis_brain]

okay, the quadrupler has to wait.

For now, I've made a motioncompensator giving you a mostly clean output, avoiding swimming static parts of the video (TV-Station logos)
and avoiding false blending in differential motioned areas.
in both cases those affected areas are replaced with a blended clip of the neighboring frames.

this means:
a pan will be super-smooth.
a ball flying through the video while the panning occurs will have some kind of stutter, because it is NOT motioncompensated.

setmemorymax(256)
# script to create doubled framerate with genmotion
# while avoiding swimming static parts and false motion generated by genmotion

loadplugin("C:\x\masktools.dll")
loadplugin("C:\x\avisynth_c.dll")
loadCplugin("c:\x\genmotion.dll")

avisource("30p.avi")
i=converttoyv12()

th=7 #threshold for static detection
th1=7 #threshold of false-motion detection

#lower those thresholds for lesser repairing


#generate statice mask
i1=motionmask(i,thy1=th1,thy2=th1,thsd=255)


# make motion global compensation of the video
f=GenMotion(i,filename="c:\deshaker.log",delta=0.5).trim(1,0)
b=GenMotion(i,filename="c:\deshaker.log",delta=-0.5)

# motion compensate the static-parts-mask
f1=GenMotion(i1,filename="c:\deshaker.log",delta=0.5).trim(1,0)
b1=GenMotion(i1,filename="c:\deshaker.log",delta=-0.5)


# mix fwd. and bckwd. compensation
g=overlay(f,b,opacity=0.5)

# mix the motioncompensated masks
g1=logic(f1,b1,mode="And")

#generate a falsemotionmask
g2=interleave(f,b).motionmask(thy1=th,thy2=th,thsd=255).selectodd().binarize().duplicateframe(0)

#merge static and false mask
g3=logic(g1,g2,mode="and") .blur(1).blur(1).binarize(threshold=1,upper=false)

# genrate blended clip out of the input to repair swimming static parts and false generated motion
h=overlay(i,i.trim(1,0),opacity=0.5)

# repair genmotion's output
j=overlay(h,g,mask=g3)


# for vis.
ih=interleave(i,h)
ig=interleave(i,g)
ij=interleave(i,j)
a=stackhorizontal(ih,ig)
b=stackhorizontal(ij,interleave(g1,g1))
stackvertical(a,b)

#stackhorizontal(g2,g1,g3)

#output the final clip
ij


my next ideas (after the quadrupler) will be:

NTSC <-> PAL conversion using this approach.
It will give super smooth pans without this annoying blending-motion on borders. while differnetial motion is converted via the standard blending method.
this will need a framerate 5 times as high as the input framerate for a NSTC -> PAL conversion.
and a framerate 6 times as high as the input framerate for a PAL -> NTSC conversion.

[morsa]

But I just wanted a converter from 30p to 24p :(
(just kidding)But in fact I just wanted that so the 120 fps approach should be fine for me, I guess......:rolleyes:

[scharfis_brain]

Yes. You have to wait a little bit.
I have to restructurize my script, to do quad framerate.

this needs 3 times more motioncompensation!

[morsa]

Thank you, really:thanks:

[scharfis_brain]

function mocomp(clip i, float amount, string file, int "th", int "th1",bool "vis")
{
th=default(th,7)
th1=default(th1,7)
vis=default(vis,false)

#generate static-area mask
im=motionmask(i,thy1=th1,thy2=th1,thsd=255)

# make motion global compensation of the video
i2f=GenMotion(i,filename=file,delta=amount).trim(1,0)
i2b=GenMotion(i,filename=file,delta=amount-1)

# motion compensate the static-parts-mask
i2fm=GenMotion(im,filename=file,delta=amount).trim(1,0)
i2bm=GenMotion(im,filename=file,delta=amount-1)


# mix fwd. and bckwd. compensation
g2=overlay(i2f,i2b,opacity=amount)
g2= vis ? g2.coloryuv(gain_u=48) : g2

# mix the motioncompensated masks
g2m=logic(i2fm,i2bm,mode="And")

#generate a falsemotionmask
g2mm=interleave(i2f,i2b).motionmask(thy1=th,thy2=th,thsd=255).selectodd().binarize().duplicateframe(0)

#merge static and false mask
g2mmm=logic(g2m,g2mm,mode="and").blur(1).blur(1).binarize(threshold=1,upper=false)

# genrate blended clip out of the input to repair swimming static parts and false generated motion
g2bl=overlay(i,i.trim(1,0),opacity=amount)
g2bl= vis ? g2bl.coloryuv(gain_v=48) : g2bl

# repair genmotion's output
g2rep=overlay(g2bl,g2,mask=g2mmm)
g2rep= vis ? g2rep.subtitle(string(amount)) : g2rep

return g2rep
}

this is the new motencompensation function

amount: float value between 0 and 1.
0 returns curr. frame
1 returns next frame
0.5 returns inbetween frame
0.25 the frame between 0 and 0.5
(and so on)

file: string to the location the deshaker.log (or whatever it is called)

th: int for false motion (differential motion) detection

th1: int for static area detection

vis: enable visualization
blue - motion compensated
red - blended (repaired areas)

usage:

general:
loadplugin("C:\x\masktools.dll")
loadplugin("C:\x\avisynth_c.dll")
loadCplugin("c:\x\genmotion.dll")

avisource("30p.avi")
i=converttoyv12()

possibilities:

framerate doubler:
i1=i
i2=i.mocomp(0.50,"c:\deshaker.log")
interleave(i1,i2)


framerate quadrupler:
i1=i
i2=i.mocomp(0.25,"c:\deshaker.log")
i3=i.mocomp(0.50,"c:\deshaker.log")
i4=i.mocomp(0.75,"c:\deshaker.log")

interleave(i1,i2,i3,i4)

30p to 24p conversion:
i1=i
i2=i.mocomp(0.25,"c:\deshaker.log")
i3=i.mocomp(0.50,"c:\deshaker.log")
i4=i.mocomp(0.75,"c:\deshaker.log")

interleave(i1,i2,i3,i4)

selectevery(5,1)

hope that helps.

[morsa]

Amazing results!!
I can't believe what I'm seeing came from 30p.Almost perfect to my eye:)
Although it has a little ghosting sometimes in sensitive areas as people heads with white hats more or less near the camera...

[scharfis_brain]

ghosting (or blending, to avoid stutter) is only present in areas with differntial motion.

but it is better live with little blending in those areas instead of haven frames full blended.

even a NTSC<->PAL conversion using this thingie is looking very smooth to me, no more annoying backward/forward motion on borders due to blending :)

theorethical:

NTSC to PAL

i1=i
i2=i.mocomp(0.2,"c:\deshaker.log")
i3=i.mocomp(0.4,"c:\deshaker.log")
i4=i.mocomp(0.6,"c:\deshaker.log")
i5=i.mocomp(0.8,"c:\deshaker.log")
interleave(i1,i2,i3,i4,i5)
changefps(50) #NO convertfps here! it will destroy the image

PAL to NTSC

i1=i
i2=i.mocomp(0.167,"c:\deshaker.log")
i3=i.mocomp(0.333,"c:\deshaker.log")
i4=i.mocomp(0.500,"c:\deshaker.log")
i5=i.mocomp(0.667,"c:\deshaker.log")
i6=i.mocomp(0.833,"c:\deshaker.log")

interleave(i1,i2,i3,i4,i5,i6)
changefps(59.94) #NO convertfps here! it will destroy the image

in practice, we first need to deinterlace the Video (maybe using motioncompensation using genmotion) and later reinterlace it.

I'll try to make a complete conversion fuction out of it.

[scharfis_brain]

@ fizick:
feature request:

could you add something like a forceprogressive-flag?

I want to fullframerate deinterlace a given video. and then do the motioncompensation.

with the current version of genmotion I only get the fields compensaten, not the whole frame.

this means, the number in the log file should be treated like progressive:

laced -> progressive
0B -> 0
1A -> 1
1B -> 2
2A -> 3
2B -> 4
3A -> 5
3B -> 6

would that be possible?

only this modification would it make possible to work without intermediate files when trying to do a standards conversion using genmotion.

[Fizick]

1. I very soon want release a new plugin for global motion compensation,
written from scratch, which will work without Deshaker.
This work take my time.
In have alpha version.

2. About force-progressive, i am not understand you:
Do you want compensate odd field to next odd,
and even to next even?

[scharfis_brain]

1) kewl! does this work in 1-pass ?

2) okay, I'll try to explain it:

- lets assume I have a made bobbed (50fps or 59.94 fps) Video out of my interlaced content.
- this bobbed video gets analysed by deshaker.
- deshaker is creating a (progressive) logfile.

- then I can do my motioncompensation in avisynth with a bobbed video to my wishes using genmotion.

but I want to pass by the first three step. I want to feed deshaker with the interlaced video (thus creating an interlaced logfile), and then want to use genmotion on the bobbed (double framerate, progressive) version of the video.

[Fizick]

1. Yes (but i do not know what is "kewl" ;) )

2. I answer later.

[scharfis_brain]

kewl = cool

same pronunciation, hehe

[Didée]

Originally posted by scharfis_brain
kewl = cool

same pronunciation, hehe
Not quite the same.

cool = [ku:l]
kewl = [kju:l]

;)

[scharfis_brain]

Okay, next time:

k(-j)ewl

[scharfis_brain]

fizick, if your new version of genmotion supports onepass gmc'ing,would this be some kind of master/slave structure?


some pseudoicode:

one=avisource("one.avi")
two=avisource("two.avi")
x=one.genmotion(analyse=true) #where x is a variable containing movement information
output=two.genmotion(analyse=false,motiondata=x)


this means, I want to use the motioninformation of video one to move video two.

this is easily possible with the current 2-pass method, but would that be possible with your next version?

[SoonUDie]

Originally posted by scharfis_brain
this is easily possible with the current 2-pass method, but would that be possible with your next version? I hope it is. It shouldn't be hard, fox example, to have the motion information from the first clip stored in a logfile like deShaker (except the file is generated at run-time).

[Fizick]

Yes, master and slave (or slaves).

[SoonUDie]

Originally posted by Fizick
Yes, master and slave (or slaves). Tsk tsk! That's not politically correct anymore in Los Angeles, where I am. The correct terms are primary and secondary.

(this is a joke!)

[Fizick]

O.K., I release new plugin. It has got name DePan -
tools for estimation and compensation of global motion (pan).
The DePan plugin must replace my experimental GenMotion C-plugin
(which uses motion data from VirtualDub Deshaker plugin log file).
The DePan works in one pass, but consist of server part (function)
and one or more clients parts (functions or its instances).
The server function estimates frames motion data and gives it to client functions on inquiry.

(to SoonUDie: I not quite understand you joke, but decided to use other, safe terminology :D )

Current version of DePan uses a special service clip as the motion data container.

DePan plugin functions:
DePanEstimate (server) - estimates frames global motion data and write it to special service clip
DePan (client) - make full or partial global motion compensation
DePanInterleave (client) - generates long motion compensated interleaved clip
DePanStabilize (client) - makes some motion stabilization

GenMotion() function is new DePan() function.
Note. The "offset" parameter of DePan() is extended version of "delta" parameter of GenMotion().

Features and limitations of current version of DePan plugin
1. Works only in YV12 color format.
2. Uses only pan motion (no zoom, rotation), but it gives advance in speed and stability.
3. The source clip must be same length as motion data clip.
4. Directly works only with progressive clips.
For interlaced sources, you must use Avisynth function SeparateFields,
with followed optional AssumeTFF, and following Weave (after motion compensation and denoising)
Plugin estimates and calculates motion from one field to neighbour (by time) field (from same or neighbour frame).
5. Not very fast, not assembler optimized.

You may download it from my site (link is on first page of thread).
It has beta (may be alpha) status.
Please, test all its bugs (i know some) and report to here.

[SoonUDie]

Thanks! I'll try it out as soon as possible.

[scharfis_brain]

the download link doesn't work :(

[SoonUDie]

Originally posted by scharfis_brain
the download link doesn't work :( There's a typo in the link. It currently says http://bag.hotmail.ru/genmotion/depan01.zip when it should be http://bag.hotmail.ru/depan/depan01.zip

[scharfis_brain]

already tried that!

I am always getting to a blueish site

Save File

but whatever I do, I am in loop, when clicking in the download-link of this blueish site :(

[scharfis_brain]

I HATE Sites being not compatible with non-IE browsers!

[SoonUDie]

Actually, I use Firefox, and I didn't have any problems... you just have to wait until the download starts on its own (10 seconds?).

[Fizick]

Sorry, now i correct the link. It was one of bugs. :(
(By the way, I use Opera browser.)
Probably, I must make a mirror (what is the recomended for free?)

[scharfis_brain]

depan 0.1 has a bug.

depan(offset=x) doesn't work for x being in the range of 1>x>-1
even the info=true screen disappears, if offset is between 1 and -1.

this means depan(offset=0.5) won't work.

[Fizick]

Released version 0.2, April 27, 2004 - bug for non-integer OFFSET values is fixed.

[scharfis_brain]

thanks!

I'll try that out.

I hope, I can realease some kind of 'smoothpan' standards conversion function (NSTC <-> PAL) within the next days
(will be nearly the same code like the one I coded for morsa's 30p to 24p conversion)

[scharfis_brain]

okay it does work.

but:
It seems to work very unprecise
I always blend fwd & bckwd compensation within my scripts for better results, but fwd & bckwd do not always match each other, thus introducing false blending.
depanestimate seems to deliver only quantisized motion information.
Example: if a pan slowly accelerates (pan is getting faster and faster), the motioncompensation doesn't adapt smoothly. It compensates in 'steps'

Depan doesn't compensate zooms.

deshakers analyse is much better compared to v0.2 of depan.

okay, enough critisism

the concept of master/slave is pretty cool!

I've made some tests with my smoothpan standards conversion.
It works! But it isn't very unusable for now :(

can't wait the next version of depan :)

would it be a copyright violation just using deshaker's analyse code within depan?

[Fizick]

The motioncompensation doesn't adapt smoothly. It compensates in 'steps'
Try subpixel=2 (bicubic). Default is =0 (nearest pixel), in next version it will be =2. It is document bug. (also i know about Depanstabilize bug with input paarameters - will be fixed later).

Depan doesn't compensate zooms.
Yes, but it may be implemented later.

deshakers analyse is much better compared to v0.2 of depan.
Yes, but it often has error with rotation.
And depan method (phase-shift) is more fast.

would it be a copyright violation just using deshaker's analyse code within depan?
What code? No any source.
May be, I shall add reading of Deshaker log, as in genmotion, but it only useful with zoom added. I do'nt want to add rotation.

[Fizick]

New DePan version 0.3, May 15, 2004 - fixed bug with DePanStabilize input parameters, set subpixel=2 (bicubic) as default, add zoom estimation and ZoomMax parameter, zoom and rotation compensation, log file input, MatchFields parameter (for best denoising).

[Fizick]

New DePan version 0.5:
improve DepanStabilize and some bug fixed.

Anybody test the plugin? Or it is fully unusable?
Any proposal for next version?

[scharfis_brain]

I do not have access to my PCs to test all the new stuff.
For at least the whole next week :(

please stay tuned. I'll post my results asap, I can get back to the faster machine...

(or: recompile for avisynth 2.08 and Pentium 1, than I can test it on my 120MHz Notebook)

[Fizick]

Version 0.6, May 28, 2004 - minor changed and documented client-server format.
DepanEstimate: add stab parameter, scenechange at sharp trust changes, range may be 0, show correlation, crop output.
DepanStabilize: Freqmax is renamed to Cutoff, add adaptive zoom, Fill borders by neibours frames.

Version 0.7, May 30, 2004 - DepanEstimate: add improve zoom estimation.

[Fizick]

I release a new version of Depan, developed about month ago.
Version 0.9, June 13, 2004 - all clients: added "Mirror" parameter to fill empty borders.

mirror parameter - fill empty borders with mirrored from frame edge pixels (instead of black):
0 - no mirror (default);
1 - top;
2 - bottom;
4 - left;
8 - right;
sum any of above - combination (15 - all ).

Mirror mode is unique but slightly strange ;)
It can give remarkable results for example for sky.

[Fizick]

Serg Belyansky,
You wrote:
Unfortunately one problem persists: any solid surface with a large number of frequent simular elements (e.g. brick wall or polka-dot dress) will exhibit an erratic motion estimation with elements disappearing and reappearing randomly. It can be partially remedied by increasing the "range" parameter of Depanestimate, but it slows the process and introduces artifacts on other surfaces. So I'm still scratching my head.

It is a hard problem with no simple and general answer. :(
But I can give one idea to you (seems i omitted it in doc).
Input clip for DePanEstimate may be not strictly same as source,
but prefiltered, or brightness-contrast adjusted, or masked by some method, or cropped, and so on.

If you get some new ideas from your head after scratching, please post here they, or at least the your scratching method. :)
And do'nt use my DeScratch filter until ! :D

P.S. It seems, i found some nice (and big!) Russian forum thread about Avisynth.
http://forum.ixbt.com/topic.cgi?id=29:9331

[scharfis_brain]

thanks for that great tool, it helps a lot denoising shaky/panning analogue camcorder video.

but there seems to be one issue:

everytime I try to enable zoom compensation, using zoommax > 1, I receive the following error MSG from AVISynth:

AVISynth read error: Avisynth: caught an access violation at 0x025d80f5, attempting to readfrom 0x025ffb10

it seems that there is a memory leak, cause, the value of setmemorymax() is in direct corellation to the framecount that is rendered, until avisynth crashes.

[Fizick]

I found some memory leakage in Depan, but only at constuctor stage (not per frame).
I can not get such errors.
May you give me more info? (Avisynth and plugin versions, script, CPU, etc)

[scharfis_brain]

using this (or similar script)

i=converttoYV12()
mdata=i.DePanEstimate(zoommax=1,improve=false,pixaspect=1.094)
i.DePanInterleave(data=mdata,pixaspect=1.094)
temporalsoften(1,10,13)
selectevery(3,1)

when I set zoommax=2 and improve=true

avisynth crashes after some frames (depends on the value of setmemorymax) with above mentioned error msg.

I am using:
newest AVS build.
newest DePan release
AthlonXP 1600+ with ASUS A7VE-266 and 768 MB of DDR-RAM

[Fizick]

I tried similar script.
At what frame number it crashes (100, 1000, 10000, 100000 )?
And for what Setmemorymax ?
Newest AVS build (20.08.2004) is buggy. See development forum.

[Fizick]

I am sorry, but i really try version 0.8 :scared:
0.9 is buggy, yes. :( (i almost did not test it )

So, good news:
New version 0.9.1 is out. Seems, zoom bug is fixed.

[scharfis_brain]

Many thanks for this bugfix :)

But: I finished my current project before you did the bugfix :(

nontheless, DePan becomes my standard aid for denoising handycam footage. it is just great, cause it is the only thing allowing low contrasted moving detail (like grey trexturized walls) being denoised while their texture remains untouched.

a blockbased motioncompensation without global motion compensation first, is not able to catch this correctly...

[Fizick]

I think about using more fast library (FFTW).

[Serg Belyansky]

Originally posted by Fizick
Input clip for DePanEstimate may be not strictly same as source,
but prefiltered, or brightness-contrast adjusted, or masked by some method, or cropped, and so on.
Yes, I tried it this way. And I must say I'm now think that I have no clear understanding of the process at all.

The motion compensation helps to match neighbor frames, right? But DePan works with global motion only, compensating only for "main" object (background or close-up), not both. This the local motion of "polka-dotted" dress should not affect it at all. And judging by undecimated debug output, it actually does not.

I tried MVTools, which is by design "local" and should compensate for _everything_, making the perfect match between frames. But that's NOT IT -- it actually makes DeSpot to work much worse, with a lot of nasty edge effects on moving subjects. The "disappearing polka-dot" problem is somewhat remedied, but I can't say that the difference is easily noticeable.

So I think I just don't understand the "motion compensation" part well. For example, what "motpn" parameter exactly do with local motion compensation performed?

Thresholding with "sign" parameter helps, but not much, since I can't set an actual threshold: the "polka dots" are not as bright as brightest parts of dirt blobs, but they are still falsely keyed and removed by DeSpot (my film dirt is bright white, scanned from negative).

(Bugreport: the "sign" parameter description in Russian manual is erratic; the values are mixed up.)


P.S. It seems, i found some nice (and big!) Russian forum thread about Avisynth.
http://forum.ixbt.com/topic.cgi?id=29:9331 [/B]

Yes, but my firewall prohibits me from posting there :rolleyes:

[Fizick]

Serg Belyansky,
Your (our) problem is serious.
Yes, DePan works with global motion only.

And DesPot is not ideal solution still.
What "motpn" parameter exactly do with local motion compensation performed? - I wrote about it in doc, you also may see a C source. :)

BTW, I do not understand, what is "polka-dots" ? :confused:
May your post some example image? (Or in russian)

If the most question now is about Despot, please return to Despot thread. May be we can improve it with some ideas (not detect very small dots or very round spots :confused: ).

Thanks for discussion!

[hartford]

Polka Dots (http://images.google.com/images?q=polka%20dot&hl=en&lr=&ie=UTF-8&sa=N&tab=gi)

[Dreassica]

My guess is that he means dot crawl and the like.. could be wrong thre.

[Serg Belyansky]

Originally posted by Dreassica
My guess is that he means dot crawl and the like.. could be wrong thre. No-no-no-no. We are talking about regular structures in an image (such as dress fabric pattern or brick wall pattern) which are prone to be partially removed by DeSpot side-effect.

Fizick, I'll try to look at the sources. Maybe the "sign" parameter can be improved by introducing the thresholds for it.

[Fizick]

I release version 1.0 of DePan.
Since version 1.0, DePan can also use more fast FFTW library version 3 (http://www.fftw.org)
as Windows binary DLL (compiled with gcc under MinGW by Alessio Massaro),
which support for threads and have AMD K7 (3dNow!) support in addition to SSE/SSE2.
It may be downloaded from ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip
For fftw using, you must put FFTW3.DLL file from this package to some directory in path (for example, C:\WINNT).

Please post test (speed) report.

[morsa]

WOW!!
A lot faster now on this Athlon XP machine I'm using to test it!!!
As always Fizick, THANK YOU!!:thanks:

(Taste the Russian advanced knowledge ):D

[Fizick]

I get some e-mail from Andreas J. and decide post it here for discussion:

I trying to use DePanEstimate for scene detection & Extraction of JPEG-FIles.
As far as I have tested, It will do what I want.
I would like to do detect the scenes an extract JPEG thumbnails in one step.
My problem is, that I havent found a way to formulate the if-clause because I Havent Found a way to get the "Trust" Value or the dx-value for each frame
As I Understand your Plugin, it is currently not possible to get these values during runtime.
Is it possible to extend DePanEstimate to do this?

My Ideas for this are:
1) getting the "trust" - value as a variable during the processing of the each frames
in AviSynth a condition like
DePanEstimate.Trust < 3.5 ? ExtractKeyframe(currentFrame):nop()
(Okay, I need todo it on every frame with EvaluateFrame() or ScriptClip() but my Scripting Experiance
ist only only 10 Hours with Avisynth ;-):
2) by adding a function "DePanIsKeyFrame" with nearly the same parameters as DePanEstimate
(as far as I understand DePanEstimate SceneChange will come if myTrust-value > Trust(CurrentFrame)
3) another possibility could ba a function (DePanExtractKeyFrames (trust = 3.5, log = "c:\myfilename, KeyFramesEvery=1500, SkipMultiKeyframes=25…)
The log will consist of lines where myTrust <= Trust (Frame) where
"KeyframeEvery" will force-add keyframes(in this case every minute 25fps*60= 1500 and
"SkipMultiKeyframes" will not allow new keyframes for x frames if another (not forced) keyframe was detected before
(it should prevent to get 25 keyframes a second…. )
The log should also contain the trust-value and a info if KeyframeEvery-conditions was positive
4) my last idea is: a extlog="c:\myFilename.log" the result is a the same as the "log=" -LogFile
plus 2 additdional values: the trust value and "Scenechange"-Info.
So it would be goot to parse the logfile with PHP.
But a second step is needed to extract the thumbnails.

A Perfect solution would be a combination out of 1) & 3) & 4)

Sorry for so many ideas, but your tool has so much power….
Do you think you will find / a way to make scene-detection & frame extraction easier?



Here is my current script:
MayBe it ist much easier as I thought…..


LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEGDecoder.dll")
global ThumbNailPath = "D:\_Szenenerkennung\videos\Bilder\"
global LogFilename = "D:\_Szenenerkennung\MyLog.txt"
myMPEGSource = MPEGSource("D:\_Szenenerkennung\videos\25.mpg")
myVideo = ReduceBy2(myMPEGSource)
myVideo = ConvertToYV12(myVideo)


function SaveFrame (myClip, myFrameNumber)
{
###Uncomment to display frame number
myClip = Subtitle( myClip, string(myFrameNumber))

### Export JPEG Image
ImageWriter(myClip, file = ThumbNailPath, start = myFrameNumber, end = myFrameNumber, type = "jpeg",info=true)
### WriteLogFile
WriteFile(LogFilename, """ "CurrentFrame: " """ ,"current_frame")
return (last)
}

#myVideo = ScriptClip(myVideo,"depanestimate(myVideo,trust=4,log=""" "scenes.log" """,info=true,show=true)")
myVideo = depanestimate(myVideo,trust=4,log= "scenes.log",info=true,show=true)



###Export KeyFrame
### a Key Frame yould only be exported when myTrust <= (DePanEstimate.Trust(frame))
myRGBVideo = ConvertToRGB24(myVideo)
ScriptClip(myRGBVideo, "SaveFrame (myRGBVideo,current_frame)")



It would make me very happy to get an Idea how this problem could be solved..

[Fizick]

My first answer to Andreas is:

I thinked about implementing scenedetection function some time ago.
For example, I can easy to realize a function DePanScenes(clip, inputlog),
which uses for input either DePanestimate motion data clip, or log file.
DePanScenes output will be clip with all pixels luma (or croma) = 0 for all ordinary frames, and = 255 for scenechanges.
This clip can be analyzed by AverageLuma() for example, and results used in ScriptClip.

But DePanEstimate make some calculation at constructor stage (sinus table for fft etc), and frame processing will be slow.

[Manao]

Thinking of this, I could also quickly make a filter from the ME engine in MVTools, which decides whether a frame is a scenechange or not. I don't know however if the scenechange I get from vectors would be more reliable than those you're getting with your filter.

Anyway, if you want to test the basic scenechange detection in mvtools, you can use the following trick :mv = source.mvanalyse(isb = false)

mask= source.mvmask(mv, ml = -1, gamma = 100)

white = source.blankclip(color = $ffffff)

black = source.blankclip(color = $000000)

scmask = conditionalfilter(mask, white, black, "AverageLuma()", ">", "0.2")

return scmask( It's a hack, if there is no motion at all, it will be considered as a scenechange, but anyway, it gives a result )

[Fizick]

New version 1.1, September 16, 2004 - Added experimental DepanScenes function.

[Ark]

I found this script on Depan help file regarding FPS conversion from 16,6 fps to 25 fps:

Avisource("kino.avi")
loadplugin("depan.dll")
i=ConvertToYV12()

data=DePanEstimate(i, range=1, trust=5)
f1_3=DePan(i,data,offset=1./3)
b1_3=DePan(i,data,offset=-1./3)
interleave(f1_3,i,b1_3)
selectevery(6,0,1,2)

But I've some movies at 15 FPS that I want to convert to 25 FPS, how do I have to change values to achieve that?

(the above script result in 22.5 FPS...)

[johnmeyer]

When trying to load Depan 1.1 or Depan 1.0, AVISynth 2.5 returns the error message:

Unable to load "depan.dll"

If I copy the version 0.91 depan.dll to the same directory, the dll loads and the script runs just fine. I have no problems loading other dlls, so I assume there is some bug in these two latest versions.

I am running XP Pro (SP1). I am running AVISynth 2.54.

Edit: I upgraded to the latest (AVISynth 2.55) and still couldn't load the two latest versions. I was hoping to use this plugin to generate slow motion from standard NTSC DV video. Can this be done using the DePanEstimate function?

[Fizick]

Ark,
Here is a possible function for framerate converting (progressive) with factor=5/3, for example from 15 fps to 25 fps :



function fps3to5(clipYV12)
{
# change FPS from 3 to 5 (or 15 to 25, or 18 to 30 and so on), i.e. with factor=5/3
# uses global motion compensation
# input must be YV12 progressive (or separated fields probably ?)
data = DePanEstimate(clipYV12)
t3_5 = DePan(clipYV12, data, offset=-2./5)
t6_5 = DePan(clipYV12, data, offset=1./5).trim(2,0)
t9_5 = DePan(clipYV12, data, offset=-1./5).trim(1,0)
t12_5 = DePan(clipYV12, data, offset=2./5).trim(3,0)
Interleave(clipYV12, t3_5, t6_5, t9_5, t12_5)
SelectEvery(15,0,1,2,3,4)
}

AviSource("e:\video.avi")
LoadPlugin("depan.dll")
ConvertToYV12()
fps3to5()

[Fizick]

johnmeyer,
I have similar reports from some users.
Probably you must have FFTW3.DLL
I will look later.
What is your CPU ?

[johnmeyer]

Probably you must have FFTW3.DLL

I'll look into it tomorrow. I have a Pentium P4 2.8 GHz.

[Fizick]

Version 1.1.1, October 07, 2004 - bug fixed with compensation near right bottom and top corners2,
change from FFTW_MEASURE to FFTW_ESTIMATE for more short init, without speed change (for power-2 windows),
compiled without /G7 flag (as before v.1.0), FPS script functions added to doc.

(not public)

Version 1.1.2, October 09, 2004 - delayed loading of fftw3.dll (now optional). Compiled with VC6.

[Ark]

Originally posted by Fizick
Ark,
Here is a possible function for framerate converting (progressive) with factor=5/3, for example from 15 fps to 25 fps :



function fps3to5(clipYV12)
{
# change FPS from 3 to 5 (or 15 to 25, or 18 to 30 and so on), i.e. with factor=5/3
# uses global motion compensation
# input must be YV12 progressive (or separated fields probably ?)
data = DePanEstimate(clipYV12)
t3_5 = DePan(clipYV12, data, offset=-2./5)
t6_5 = DePan(clipYV12, data, offset=1./5).trim(2,0)
t9_5 = DePan(clipYV12, data, offset=-1./5).trim(1,0)
t12_5 = DePan(clipYV12, data, offset=2./5).trim(3,0)
Interleave(clipYV12, t3_5, t6_5, t9_5, t12_5)
SelectEvery(15,0,1,2,3,4)
}

AviSource("e:\video.avi")
LoadPlugin("depan.dll")
ConvertToYV12()
fps3to5()



Thanks a lot Fizick, I'll try thah script tomorrow! :)

[johnmeyer]

fftw3.dll fixes the problem. Thanks!

[scharfis_brain]

Bug in depanstabilize found:
it produces dark-green frames (I think Y=U=V=0 )sometimes.
also it makes one or two lines at the bottom looking dark green.

the error is reproduceable, but I didn't tried it with another script for now.

[Fizick]

:script: :)

[scharfis_brain]

you won't like it:


setmemorymax(512)
loadplugin("d:\x\masktools.dll")
loadplugin("d:\x\depan.dll")
loadplugin("d:\x\removedirt.dll")

function repairfieldswap(clip c)
{d=c.swapfields().crop(0,0,0,-1)
d=stackvertical(d.crop(0,0,0,1),d)
e=c.converttoyv12()

global edges=e.yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)
global swapedges=e.swapfields().yv12convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0,usemmx=true)

ConditionalFilter(e, c, d, "float(averageluma(edges)-averageluma(swapedges))", "lessthan", "0.0")
}
a=avisource("capture (2004-11-14 at 14-34-24).avi")
b=avisource("capture-55630.avi").converttoyuy2().levels(16,1,235,0,255)
c=avisource("capture-81443.avi")
d=avisource("unblended.avi").changefps(25).killaudio().addborders(2,0,6,8)

a1=a.trim(0,55630)
a2=a.trim(55638,81442)
a3=a.trim(81447,0)
a1+b+a2+c+a3
trim(580,100296)
assumetff()
doubleweave().selectodd()
audiox=last
last.killaudio()
last.trim(0,2205)+d+last.trim(3074,0)
audiodub(last,audiox)
r=2
c=2
mergechroma(crop(0,c,0,0).addborders(0,0,0,c))

repairfieldswap()


crop(2,0,-6,-8)

converttoyv12()
x=depanestimate(last,pixaspect=1.094,trust=0.0,range=1)
depaninterleave(last,data=x,next=r,prev=r,mirror=15)
removedirt()
selectevery(r+r+1,r)

de0=last
de1=de0.depanstabilize(data=x,dxmax=12,dymax=12,zoommax=1.0,rotmax=0.0,pixaspect=1.094,mirror=15) #next=12,prev=12)
de1.trim(0,190)+de0.trim(191,540)+de1.trim(541,0)

[Fizick]

Confirmed :(
The bug was becouse of infinite calculated shifts for improperly Depanestimate() parameters setting.
Bug fixed in new version 1.1.3. Now base frame reseted.
Thanks.
Try!

[scharfis_brain]

Wow, you're damn fast in fixing bugs!

but can you explain this:
"The bug was becouse of infinite calculated shifts for improperly Depanestimate() parameters setting."

a little bit more detailed?

what are infinte shifts?
why improper settings?

[Fizick]

1. improper settings - some scenechange was not found,
and resulted in very big false pan by Depanestimate.

2. Depanstabilize try calculate shift for these false data,
and internal get divergence of caclulation (infinite shift).

[smok3]

iam trying to stabilize rotation using DePanStabilize, but it is a nogo no matter what i use for 'rotmax' value..., any ideas?

[Fizick]

Recently I developed first beta (alpha) version of MVtools to Depan interface, as MVDepan function of MVTools.

The global motion extraction algorithm is still not very clever and robust in this version, but it works as idea.

But it can estimate rotation too.

Try latest version on MVTools 0.9.8.2 and above
http://forum.doom9.org/showthread.php?s=&threadid=82015&perpage=20&pagenumber=3

[Fizick]

Version 1.1.4, December 15, 2004 - damping parameter in DePanStabilize is now variable (was accidentally fixed =0.9 in all previous versions :scared:

[Fizick]

Version 1.1.5, December 31, 2004 - bug fixed in DepanEstimate (erroneous motion data) for fftw=true with show=false and info=false

I really wander why it bug was not fond before
(it present since v.1.1)

Happy Mew Year!

[scharfis_brain]

I would really like to know, how a mewing year sounds alike ;)

Anyway. thanks for the fixed version :)

[nmap]

Hello Fizick,

Excellent work on a wonderful plugin! I do have but one problem, it may be due to my own stupidity though.

Glabally speaking, if I use

mdata = DePanEstimate(range = 4, trust=5, dxmax=8, dymax=8, improve=true, fftw=true)
DePanStabilize(data=mdata, dxmax=8, dymax=8, prev=1, next=1, info=false, cutoff=0.5)

I still obtain black bars where the panning occurs. Would this be because I've limted the dx and dy? I'd really like to remove the black bars if possible.

Thanks
nmap

[scharfis_brain]

increase the next and prev parameters

[Fizick]

nmap,
it is most hard question, we do no know what must be in place of black borders.

try use "addzoom", and "mirror" to fill it.
I often use use greater "cutoff" = 2.0

Note. The used values of dxmax and dymax are too low,
try defaults.

Lucky depaning! :)

[Fizick]

Version 1.2, April 1, 2005 - added fitlast parameter to fit some last frames range to original position


Version 1.3, April 29, 2005 - added blur parameter to somewhat hide the sharp mirrored details; blur is horizontal only, at left and right border.

[Fizick]

Version 1.4, May 7, 2005 (published May 29)- DePanStabilize: Zoom adaptive mode addzoom is improved. Adaptive zoom decreasing rate is slower than zoom increasing rate now. Thus, the black empty borders are decreased, and zoom value is more stable now.

[vinetu]

and now the WWW button is... unvisible! (http://www.avisynth.org/fizick/)
:)

[Seed]

and now the WWW button is... unvisible! (http://www.avisynth.org/fizick/)
:)

I know that your "invisible" link links to his homepage mirror. But just for information, if one click on the username (e.g. "Fizick") insie the left pane, a box will slides out and it contains the usual user profile links, including the link to homepage.

[Fizick]

Yes, this is a great forum enhancement. :angry:

BTW, DePab version 1.4.1 released (bug fixed with log)

[SCIF]

There is new version DePan v.1.5 (http://bag.hotmail.ru/depan/depan150.zip) - Stabilizer adaptive zoom mode improved more.
Thanks to Fizick for regular improve!

[J&J]

Fizick,

rotation.......
in the near future?

I still use deshaker 1.6 but waiting for Depan's next step....

:)

[Fizick]

J%J,
Did you try MVDePan?
http://forum.doom9.org/showthread.php?p=580027#post580027

Hm, I now found that MVDePan info is missed from Depan 1.5 English doc after edition :confused:
So, i restore it and relese a new 1.5.0 zip archive (the same binary DLL).

[J&J]

Wow, MVDepan can work!
But I can't get the stable outcome as Deshake.

My Avs :


LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\deflicker.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\depan.dll")
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\MV\MVTools.dll")

sourcefile="CIMG0297"

BaseLayer=DirectShowSource(sourcefile+".avi",fps=29.970).AddBorders(96,96,96,96)

vectors = BaseLayer.MVAnalyse(isb = false,lambda=40000)

globalmotion = BaseLayer.MVDepan(vectors, pixaspect=1, thSCD1=40000,thSCD2=40000,zoom = false,error =90)
Midway=DepanStabilize(BaseLayer, data=globalmotion, cutoff=2.0, mirror=0, pixaspect=1).deflicker(percent=99,scene=25)

Return(Midway)


Thanks for your answering.....

[videoFred]

Wow, MVDepan can work!
But I can't get the stable outcome as Deshake.

Yes, MVDepan works better than Depan only, because now we have not just global motion compensation but also compensation of moving objects etc..

:D Thank you, Fizick :D
:D Thank you, Manao :D

I must do many more tests, but it seems to be a huge improvement.
But I have noticed the rotation estimation is not 100%. (sometimes in the wrong direction.)

Also, on scenes with very fast moving objects and fast panning at the same time sometimes there is a jump in time beween two frames. But this is easy to solve with very low thSCD1 parameter. But then stabilising is not so good anymore.

Yes, Deshaker is still a little better, but needs two passes. On the other hand, too much stabilising looks artificial sometimes, and you get lots of blurred frames.

Resumed: I am verry happy with MVDepan, I think I'm gonna use it on all my files in the future. But if someone has a better solution for the time-jump between frames, I would be even more happy.

Fred.

[Fizick]

now we have not just global motion compensation but also compensation of moving objects etc..
No, we have global motion only with MVDePan.

[videoFred]

No, we have global motion only with MVDePan.

Realy???

You made it.. You know it.
But movement is smoother with MVDepan, then why?

Fred.

[Fizick]

I release new version DePan 1.6, August 5, 2005. Here is changelog:
- added YUY2 support (It is the main addition);
- disabled improved=true mode in DePanEstimate. (It seems, it was always broken );
- changed default subpixel=1 (bilinear) for DepanInterleave as quite sufficient for denoising and more fast

[Fizick]

New Version 1.7, September 5, 2005 - DePanStabilize: added parameter initzoom - minimal zoom;
changed limits dxmax, dymax, zoommax, rotmax from hard to soft with larger slope non-linearity.
Changed cache.

[scharfis_brain]

can you add an expanding feature to depanstabilize?

so I can do some kind of panorama out of quick pans.

[Fizick]

I do not understand you.

[scharfis_brain]

When I process VHS, I need to crop away 8px around the image, because there is garbage at this image borders.
So 704x576 becomes 688x560.
This video I feed into depanstabilize.
It is set up to use prev. & next frames to fill the borders.
After this I need to add an 8px border round the image to get 704x576 again.

Is it possible to skip the addborders? So that I get more visible image due to GMC from depanstabilize in shaking scenes?

EDIT: I do not mean a zoom here!

[Fizick]

We can not fill all (and whole) borders so.

[scharfis_brain]

with deshaker I can output a bigger than the input frame without zooming.

Assume an 320x240 input video (Digital photo camera)
http://home.arcor.de/scharfis_brain/samples/deshaker-expand.png

the resolution at the right side of the panel is set to 640x480 (doubled height & width). to compensate zooming, the extra zoom factor is set to 0.5 .

This leads to an expanded frame without resizing and without constant black borders (every movement fills them!)
, but for sure not all borders are filled. I am aware of that.

I would be happy to see this featur in depan, too :)

[Fizick]

So, in Deshaker you get big output frame, with main video in central part only, with big black (partially filled) borders?
So, you want, that output video from depanstabilize will have more width and height size, than input?
it is good for deshaker with many frames used for filling.
Some time ago i think about it, but for more simple boundary proccesing.
Some code re-design will be needed.

May be it is better to add "cropping" option to depan?
and use the "cropping" boundaries as limited mirror positions or solid fill.
Of course, you may use external Avisynth command for these now.
May be i can add some partial fill by previous frame too. But i am not sure.

[videoFred]

I use Depan for frame rate conversion, now.
With very nice results.

AssumeFPS(12.50) > ChangeFPS(25.00) >> Depan Stabilise
Gives very s-m-o-o-t-h panning.

But I do not understand the 'cutof' parameter :(
Suppose I set it to 2HZ.
2HZ of what?

I also do not understand the 'damping' parameter :(

But I begin to understand all the other parameters :D

Fred.

[Fizick]

I use innertial method. consider camera mounted on some inertial platform, with some damped spring (not fixed).
there is some intrinsic frequency for this mechanic system as oscillator.
and threre is its damping (relative factor).
Cutoff frequency is really not strict cutoff, but simply some characteristic frequency value.
Higher frequencies are suppressed. The higher, the more.
(it is simplified algo).

2 Hz is real frequency of shaking.

[loba]

2 Hz is real frequency of shaking.
Does it mean 2 periods per sec?

[Chainmax]

A long time ago, I read a recommendation from scharfis_brain to Boulder of using DePanInterleave in order to improve a script, saying that it helped a lot on video stuff as it usually contains panning/zooming. I have a few very crappy short clips from a digital camera and want to make something discernable out of.

In the previous thread I made about this, the following recommendations were made to me:

i=ConvertToYV12()
mdata=DePanEstimate(i)
r=2 # interleaveradius
DePanInterleave(i,data=mdata,prev=r,next=r)
yourfiltershere()
selectevery(r+r+1,r)
DePanStabilize(last,data=mdata)

Code:
i=ConvertToYV12()
mdata=DePanEstimate(i)
r=2 # interleaveradius
DePanInterleave(i,data=mdata,prev=r,next=r)
yourfiltershere()
i=selectevery(r+r+1,r)
mdata=DePanEstimate(i)
DePanStabilize(i,data=mdata)

None of those options seem to work, in both cases black borders appear and change size in a quite dizzying manner. I can provide one of the clips for you guys to examine if needed.

[johnmeyer]

Are you trying to take 10 fps progressive "video" from a still camera and change its frame rate? On what device do you plan to view it? (NTSC TV, HD TV, computer monitor, etc.).

I have had good luck with Motionperfect as a way to introduce intermediate frames in order to smooth MJPEG video from Canon still cameras. I haven't gotten around to doing it, but I have a 90% complete AVISynth script that accomplishes more or less the same thing using MVTools.

[Chainmax]

The sources are indeed extremely crappy 160x112 videos from a still camera, but the source is 25fps and I don't intend to change the framerate, just deshake the badly done pans. After that, the stream would be cleaned up as much as possible while resizing to 320x256, borders will be added, and it will all be encoded to MPEG2 in order to author a miniDVD.

[Fizick]

Chainmax,
Both your script are generally correct (use first, i.e. second simply slower).
But you must try tune optional parameters.
1. Firstly check, that depanestimate make correct motion estimation. Temporary add "return mdata" line to script after depanestimate,
and add parameter to it:

i=ConvertToYV12()
mdata=depanestimate(info=true, trust=4)
return mdata

Check data, base frame must be changed on scenechanges only.
If not, decrease the "trust" value.
2.Now tune depanstimate (remove "return mdata" of course)
It can not create perfect picture. DEPan does not know, what must be in place of moved borsers (= empty black spaces).
But you may try add some data from previous frames, use mirror, or zoom.
Read DePanStabilize documentation. Then i will aswer your questions.
3. You also may conside apply motion stabilization after resising (320x256) for more smooth results.

[Chainmax]

I see, thanks for the suggestions.

[Chainmax]

I didn't understand how to tweak the trust parameter, but after some experimenting I found that lower values increased the thickness of the black borders, so I raised it until there were no black borders. I also put the DePanStabilize line after the resize as per your recommendation. This is the script I'm currently using:

i=ConvertToYV12()
mdata=DePanEstimate(i,trust=23)
r=2
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
ffdshow("default")
SelectEvery(r+r+1,r)
Lanczos4Resize(320,256)
DePanStabilize(last,data=mdata)
LimitedSharpen(strength=500)
AddGrain(10,0,0)
AddBorders(16,16,16,16)

Thanks for the help :).


P.S: how do different values for r affect the result?

[Fizick]

No,no! It is wrong! :)
1. You really disabled the motion compensation when you rise trust so much.
So, why use DePan?
Decrease trust to default!
2. If you resize clip, its motion is changed, so you MUST use second DePanEstimate.
Or resise at begining of the script (before first estimation and denoise), but it may be not good.
3. Add prev=r,next=r,mirror=15 to DePanStabilize too.
4. r is frame range of motion estimation. It must be used for estimation too.
ffdshow("default") is denoiser? I do not know how good is it.
But try use r=1.

Try this for example:

i=ConvertToYV12()
r=1
mdata=DePanEstimate(i,trust=4,range=r)
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
ffdshow("default")
SelectEvery(r+r+1,r)
Lanczos4Resize(320,256)
i=last
mdata=DePanEstimate(i,trust=4,range=r)
DePanStabilize(i,data=mdata,cutoff=1,prev=1,next=1,mirror=15)
LimitedSharpen(strength=500)
AddGrain(10,0,0)
AddBorders(16,16,16,16)


may be you will need in initzoom

[Fizick]

What video does you try to process (link to short clip)?

[Chainmax]

So, the more trust the more compensation? No wonder the borders disappeared at trust=23 :o.
ffdshow("default") is a line that allows you to use whatever postprocessing options you select in ffdshow's "video decode configuration" section. In this case, I was using SPP deblocking at strength 100%. It's not good as it loses too much detail, but it's the only deblocker (and I tried lots) that could get rid of the horrible blocking in the clip. I'll try your suggestion and report back. In the meantime, you can download one of the clips here (http://www.31012.com/~azulftp/MOV01825.MPG).

[Fizick]

"trust" parameter value is threshold. If real trust of current frame is below "trust", motion compensation will be disabled.

P.S. Why you sometimes do not disable your long signature? I already read it.

[Chainmax]

Well, I tried your suggestion and the black bars are indeed gone, but since they are so thick the mirroring becomes very evident. I guess I should stick to using

i=ConvertToYV12()
mdata=DePanEstimate(i)
r=2 # interleaveradius
DePanInterleave(i,data=mdata,prev=r,next=r)
yourfiltershere()
selectevery(r+r+1,r)
DePanStabilize(last,data=mdata)

and if that doesn't work (like in this case) then just accept that motion compensation can't be done.

[Chainmax]

By the way, I'm going to try the combination on another video I have, would this script

i=ConvertToYV12()
r=1
MData=DePanEstimate(i)
DePanInterleave(i,Data=MData,Prev=r,Next=r,Mirror=15)
DeDot()
FixChromaBleeding()
Crop(whatever,align=true)
Cnr2("xxx",4,5,255)
DeBlock_QED()
DeGrainMedian(mode=0)
SelectEvery(r+r+1,r)
GaussResize(656,448,p=100)
DePanStabilize(last,data=mdata,Prev=r,Next=r,Mirror=15)
DeHalo_Alpha()
LimitedSharpen(SMode=4,LMode=3)
AddBorders(24,16,24,16)

be ok?

[Fizick]

1. Motion compensation (for best denoising) and motion stabilization (deshaking) are different things. So, if you want decrease mirrored borders (and the strength of motion stabilization), try increase cutoff parameter (of DePanStabilize in my suggestion script) to about 4. Only high frequency vibration (above cutoff) will be stabilized.

2. Too many questions. But your second script is not correct.
2.1 You must use second DeOanEstimate after resize. Do you understand the my script?
Please, re-read my post http://forum.doom9.org/showthread.php?p=749324#post749324
2.2 If you use many temporal denoisers, else increase the r parameter
or use DePaninterleave(data=mdata).SomeTemporalFilter(...).Selectevery(3,1) block for every temporal filter.
2.3 Why crop?

[krieger2005]

I have noticed a strange behaivor with depan. I used it on a relativly static scene and set "trust" to 2. Then i let show me the info of "DepanEstimate". The "strange" was, that when info show a trust below 2 it show also "SCENE CHANGE!". When trust was above 2 it show me not scene-change. I thought, that on high motion/many changes it would be classified as "scene-change" and not on low motion/less changes in the frames.

Here my script:
h=height
i=StackVertical(SelectEvery(3,0), SelectEvery(3,1), SelectEvery(3,2))
i=RemoveDirt(i,mthreshold=80, pthreshold=3, tolerance=12).RemoveGrain(17)
i=interleave(crop(i,0,0,width,h), crop(i,0,h,width,h), crop(i,0,h*2,width,h))

mdata=DePanEstimate(i,range=2, trust=2, info=true)

[Fizick]

It is by design.
trust - limit of relative maximum correlation difference from mean value at scene change.

[krieger2005]

This is not logical for me. Can you explain what you exactly thought when you implement this this way? Why do you make it so?
Sorry, but this sentense i does not unterstand: limit of relative maximum correlation difference from mean value at scene change

[Fizick]

To detect scenechange and not compensate (stabilize) motion across this scenechange.
Frames correlation is the best way to detect scanenange.

[krieger2005]

But why the filter make something to a "scene-change" when there is no movement? Again: When i set a value to "trust" the frame correlation must be lower than this "trust"-value to be a scene-change? Is it not the different way, that the frame correlation must be higher than the "trust"-value to became a scene-change, because lower frame correlation values mean lower movement, or not? Sorry, i'm a little bit confused.

[Fizick]

Current frames correlation must be lower than the "trust" parameter value to became a scene-change.
You get different processing?
I am confused.

[krieger2005]

You are right. It seems to be something with the clip. Thanks

[Beave]

I posted my problem in two other threads as well, but I still wanna ask here, because I tried your plugin.

I have a few 720p capture that shakes about one line up and down. So the whole image moves by one pixel. I don't wanna correct any other shaking. I used:
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata, dymax=1, dxmax=0, info =false)

Now not all the shaking is caught and it seems only parts of the image move. But it is smoother then before.
Could I only specify for example the bottom 20 lines for finding the shaking? There is a small black edge where I can see the picture moving quite easily.

Sorry for bothering you and thank you for your great plugins!

[Beave]

I'm getting closer to solving my problem. I used this script to first analyze only the bottom border:

mpeg2source("test_720p.d2v")
Trim(0,400)
i = ConvertToYV12()
j = ConvertToYV12().crop(0,712,-0,-0)
mdata = DePanEstimate(j,info=false)
DePanStabilize(i, data=mdata, dymax=1, dxmax=0, info=true)

Now I can see the data being diplayed on the video. the "dy" is either between -0.15 and +0.10 or between +0.25 and +0.55 when the video jumps. Now I now the values, but how do I translate that into my DePanStabilize function? So I have the sweet spot at around 0.18 and I want the picture to shift one pixel up when going over that threshold. How do I do that?

Thanks for the help in advance!

[johnmeyer]

I want to only compensate in the X direction. I thought I could do this by setting dymax=0, but this has no effect. I describe the reasons for this script in this thread: http://forum.doom9.org/showthread.php?t=109687&highlight=depan

Here is the script:# Script to recover film frames from film projected on shutterless 16mm projector.
# Second Pass
# Copyright 2006 John H. Meyer
# Revision May 9, 2006
#-----------------------------

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
loadPlugin("c:\Program Files\AviSynth 2.5\plugins\Depan.dll")
loadplugin("c:\Program Files\AviSynth 2.5\plugins\MultiDecimate.dll")

AVISource("D:\OPRFHS\Intersquad 1st reel)0002.avi")

converttoYUY2(interlaced=true)
AssumeBFF()
tfm(display=false,micout=2,mode=0,cthresh=45,mi=1600,pp=6,metric=0,field=1,micmatching=2,slow=2,blockx=256,blocky=256,sco=-1,debug=false,input="d:\tfm.txt")

a=MultiDecimate(pass=2)
b=AssumeFrameBased(a)
c=b.separatefields()
d=c.colorYUV(autogain=true)

e=interleave(blankclip(c),selectodd(c),selecteven(c),blankclip(c))
f=interleave(blankclip(d),selectodd(d),selecteven(d),blankclip(d))

mdata2 = DePanEstimate(f,range=1,dxmax=32,dymax=0,pixaspect=0.9091,info=false)
g=e.Depan(data=mdata2,matchfields=false,offset=1,info=false)
h=g.selectevery(4,2,3)
i=h.ComplementParity().weave()
AssumeFPS(18)

i

As you can see, I use an exposure-compensated version (d) to generate the depan parameters, and then use these parameters to compensate the original. I then force a scene change after each pair of fields by inserting two blank fields (thanks to Avil for that hint). Thus, only the second of each pair of fields is shifted, and it only uses the information between from the previous field, which is what I want.

The problem is that this second field is shifting vertically as well as horizontally. I have tried using small non-zero settings for dymax, but it doesn't seem to do anything.

The depan.dll reports version 1.7.0.0, and is dated 9/5/2005.

I also am trying to figure out how to get the framecount back to where it should be. The Interleave function with the blank frames added doubles the frame count so I end up with the second half of the video being duplicates of the last frame. I can probably figure that out eventually, but the dymax has got me stumped.

[Fizick]

Probably I can add option "not estimate dy" in next version (in preparation, with some improvement in DePanstabilization), if you are still interested in it.

[Alain2]

some improvement in DePanstabilization
I am definitely, I have some bugs with current version (or I don't understand what I am doing with parameters, which is also possible but i doubt it)
Thank you Fizick :)

[johnmeyer]

It would help to be able to turn off dy, but I have a workaround I've created, so I am OK for now.

[JuanC]

Probably I can add option "not estimate dy" in next version (in preparation, with some improvement in DePanstabilization), if you are still interested in it.It is really interesting!It would help to be able to turn off dy, but I have a workaround I've created, so I am OK for now.I will appreciate if you share how you managed to work around it.

[johnmeyer]

I will appreciate if you share how you managed to work around it.

My workaround is very specific to my application. You have to go back and look at my earlier posts, but I am transferring film to video using a patent-pending process that I developed where I remove the shutter from a film projector, videotape by pointing the camera directly at the lens, and then, using software available for AVISynth, remove redundant frames, and frames where the film is being pulled down. This leaves me with two fields of video for every frame of film. Unfortunately, when a film projector is run at full speed, the film never quite comes to rest in the film gate (it is only there for a few milliseconds before the next frame of film is pulled down). I was therefore trying to remove the residual horizontal motion. I preferred to leave the residual vertical motion in order to get the proper offset between fields (they need to be offset vertically by one scan line). However, since I couldn't get dy=0 to work, I just let Depan line them up vertically, and then used AVISynth commands to move the odd fields by one scan line. The results are not quite as good as I wanted, but better than if I don't do the correction.

[Chainmax]

I am trying to make a conversion from 15fps to 23.976fps using DePan, but even after reading the corresponding portion of DePan's readme several times, I haven't been able to figure out how to do it. Can someone help me out?

[Fizick]

No, but here is simple fps doubler (from 15 to 30):

data = DePanEstimate()
f1_2 = DePan(data=data, offset=1./2)
Interleave(f1_2,last)

[Chainmax]

How about from 15fps to 23.976? Would something along the lines of:

i=last
data=DePanEstimate(i,range=1,trust=5)
f=DePan(i,data,offset=x)
b=DePan(i,data,offset=-x)
Interleave(f,i,b)
SelectEvery(???)

be ok? If so, what should I substitute x and ??? for?

[Fizick]

Look to fps conversion examples in DePan doc and try understand them.

[Chainmax]

I'm not being lazy, the framerate changes appear out of the blue in the readme, and the only two examples (2-to-3 and 3-to-5) are too different for a common template to be extracted from them. About the only thing I see in common is that both examples create some kind of backward and forward information, but that's it.

[johnmeyer]

Here's some code I used quite awhile ago. Haven't tested it with the latest version. However, it worked when I last used it. Hopefully it will help you fill in the gaps in your knowledge and get you going.#Slow motion
loadplugin("c:\Program Files\AviSynth 2.5\plugins\depan.dll")
AVISource("E:\Volleyball\Volleyball (Soldedad)\Volleyball (soledad)0022.avi")

converttoYV12()
AssumeBFF()
fps3to5()


function fps3to5(clipYV12)
{
# change FPS from 3 to 5 (or 15 to 25, or 18 to 30 and so on), i.e. with factor=5/3
# uses global motion compensation
# input must be YV12 progressive (or separated fields probably ?)
SeparateFields(clipYV12)
data = DePanEstimate(clipYV12)
t3_5 = DePan(clipYV12, data, offset=-2./5)
t6_5 = DePan(clipYV12, data, offset=1./5).trim(2,0)
t9_5 = DePan(clipYV12, data, offset=-1./5).trim(1,0)
t12_5 = DePan(clipYV12, data, offset=2./5).trim(3,0)
Interleave(clipYV12, t3_5, t6_5, t9_5, t12_5)
SelectEvery(15,0,1,2,3,4)
weave()
}

[Chainmax]

Thanks, but that 3-to-5 example is in DePan's readme, and the only other example (2-to-3) is much too different to extract a general framework from them.

[johnmeyer]

I guess none of know what you want. Yes, this was from the readme, but I actually used it, and this script is what I used. In other words, this is a working script. Isn't that what you wanted?

[Chainmax]

Not exactly, I need a 15-to-23.976 script that doesn't result in the need of resampling/repitching the source audio.

[scharfis_brain]

that won't work with depan.

[Chainmax]

I assume 15-to-29.97 won't workt with DePan either, am I stuck with duping/interpolating frames with ChangeFPS then?

[scharfis_brain]

you may upinterpolate it to 150 fps (15fpsx10) and then use cangefps(29.97).
so the remaining jitter is about 1/150 sec

[Chainmax]

You mean extrapolation to 150fps and then ChangeFPS(23.976), right? And what should I use for the extrapolation?

[scharfis_brain]

it is still interpolation.

modify fps3to5 so that it delivers 1/10th steps to achieve 10x of the input framerate

[krieger2005]

... and then use cangefps(29.97).

... and then ChangeFPS(23.976), right?
No comment.

[Chainmax]

it is still interpolation.

modify fps3to5 so that it delivers 1/10th steps to achieve 10x of the input framerate

I really have no clue how to do it, 3to5 has 3_5, 6_5, 9_5 and 12_5 but it really isn't clear why 4 steps are made and why the first numbers of each step are multiples of 3. I also don't understand the logic behind the offsets on each step or the SelectEvery line.


[edit]If the final conversion is going to be to 23.976, would it benefitial to do the up-interpolation to 120fps instead?

[scharfis_brain]

the more up-interpolation, the less the remaining jerkyness due to the 1000/1001 factor.

[Chainmax]

But 119.88 (120) is a multiple of 23.976 (24), whereas 150 is not. Does that not make a difference when the filter tries to calculate which of the up-interpolated frames to discard?

[Alain2]

That's why sharfis is not talking of an exact convertion, he says you'll have remaining jerkiness. With the depan way as I understand from sharfis and the 3to5 script, you can only divide by decimal numbers the gap between 2 frames to introduce the interpolated ones. This is not compatible with 15 to 23.976 (or more precisely 24000/1001, and this is not an decimal number)

Have a look in the avisynth doc to what changefps does... You'll see why with the depan method "the more up-interpolation, the less the remaining jerkyness due to the 1000/1001 factor."

[Chainmax]

I see. So, what would the modified fps3to5 with 1/10th steps to achieve 150fps be?

[Chainmax]

Alain2? scharfis_brain?

[scharfis_brain]

I have no time right now.

Why aren't you trying it out yourself?

[Chainmax]

Because like I said several times before on this thread, the docs are not clear enough in that regard, and the two framerate examples are too different for me to extract a common principle that would allow me to build a function with your suggestion.

[scharfis_brain]

what about trail & error then? That is what I do, when something isn't clear to me.

I remember made a similar conversion function when depan came out
(50 -> 300 -> 59.94 fps)

And the docs are clear IMO.

hint: leave out the selectevery - line.
Its job will be done by changefps afterwards.

[Chainmax]

It's not like I haven't been trying, but whatever.

[Fizick]

Chainmax,
Do you clear understand my third (simplest) example, which is in my post:
http://forum.doom9.org/showthread.php?p=840636#post840636

If not, other discussions is useless.
And did you try this doubler? Do you like it results at 30 fps (smooth)? If not, you must not spend you time with 24 fps.

[Chainmax]

I understand that example, but it only interleaves one component instead of two and it results in 30fps. I need either 23.976 or 29.97 fps output without having to resample/repitch audio.

[krieger2005]

You must find two numbers (not floats). The first one is the factor how much more picture you want to get an (not-float frame-rate) (let's say in your case 7992, because 15*7992=119880fps (or 999 -> 15*999=14985)). The second factor defines which frames of this 119880 you want have in your movie (in your case: 5000, because 119880/23.976=5000 (or 625-> 14985/23.976=625)). And then you have your 23.976fps.

So, in first case you must find a factor, to push up your framerate to a number (not float) framerate, which can be can be devided in the second step by a number (not float) to get 23.976. These two numbers must be as low as possible. Here 999 and 625 would work. This mean an interleave of 999 frames, where you take 625 of it.

[Alain2]

I understand that example, but it only interleaves one component instead of two and it results in 30fps. I need either 23.976 or 29.97 fps output without having to resample/repitch audio.
krieger2005 is right, and this is working only for decimal numbers, its not possible to change from a decimal framerate to a float framerate 24000/1001 using discrete numbers of intermediate interpolated frames ; but why can't you just interpolate to 24fps and then slow down to 24000/1001? I don't think you will notice the pitch change, and it will but easier to implement (15*8=120=24*5, thus interleave 7 frames between original ones, and select 5)

[Chainmax]

It's just that I was already trying to make a 5.1 mix of the mono track and having to resample/repitch means I'll have to do it all over again...*sigh*

Fine, I cave :(.

[edit]Not good. Using DePan creates huge black borders. What other alternatives do I have?


One more thing then: I'm now trying to have a small 15fps clip converted to 23.976fps but have it play at half the speed. I tried this:

fps3to5()

AssumeFPS(23.976)

SelectEven()

ChangeFPS(23.976)

But it doesn't seem to play at half speed. What am I doing wrong?

[johnmeyer]

You could try MotionPerfect from Dynapel. They have a free trial, I believe.

[scharfis_brain]

Not good. Using DePan creates huge black borders.It has parameters to avoid that!

I repeat mysel. it is possible to do your task
pseudocode may be like this:

input=xxxsource("blah")

vectors=depanestimate(input, bla bla bla)

input_1_of_10 = input
input_2_of_10 = input.depan(+1/9)
input_3_of_10 = input.depan(+2/9)
.
.
.
interleave(input_1_of_10, input_2_of_10, ...)
changefps(23.976)


WHAT is so HARD about this task?!?
That is basically typed from my mind.

[videoFred]

WHAT is so HARD about this task?!?
That is basically typed from my mind.

Hello Scharfis,

It's simple for you....

But you and some other members here have this special mathematic talent.
Be happy with it, it's a gift! I admire this talent very much!

But we, mortal people :p , do not have this talent.
You must understand this.
So it would be good if someone would explain more about this.

Fred.

[scharfis_brain]

what is so hard about simple multiplication and division? and also some calculations with the rule of three?

pupils are learning this in 5th grade...

[videoFred]

what is so hard about simple multiplication and division? and also some calculations with the rule of three?

pupils are learning this in 5th grade...

It's not about the multiplication and division, but where and how to use it.
Working with frames, fields, frame rates etc can look very complicated if you are not used to do it.

Fred.

[Fizick]

Take a white paper and draw a time graphs of shifted frames. It must help.

[Chainmax]

Thanks for your example scharfis_brain, I made a 16-step function that extrapolates to 240.002fps, then I use AssumeFPS(239.76) and set the final framerate with ChangeFPS. Motion appears to be acceptably smooth and as far as I can see there are zero artifacts. I am extremely happy and I apologize if I came out as too insistent. I have but two more questions:

1) Will using ChangeFPS(23.976) and then using pulldown or ChangeFPS(29.97) directly make any difference in motion fluidity in the end result mpeg2? Empirically, both seem pretty much equal to me, but I'd like to know a theoretical reason.

2) Would using SelectEvery yield better results than using ChangeFPS?

[scharfis_brain]

you don't need assumefps follower by changefps.

1)just use changefps(23.976) and become happy.
you will not notice the small error of 23.976 <-> 24.000 fps.
but do NOT use changefps(29.97 ) after changefps(23.976) it will make your video jerky.
either use 23.976 fps MPEG2 with pulldown, or go straightforward to 29.97 fps and encode it as 29.97 fps MPEG2

2)selectevery only can reflect integer decimations, while changefps can do arbitrary decibmations.

[Chainmax]

...
but do NOT use changefps(29.97 ) after changefps(23.976) it will make your video jerky.
either use 23.976 fps MPEG2 with pulldown, or go straightforward to 29.97 fps and encode it as 29.97 fps MPEG2
...

Oh, you misunderstood me. What I meant is the second thing: either ChangeFPS(23.976)+Pulldown or directly ChangeFPS(29.97). The former is probably better anyway because the intermediate huffyuv file will occupy much less. Again, thanks for your advice :).

[scharfis_brain]

go directly 29.97.

this won't introduce telecine judder.

[Chainmax]

Ok. The final MPEG2 was done extrapolating to 240.002fps, then using AssumeFPS(239.76) and ChangeFPS(29.97). If I mux that with the original audio, will there be any noticeable A/V asynch or will it be ok?

[scharfis_brain]

there will be A/V async.
remove assumfps and it will be okay.

[oddball]

Hey guys I need some help. I really cannot understand why it's so difficult to fix this problem. I have a movie clip that moves up and down by a few pixels every few frames (Something like every 15 frames but not always). I just need to stablize the up down motion. I thought it would be simple but it appears not.

The depan command I tried was

i = ConvertToYV12()
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata, dxmax=0)

But the frame moves up and down by mad amounts every few frames.

In another thread someone suggested I use a cropped clip for motion estimation and apply that to the second clip or somesuch. Damned if I know what he means. The clip has a logo on it that I could use as a guide to help with the motion detection. Since that logo is the same all the way through the clip and it too moves up and down every few frames I thought it would be a matter of applying a filter that detects that area when it moves up and down.

I don't know very much about all these Avisynth commands apart from the basics. The Depan manual seems rather cryptic to someone who does not do much Avisynth work.

Can someone lead me to a solution?

Thanks.

EDIT: OK I tried something from some earlier postings. It seems to work a lot better. No wild motion with big frames appearing this time. Is this OK?

i=ConvertToYV12()
r=1
mdata=DePanEstimate(i,trust=4,range=r)
DePanInterleave(i,data=mdata,prev=r,next=r,mirror=15)
SelectEvery(r+r+1,r)
i=last
mdata=DePanEstimate(i,trust=4,range=r)
DePanStabilize(i,data=mdata,cutoff=1,prev=1,next=1,mirror=15,dxmax=0,dymax=0)

Awwww crap. Just realised it did nothing to stop the up and down motion. Back to square one.

Nope. I think I nailed it. It needs a few frames to figure out the motion then it stabilizes it rock solid. I removed what looks like a repeat line from that code too.

EDIT: Well it does not seem to stop the very slight up and down motion every few frames at all. Very frustrating.

[oddball]

I'm getting closer to solving my problem. I used this script to first analyze only the bottom border:

mpeg2source("test_720p.d2v")
Trim(0,400)
i = ConvertToYV12()
j = ConvertToYV12().crop(0,712,-0,-0)
mdata = DePanEstimate(j,info=false)
DePanStabilize(i, data=mdata, dymax=1, dxmax=0, info=true)

Now I can see the data being diplayed on the video. the "dy" is either between -0.15 and +0.10 or between +0.25 and +0.55 when the video jumps. Now I now the values, but how do I translate that into my DePanStabilize function? So I have the sweet spot at around 0.18 and I want the picture to shift one pixel up when going over that threshold. How do I do that?

Thanks for the help in advance!


Same problem. Fizik how do I compensate for a vertical dy = 0.01 to 0.04 pixels frame motion?

Here is a screenshot of DePanStabilize output.

http://img213.imageshack.us/img213/4462/motionpm8.png

[Fizick]

Please do not cross-post.
Are your source is progressive? If yes, here is the steps:

1. find appropriate crop values to remove almost all besides logo. Not crop logo at any frame. And use cropped width and height as power of 2 (i.e. 16, 32, 64, 128, 256)

mpeg2source(...)
c=crop(100,200,300,400)
return c

2. Try analyse the motion
mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dx=0)
depan(last,data=m, info=true)

(logo is very small to use directly depanestimate(c, info=true))

(You can also produce log file to analyze and process it somehow)

dy = 0.01 to 0.04 pixels is very very little values :)
Probably you will get around 3.0 to 10.0

Chack scenechange too.

4. Next step is stabilization.
mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dx=0)
depanstabilize(last,data=m, info=true,cutoff=2)

Try tune some parameters, for example cutoff

But I am not sure that DepanStabilize can give usable result for your case. (NO WARRANTY :) )

[oddball]

When I open up the Avisynth in Vdub I get an error.


Script error:depanestimate does not have a named argument "dx"

I am using 1.9.0. dx=0 does not work.

[Fizick]

NO WARRANTY ;)

may be dxmax ? (I did not try it)

[oddball]

NO WARRANTY ;)

may be dxmax ? (I did not try it)

Hey it's your plugin lol :D

Oh well looks like the warranty is void. Depan simply cannot deal with small vertical movements. Thanks anyhow.

[Fizick]

I did not try that script right now...
(ones upon a time i tested similar)

what dy shift have you got?

[oddball]

It shifts between 0.01 to 0.04 up and down. Using the logo was no good. Depan was not sensing any movement on it. If I use the previous posters line 712 which is pretty much where my bottom border is then it detects the small motion up and down. It detects it but DepanStabilizer cannot move the frame by such small increments up and down to correct the vertical motion.

[Fizick]

I am sure you still do get right settings for Depanestimate.
What is real shift?
0.04 pixels is invisible.
Post your real script like it:

mpeg2source(...)
c=crop(100,200,300,400)
m=depanestimate(c, dxmax=0,info=true)

And why you do not use v1.9.1 ? ;)

[oddball]

I initially tried

DirectShowSource(video.m2v)
c=crop(1058,570,76,80)
m=depanestimate(c, dxmax=0)
depanstabilize(last,data=m, info=true,cutoff=2)

But it does not detect any movement whatsoever where the logo is (It is correctly set as i can see the logo only when I add 'return c').

So I tried.

c=crop(0,712,0,0)
m=depanestimate(c, dxmax=0)
depanstabilize(last,data=m, info=true,cutoff=2)

Like a previous person used and it detects the movement and displays it in Depan's output but it does not stop the movement up and down.

I am using 1.9.1 now. No difference though.

[oddball]

Would it be helpful if I uploaded a clip from the movie for you to look at?

[Fizick]

Logo is not moving or depanestimate do not detect its movement?

upload short compressed clip.

[Fizick]

if movement between frames is always really so little, and it slow accumulates to large values, then some new specific algorithm must be implemented (for example, motion estimation across large interval). But I have no plan to do it right now.

[AVIL]

Hi,

A mad idea. If you suffle the frames, do depaning and reorder it ¿could result?.

As example:

a=clip.selectevery(10,0,5,1,6,2,7,3,8,4,9)
v=a.depanestimate()
d=a.depanstabilize(v)
d.selectevery(10,0,2,4,6,8,1,3,5,7)

[scharfis_brain]

I get some weird greenish tint when I try to use depanstabilize at default values.

It is best noticable with greyscale videos.

Is this issue known?

[odyssey]

I have trouble writing a log file. What's the correct syntax for it?

I'm trying to stabilize some of the very old disney shorts, which are badly transferred resulting in a very jumpy clip. Base frames seems to be detected quite well, but i'm not so happy with the correction itself. I've been tweaking some of the parameters with different results. My main concern is that when i specify dxmax/dymax in both DePanEstimate and DePanStabilize, there are still cases where it corrects with insane values.

I dislike that it focuses so much on the center, because it appearently often detects a moving character as a reference instead of the actual scene.

I don't really understand the way this works. I would expect it to take the base frame and use that frame as a reference for correcting other frames, but as I understand, it only corrects based on the range, that is the surrounding frames (is this correct?).

Can someone enlighten me? Also I don't understand the meaning of parameters cutoff, damping and trust.

[odyssey]

Bump. Could someone please clear up my confusion in the last post?

[DarkT]

Can't help you there, but I share your pain - I messed with the whole stabilization thingy for a couple days, and gave up on it - no matter the settings, sometimes it messed up big-big, detected stuff incorrectly, and so on... The only solution I got to was use really small values, and then record frames where it mis-detects, and then, theoretically, not apply it to those places... Which is all too much work then I'd care to do, so I dropped it...

Edit:

In case you wonder what I mean by small values, here's what I used:

i = ConvertToYV12()
mdata = DepanEstimate(i,dxmax=2,dymax=2,range=3,stab=1.0)
DePanStabilize(i, data=mdata)

[johnmeyer]

If you don't mind using Deshaker instead, it does a VERY good job of motion stabilization, if that is the only thing you need to do. I wrote a very detailed guide several years ago which you can find here:

http://www.sundancemediagroup.com/articles/deshaker_guide.htm

It was then ripped off and posted at dozens of other places around the Internet, so you can easily find it elsewhere (in case you have problems linking to the above site). Just Google my name and "Deshaker."

[DarkT]

Ah, thanks, I was put off by the author's original guide, he wrote there:

"Deshaker works best for video shot using high shutter speeds (at least 1/200 sec or so). Otherwise you might get motion blur without motion in the deshaked video. Also, the matching will work better."

I need it to stabilize ANIME - should I still give it a try?

[johnmeyer]

I need it to stabilize ANIME - should I still give it a try?
I do a LOT of work with film, both film that I transfer with workprinter as well as film that I inverse telecine back to 24p. Deshaker works wonders eliminating gate weave/judder (the slight motion introduce because the film doesn't come to rest exactly the same way from one frame to the next, either in the camera when it is being shot, or in the projector when it is being shown). In fact, the guy that wrote Deshaker even added a few settings specifically for a guy that wanted to get rid of this problem, but without affecting the global motion. In general, you want to use really "mild" settings so that only the small motion from gate weave is eliminated.

[odyssey]

Great. I'll look into it :D

Addendum:
Another thing, I've messed slightly around with the Virtualdub filter plugin and Deshaker in avisynth. Is it possible to set the nessesary parameters all inside a script and avoid the vdub step?

Depan should be able to read these log files (although I haven't had any success with it yet), would it make any difference, for Depan to do the 2nd pass instead of deshaker?

[johnmeyer]

I wrote a script for the editing program, Sony Vegas. That script communicates with Deshaker through the VirtualDub scripting language. If you want to take a look at what I did, you can download that script here:

http://www.lawrence-arnold.com/Vegas/DeshakerScript.htm

The .js portion is the script for Vegas which you can read in Notepad. If you go about 3/4 of the way down in that script you will see the call I make to VirtualDub and will see how I pass parameters into the VirtualDub script. It is a real kludge because the VirtualDub scripting was something that was added as an afterthought and has never really been enhanced or supported. However, the kludge works and I have a lot of happy Vegas users, especially since Deshaker actually works better than most of the commercial solutions.

[DarkT]

Thanks a bunch, I'll be trying it out as soon as I encode my next messy episode.

[johnmeyer]

After you run the "deshake.exe" file (which is just a zip file), this will put all the files in the c:\deshake directory. That is all it does (no registry changes or anything else). As soon as the files are unzipped, immediately open the readme.txt file. It will tell you (I think) everything you need to know for your purposes.

[odyssey]

Since this is the Depan-thread, I would really like to know how i'm able to load (and save) a log-file as described in on the web page.

It's interresting, if i'm able to scan the video with eigher Depan and Deshaker, but use Depan only for stabilizing the image, especially if i'm able to make corrections to the log-file.

[johnmeyer]

I would really like to know how i'm able to load (and save) a log-file as described in on the web page.
If you are talking about using the Deshaker log file for other plugins/filters, I have seen that discussed in many places, including the home page for each of the filters and here on the forum. Try doing a search for "Deshaker" AND the name of the filter that you want to have read the Deshaker log file.

[Fizick]

odyssey, syntax is simple as it can be:
DePanStabilize (source, data=source, ..., inputlog="deshaker.log")
So, you are able to read (at last old Deshaker 1.x versions - i did not test recent).
But log file does not contain "stabilized" data. It contains estimated motiion data only.
so, you cant change intrenal stabilization algo

[odyssey]

odyssey, syntax is simple as it can be:
DePanStabilize (source, data=source, ..., inputlog="deshaker.log")
So, you are able to read (at last old Deshaker 1.x versions - i did not test recent).
But log file does not contain "stabilized" data. It contains estimated motiion data only.
so, you cant change intrenal stabilization algo

Why is a data source needed, if an input file is specified? I thought you could get past the estimation part, and concentrate on stabilizing?

[Fizick]

its dummy data

[odyssey]

its dummy data

So it could be anything? How can I send dummy data without wasting CPU?

[Fizick]

any way :)

[odyssey]

Got it ;)

At which stage is the BASE frame detected? I have trouble with some scenes where it cannot find a BASE frame and just counts on, and other scenes where a BASE frame is picked too often.

It seems like the log file contains just cordinates, but they are not equal to the cordinates displayed with info=true.

It would be great if I were able to get a pretty good estimation logged and are able to edit out a few wrongs manually.

This is the intro of one of the old disney shorts i'm working on (http://www.megaupload.com/?d=D2T5QSGI), and I'd assume it's an easy task for any image stabilizer as no elements in the picture are not moving (just the film). Still I have problems with the duck-image and the title, where BASE frame is simply ignored and no correction made.

[anton_foy]

I have tried depan very little but here is ( IMHO ) a great way to convert 50i footage to look like it has been shot at 15 fps, "silent film"/super-8 -style.


input = input.nnedi()

vectors = depanestimate(input)
input_1_of_12 = input.duplicateframe(0)
input_2_of_12 = input.depan(vectors,offset= 1./12.,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset= 2./12.,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset= 3./12.,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset= 4./12.,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset= 5./12.,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset= 6./12.,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset= 7./12.,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset= 8./12.,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset= 9./12.,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10./12.,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11./12.,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,
\ input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)



ChangeFPS(15,linear=false)
assumefps(25,true)


I have not seen any artifacts when using this script, very cool!


Edit: mostly silent film was shot at 16 fps rather than 15 but it has occured.

[johnmeyer]

That's a great script. Now if only someone could post one that would do a good job of taking 30p video taken with the "movie" function on still cams, and turn it into decent 60i. I've tried various things with mvtools (which may or may not be a better tool for this than DePan), but so far have not gotten consisten results.

[odyssey]

Could you please try to describe the use of cutoff and damping values? I don't understand how to use these, but I think they have a significant impact on my trouble.

[Fizick]

do not touch damping.
try firstly play with cutoff.
this thread is 12 pages only :)

[odyssey]

do not touch damping.
try firstly play with cutoff.
this thread is 12 pages only :)

I did, but still don't really understand it. If I set it to 0.1, the intro is almost perfectly stable, but throughout the movie it only corrects few frames at a time.

Setting it to an insanely high value (say 100) makes the picture shake around randomly with insane correction values - Why is that?

Regardless of any value I put, the duck-image is not detected as a scene - Why not?

[Fizick]

I can not use megaupload in Russia.
My internet is not cheap (about 1 USD per 10 MB). :(
So, I do no know nothing about your "duck" :)
As I wrote above, DepaStabilize was designed not for movie, but for camcorder video.
Try apply DePanstimate on cropped border only of your films if it present and stabilize whole frame.
Try watch "trust" values and scenedetection.

[odyssey]

How do i watch trust values?

[Fizick]

info.

[odyssey]

info.

I see the trust value but have no idea how to read it. Also I don't understand how it relates to scene changes - I have a clip where even identical frames are recognized as "SCENE CHANGE!" with default trust=4. Setting trust below 1 gives a very steady picture, but it seems odd to use such low values.

[Fizick]

I see your message, but i have no idea how to read it :)
try live with this odd values

[odyssey]

I see your message, but i have no idea how to read it :)
try live with this odd values

Can you explain what the value represents?

[Fizick]

some measure of correlation.
I can not to add almost nothing to plugin doc and info collected in this (and some other?) thread at forum.

[Fizick]

But now I can release version 1.10 ;)
DepanStabilize: added new 'average' method,
small bug fixed in edge interpolation.

(But I am still not ready to rewrite many very old functions.)

[odyssey]

Would it be possible to implement a "invert mask" flag, so estimate will work on borders instead of central elements?

[Roco]

Is it possible with these plugins to achieve an almost absolute stabilization with less than a pixel resulted shaking if you define a static object in the scene?
The camera is shooting almost stable, but I want to change that to the degree that the shaking won't be detected visually. There are static elements on the background and also moving ones. The camera is actually on a tripod but due to strong wind or my hand holding a button, or a not so stable tripod etc, there is some shaking.
I have tried some scripts but I can't make it that stable (far from that) but maybe I'm missing something.
For example I tried the following where I defined a focus-window of a stable object:

AviSource("Final.avi")
source = ConvertToYV12()
focus = source.Crop(1108,916,512,162)
vectors = focus.MVAnalyse(isb = false, pel = 2)
globalmotion = source.MVDepan(vectors, zoom=false, rot=false, pixaspect=1.0, thSCD1=400)
DepanStabilize(source, data=globalmotion, cutoff=0.05, mirror=0, pixaspect=1.0)

[johnmeyer]

Have you tried Deshaker (VirtualDub plugin)?

[Roco]

Have you tried Deshaker (VirtualDub plugin)?

Yes I have, with no success so far... The online documentation doesn't make much sense without doing an infinite number of tests...

[Roco]

After testing many other apps & plugins including After effects, I came to the conclusion that there is no software for such a task -there is not enough technology today for such a precise stabilzation, especially automatic, even if you show them the static object(s).

[mikeytown2]

After testing many other apps & plugins including After effects, I came to the conclusion that there is no software for such a task -there is not enough technology today for such a precise stabilzation, especially automatic, even if you show them the static object(s).

If your feeling adventurous you could try cinelerra
http://heroinewarrior.com/cinelerra/cinelerra.html#MOTION
http://forum.doom9.org/showthread.php?t=135040

Example
http://youtube.com/watch?v=3jYFJw1dT18

Another Example
http://www.vimeo.com/302809
"910 frames shot by the mighty Canon EOS 20D. 2 motion tracking passes and color timing in Cinelerra. The changing brightness made automatic motion tracking impossible, so manual tracking was used" This vid looks like it could have used HDRAGC http://strony.aster.pl/paviko/hdragc.htm

[Roco]

If your feeling adventurous you could try cinelerra


Not that adventurous! I'd rather go to the Jungle for treasure hunting like Indiana Jones!!! :p

Seriously now, after your post, I went there and ... I came back!
I installed Ubuntu Studio and then Cinelerra, but the number of issues I encountered made me running back to Billy's Windows (unfortunately)! Half of the times I booted on Ubuntu Studio, Gnome or something didnt started, also before I run Cinelerra I had to type some strange instructions in a console to free more ram (which reminded me the old days of DOS back in 1994 when I bought my first PC), I couldn't always find my partitions (I never understood why it was so hard), when I did find them the playback of most files was sluggish, although I did install all the updates and additional codecs/players. Also the Cinelerra's interface was a bit problematic and it couldn't load my raw avis. So I didn't actually tested that stabilization feature but I don't regret it. If I did, I might become ...unstable! :)

Thanx for the links though!

I have heard something for Fusion and Nuke, I wonder if anyone knows how good is that software for stabilization... I have some respect for Macs...

[johnmeyer]

Yes I have, with no success so far... The online documentation doesn't make much sense without doing an infinite number of tests...

Well, I actually did write a manual for this which you'll find in lots of places (people just took it and copied it). Here's one of the authorized posts:

http://www.sundancemediagroup.com/articles/deshaker_guide.htm

Most of this was designed to help someone who had never used VirtualDub, so it may be too basic for you. However, it provides some initial settings that work pretty well.

In our case, you are trying to remove extremely tiny shaking. If you do some Google searching on "Deshaker" and "film," you may come across some posts from many years ago that discuss the settings to use in order to remove the "gate weave" (sometimes called gate jitter) that all film exhibits because of the fact that each frame of film, both in the camera and then in the projector, doesn't come to rest in exactly the same position as the previous frame. This can very effectively be removed by Deshaker.

[g-force]

Roco,

My very first post is dedicated to your problem!
I've had a lot of success with the following script that I created for a very similar problem.

Let me know how this works!


orig=last
temp=orig.TemporalSoften(7,255,255,25,2)
rep=repair(temp,orig,mode=16).TemporalSoften(1,255,255,25,2)
source=Interleave(rep,orig,rep)
mdata=DePanEstimate(source, range=1, trust=0, dxmax=1, dymax=1)
DePan(source, data=mdata, offset=1)
SelectEvery(3,2)

You will need repair from here:

http://home.pages.at/kassandro/RemoveGrain/

You may have to play around with dxmax and dymax for your application, and then you'll have to crop the edges by the appropriate amount. I may write a function for this if it works for more people than just myself.

-G

[Roco]

johnmeyer,
Thanx for the response, you have done a great job with the guide, also congratulations to the author of Deshaker!

It worked very good, to a degree that I wouldn't mind for some slight instabilities, but unfortunately not at the resolution I wanted which is 3008x2000. While it passed the first pass succesfully, at the second pass, no matter what I tried, it causes the following runtime error:

http://forum.doom9.org/attachment.php?attachmentid=8191&stc=1&d=1204928117

I wonder if the author could fix this...

g-force,
Welcome to the forum and thank you very much for the suggestion, but at some basic tests I did, the script didn't work for me as expected, but I will experiment a bit more with the parameters.

[g-force]

g-force,
Welcome to the forum and thank you very much for the suggestion, but at some basic tests I did, the script didn't work for me as expected, but I will experiment a bit more with the parameters.

Thanks for the warm welcome! I didn't even think that your source is way larger in resolution that what I'm used to. You may need to increase dxmax and dymax as a result.

Hope you get something that works!

-G

[g-force]

Roco,

have you had a chance to play around with this yet? I can take a whack at it if you want to post a clip.

-G

[Roco]

Roco,

have you had a chance to play around with this yet? I can take a whack at it if you want to post a clip.

-G

Thank you very much g-force for your interest and your offer, but I cannot post a sample of that material (I wish I could). I was occupied with some other stuff until today so I will know in about 2-3 days what solution will be more suitable (if any) and I will let you know.

[g-force]

Thank you very much g-force for your interest and your offer, but I cannot post a sample of that material (I wish I could). I was occupied with some other stuff until today so I will know in about 2-3 days what solution will be more suitable (if any) and I will let you know.

No worries. Let me know what you find out!

-G

[thetoof]

Little question about pixaspect=

I use a LOT this very useful script by g-force (turned into a function for convenience) before encoding anamorphic (so I don't resize to square pixel aspect ratio).

function stab (clip orig, float "pixaspect"){
pixaspect=default(pixaspect,1)
orig
temp = last.TemporalSoften(7,255,255,25,2)
Interleave(temp.Repair(last.TemporalSoften(1,255,255,25,2)),last)
DePan(last,data=DePanEstimate(last,trust=0,dxmax=10,dymax=10),offset=-1, pixaspect=pixaspect)
return SelectEvery(2,0)}

It is said in the docs (actually, I saw that in the MVtools docs) that the pixaspect should be 1.094 for standard PAL and 0.911 for standard NTSC.

However, this calculation yields to other values... where did I mess up??
NTSC
720x480 is actually 640x480 (4:3)
y * 720 = 640
y = 0.8888....

720x480 is actually 853.3333x480 (16:9)
y = 1.185185185.....

PAL
720x576 is actually 768x576 (4:3)
y * 720 = 768
y = 1.06666........

720x576 is actually 1024x576 (16:9)
y = 1.42222.....

edit: and (why) is it such a big deal to use pixaspect=1 even if I'm filtering an anamorphic clip??

[kemuri-_9]

It is said in the docs (actually, I saw that in the MVtools docs) that the pixaspect should be 1.094 for standard PAL and 0.911 for standard NTSC.

However, this calculation yields to other values... where did I mess up??


it seems that those calculations are more seeming to be based on a source with a width of 704 not 720
so
640/704 ~ .910
768/704 ~ 1.10

the aspect numbers you posted/quoted result as:
640/.911 ~ 703
768/1.094 ~ 702

so it seems that these are fairly close to being based on 704,
but not quite, so maybe some rounding error?

probably based on 704 to account for cropping on 'typical' overscan (which may or may not actually be true for what you're dealing with)

[Fizick]

about aspect you may read "Analog capture guide" http://forum.doom9.org/showthread.php?t=78797

[g-force]

Fizick, correct me if I'm wrong, but DePan should only need to know about pixaspect for Zoom or Rotation, both of which the script thetoof posted doesn't use.

[Fizick]

g-force, only if rotation is not 0.
So, you are wrong (partially) :)

[g-force]

I think I understand. I'll try one more time:

Fizick, correct me if I'm wrong, but DePan should only need to know about pixaspect for Rotation, which the script thetoof posted doesn't use. ?!?!

-G

[Fizick]

g-force,
yes, it is correct.

but IMHO, all this aspects is not so important practically :)

[gaillague]

Hello all,
This is my first post ever and I want to thank the community, but this question is perhaps especially for Fizick.

I have problems with DePanStabilize with the average method method=1
Many parameter do not longer work when i use before mdepan() or mvdepan() for global motion estimation.

eg dxmax,dymax,rotmax,addzoom, zoommax

However they work fine when i use DepanEstimate() for the global motion estimation but it's less interesting for me because there is no rotation estimation.

It seems to me that the average method give a very good deshaking, close to the Vdub Plugin Deshaker.
Unfortunately without all this parameters, it's very hard to tweak precisely the motion stabilization and the zoom.

exemple of non working script
a=last
vectors = a.MSuper().MAnalyse(isb = false,blksize=8)
globalmotion = a.MDepan(vectors)
a1 = a.DePanStabilize(data=globalmotion,method=1,cutoff=0.5,dxmax=0,dymax=0,rotmax=0,addzoom=true)
return a1

thanks in advance

[Fizick]

dxmax=0 (and dymax=0, rotmax=0) should not work for method=1 for any vector source (MDepan, DePanEstimate).
The current code contains division of dx by dxmax, etc
It was unclear for me, how to process special case dxmax=0, etc

[gaillague]

thank you for reply

dxmax=0 (and dymax=0, rotmax=0) was only an (extrem) exemple but dxmax =x (and dymax=y, rotmax=z ) do'nt work too (with method=1 and M(v)Depan as vector source.
The stabilisation works (damn good) but no correction is possible.
The addaptive zoom do'nt work too.

DePanStabilize(data=globalmotion,method=1,dxmax=10,dymax=20,rotmax=0,addzoom=false)
or
DePanStabilize(data=globalmotion,method=1,dxmax=60,dymax=60,rotmax=30,addzoom=true)

are giving absolut identical results.

[Fizick]

gaillague,
Oh, yes, dxmax, etc are disabled for method=1. It seems I did not get good result that time.
But it is not dependent on vector source. DepanEstimate should not work too.
You should wait some more advanced developer for motion stab. Sorry.

[gaillague]

You should wait some more advanced developer for motion stab :rolleyes:

I'll wait....
But it is not too annoying, the tool works anyway very fine :)

In any case,thanks very much for all your work.
:thanks::thanks:

[Fizick]

I will fix the doc about dxmax, etc.
But addzoom should work with method=1

[gaillague]

addzoom is not working too :(
always with method=1 & MDepan as vector source

mvtools2 v 2.4.2
Depan v 1.10.0

[Fizick]

I found that I use DePan v1.10.1, January 27, 2008 :)
Seems, I forget to publish it.
Changes is minimal (fixed bug with blur mode for subpixel=1), but V1.10.0 really does not work with addzoom for some strange reason.
I will upload the update

[gaillague]

Yes it works now
thanks Fizick
:thanks:

[Mini-Me]

Sorry to necro this thread, but is the source code available for Depan/DepanStabilize?
DepanEstimate includes the source with the binary, but Depan/DepanStabilize do not.

Thanks :)

[Fizick]

Well, i am totally out of avisynth world now. :(
So, source Depan code is released today. But it was my first plugin, and probably the worst code ever written :)
Good luck!

[Mini-Me]

Well, i am totally out of avisynth world now. :(
So, source Depan code is released today. But it was my first plugin, and probably the worst code ever written :)
Good luck!

Don't worry; I think everyone believes their early code to be the worst code ever written. :D I'm glad you decided to make it available anyway. Thank you!

[videoFred]

Well, i am totally out of avisynth world now. :(

I'm sorry to hear this... why?


So, source Depan code is released today. But it was my first plugin, and probably the worst code ever written :)


Well, it's the best stabilizer for Avisynth and I got it perfect working now! After many tests, I have found that correct levels for the base clip are very important. So I'm feeding DePanEstimate with a special tweaked clip. I'm using colorUYV(autogain=true) for this. ColorYUV(autogain=true) is often wrong, but it works very good for DePanEstimate().

The trust parameter is very important too. It sets the scene change detection.

I have tested this on many different clips and the result is almost always 100%, with trust=1.5.

I will upload my new stabilizing script here today. :)

Fred.

[fluor]

I just discovered this fantastic plugin and it does wonders for my digital camera movies! by just cropping 20 pixels around I get a much more stable picture on many of my movies!

I use the following for some of my canon camera movies

loadplugin("C:\Util\canon2mkv\ffms2.dll")
FFVideoSource("c:\full-path\MVI_9766.avi")
loadplugin("C:\Util\canon2mkv\hqdn3d.dll")
loadplugin("C:\Util\canon2mkv\DePan.dll")
loadplugin("C:\Util\canon2mkv\DePanEstimate.dll")
i = ConvertToYV12()
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata,method=0,dxmax=20,dymax=20,rotmax=0)
crop(20,20,-20,-20)
hqdn3d()


:thanks: :thanks: :thanks: !!

[SSH4]

Hmm, i don't undestand why, but blur in DePanStabilize 1.101 not work for me.
Made it any size from 10 to 150 and don't see any difference :(


vectors = MSuper().MAnalyse(isb = false,blksize=8, overlap=4,search=5)
globalmotion = MDepan(vectors, pixaspect=0.911, thSCD1=500)
DepanStabilize(data=globalmotion, cutoff=0.5, mirror=15,rotmax=5,prev=0,next=0,blur=50, pixaspect=0.911,subpixel=2,method=1)

[Fizick]

SSH4,
sorry. Blur is not implemented for rotation in current version 1.10.1.
Try MDepan (..., rot=false)

Also I discovered, that blur was accidentally not implemented at left side for subpixel=1.

[Hat3L0v3]

Can somebody help me to figure out why depan filter crashing my Avs(AvsPmod)?
I want to use DePanStabilize from mvtools pack but it crashes.
system and others:
win10 x64
avs+ 3.7.3 r3825 x86-64
AvsPmod_v2.7.3.2_.Windows_x86-64
mvtools-2.7.45 (.dll`s x64)
vcredist last time installed from here (https://github.com/abbodi1406/vcredist) (v0.64.0) (can't remember what versions I had before)

(libfftw3f-3.dll and libfftw3l-3.dll) from FFTW - 3.3.10
x64.dll into System32 folder (x32.dll on SysWoW64)

I remember before updating avs+ from 3.7.2 to r3825 I tried different stabilizers and depan was causing "access violation" error (same as TemporalDegrain2 with postFFT=2 option). Now they both just crashing without errors.

[StainlessS]

libfftw3f-3.dll, but not libfftw3l,-3.dll
also can copy renamed libfftw3f-3.dll to system32/fftw3.dll
Some filters like one name, some the other, and some will accept either.

Failing that, post script.

[kedautinh12]

libfftw3f-3.dll, but not libfftw3l,-3.dll
also can copy renamed libfftw3f-3.dll to system32/fftw3.dll
Some filters like one name, some the other, and some will accept either.

Failing that, post script.

Change name x64 libfftw3f-3.dll to fftw3.dll and copy to system32. Don't need rename libfftw3f-3.dll already in system32. I had both libfftw3f-3.dll and fftw3.dll in system32, don't have error with both already in system32

[StainlessS]

Well, result is same thing. [provided those in system32 are same version as new ones]

[Hat3L0v3]

libfftw3f-3.dll, but not libfftw3l,-3.dll
also can copy renamed libfftw3f-3.dll to system32/fftw3.dll
Some filters like one name, some the other, and some will accept either.

Failing that, post script.

So I need just single libfftw3f-3.dll in my system32 folder if I using x64 version avs/avsp? I deleted all other .dll`s - still crashing.

[kedautinh12]

So I need just single libfftw3f-3.dll in my system32 folder if I using x64 version avs/avsp? I deleted all other .dll`s - still crashing.

You need both libfftw3f-3.dll and fftw3.dll in system32 too. And fftw3.dll same with libfftw3f-3.dll (change name libfftw3f-3.dll to fftw3.dll)

[kedautinh12]

If still errors, you can check with dependencies, drag mvtools.dll, fft3dfilter.dll, neo-fft3d.dll to this app to check error
https://github.com/lucasg/Dependencies/releases

[Hat3L0v3]

If still errors, you can check with dependencies, drag mvtools.dll, fft3dfilter.dll, neo-fft3d.dll to this app to check error
https://github.com/lucasg/Dependencies/releases

after I drag it to app I don't really undestand where to look for errors. Like this (https://i.imgur.com/cztKw55.png)one? the only red I found.

[kedautinh12]

after I drag it to app I don't really undestand where to look for errors. Like this (https://i.imgur.com/cztKw55.png)one? the only red I found.

Yes, you missing this .dll. You need download this x64 ver .dll and copy to system32

[kedautinh12]

after I drag it to app I don't really undestand where to look for errors. Like this (https://i.imgur.com/cztKw55.png)one? the only red I found.

My mistake. You need check depan.dll and depanestimate.dll too

[Hat3L0v3]

My mistake. You need check depan.dll and depanestimate.dll too

I have list (https://i.imgur.com/WPVEgU8.png) of this .dll`s missings. Don't really know how it can helps and where to fix them (if it needed)

[kedautinh12]

You can search .dll missing and download here
https://www.solvusoft.com/en/files/missing-not-found-error/dll/

[Hat3L0v3]

You can search .dll missing and download here
https://www.solvusoft.com/en/files/missing-not-found-error/dll/

this site not work properly.
I open other .dll that works for test it have same .dll missing but it works fine. so maybe not all of them I need to repair.

[kedautinh12]

Ok, i use this fftw3.3.10 and didn't had error (you can copy all x64 .dll to system32)
https://drive.google.com/file/d/1PF9SVH6iWvmkv9Vwbw-yfa7u6BDYqOj6/view?usp=drivesdk

If error still exist, try search: add or remove program -> unistall all vc++ version and install abbodi1406's vc++ again

[Selur]

Just as a side note:
Instead of copying fftw dlls into the system folder you could also load it inside the Avisynth script using LoadDLL (http://avisynth.nl/index.php/LoadDLL). :)
Here's an example:

# LoadPlugin("I:\Hybrid\64bit\Avisynth\avisynthPlugins\LoadDll.dll") # only needed when auto-loading is disabled.
LoadDLL("I:\Hybrid\64bit\Avisynth\avisynthPlugins\fftw\libfftw3f-3.dll")

(Hybrid uses this for ages. Hybrid disables the auto-loading and loads everything explicitly for more control over what is used.)

Cu
Selur

[StainlessS]

Found this on TechNet
https://social.technet.microsoft.com/Forums/WINDOWS/en-US/8f58e95f-60c5-44ac-85b5-2531914ac738/missing-all-apimswin-dlls-many-apps-will-not-start-including-vc2017redistx64-and?forum=WinServerPreview

Question
Sign in to vote
0
Sign in to vote

Thank you for the pointer to the VC++ 2015-2019 v14.23.27820.0 runtime files that will start and run on Win 10 v1903. Obviously these have been compiled with different dependencies than the VC++ 2015-2019 v14.21.27702.2 runtime files that I have had since v1803 of Win 10.

Although, installing these 2015-2019 C++ runtimes has not solved the issue.

I have read elsewhere on this forum:

"You are not missing those DLLs.

Those DLLs are Windows's implementation detail and are subject to change at anytime. Basically Microsoft started moving around APIs starting from Windows 8. For example, APIs in Windows 8's api-ms-win-core-file-l1-2-0.dll got moved to api-ms-win-core-file-l1-2-1.dll in Windows 8.1. Old software still work because the dlls developer linked to are now just placeholders redirecting calls to the actual implementation. But nobody should link to the implementations directly, as that would be defeating the purpose of having such a redirection. Any software that report those dlls are missing are failing to accommodate the redirection (e.g. reporting a delay-loaded dependency as hard one)."

that v1903 has replaced the direct calls to all of those DLLs with a handful of stub DLLs that now must be called to be redirected to the corresponding function provided by the original DLLs.

In my opinion, it was very rude of Microsoft to make this change to v1903, with no workaround, as it ruins several years of programs that were created using VC++.

Including Microsoft's own VC++ runtime installation files which were obviously recompiled with new dependencies to now function correctly on v1903.

If anyone knows of a workaround other than regression to v1803 of Win 10, please reply.

Thank you in advance.

Looks like M$ ****'ed it up a bit.

Those error messages you have [missing dll's], might be bogus.

Near end of that technet thread, seems that something you downloaded, might not have been installed correctly (maybe the AIO runtimes updater),
due to M$ preventing it from running properly because it was downloaded from an 'unknown' computer. (ie internet).

Also [YOU MUST] see here:- https://forum.doom9.org/showthread.php?p=1872609#post1872609

EDIT: to do with this stuff [see above D9 link]:
https://i.postimg.cc/13hhcYbq/Untitled.jpg (https://postimages.org/)

EDIT: Added a Regedit fix for 'unknown' computer file blocking thing at above D9 link.
There is also a GpEdit.msc fix at above link (Windows Pro only).
Suggest [YOU MUST] get the NirSoft AlternateStreamsViewer mentioned at above link. and search for blocked files over your whole system
(I had hundreds of files blocked without knowing, when I first discovered problem).

[Hat3L0v3]

Ok, i use this fftw3.3.10 and didn't had error (you can copy all x64 .dll to system32)
https://drive.google.com/file/d/1PF9SVH6iWvmkv9Vwbw-yfa7u6BDYqOj6/view?usp=drivesdk

If error still exist, try search: add or remove program -> unistall all vc++ version and install abbodi1406's vc++ again
Don't understand why you think fftw3 .dll`s are the problem. I deleted all of them, its still crashing.


Also [YOU MUST] see here:- https://forum.doom9.org/showthread.php?p=1872609#post1872609

scanned disc C with AlternateStreamView show nothing except some torrent files.

I delete manually all vc++ that was installed on pc and run depan again - still crashing.
Guess I need delete avs+ 3.7.3 r3825 and go back for 3.7.2 stable version.

upd. After installing avs+ 3.7.2 stable start getting Access Violation error, not a crash.

[StainlessS]

Have a look at your Windows Event Logs.
(Right Click "My PC" (or whatever its called {I'm not currently on Windows}, and click on Manage)

[Hat3L0v3]

Have a look at your Windows Event Logs.
(Right Click "My PC" (or whatever its called {I'm not currently on Windows}, and click on Manage)

you mean Event Viewer ? and what I need looking for?

upd. I download FullEventLogView, but it doesn't show anything new when I recreating Access Violation error.

[Hat3L0v3]

Sorry guys, my bad. There was nothing wrong with setup or .dll
I just didn't know that Depan require data from DePanEstimate in order to work correctly. So it crashed avsp every time. Usually filters lead to the place if something goes wrong. Didn't expect such behavior.
So I'll go run some tests to find settings that works better for me. Because stab3 helps me with shakings on static scenes (where backgound static and character (or something) are shaking) but also ruin another static scenes where there's no shaking (by making background follow character movement so all scene start shaking for exampe). So it takes me alot of time to watch and trim manually whole episode and apply stab3 only on scenes where it needed.

[kedautinh12]

I was said you about DePanEstimate.dll but you won't notice it
https://forum.doom9.org/showthread.php?p=1981691#post1981691

[pinterf]

I was said you about DePanEstimate.dll but you won't notice it
https://forum.doom9.org/showthread.php?p=1981691#post1981691
The dlls were perfectly fine. The avs script did not pass the result of DepanEstimate to Depan. Depan was called without any parameters.