motion stabilize before or after "processing" ?

[hydra3333]

Seeking advice on at what point in a script to motion stabilize a handheld PAL 16:9 DV camera "fairly ordinary and lowish light and quite noisy" interlaced capture - before or after "other processing". The (untested) code below does it just after deinterlacing but before denoising and reinterlacing.

What positioning do you advise for the motion stabilization ?


# for the moment, do a "quick" deinterlace first
AssumeBFF()
yadifmod(mode=1, edeint=nnedi2(field=-2)) #mode=1 means double framerate,field=-2 means double rate (alternates each frame)
# try one of these for different deinterlacing http://forum.doom9.org/showthread.php?p=1374359#post1374359

# see example at http://avisynth.org.ru/mvtools/mvtools2.html#examples
blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
super0 = MSuper(pel=2,chroma=true,sharp=2)
vect = MAnalyse(super0,isb=false, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
globalmotion = MDepan(vect, pixaspect=1.094, thSCD1=400, zoom=true, rot=true, range=1) #1.094 for standard PAL
DepanStabilize(data=globalmotion, rotmax=5, cutoff=2.0, mirror=15, pixaspect=1.094) #1.094 for standard PAL

blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
super = MSuper(pel=2,chroma=true,sharp=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,plane=4)

HDRAGC(corrector=0.8, reducer=2.0, black_clip=1.0)
#HDRAGC(coef_gain=0.1, min_gain=0.1, max_gain=0.5, coef_sat=1.0, corrector=0.8, reducer=2.0, black_clip=1.0)
LimitedSharpenFaster(smode=4,strength=100)

AssumeBFF()
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
AssumeBFF()



edit: I see videofred does it at the top of his script, but with DePanEstimate (which I don't have) on a clip which has things done to it...
stab_reference= source1.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()
mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
Should I unravel what it means and do something equivalent too ?

[g-force]

I've never had much success with the method videoFred uses, or with what you are doing in your script, but the placement is good (after de-interlacing, before Mo-Comped denoising)

search out stab() function for something may may or may not work better.

-G

[2Bdecided]

No, stab() isn't meant for camcorder footage at all. VirtualDub Deshaker is.

Cheers,
David.

[pirej]

I tried to extract the stabilizing part from videoFred's script for "8mm film restoration", and becouse i'm a beginner in using avisynth a'm happy i succeeded :)
Here is the script.
http://www.mediafire.com/?1zzzcjyn2qw
And the result is pretty good.
Here is the short stabilized clip(mjepg 3.6mb).
http://www.mediafire.com/?zhmynixxjzm
And a amazing (in my opinion) restoration job of VERY bad vhs tape, using videoFred's script(screenshot)
http://www.mediafire.com/i/?nt5t2ny25ju
So my point is that videoFred's order of stabilizing and "processing" works really good!

[markanini]

Thanks pirej!
Can't wait to try on some handheld digicam footage and compare to deshaker which is a dog to configure. Stab didnt work especially well for shaky handheld footage IME.

[videoFred]

Here's my stabilization method as a script, without anything else:

# film stabilization script by videoFred. C2010
#====================================================================================


film="C:\Documents and Settings\Eigenaar\Mijn documenten\films\Dirk\videofred_002.avi"


result="comp" # stab= stabilized clip
# stab2= stabilized and cropped clip
# comp= comparison orgiginal vs stabilized clip



#STABILIZING PARAMETERS
#----------------------------------------------------------------------------------------------
maxstabH=30
maxstabV=30 #maximum values for the stabiliser (in pixels) 20 is a good start value

est_left=20 est_top=20 est_right=20 est_bottom=20 #crop values for special Estimate clip

est_cont=1.0 #contrast value for special Estimate clip

CLeft=20 CTop=20 CRight=20 CBottom=20 #crop values after Depan



# END VARIABLES, BEGIN SCRIPT
#=============================================================================================


SetMemoryMax(800) #set this to 1/3 of the available memory


Loadplugin("Depan.dll")
LoadPlugin("DepanEstimate.dll")
Loadplugin("mt_masktools.dll")


source1= AviSource(film).converttoYV12()




#STABILIZING
#................................................................................................
stab_reference= source1.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).
\MT_binarize(threshold=80).greyscale().invert()

mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstabH,dymax=maxstabV)
stab=DePanStabilize(source1,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,mirror=15)

stab2= stab.crop(CLeft,CTop,-CRight,-CBottom)
stab3=DePanStabilize(source1,data=mdata,cutoff=0.5,dxmax=maxstabH,dymax=maxstabV,method=1,info=true)

WS= width(stab)
HS= height(stab)
stab4= stab3.addborders(10,10,10,10,$B1B1B1).Lanczos4Resize(WS,HS)

show_reference= overlay(source1,stab_reference,x=est_left,y=est_top).
\addborders(2,2,2,2,$FFFFFF).Lanczos4Resize(WS,HS)




#SPECIAL COMPARISON CLIP FOR TESTING THE STABILIZER
#.......................................................................................................
comp= stackhorizontal(subtitle(show_reference,"baseclip for stabilizer -only the B/W clip is used",
\size=32,align=2),subtitle(stab4,"test stabiliser: dx=horizontal, dy=vertical",size=32,align=5))


Eval(result)

est_cont is a very important parameter: play with it :)

Fred.

[Didée]

stab_reference= source1.crop(est_left,est_top,-est_right,-est_bottom).tweak(cont=est_cont).
\MT_binarize(threshold=80).greyscale().invert()

Technically okay, but a bit more complicated than necessary. That's four filters. You need only one. Three are superfluous.

invert() can be achieved by swapping the "upper" argument of mt_binarize. Default is false, so setting it to true will replace invert().
greyscale() can be done by mt_binarize through the U,V parameters.
Tweak is not necessary, because it is just a linear scaling. Since it's followed by a fixed-threshold in mt_binarize, you can just as well kick the tweak() out, and adjust the binarize threshold instead.

So, the whole

tweak(cont=est_cont).MT_binarize(threshold=80).greyscale().invert()

melts down to

mt_binarize(threshold=80/est_cont,upper=true,U=-128,V=-128)

One filter instead of four.

Now, this is a pretty fast script sniplet where it doesn't matter too much. I hope the rest of your script isn't of similar complicated-ness, especially in the more time-critical parts... ;)

[videoFred]

Technically okay, but a bit more complicated than necessary. That's four filters. You need only one.

Thank you for correcting this, Didée!
I'm an end user, you are ....:eek: :)


I hope the rest of your script isn't of similar complicated-ness, especially in the more time-critical parts... ;)

Please be my guest to have a fast look at it, if you have time for this of cource.

http://forum.doom9.org/showthread.php?t=144271

Fred.

[Didée]

I had a glimpse at it back in the past ... frankly, I found the scripting style / script flow a bit unintuitive for me, so I let it be. ;)
My thoughts currently circle rather around TGMC (20-40% faster, perhaps make it 'lossless', ... )
And every day, the paper is crying for me, too ... (see sig.)

[hydra3333]

Thanks.
Is MDepan different to a more useful extent than DePanEstimate ?
Just checking ...
clip stab2 isn't used ?
clip stab isn't used (only to get width and height) ?

So for the sample code at top, the stabilizing bit becomes like this in order to improve motion detection ?
I'm not sure if the "=MAnalyse" should be "=stab_ref_clip.MAnalyse" (I don't think it should be).
# see example at http://avisynth.org.ru/mvtools/mvtools2.html#examples
blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
stab_ref_clip=last.mt_binarize(threshold=80/est_cont,upper=true,U=-128,V=-128)
super0 = MSuper(stab_ref_clip,pel=2,chroma=true,sharp=2)
vect = stab_ref_clip.MAnalyse(super0,isb=false, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
globalmotion = stab_ref_clip.MDepan(vect, pixaspect=1.094, thSCD1=400, zoom=true, rot=true, range=1) #1.094 for standard PAL
DepanStabilize(data=globalmotion, rotmax=5, cutoff=2.0, mirror=15, pixaspect=1.094) #1.094 for standard PAL

[videoFred]

I had a glimpse at it back in the past ... frankly, I found the scripting style / script flow a bit unintuitive for me, so I let it be. ;)

Haha.. I see.. But the script gives the results I want :)


And every day, the paper is crying for me, too ... (see sig.)

I vote for Kermit the Frog :)

Fred.

[videoFred]

Thanks.
Is MDepan different to a more useful extent than DePanEstimate ?

I do not know... I have never tested MDepan


clip stab2 isn't used ?
clip stab isn't used (only to get width and height) ?

If you want stabilized but not cropped output, use stab. If you want stabilized and cropped output, use stab2.


I'm not sure if the "=MAnalyse" should be "=stab_ref_clip.MAnalyse" (I don't think it should be).


Well, the trick with my method is to do the analyse on the prepared clip and apply the result on the original clip. If MAnalyse() is the same as DePanEstimate() then you must do the MAnalyse on the stab_ref_clip indeed. Otherwise the stab_ref_clip has no use at all.

PS: you must change your script according to this.. As far as I can see, your script is returnig the stabilized stab_ref_clip! It should return the original clip, but stabilized with the information from the prepared clip.

Fred.

[hydra3333]

Well, Fizick's example at http://avisynth.org.ru/mvtools/mvtools2.html#examples with interlaced and his new mvtools2 is :-
AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc
AssumeTFF().SeparateFields() # set correct fields order
vectors = MSuper().MAnalyse(isb = false)
globalmotion = MDepan(vectors, pixaspect=1.094, thSCD1=400)
DepanStabilize(data=globalmotion, cutoff=2.0, mirror=15, pixaspect=1.094)
Weave()

so I'd thought "if I remove the interlacing bits, I must be close".

edit: searched and found another example which seemed to clarify ...edited script above (http://forum.doom9.org/showthread.php?p=1376831#post1376831) to put "stab_ref_clip." in front of MAnalyze and MDepan. Still not sure about putting it in front of MAnalyse, it doesn't feel right...
# see example at http://avisynth.org.ru/mvtools/mvtools2.html#examples
blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
stab_ref_clip=last.mt_binarize(threshold=80/est_cont,upper=true,U=-128,V=-128)
super0 = MSuper(stab_ref_clip,pel=2,chroma=true,sharp=2)
vect = stab_ref_clip.MAnalyse(super0,isb=false, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
globalmotion = stab_ref_clip.MDepan(vect, pixaspect=1.094, thSCD1=400, zoom=true, rot=true, range=1) #1.094 for standard PAL
DepanStabilize(data=globalmotion, rotmax=5, cutoff=2.0, mirror=15, pixaspect=1.094) #1.094 for standard PAL

edit2: I really do think I should remove the stab_ref_clip. - can anyone please confirm ?

[videoFred]

so I'd thought "if I remove the interlacing bits, I must be close".

And you have done this well. But using a special prepared reference clip is different. It was a hint from Fizick - it must be somewhere on his site - and I have tested this in practice.

Fred.

[hydra3333]

Thanks, I'll give it a whirl and see. I don't think that MAnalyse() is the same syntax/usage as the DePanEstimate() which you use, but I may be wrong... Fizick's web pages seem to indicate they are different.
http://avisynth.org.ru/mvtools/mvtools2.html
http://avisynth.org.ru/depan/depan.html

[videoFred]

edit2: I really do think I should remove the stab_ref_clip. - can anyone please confirm ?

Of cource you must remove it. MAnalyse is already using a clip: super0.

Fred.

[hydra3333]

Thanks Fred. So the full script becomes

# for the moment, do a "quick" deinterlace first
AssumeBFF()
yadifmod(mode=1, edeint=nnedi2(field=-2)) #mode=1 means double framerate,field=-2 means double rate (alternates each frame)
# try one of these for different deinterlacing http://forum.doom9.org/showthread.php?p=1374359#post1374359

# see example at http://avisynth.org.ru/mvtools/mvtools2.html#examples
blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
stab_ref_clip=last.mt_binarize(threshold=80/est_cont,upper=true,U=-128,V=-128)
super0 = MSuper(stab_ref_clip,pel=2,chroma=true,sharp=2)
vect = MAnalyse(super0,isb=false, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
globalmotion = stab_ref_clip.MDepan(vect, pixaspect=1.094, thSCD1=400, zoom=true, rot=true, range=1) #1.094 for standard PAL
DepanStabilize(data=globalmotion, rotmax=5, cutoff=2.0, mirror=15, pixaspect=1.094) #1.094 for standard PAL

blksize=8 # 4, 8 or 16 ( default is 8 ). Larger blocks are less sensitive to noise, are faster, but also less accurate.
overlap=4 # overlap value (0 to 4 for blksize=8) Must be even and less than block size
dct=0 # use dct=1 for clip with light flicker
super = MSuper(pel=2,chroma=true,sharp=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct, chroma=true)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,plane=4)

HDRAGC(corrector=0.8, reducer=2.0, black_clip=1.0)
#HDRAGC(coef_gain=0.1, min_gain=0.1, max_gain=0.5, coef_sat=1.0, corrector=0.8, reducer=2.0, black_clip=1.0)
LimitedSharpenFaster(smode=4,strength=100)

AssumeBFF()
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
AssumeBFF()

[g-force]

No, stab() isn't meant for camcorder footage at all. VirtualDub Deshaker is.

Cheers,
David.

maybe not at it's default settings, but you just need to bump up dxmax and dymax for shakier video.

##############################################################################
#Original script by g-force converted into a stand alone script by McCauley #
#latest version from December 10, 2008 #
##############################################################################

function Stab (clip clp, int "range", int "dxmax", int "dymax") {

range = default(range, 1)
dxmax = default(dxmax, 4)
dymax = default(dymax, 4)

temp = clp.TemporalSoften(7,255,255,25,2)
inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp)
mdata = DePanEstimate(inter,range=range,trust=0,dxmax=dxmax,dymax=dymax)

DePan(inter,data=mdata,offset=-1)
SelectEvery(2,0) }

[ChrisW77]

Interesting thread.
Is it possible to correct up and down bobbing (vertical stabilizing ?) only ?
I have some camcorder footage, shot on a older cam (2001-2003 ish era using mini tapes) by my Dad, of my Sister's little one. All scripts I've tried, correct the vertical stabilizing nicely, but the moment the camera pans to the left or right, the scripts I've tried try to stabilize the panning as well.
I can't post any footage, because they're private home videos, but is this possible ?
Sorry, don't mean to hijack the thread, or anything.

[videoFred]

Hey Chris,

What you need is to analyse only a small area where the jitter is very obvious, then apply these vectors to the entire picture. My script is special made for this.. Try!

Fred.

[hydra3333]

Just wondering, some people seem to use the DePanEstimate function last updated January 27 2008, whilst the more recent version of MVtools2 last updated 8.11.2009 suggests using the newer MSuper/MAnalyse/MDePan.

Would it be fair to guess that the newer MSuper/MAnalyse/MDePan functions would incorporate more recent updates and motion detection approaches than the older DePanEstimate functions and maybe some other things like multithreading ?

[ChrisW77]

Hey Chris,

What you need is to analyse only a small area where the jitter is very obvious, then apply these vectors to the entire picture. My script is special made for this.. Try!

Fred.

Many thanks, Fred, I'll have a go at your script,

Cheers,

[g-force]

Interesting thread.
Is it possible to correct up and down bobbing (vertical stabilizing ?) only ?


sure, use stab and set dxmax to 0.


Hey Chris,

What you need is to analyse only a small area where the jitter is very obvious, then apply these vectors to the entire picture. My script is special made for this.. Try!

Fred.

really? wouldn't the small area also contain the pans and try to correct them?

-G

[videoFred]

really? wouldn't the small area also contain the pans and try to correct them?

Fast pans are very difficult to correct indeed. Sometimes it is better to leave them as they are. It all depends on the source of cource. There are no general rules here, every scene is different.

PS: because it's so difficult to predict the behaviour of the stabilizer, I have added the split screen: original on the left, stabilized on the right. Depan runs in real time on any recent computer, so this split screen is ideal for fine tuning the stabilizer. The split screen also shows the position of the cropped analyse clip on the left and all the Depan info on the right.

Fred.

[travolter]

Hi guys!

Im trying to do frameratedoubling + motion stabilize realtime. (anyone have a script ready?)
(the last powerdvd10 implement these 2 features realtime when playing videos and not cpu heavy.. so I wonder if I can do the same with avisynth)

There is a simple and low CPU script for motion stabilize?