Is VirtualDub and DeShaker still the preferred way of stabilizing motion?

[Chainmax]

A long time ago, when I asked about this, DeShaker was the most recommended option. I tried some suggested DePan settings but they never really satisfied me. Has anything been created on the Avisynth side that produces at least as good results as the VDub route?

[g-force]

Here is a script that I used that worked quite well!

temp=orig.TemporalSoften(7,255,255,25,2)
rep=Repair(temp,orig,mode=16).TemporalSoften(1,255,255,25,2)
source=Interleave(rep,orig,rep)
mdata=DePanEstimate(source, range=1, trust=0, dxmax=1, dymax=1)
DePan(source, data=mdata, offset=1)
SelectEvery(3,2)

You'll need to crop afterwards.

-G

[scharfis_brain]

I still recommend Deshaker + VDub.
Depanstabilize isn't as flexible as deshaker in difficult situations.

For best image quality I suggest to prepare an interlaced video like this:

1) xxxsource("blah.xxx") # load the video
crop(...) #cut away dirty borders
yadifmod(mode=1, edeint=nnedi(field(-2)) #bob deinterlace it without stairstepping to 50 or 59.94fps
converttorgb24(matrix="PC.601") # convert to VDubs colourspace without introducing banding.

2) serve this to VDub or save it as lossless compressed RGB Video.

3) Apply deshaking and save it as lossless compressed deshaked RGB Video.

4) load the video into AVISYnth and apply
converttoyuy2(matrix="PC.601") to revert it to YUV colourspace without banding.

5) Do any further processing you like.
(eg. Denoising, Colour adjustment, sharpening)

6) Apply re-interlacing, if necessary
separatefields().selectevery(4,0,3).weave()

[Boulder]

How about those older videos where there is a small (usually vertical) jitter of a couple of pixels here and there, is there anything that could be used to stabilize them?

[scharfis_brain]

this issue can be well treated with depanstabilize and very low limits for vertical compensation and near to zero limit for horizontal compensation.
I used depanstabilize to de-jitter some old film footage before.
It works quite well for this purpose.

But for handheld camera deshaking I don't recommend depanstabilize.

[g-force]

I've never been able to dial in the settings for DePanStabilize to be of any use. Doesn't fix quick jitter, and tracks slower pans too well. Notice that my script doesn't use DePanStabilize though! You'll only know how well it works if you try it.

-G

[Chainmax]

The filterchain for a clip I'm processing is as follows:

Deblock_QED()

source=last
denoised=DeGrainMedian().FFT3DFilter(sigma=6,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
backward_vec2 = MVAnalyse(denoised,isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = MVAnalyse(denoised,isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = MVAnalyse(denoised,isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = MVAnalyse(denoised,isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
MVDegrain2(source,backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=300,idx=2)

FFT3DFilter(sigma=3,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)

LimitedSharpenFaster(SMode=4)

nnediresize2x(true,true,true)

Spline36Resize(320,240)

dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,Strength=200)
Soothe(sharp,dull,25)

Tweak(sat=1.4)

gradfun2db(thr=2.4)

AddGrainC(5,2)

Levels(0,1,255,16,235)


I was intending to make a lossless intermediary file with the filterchain up to the Tweak line, deshake that, then apply gradfun2db, addgrainc and levels to the stabilized video. Would that be ok or do you suggest me a different point in the filterchain to make the intermediary file at?

[g-force]

The filterchain for a clip I'm processing is as follows:
I was intending to make a lossless intermediary file with the filterchain up to the Tweak line, deshake that, then apply gradfun2db, addgrainc and levels to the stabilized video. Would that be ok or do you suggest me a different point in the filterchain to make the intermediary file at?

If you deshake first, the filters in your "denoised" will be more effective, and you may be able to just use the FFT3d that you are applying after the mo-comp stage instead. Here's my suggestion:

orig=Deblock_QED()

temp=orig.TemporalSoften(7,255,255,25,2)
rep=Repair(temp,orig,mode=16).TemporalSoften(1,255,255,25,2)
source=Interleave(rep,orig,rep)
mdata=DePanEstimate(source, range=1, trust=0, dxmax=1, dymax=1)
DePan(source, data=mdata, offset=1)
SelectEvery(3,2)

#you will need to crop an appropriate amount here and maybe resize to make MOD16

source=last.FFT3DFilter(sigma=3,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16) #just use your last fft3d here instead
backward_vec2 = MVAnalyse(SOURCE,isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = MVAnalyse(SOURCE,isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = MVAnalyse(SOURCE,isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = MVAnalyse(SOURCE,isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
MVDegrain2(source,backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=300,idx=1) #now all idx's can be the same

LimitedSharpenFaster(SMode=4)

nnediresize2x(true,true,true)

Spline36Resize(320,240)

dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,Strength=200)
Soothe(sharp,dull,25)

Tweak(sat=1.4)

gradfun2db(thr=2.4)

AddGrainC(5,2)

Levels(0,1,255,16,235)

[videoFred]

The 'trick' with Depan is: use a different (cropped)clip for DepanEstimate. What to crop? It depends on the scene.. Focus on something that should not move... A house or something. I use tweak(bright=-100,cont=2.0) on this clip too.. Somehow this helps, too.

Also, I nearly always set cutoff to 0.5 and trust between 1 and 1.5.

The same values for dxmax and dymax both in DepanEstimate() and in DepanStablize helps... (30 is a good value)

Sometimes I get better stabilizing with Depan then with Deshaker.. It realy depends on the source.

Fred.

[g-force]

The 'trick' with Depan is:
Fred.

Actually, you mean DePanStabilize. ;) The trick with DePan (not DePanStabilize) used as a stabilizer is to get a steady video by another means such as TemporalSoften, and then get the motion vectors between that and your jerky source. The advantage is that you don't have to worry about trust values, cutoff frequency etc. which need to change from scene to scene depending on if the pan is desired or not.

-G

[videoFred]

Actually, you mean DePanStabilize. ;) The trick with DePan (not DePanStabilize) used as a stabilizer is to get a steady video by another means such as TemporalSoften, and then get the motion vectors between that and your jerky source.

Ah! I see..... I will give that a try.. :)

Fred.

[videoFred]

@ G-Force:

I have tried your script and it works remarkable well for small and high frequence jitter. But not if we need to move the frame more.. Then the system I mentioned above here works better.

Fred.

[g-force]

@ G-Force:

I have tried your script and it works remarkable well for small and high frequence jitter. But not if we need to move the frame more.. Then the system I mentioned above here works better.

Fred.

Fred, you described it perfectly (small, high frequency jitter)! And yeah, it won't even touch larger pans (could be good or bad depending on what you need to do). I'm glad it might have some use for others!

-G

[Chainmax]

The clip is mostly static, but there are two high speed pans in it. I'll try your method and report back, thanks for the suggestion, g-force :).



[edit]Doesn't work, it creates an ugly sort of artifacting in a single frame, looks sort-of like a grid. I'll give DeShaker a shot now, but will try your suggestion (deshaking a deblocked version, then filtering that, right?).

[CruNcher]

Would you say this is more efficient then Depan or Deshaker?
http://www.avid.com/video/Composer/index.asp?currVideo=5

or this?
http://www.infotoday.com/eventdv/mercalli/mercalli.html

how about Autodesks Stabilizer?
http://download.autodesk.com/media/MNE/qt/c4_general_1_demo_380k.mov

[Malcolm]

German magazine c't compared 4 video stabilizers in it's issue 05/2008. They tested the following products:
- Mercalli Expert
- Premiere Elements 4
- Studio Version 11
- Gunnar Thalins Deshaker v2.2

Here's the table with results:
http://img168.imageshack.us/img168/5490/deshaking01smallmi5.th.jpg (http://img168.imageshack.us/my.php?image=deshaking01smallmi5.jpg)

The table contains ratings for the following features:
- stabilizing camera turns
- stabilizing continuous shaking/jitter
- stabilizing closeup scenes
- stabilizing single shakes
- stabilizing with default settings

[mikeytown2]

@malcom, nice find. If I'm reading the table correctly, If you want the best results, use Gunnar Thalins Deshaker. If you want a faster close second, use Mercalli Expert.

Cinelerra also does image stabilization. Importing clips into Cinelerra is it's main drawback though.
http://forum.doom9.org/showthread.php?p=1108307#post1108307

I've been using Deshaker for a long time now... IMHO it's very hard to beat, it's just SLOW. I've been wondering if i can use FQSharp for those frames that get blurred http://avisynth.org/vcmohan/FQSharp/FQSharp.html
http://forum.doom9.org/showthread.php?p=1107097

[g-force]

[edit]Doesn't work, it creates an ugly sort of artifacting in a single frame, looks sort-of like a grid. I'll give DeShaker a shot now, but will try your suggestion (deshaking a deblocked version, then filtering that, right?).

Wanna post the clip so I can take a shot at it?

-G

[Chainmax]

The video is a .3GP I haven't figured out the framerate of yet, so anything i managed to extract yet either plays too slow of too fast.

[Chainmax]

15fps seemed to be the one that yielded the more natural seeming motion except for one spot where things moved too fast. Therefore, I losslessly encoded a small sample. The chain is only loading through DirectShowSource using the fps=15,convertFPS=true switches. You can download it [link removed].

[scharfis_brain]

would you please also upload the original §GP file (including the two fast pans)?

[Chainmax]

I don't think the people in it will mind, this will end up online after all. [link removed] it is.

[Chainmax]

So, what do you think? Can stabilization be done satisfactorily?

[scharfis_brain]

that's what I got after three steps:
http://rapidshare.com/files/102933724/MOV00003-xvidq2.avi.html

1st: luminance correction, deblocking, and interpolation to 704x576 using 4 times nnedi()
2nd: deshaking
3rd: heavy denoising, resize to 640x480, and compression to xvid

[Chainmax]

Thanks :).

The mirrored borders are very noticeable, so I think I'll skip deshaking for this particular case. I like the output, could you please include the script you used, indicating at which point deshaking was done?

[2Bdecided]

Have you tried fixmyvideo.com? It won't do all you want, but it might help.

Cheers,
David.

[scharfis_brain]

@Chainmax. I used several scripts.
Also you don't see mirrored borders. Instead it is the border compensation from prev/next frames.

Without stabilizer, you can use one script, for sure.
BUT you have to convert the mov to a AVI first, cause directshowsource chokes on this clip bad time for me.

loadplugin("c:\x\nnedi.dll")
loadplugin("c:\x\dgdecode.dll")
loadplugin("c:\x\mvtools.dll")
loadplugin("c:\x\ttempsmooth.dll")

blah_source("MOV00003.3gp")
deblock(quant=40)
levels(0,1,255,5,235,coring=false)
nnedi(dh=true, field=0).turnright().nnedi(dh=true, field=0).turnleft()
nnedi(dh=true, field=0).turnright().nnedi(dh=true, field=0).turnleft()
s=last
bv3 = s.MVAnalyse(isb = true, truemotion=true, delta = 3, idx = 10,blksize=16)
bv2 = s.MVAnalyse(isb = true, truemotion=true, delta = 2, idx = 10,blksize=16)
bv1 = s.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 10,blksize=16)
fv1 = s.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 10,blksize=16)
fv2 = s.MVAnalyse(isb = false, truemotion=true, delta = 2, idx = 10,blksize=16)
fv3 = s.MVAnalyse(isb = false, truemotion=true, delta = 3, idx = 10,blksize=16)

fc3 = s.MVFlow(fv3, idx=10, thSCD1=500)
fc2 = s.MVFlow(fv2, idx=10, thSCD1=500)
fc1 = s.MVFlow(fv1, idx=10, thSCD1=500)
bc1 = s.MVFlow(bv1, idx=10, thSCD1=500)
bc2 = s.MVFlow(bv2, idx=10, thSCD1=500)
bc3 = s.MVFlow(bv3, idx=10, thSCD1=500)

interleave(fc3, fc2, fc1, s, bc1, bc2, bc3)
ttempsmooth(maxr=3, lthresh=40, cthresh=70)
selectevery(7,3)
bicubicresize(640,480)

[g-force]

My avisynth routine can't even touch the amount of shake you have. I think you're best off with scharfis_brain's suggestion.

-G

[Chainmax]

scharfis-brain: thanks for the script. Could you psot a screenshot with your DeShaker settings as well? I got nowhere near the results you had in that regard. Also, how do you recommend me to convert to AVI if Avisynth can't be used?

[scharfis_brain]

I don't know anymore, which settings I used...
Also it may be, that I altered the deshaker.log manually by deleting false detected motion.

1)prepare a rgb (-compressed) (DO NOT convert to YUV!!!) file for deshaker with this script:

blah_source("MOV00003.3gp")
deblock(quant=40)
levels(0,1,255,5,235,coring=false)
nnedi(dh=true, field=0).turnright().nnedi(dh=true, field=0).turnleft()
nnedi(dh=true, field=0).turnright().nnedi(dh=true, field=0).turnleft()
converttorgb(matrix="PC.601")

2) apply deshaker to to the RGB-video and save as rgb-file again

3)denoise using this script:
avisource("deshaked-rgb.avi")
s=last.converttoyv12(matrix="PC.601")
bv3 = s.MVAnalyse(isb = true, truemotion=true, delta = 3, idx = 10,blksize=16)
bv2 = s.MVAnalyse(isb = true, truemotion=true, delta = 2, idx = 10,blksize=16)
bv1 = s.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 10,blksize=16)
fv1 = s.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 10,blksize=16)
fv2 = s.MVAnalyse(isb = false, truemotion=true, delta = 2, idx = 10,blksize=16)
fv3 = s.MVAnalyse(isb = false, truemotion=true, delta = 3, idx = 10,blksize=16)

fc3 = s.MVFlow(fv3, idx=10, thSCD1=500)
fc2 = s.MVFlow(fv2, idx=10, thSCD1=500)
fc1 = s.MVFlow(fv1, idx=10, thSCD1=500)
bc1 = s.MVFlow(bv1, idx=10, thSCD1=500)
bc2 = s.MVFlow(bv2, idx=10, thSCD1=500)
bc3 = s.MVFlow(bv3, idx=10, thSCD1=500)

interleave(fc3, fc2, fc1, s, bc1, bc2, bc3)
ttempsmooth(maxr=3, lthresh=40, cthresh=70)
selectevery(7,3)
bicubicresize(640,480)

[Chainmax]

So you are loading the .3GP directly in Avisynth. Doesn't that mean blah_source is actually DSS as it's the only one that is able to load it? If so, how do you avoid it choking on the .3GP?

[scharfis_brain]

I loaded it using DSS alone and saved it as avi.