Video stabilization (deshake)

[szabi]

Hi

I tried to stabilize my video, however I could not get what I expected.
Video source is 3840×2160@29.970, mp4.
First I try to use deshaker (v3.1) plugin in virtualdub.
Altered setting from default in 1st pass, scale:Full
Altered setting from default in 2nd pass, edge compensation:None
Because the final video going to be cropped any zoom is not needed.
I was not happy with outcome.

Second try to use DePanStabilize.
LoadPlugin("C:\My Program Files\MeGui\tools\lsmash\LSMASHSource.dll")
LSMASHVideoSource("D:\Camera\deshake-test\VID_20190606_123345.mp4")
i = ConvertToYV12()
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata)
crop(142, 80, -142, -80)
LanczosResize(1920,1080)

I was not happy with result so I modified the script based on what sample I did find on internet:
i = ConvertToYV12()
mdata = DePanEstimate(i, range=2, trust=0, dxmax=5, dymax=5, zoommax=1, log="F:\depanlog.log")
DePanStabilize(i, data=mdata, dxmax=30, dymax=15, zoommax=1, rotmax=0, inputlog="F:\depanlog.log")
crop(142, 80, -142, -80)
LanczosResize(1920,1080)
It did not improve anything, even the created logfile was empty.

Last I did try also came from google search.
i = ConvertToYV12()
vectors=MSuper().MAnalyse(isb=false,blksize=8, overlap=0,search=5)
globalmotion=MDepan(vectors,pixaspect=1,thSCD1=500)
DepanStabilize(data=globalmotion,cutoff=0.5,mirror=15,prev=0,next=0,blur=50,pixaspect=1,subpixel=2,method=1)
crop(142, 80, -142, -80)
LanczosResize(1920,1080)
Unfortunately this one has crashed whatever I did try.

I created a stack video about each try including one video which only cropped and resized:
Video stabilization (deshake) by Avisynth (https://vimeo.com/340744626)

The original video is available here: VID_20190606_123345 (https://drive.google.com/file/d/1MxGkShwqsHZNx-80ZkquqcH_-IZG-8Jh/view?usp=sharing)
I would like to ask help what should I do to get a correct result?

Regards
Szabi

[ChaosKing]

Need more info. Which avs version 32/64bit? Latest dll versions?
No problems here with 32bit avs+. But ran into memory issues with prefetch in avs 32bit. (with last script)

[wonkey_monkey]

Being a wide angle video and having what I think is some moderate-to-strong rolling shutter are probably what's stopping you get a better result. Have you tried enabling the Rolling Shutter setting on Deshaker?

It's either that or there's some other weird processing going on with that video, because there are times when it looks like the clouds are out of sync with the ground.

[johnmeyer]

I tried with Deshaker 3.0 but couldn't get it to work. I then stabilized it using an old version of Mercalli that is built into Vegas. I didn't play around with the settings much, so I assume that some of the funkiness in the clouds and around the edges of the video could be improved. Also, my Mercalli is almost a decade old and their newer versions are far better. It is expensive, but is probably the best stabilizer out there. You can download the trial and see if it is worth the (considerable) expense to purchase.

Here's a link to the result (which I down res'd to HD just to make it easier to deal with).

Stabilized Video (https://www.mediafire.com/file/85f8kj5a8ehcspk/Temp_Stabilized_Result_%281920x1080%29.mp4/file)

[poisondeathray]

As John mentioned by John- newer versions of mercalli are improved. Most notably - they have rolling shutter correction modes.

Here is the result using the default setting for rolling shutter - vibrations/wobble (also downscaled to 1080p). You can play with the settings and probably get slightly better results. Or adjust more smoothness (but more zoom)

Free , open source software is still lacking in terms of RS correction . Deshaker has partial RS correction, but not generally not as good in those scenarios as mercalli or warp stabilizer
https://www.mediafire.com/file/o3ickg7bqqdzbck/mercalli_rs_vibrations_wobble.mp4/file

[pinterf]

in the non-working example clip "i" is not used

[szabi]

Need more info. Which avs version 32/64bit? Latest dll versions?
No problems here with 32bit avs+. But ran into memory issues with prefetch in avs 32bit. (with last script)

First I tried official Avisynth (https://sourceforge.net/projects/avisynth2/). This is the version I use since came out without problem until now.
In this case it did not worked neighter with ffms nor with lsmash (index was done my megui).
Installed AviSynth+ (https://github.com/pinterf/AviSynthPlus/releases). Now ffms still crash however lsmash worked expect the last one.
Presently 9 different Avisynth (https://forum.doom9.org/showthread.php?t=172124) exist? Really?
For me it is really confusing and it even worse by complicating x86 or 64bit.

Being a wide angle video and having what I think is some moderate-to-strong rolling shutter are probably what's stopping you get a better result. Have you tried enabling the Rolling Shutter setting on Deshaker?

It's either that or there's some other weird processing going on with that video, because there are times when it looks like the clouds are out of sync with the ground.
There is no other process on original video it is as camera recorded.
I did try rolling shutter set on deshaker, it gives a slightly better result.

.....my Mercalli is almost a decade old and their newer versions are far better. It is expensive, but is probably the best stabilizer out there.....

Checking the prices of mentioned software seems more recommended to buy a gimbal.

Here is the result using the default setting for rolling shutter - vibrations/wobble (also downscaled to 1080p)....

Free , open source software is still lacking in terms of RS correction . Deshaker has partial RS correction, but not generally not as good in those scenarios as mercalli or warp stabilizer
https://www.mediafire.com/file/o3ickg7bqqdzbck/mercalli_rs_vibrations_wobble.mp4/file
This is the best result until now.
I stacked it to the new comparison video:
Video stabilization (deshake by VirtualDub) and Mercalli (https://vimeo.com/340847605)

As conclusion there is no free software wich could provide better result what I achieved by Deshaker and DePanStabilize until now and there is no better settings for DePanStabilize neighter.
Thanks for your help.

Regards
Szabi

[szabi]

in the non-working example clip "i" is not used

Modifiy it to the following also has crash result:
ConvertToYV12()
vectors=MSuper().MAnalyse(isb=false,blksize=8, overlap=0,search=5)
globalmotion=MDepan(vectors,pixaspect=1,thSCD1=500)
DepanStabilize(data=globalmotion,cutoff=0.5,mirror=15,prev=0,next=0,blur=50,pixaspect=1,subpixel=2,method=1)
crop(142, 80, -142, -80)
LanczosResize(1920,1080)
The problem seems the various avisynth with complicating 32 or 64bit plugins.
It would be nice to try 64bit ffms or lsmash indexing but there is no tool to do so.

[ChaosKing]

The problem seems the various avisynth with complicating 32 or 64bit plugins.
It would be nice to try 64bit ffms or lsmash indexing but there is no tool to do so.
64bit ffms2 https://forum.doom9.org/showthread.php?p=1870563#post1870563

[StainlessS]

From Here:- https://forum.doom9.org/showthread.php?p=1876357#post1876357


For anyone else wanting to try the multi-source loader script function, here is a 7zip compressed file (~31MB: AVI_LSMASH_FFMS2_SOURCE.7z):-
http://www.mediafire.com/file/2kuq0flzz145ct8/AVI_LSMASH_FFMS2_SOURCE.7z.7z/file
Above 7zip file using Ultra Compression, so may need latest version 7Zip if yours is old (saved an extra 10MB).

Contains both x86 and x64
FFMS2 (XP compatible [C Filter NOT CPP, but autoloads OK in AVS+ without LoadCPlugin call])
LSmash(NOT XP Compatible, CPP filter)
Multi-Source Script function [ GetSeq() ], tries AVI, both LSmash filters and FFMS. (and dubs audio to video)
Also contains RT_Stats latest, v2.00 Beta 12(used by the multi-source loader script).



EDIT: The above zip also has a 15MB MP4 demo clip (heima_720p_500.mp4, demo'ing x264 from some years ago, 720x500p 4 mins).
Guess I forgot to remove from directory before zipping up.
Zip size 31MB, unzip about 113MB(ffms dll's are quite big.
The included script will try AVIsource if AVI file, then LSMash ISO source if ISO file, then FFMS if still not opened ok, then lastly LSMash non ISO source filter.
Requires RT_Stats which is included in zip.

EDIT: Think all source dlls are 10 bit capable.

EDIT: Here script. Can extract function only to eg GetSeq.avsi in plugins and then just use GetSeq(FileName) to load clip.
As it stands, does not use any fancy LSmash or Ffms2 args.

GetSeq.avsi

Function GetSeq(String vFn,String "aFn") {
Function IsISOFileName(String s) {s=RT_GetFileExtension(s) Return(s==".mov"||s==".mp4"||s==".m4v"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}
Function IsRiffFileName(String s) {s=RT_GetFileExtension(s) Return(s==".avi"||s==".wav")}
myName="GetSeq: " aFn = Default(aFn,"") aFn = (aFn=="") ? vFn : aFn
Assert(vFn!="",RT_String("%sVFn Cannot be ''",myName)) vFn = vFn.RT_GetFullPathName IsRiffV = vFn.IsRiffFileName IsIsoV = vFn.IsISOFileName
Assert(aFn!="",RT_String("%saFn Cannot be ''",myName)) aFn = aFn.RT_GetFullPathName IsRiffA = aFn.IsRiffFileName IsIsoA = aFn.IsISOFileName
c=0 a=0
Try { # 1st Try specialized AVI
c = (IsRiffV) ? vFn.AviSource : NOP
(c.IsClip) ? RT_DebugF("AviSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
a = (c.IsClip && c.HasAudio && aFn==vFn) ? c : (IsRiffA) ? aFn.WavSource : NOP
(a.IsClip && a.HasAudio) ? RT_DebugF("%s opened Audio\n '%s'",(c.IsClip && c.HasAudio && aFn==vFn)?"AviSource":"WavSource",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try specialized ISO LSMash
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (IsIsoV) ? LSMASHVideoSource(vFn) : c
(IsIsoV) ? RT_DebugF("LSMASHVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (IsIsoA) ? LSMASHAudioSource(aFn) : a
(IsIsoA) ? RT_DebugF("LSMASHAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try FFMS2
Try {
IsOpenV=(c.IsClip && c.HasVideo)
(!IsOpenV) ? FFIndex(vFn) : NOP
c=(!IsOpenV) ? FFVideoSource(vFn) : c
(!IsOpenV) ? RT_DebugF("FFVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a=(!IsOpenA) ? FFAudioSource(aFn) : a
(!IsOpenA) ? RT_DebugF("FFAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

# Next try LSmash non ISO Video
Try {
IsOpenV=(c.IsClip && c.HasVideo)
c = (!IsOpenV && !IsIsoV) ? LWLibavVideoSource(vFn) : c
(!IsOpenV && !IsIsoV) ? RT_DebugF("LWLibavVideoSource opened Video\n '%s'",vFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }
Try {
IsOpenA=(a.IsClip && a.HasAudio)
a = (!IsOpenA && !IsIsoA) ? LWLibavAudioSource(aFn) : a
(!IsOpenA && !IsIsoA) ? RT_DebugF("LWLibavAudioSource opened Audio\n '%s'",aFn,name=myName) : NOP
} catch (msg) { RT_DebugF("Catch: %s",msg,name=myName) }

Assert(c.IsClip, RT_String("%s failed open on\n '%s'",myName,vFn))
(!c.HasAudio && a.IsClip && a.HasAudio) ? AudioDubEx(c,a) : c
(!HasAudio) ? RT_DebugF("Audio failed Open on\n '%s",aFn,name=myName) : NOP
return Last
}


Client

FN=".\heima_720p_500.mp4"
#FN="10Bit_Standard8_Scan_16FPS.avi"

GetSeq(FN)
Info
Return last


Viewing debug output via debugview(Google) will show which source filter was used to open clip.

[szabi]

.... You can download the trial and see if it is worth the (considerable) expense to purchase.

After registration I check the trial version.
When analyzing the 2nd step (Correct CMOS distortions) always crash.
Failure message : "Analyzing or creating the shadow-video has failed"
Nothing comes easy.

EDIT: Exporting the video by Lagarith Lossless Codec with resizing to 1920x1080 helped to feed it with Mercalli.
Seems 4k/mp4 video not welcomed or my PC has not enough power.

[johnmeyer]

EDIT: Exporting the video by Lagarith Lossless Codec with resizing to 1920x1080 helped to feed it with Mercalli.
Seems 4k/mp4 video not welcomed or my PC has not enough power.The 4K size does present some problems.

One trick you could use would be to create a temporary video that is identical to the original, but down-res'd to 1920x1080, or even lower. Run that through the Deshaker first pass. Then, open the original file, load the LOG file created in the first pass, and see if Deshaker will process the original file using the motion vectors created on the lower-res version in the first pass.

[szabi]

Hi

I cropped the video by virtualdub to 3556x2000 and saved to avi through Lagarith Lossless Codec and did cut 1 minutes segments.
After the Mercalli did eat them.
It was so long time ago when I could not use script rather than working lossless. :)

Thanks for help.
Regards
szabi