Deshaker Comparison

[MysteryX]

I've been doing some comparison between Deshaker (http://www.guthspot.se/video/deshaker.htm) and Mercalli ProDad v4 (http://www.prodad.com/home/products/video_optimization/mercalli_v4,l-us.html). I wanted to share my results.

First, Deshaker is a VirtualDub plugin and is hard to use. I'm trying to program it within my own software to automate the process but am encountering a few challenges.

Mercalli ProDad is a commercial software. It used to integrate within Adobe Premiere but v3 and v4 are stand-alone software.

So here are the results, for those interested.

Deshaker
Pros:
- Cleaner image
- Better efforts at reconstructing borders and not zooming unecessarily
Cons:
- Difficult to configure and use
- Slow
- Some wobble or jello effects

Mercalli ProDad
Pros:
- Much easier to use
- Much faster
- Wobble and jello effects are still there but weaker
Cons:
- Image is blurrier
- It zooms a lot more
- Requires a lossy re-encoding of the file -- not ideal in a video processing chain

I am aware that some of the settings can be tweaked in both software to improve the results. Here I have gone mostly with default settings with just minor customization.

I would have expected Mercalli ProDad to do a much better job because it analyzes the movements in 3D and is a lot more sophisticated.

Overall, Deshaker wins. It will be worth spending more time to program its easy integration within an AviSynth script.

However, whenever I use "fill borders with previous or future frames", it fails to render the video. First the video processes, then the output jams on a frame, and after being jammed on the same frame for a while, it aborts the encoding. Will have to investigate.

There was a bug in AviSynth+ where it crashed in the destructor and I fixed it. You need this updated VDubFilter.dll. (https://github.com/pinterf/AviSynthPlus/pull/4/commits/78efa4e1dfbee153b05b669f9cacacb7b7661f37)

I'll have to keep testing whether the issues I'm seeing are related to AviSynth+ or I'm doing something wrong.

[shekh]

Interesting subject. When it comes to testing codecs everyone have test patterns and psnr graphs, but all I can find for deshaking is "looks good".

What kind of shakiness are you trying to repair?
Supposedly strong part of Mercalli4 is 3d and cmos repair, but it is very slow and actually when I run single test it did not repair anything and created artifacts.

[MysteryX]

You can't really test a deshaker with test patterns and graphs; unless you take a video of a sheet of paper having those patterns.

I have a 1080p video testimonial recorded from a compact camera. The video is quite noisy and slightly shaky (hand-held); with a part where I'm moving the camera around so it's then very shaky. For the part that is fixed, I find it works best with Deshaker when I only detect rotation and disable "detect zoom" and "detect scene change", but then when I start moving around it causes the image to skip quite a lot so "detect zoom" must be enabled.

I just gave Mercalli another try. This time I disabled Smooth Camera Path and reduced "avoid borders" from 40 to 22, and reduced "pan shot smoothing" from 40 to 29. Most importantly, I used the preset "Vibrations/Wobble". Compared to Deshaker, the first pass took quite a bit longer, while the second pass was still fast.

Result? The over-zoom was gone. I reduced the settings too much and it is still a bit shaky, but even there, Deshaker's video that is zoomed in more has a clearer picture.
- Mercalli loses details
- Most of the wobbles are now gone; but there are a few left that are even bigger (or they just stand out more?)
- *there is a slight yellow decoloration*

The workflow that is working best so far is
1. Convert video from PC colors to TV colors and run Deshaker's first pass from the source
2. Run SMDegrain(prefilter=4) on the source, then convert from PC to TV colors, then run Deshaker's second pass.

OR

1. Run SMDegrain(prefilter=4), convert from PC colors to TV colors, and save to AVI
2. Import into Mercalli, re-encode as H264 with highest quality.

I tried importing an AVS file into Mercalli. That didn't work.

Based on my source video, my goal is to have a video that fixes the slight shaking of the first part without degrading the quality, and that handles the camera movements of the second part nicely without too much wobbles. I think that's a good test video.

[MysteryX]

For Deshaker, I had an idea. To deal with Wobbles, you can go through the clip (without SMDegrain it renders quite fast), see what specific frames have wobbles, and delete those frames from the Log file. In my software I might add an option to enter the list of frames to discard. But then, if the video is zoomed, will it "break the zoom" and completely break the video sequence?

Now I'm testing this script. Second argument is "Pass". You must first run with 2nd argument set to 1 which generates a log file, then run again with it set to 2. Only run SMDegrain on the 2nd pass.

Here I'm setting "Fill borders with previous and future frames" and it's working. It's just that when playing in MPC-HC, the first 30 frames are black, but I'm not getting the crash.


LoadVirtualDubPlugin("deshaker.vdf", "deshaker", preroll=0)
Trim(0, 500)
SMDegrain(prefilter=4, tv_range=false)
ConvertToRGB32(matrix="PC709")
deshaker("19|2|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|1|3|2|8|30|300|4|F:\Job_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|1|1|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")


Oh... I think VirtualDub informed me of the crash I was getting in my previous encodings: "Out of memory"

[johnmeyer]

As you may know, I wrote an extensive guide to Deshaker, over ten years ago. I then wrote a series of Vegas and Virtualdub scripts which completely automate the stabilization process so you can batch stabilize individual scripts with Deshaker. Batch stabilization is something you still cannot easily do with Mercalli.

Having established my "creds," let me say that in most cases, Mercalli produces vastly better stabilization. There are exceptions, and I still use Deshaker on some footage, but not often.

The one thing that puzzles me is your report that Mercalli is creating footage that is less sharp. That has not been my experience. It makes me think that there is something wrong in your Mercalli workflow. I do not have the latest version, and I use the plugin for Vegas, so I may not be able to duplicate what you are doing.

The usual reasons for blurred footage is zooming. Even the smallest amount of zoom can end up softening detail. It sounds like you are using the Deshaker modes that use border fixing and don't zoom. This preserves detail and sharpness. It is important to make sure that every last vestige of zooming is turned off in Mercalli or you will indeed get a softer image. Since Mercalli's border fixing didn't used to be as good as Deshaker, it was often undesireable to completely eliminate zooming because the edges ended up looking too weird. I have heard that V4 has better edge correction. I have been meaning to purchase the SAL version, but haven't gotten around to it yet.

So, make sure that you really do have all zooming turned off in Mercalli.

[MysteryX]

For Deshaker, I can improve on memory limitations with something like this; but it does bring quite a few limitations that will be a hassle to program around for automatic script generation. But this script works. However, I can't use MT and CPU usage is only 50-60%.

P="Encoder\"
AddAutoloadDir(P, true)
MP_Pipeline("""
P="Encoder\"
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"MVTools2.dll")
Import(P+"SMDegrain.avsi")
LoadPlugin(P+"KNLMeansCL.dll")
LoadPlugin(P+"Dither.dll")
Import(P+"Dither.avsi")
file="F:\NaturalGrounding\Convert\TestimonialAndreas.MOV"
LWLibavVideoSource(file, cache=True)
Trim(0, 500)
SMDegrain(prefilter=4, tv_range=false, lsb=true)
### prefetch: 5, 5
### ###
""")
LoadVirtualDubPlugin(P+"deshaker.vdf", "deshaker", preroll=0)
ConvertToRGB32(matrix="PC709")
deshaker("19|2|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|1|3|2|8|30|300|4|F:\Job_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|1|1|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")


Here are some comparison images on the first frame of the video.

Original
https://s32.postimg.org/ffkuofiu9/Deshake_Original.png (https://postimg.org/image/ffkuofiu9/)

AVI after SMDegrain (which served as the source for Mercalli)
https://s31.postimg.org/nbibceyvr/Deshake_Degrain.png (https://postimg.org/image/nbibceyvr/)

Mercalli - no zoom, "Vibrations/Wobble" mode
https://s32.postimg.org/qdw46m7fl/Deshake_Mercalli.png (https://postimg.org/image/qdw46m7fl/)

Deshaker
https://s32.postimg.org/5fpy8j7kx/Deshake_Deshaker.png (https://postimg.org/image/5fpy8j7kx/)

In the Mercalli screenshot, the wood loses the sharpness of its grain and there is a yellow decoloration. I have run SMDegrain(prefetch=4) on both. In the case of Mercalli, the first script was exported as AVI.

[poisondeathray]

RE: color discoloration

1) What kind of "AVI" did you export for import into Mercalli ?
2) What export settings are you using ?
3) How are you taking screenshots ?

YV12 Colorbars in UT Video 709 imported and exported come out the same in terms of color. So I'm thinking there are some issues with your workflow for mercalli

[MysteryX]

Source video is PC709. I degrain, convert color range PC->TV, and export as Rec709 UT Video AVI using FFMPEG.

From Mercalli, I export with H264(mp4) High Quality.

To take screenshots, I open in MPC-HC and "Save Image as PNG"

I rendered 2 videos with Mercalli. The first one with Standard preset doesn't have the same decoloration... or perhaps just very lightly. The second one with "Vibrations/Wobble" preset has much stronger decoloration (or coloration?)

[poisondeathray]

Mercalli uses something close to Rec601 when converting RGB<=>YUV , similar to vdub deshaker, when you don't control the conversion. But there is still a slight discoloration when corrected for , so it's not exactly Rec601. You can use prores if you want higher quality . It's unfortunate that they don't allow you to use lossless codecs

[ajp_anton]

What about MDepan with DepanStabilize? Has served me well so far.

[MysteryX]

Mercalli uses something close to Rec601 when converting RGB<=>YUV , similar to vdub deshaker, when you don't control the conversion. But there is still a slight discoloration when corrected for , so it's not exactly Rec601. You can use prores if you want higher quality . It's unfortunate that they don't allow you to use lossless codecs
It would be *great* if Mercalli could open up AVS script and export in a lossless format. Besides that, it does have a few drawbacks; one of which is being a considerably expensive program.

But then, Deshaker also definitely has its drawbacks. Is there a way to fix Mercalli's decoloration and loss of quality? The decoloration can easily be fixed afterwards; but the loss of quality I guess it's just the way Mercalli works.

What about MDepan with DepanStabilize? Has served me well so far.
Had not heard of DePan.dll (http://avisynth.org.ru/depan/depan.html). Does anyone have experience with this and how it compares with Deshaker?

Features and limitations

1. Works only in YV12 and YUY2 color formats.
2. Only pan and zoom motion (no rotation) are estimated, but it gives advance in speed and stability. Zoom estimation is not very precise.
3. The source clip must be same length as motion data clip.
4. Directly works only with progressive clips. For interlaced sources, you must use AviSynth following function SeparateFields and followed Weave
(after motion compensation and denoising), with AssumeTTF and AssumeBFF (both may be needed for odd fields offset). Plugin estimates and calculates motion from one field to neighbor (by time) field (from same or neighbor frame). For preserving fields order (dominance) and best denoising, set parameter MatchFields=true.
5. Mirror mode is unique but slightly strange :-). The blur is some workaround to hide sharp mirrored details. Note: blur is not implemented for rotation as yet.
6. Not very fast, not assembler optimized.
7. Tested with Avisynth 2.5.3, 2.5.4, 2.5.5, 2.5.6, 2.5.7, 2.5.8, 2.6.0.
8. Old versions DePanEstimate used free FFT2D code by Takuya Ooura (http://momonga.t.u-tokyo.ac.jp/~ooura/index.html)
Now DePanEstimate uses only more fast FFTW library version 3 (http://www.fftw.org)
as Windows binary DLL (compiled with gcc under MinGW by Alessio Massaro),
which support for threads and have AMD K7 (3dNow!) support in addition to SSE/SSE2.
It may be downloaded from ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip
For using, you must put FFTW3.DLL file from that package to some directory in path (for example, C:\WINDOWS\SYSTEM32). DePanEstimate will NOT work without it!
9. For best results, you may temporary add Info parameter, analyze info and tune some parameters (Trust, dxmax etc).
10. You may use not strictly same clips for motion estimation and compensation, for example try add some brightness-contrast adjusting, pre-filtering, masking, cropping to input clip used for motion estimation only (and use different processing for output compensated-stabilized results).

#2 may be a deal-breaker. The first part of the footage for the most part only has slight rotation; no zooming and only minor panning.

[Reel.Deel]

There's also Deshaker3D (http://forum.doom9.org/showthread.php?t=171051).

[MysteryX]

One of the drawbacks of Deshaker is the need to manually configure the Rolling Shutter value to "God Knows What"; or perhaps this gives a quality advantage over Mercalli doing it automatically?

It appears there has been several other attempts at creating better deshakers: Deshaker3D (http://forum.doom9.org/showthread.php?t=171051) and Lens Transform Tool (http://forum.doom9.org/showthread.php?p=1747268#post1747268). Neither of which support Rolling Shutter Correction. Both of which also use Deshaker's first pass. To process the video in 3D, wouldn't it have to better analyze the movement axis in the first pass instead of only changing the 2nd pass? Deshaker3D also misses many of the features of Deshaker. Has anyone seen good improvements with it still? It seems these are mostly "work in progress" that saw no further development. Mercalli ProDad definitely wins on this aspect.

[MysteryX]

On the same vein of comparing commercial products with free products, here's a comparison of Adobe Premiere CC 2015.1's new Optical Flow with SVP.

SVP
https://s31.postimg.org/4musvzzqf/Motion_SVP.png (https://postimg.org/image/4musvzzqf/)

Adobe Premiere Optical Flow
https://s31.postimg.org/sp71d4qzr/Motion_Optical_Flow.png (https://postimg.org/image/sp71d4qzr/)

The clip isn't exactly the same; the SVP clip is zoomed in a little bit more, but it's good enough for a comparison. Optical Flow creates UGLY artifacts.

SVP team wins!

I'll steal the words of McAfee when he ridiculed the FBI computer engineers: "Some people are born geniuses and these people can't be 'made'. Get a few of my people together and they'll do what hundreds of well-trained computer engineers can't ever do." (he didn't use these exact words but that was the general idea)

I'm all for developing products in a sustainable way and creating businesses, and this gives me faith in the Open Source path.

[MysteryX]

When encoding with Deshaker and "Fill borders with previous and future frames", I get this error

Could not allocate video frame. Out of memory. memory_max = 1073741824, memory_used = 900757717 Request=6220831

Weird thing is that the memory usage is only 1.4GB. Is there another work-around than going the MP_Pipeline route?

In defense of Mercalli, the AviSynth Virtual File System (http://forum.doom9.org/showthread.php?t=133313) may allow piping directly from AviSynth into Mercalli without first re-encoding.

[poisondeathray]

For the memory issue, is that with avs script and filters, or did you "bake" it to an intermediate? Splitting it out would reduce memory issues

For mercalli (and all stablizers), they can all soften to picture to an extent, even without zoom. x,y subpixel translation or rotations can also soften the picture, dewarping /cmos correction (if the program has it) can also soften the picture. Some of what you're seeing is going to be compression related issues if you're using h264. It's too bad that version doesn't allow lossless export.

You can indirectly use avs with anything, using avfs . I've never come across a Windows program that won't accept the "fake" AVI. But some receiving programs require RGB24 or 32 (ie. add ConvertToRGB24 or 32 to the end) , but many will accept as is

Other popular options for generic stabilization : ffmpeg has libvidstab, and there is also warp stabilizer in Adobe .

There are also other categories of stabilizers, but they are meant for tracking and compositing VFX work, not generic use hand held shots. They usually require more manual input

[MysteryX]

I get best results by running in that order
1. SMDegrain
2. Deshake
3. InterFrame (SVP)

Perhaps MP_Pipeline is the best option then... with the hassles that will go with it. Or without it, it works fine without "use previous and future frames" on 1080p content.

[shekh]

Lens Transform Tool is work in very slow progress (too difficult to do part time) and not in generally usable state now.
I worked on improving analysis pass. Deshaker and libvidstab search for 2d motion and therefore they find no solution with any amount of perspective. What I was doing is to instead solve for 3d rotation, and it kindof works.

[pinterf]

I will soon release a DepanEstimate/DepanStabilize port.
64 bit
YV16 and YV24 support
Native 16 bit (yuv420p16.. yuv444p16) when used with the next (not yet released) avisynth plus.

[MysteryX]

I will soon release a DepanEstimate/DepanStabilize port.
64 bit
YV16 and YV24 support
Native 16 bit (yuv420p16.. yuv444p16) when used with the next (not yet released) avisynth plus.


Only pan and zoom motion (no rotation) are estimated

Will it be the same or will yours detect rotation?

[pinterf]

Use mvtools depan instead.

[Reel.Deel]

Other popular options for generic stabilization : ffmpeg has libvidstab,...

libvidstab IMO is a bit easier to use than Deshaker/Depan/MDepan, maybe one day someone will port it over to AviSynth.


I will soon release a DepanEstimate/DepanStabilize port.
64 bit
YV16 and YV24 support
Native 16 bit (yuv420p16.. yuv444p16) when used with the next (not yet released) avisynth plus.

Sounds great. :)

[MysteryX]

Other popular options for generic stabilization : ffmpeg has libvidstab
vid.stab should be rather easy to integrate into AviSynth. The full source code is easily available here. (http://public.hronopik.de/vid.stab/) Question is: how good of a job does it do? Their sample video is already blurry so it's hard to see whether it blurs even more, and by reading the description, I got the feeling that it heavily damages the video with blurs. It needs to be tested.

Neither DepanStabilize, nor MvTools Depan have rolling shutter compensation, and that's a big drawback. How much of a drawback is this? This also would require testing.

It still seems Deshaker is the most mature option. As for the MP_Pipelines side-effects, it might not be that bad. Side-effects are:
- Cannot use AddAutoloadDir -> use LoadPlugin and Import for each file
- No audio support -> merge the audio at the end and perform the same trimming to put audio/video in sync

Then I move the entire script within MP_Pipeline and add ### ### above and below the Deshaker line, and at the end of the script. That wouldn't be too bad to implement and the script will then behave the same way within my processing chain (especially regarding the audio).

[Reel.Deel]

vid.stab should be rather easy to integrate into AviSynth. The full source code is easily available here. (http://public.hronopik.de/vid.stab/) Question is: how good of a job does it do?

Here's another video (in this sample the output seems to have some wobble): https://vimeo.com/134488187

In this sample the output is much better: https://www.youtube.com/watch?v=kYc7TU-5U00

[MysteryX]

Here's another video (in this sample the output seems to have some wobble): https://vimeo.com/134488187

In this sample the output is much better: https://www.youtube.com/watch?v=kYc7TU-5U00
Wow, first video has horrible warping effects!! Deshaker won't handle that. I don't think any of the AviSynth methods will do anything good with such heavy fish eye effect. There is heavy jello effect but I'm impressed it's able to do something decent on that one! Or... is it a regular video than then got distorted? I'd like to see the original video whether it has fish lens effect or not.

Mobius Lens C has heavy fish eye effect (https://vimeo.com/117524310)

Second video actually looks very good. The image keeps the clarify of its details.

[poisondeathray]

maybe some source videos should be posted, so everyone is on the same page and various settings / methods can be tested

rolling shutter can be very difficult to treat. IMO, mercalli is the best "generic" stabilizer for that. There are other commerical filters / methods that deal with it specifically, but not perfectly either

[poisondeathray]

There are many comparisons and reviews online of different methods, including commercial ones, but they are usually split or quad screen and heavily compressed by YT or Vimeo etc.... so examining characteristics like detail loss /blur is more difficult

Some linux only ones too in kdenlive, I'm not familiar with those - anybody try them out ?
https://www.youtube.com/watch?v=MfHGH2Mnxls

[MysteryX]

Good idea to use a common video; if someone has a good one to share.

I'm testing VidStab. Performance is good with default settings.

Pass 1: the analysis file for a 4m22 1080p video is 48MB, which indicates it collects a LOT more data to work with than Deshaker

ffmpeg -i "Source.avi" -vf vidstabdetect -f null -


Pass 2:
Smoothness=20.

ffmpeg -i "Source.avi" -vf vidstabtransform=smoothing=20:input="transforms.trf" -c:v libx265 -preset medium -crf 16 "Output.mp4"

https://s31.postimg.org/kbsmaj9fr/Deshake_Vid_Stab.png (https://postimg.org/image/kbsmaj9fr/)
Decent preservation of details despite considerable zoom-in.


Smoothness=20, Bicubic.

ffmpeg -i "Source.avi" -vf vidstabtransform=smoothing=20:input="transforms.trf":interpol=bicubic -c:v libx265 -preset medium -crf 16 "Output.mp4"

https://s32.postimg.org/szjkui99d/Deshake_Vid_Stab_Bicubic.png (https://postimg.org/image/szjkui99d/)
Better preservation of details.


They recommend applying unsharp but I think we're better to look at the raw results; AviSynth is best suited to add sharpness back. There isn't much loss of details so not much sharpening is required at all.

It doesn't seem to be compensating for rotation so my video is left with a "boating" effect.

I'm doing an encoding of the full video (from the same AVI as I used with Mercalli) and will post once it is completed. Let's see how it does with the more difficult scenes and how it deals with wobbles.

Edit: it should be compensating for rotation

maxangle
Set maximal angle in radians (degree*PI/180) to rotate frames. Default value is -1, meaning: no limit.


Edit2: Test results. VidStab has the worst wobble effects compared to Deshaker and Mercalli. It's much worse. Just like Mercalli, it also zooms in a LOT.

Deshaker is still the best tested so far. I believe Mercalli uses linear scaling. It would be nice if it could use bilinear.

[shekh]

Wow, first video has horrible warping effects!! Deshaker won't handle that. I don't think any of the AviSynth methods will do anything good with such heavy fish eye effect. There is heavy jello effect but I'm impressed it's able to do something decent on that one!

This is nothing exciting, Deshaker can do the same. You just have to raise error tolerance because vectors at different places in image dont align.
This partially works because distortion is radial and differences in vectors cancel each other but this falls apart when frame has strong asymmetry.

[shekh]

Pass 1: the analysis file for a 4m22 1080p video is 48MB, which indicates it collects a LOT more data to work with than Deshaker

Both create a grid of motion vectors (you can select which density).
Deshaker does not save these vectors and analyze them inplace, where vidstab is probably saving all vectors and analyze them at pass 2.

[poisondeathray]

Can you upload the video you're working on ?

Or if not, How is this for a test video? 10sec 44MB , handheld 1080p60 , in camera stabilization off, mild RS artifacts . It's pretty typical these days for a point and shoot

video preview
https://www.youtube.com/watch?v=D4IqMXjlOFY

actual page canon GX9 review
http://www.imaging-resource.com/PRODS/canon-g9x/canon-g9x-field-test.htm

direct link original file
http://www.imaging-resource.com/PRODS/canon-g9x/FULLRES/yvid_MVI_0731_day_nostab.MP4

It's full range video, so if your decoder or program that you're using doesn't clamp to legal range, do with with something like levels or smoothlevels so everyone is on the same page .
eg. smoothlevels(preset="pc2tv")

Don't correct for anything else or make levels/color adjustments since this the object is to test stabilization

[MysteryX]

OK I can post my video. You can use it for testing; just don't actually watch the video unless you can handle what's actually being said in it. It can be a paradigm shock for most technical people.

Disclaimer: by watching this video, you agree to not verbally assault me or criticize the video in any way.

>> Download here (https://mega.nz/#!bYA1zYYD!XTiUHsE3oivZSF4HduhrqPxEA7YPo0mDJhRaPIyd7Tg)
4m42, 1080p, 210MB

I ran Deshaker(prefilter=4) and converted from PC range to TV range, and encoded to x265 with crf 16.

First part is steady and I want to preserve the quality. Then the camera is moving around and I want to handle it nicely.

[MysteryX]

Alright I might be able to do something with Deshaker and MP_Pipeline. I created a function in my program to convert the script into multiple processes with a single call at the end, and it will re-insert the audio to match the video, even if applying trimming or changing the playback speed. Ah... the joys of auto-generating scripts.

From here I'll be able to implement it nicely; unless someone find another plugin doing a better job. So far this one works best.


P="Encoder\"
LoadPlugin(P+"MP_Pipeline.dll")
SetMemoryMax(1)

MP_Pipeline("""
### inherit start ###
P="Encoder\"
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"Dither.dll")
Import(P+"Dither.avsi")
LoadPlugin(P+"masktools2.dll")
LoadPlugin(P+"VDubFilter.dll")
LoadVirtualDubPlugin(P+"deshaker.vdf", "deshaker", preroll=0)
LoadPlugin(P+"SmoothAdjust.dll")
file="TestimonialAndreas.MOV"
### inherit end ###

LWLibavVideoSource(file, cache=True)
AssumeFPS(22.7771998405457, true)
Trim(1438, 0)
Dither_convert_8_to_16()
SmoothLevels16(preset="pc2tv")
Dither_convert_yuv_to_rgb(output="rgb32", matrix="709", noring=true, lsb_in=True, mode=6)
### prefetch: 3, 3
### ###
deshaker("19|2|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|0|0|2|8|30|300|4|Preview_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|1|1|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")
ConvertToYV12(matrix="Rec709")
### prefetch: 30, 30
### ###
""")

SourceFps=23.9759998321533
A=BlankClip(1,1,1).AudioDub(LWLibavAudioSource(file, cache=True))
Blank=BlankClip(int(A.AudioDuration*SourceFps), 4, 4, "YV12", SourceFps)
A=AudioDub(Blank, A)
A=A.Trim(1438, 0)
A=A.AssumeFPS(22.7771998405457, true)
AudioDub(last, A)

[poisondeathray]

hahaha sorry confess that I watched it... uhh Maybe you shouldn't have posted it. Just joking :)

It's a bit lengthy clip for a test clip. So you went with deshaker ? How did it look in the end ? Did you try other mercalli settings ? Maybe splitting it up into sections would help, since the clip characteristics are very different from the 1st part and 2nd part

In my experience, mercalli does a better job for general use but it's just a general observation ; you're going to have content where warp stabilizer does a better job, or deshaker does a better job on other clips, or some other stabilizer with custom settings does a better job. For example, foreground objects can skew the analysis in certain scenes. e.g. a car drives by and the stablization sort of follows because the tracking follows the car not the background. There is no way to block out the car or objects in deshaker or mercalli (deshaker has the exclude pixels or crop, but it's very crude), but it's possible in other programs (manual intervention, mattes and roto masks, you can guide motion estimation or delete /modify track points. You can tell what / where to track or exclude or include) . My point is I would hesitate to make broad based conclusions based on 1 clip.

Did anyone try the 10 sec clip posted above? I can't get deshaker to give decent results on that, maybe the rs settings or some other settings need to be adjusted ; I tried 80 and 88. But mercalli and warp stabilizer handle it well

[MysteryX]

Here's the final video (https://www.youtube.com/watch?v=VgLb7kOnmyA)

I confess that this version has a little bit too much chroma sharpening due to resizing chroma back from YV24 with Spline36 instead of Bicubic. Didn't want to re-encode just for that.

With Deshaker, it's possible to configure the first pass differently for various parts of the clips, and I intent to make that easy to configure within my software. "starting at frame 0, apply these settings, starting at frame 500, apply these settings", and then I can run pass 1 for each segment automatically. Manually, however... forget it.

Deshaker does less blurring and less zooming. Mercalli probably handles difficult scenes better than anything else and avoids wobble and artifacts.

[poisondeathray]

Here's the final video (https://www.youtube.com/watch?v=VgLb7kOnmyA)

Hey, you look better in the "final video" :) Maybe something's different :P




Deshaker does less blurring and less zooming. Mercalli probably handles difficult scenes better than anything else and avoids wobble and artifacts.

It's adjustable in deshaker and mercalli and other programs. Maybe the default settings do less zooming , but maybe they are less stable. It's difficult to compare "apples to apples", unless you do several runs at approximately the same zoom and stability level, then compare. Furthermore, "blurring" isn't only a function of zoom or scaling algorithm; subpixel adjustments and "dewarping" can make image softer. If deshaker isn't "fixing" the wobble or warping as much, it might look "sharper".

Some people even complain about "bicubic" in deshaker being too soft. Well you can "roll" your own scaling algorithm, just don't auto scale - but it requires a 2nd step. Same with other programs. Some people might not want to or don't have time to.

Mercalli in the standalone version doesn't allow lossless export. Prores is going to be closer than HEVC or h.264 options, so some of the "blurring" you're seeing might be from that

[Groucho2004]

OK I can post my video. You can use it for testing; just don't actually watch the video unless you can handle what's actually being said in it. It can be a paradigm shock for most technical people.

Disclaimer: by watching this video, you agree to not verbally assault me or criticize the video in any way.
Neither assaulting nor criticising - I'm just in awe how you get these people to pay for this bullshit. They must have been in a very bad place, vulnerable and desperate.

[MysteryX]

OK I took a look at your video. It's not that bad with Deshaker but it does have quite a few distortions. Deshaker does good with zoom, rotation and panning, but doesn't do well with 3D perspective. Yours is very hard on 3D perspectives.

Perhaps we just have to accept that Deshaker doesn't do good with 3D perspective? I don't know whether some settings can be tweaked to improve in that case.

Here's the encoded video with Rolling Shutter 88%, Adaptive Zoom and default settings. (https://mega.nz/#!aARxVCpJ!GSVwzrlb0kaVlHr1unHcE6TKah354PbA7NBLcNarboE)

In this case, perhaps running the first pass with Deshaker and the 2nd pass with the other plugin that supported 3D perspective would give better results.

Mercalli V4 isn't doing a good job on this one either. (https://mega.nz/#!PVwFwKyZ!mIbQvL6h919-vM5LMiE6mSIjaXnXb8i1bMJSzh93qp0) It also doesn't handle the PC color range well. I prefer the Deshaker version.

My sample is good to improve an easy case and compare the quality; yours is good to test a difficult case.

[MysteryX]

Neither assaulting nor criticising - I'm just in awe how you get these people to pay for this bullshit. They must have been in a very bad place, vulnerable and desperate.
Groucho, the disclaimer was specifically for you and The Fluff. Glad we're on the same page here :) Let's keep the discussion technical and on-topic.

[MysteryX]

Deshaker3D (http://forum.doom9.org/showthread.php?t=171051) does the worst job. Horrible distortion. We can't talk about the "best" as a forum rule, but can we talk about the "worst"?

My software half-implementation really does make running Deshaker a LOT easier already, although the integration isn't yet finished.

Here's with Deshaker3D. (https://mega.nz/#!XdQRBKgb!KKwZklv8paEzMaCFRTwAknf4x3HZP3XXxN6p5uXcShE)

So far Deshaker still wins. It might be possible to improve the result by tweaking the settings.

[poisondeathray]

OK I took a look at your video. It's not that bad with Deshaker but it does have quite a few distortions. Deshaker does good with zoom, rotation and panning, but doesn't do well with 3D perspective. Yours is very hard on 3D perspectives.

Perhaps we just have to accept that Deshaker doesn't do good with 3D perspective? I don't know whether some settings can be tweaked to improve in that case.

Here's the encoded video with Rolling Shutter 88%, Adaptive Zoom and default settings. (https://mega.nz/#!aARxVCpJ!GSVwzrlb0kaVlHr1unHcE6TKah354PbA7NBLcNarboE)

In this case, perhaps running the first pass with Deshaker and the 2nd pass with the other plugin that supported 3D perspective would give better results.

Mercalli V4 isn't doing a good job on this one either. (https://mega.nz/#!PVwFwKyZ!mIbQvL6h919-vM5LMiE6mSIjaXnXb8i1bMJSzh93qp0) It also doesn't handle the PC color range well. I prefer the Deshaker version.

My sample is good to improve an easy case and compare the quality; yours is good to test a difficult case.





mercalli (rec601 adjusted, but color is still a bit "off")
https://www.mediafire.com/?7o2opoxdn80llo9

warp stabilizer
https://www.mediafire.com/?tm0v1d4u34am8g3

Maybe you didn't use the right settings for mercalli ? It looks like you have some tilt disabled ?

What's disconcerting for the viewer, is the y-axis rotation in the deshaker processed video. I couldn't find suitable deshaker settings to get rid of that y-axis rotation. Maybe someone knows better deshaker settings. I'm guessing some of the wobble is interfering with it's analysis

Sure you can adjust many of the settings. If you want less zoom/more zoom, less tilt, more smooth, etc....same with deshaker. It's difficult to cover all the bases in a simple comparison, but I would argue the mercalli and warp stabilizer video are more "pleasant" to watch

Another disconcerting thing is the motion blur in the source. When the frames are aligned and stabilized, blur in the source causes it to sort of go in/out of focus. There is not auto fix for this, you would have to use deconvolution filters manually on affected frames to undo the directional blur

If you take 10 random clips, of consumer handheld video. In my experience 9/10 will look better with mercalli than deshaker. Not massively better, just marginally better. Probably more than 9/10 because these days almost all sensors are CMOS, and minor wobble issues that are handled by mercalli or warp stabiilizer, aren't handled by
deshaker doesn't handle as well. But even if you go back 10 years , and only look at CCD videos from DV camcorders (no CMOS wobble), mercalli did a better job overall IMO. The gap has just widened now because of RS issues

I haven't tried deshaker3d, but it might be worth a shot

[shekh]

pdr, do you by chance have adobe lens profile for that canon G9X lens in lcp format?

[poisondeathray]

pdr, do you by chance have adobe lens profile for that canon G9X lens in lcp format?

nope, I don't own that camera, it's just a random example

[MysteryX]

You're right, Tilt was disabled in Mercalli. I was using Standard preset.

Here's another shot with the preset Vibrations/Wobble. (https://mega.nz/#!jRYhkbwB!lTXLCCTTXyR4gkh9RVxIXlv-U76NgBJRnejQbhS79Rs) This one looks pretty good!

Here's with preset Intensive Repair #1. (https://mega.nz/#!fcQDAaza!g_scaQs1U2BHzpBgyn6pMOM0LW9p7PutctrKYCYo43M) This one is definitely sharper, but it skips around.

I just found a video comparing Mercalli, Deshaker and Warp Stabilizer
https://www.youtube.com/watch?v=2kOu_XenEx8

From that video, it's clear that Mercalli gives good results, Deshaker gives clearer image and Warp Stabilizer blurs more.

[poisondeathray]

I gave a try with "Intensive Repair" and it threw a generic failure message.


What was the message exactly ?




I just found a video comparing Mercalli, Deshaker and Warp Stabilizer
https://www.youtube.com/watch?v=2kOu_XenEx8

From that video, it's clear that Mercalli gives good results, Deshaker gives clearer image and Warp Stabilizer blurs more.

It's not clear to me, because we don't know the processing lineage or what was done exactly. Maybe the guy exported warp stabilizer with low bitrate h264 for example. We don't know what settings were used (maybe he could have used better settings for all of them)

[shekh]

I run the yvid_MVI_0731_day_nostab.MP4 through SynthEyes to get fov, and it gave me ~61deg horizontal.
I wanted to post a video as stabilized by SynthEyes but it really looks like crap, most frames are either motion blurred or skewed by rolling shutter (so I assume it is not worth looking at).

[poisondeathray]

I run the yvid_MVI_0731_day_nostab.MP4 through SynthEyes to get fov, and it gave me ~61deg horizontal.
I wanted to post a video as stabilized by SynthEyes but it really looks like crap, most frames are either motion blurred or skewed by rolling shutter (so I assume it is not worth looking at).

Syntheyes is a fantastic tracker for match moving but the wrong tool for the job.

Just like deshaker, mercalli would be the wrong tool for tracking and VFX shot compositing

[shekh]

I agree SynthEyes is not usable for generic stabilization and not for this specific shot. But it does have perspective and distortion aware projection, so can demonstrate vastly better effect than deshaker in other scenes.
hugin is not stabilization tool either https://gfycat.com/CriminalAromaticEsok

[wonkey_monkey]

Ugh... what's the recommended way to open h.265 in Avisynth?

It doesn't play in VLC... plays badly in MPC-HC...

Would it be too "old-mannish" to ask for nothing older than h.264?

I don't see a huge difference between Deshaker and Deshaker3D. Don't forget that their parameters aren't comparable so you may be applying different amounts of smoothing, and that Deshaker3D doesn't do rolling shutter. It wasn't meant to be much more than a proof of concept when released.

Groucho, the disclaimer was specifically for you and The Fluff. Glad we're on the same page here Let's keep the discussion technical and on-topic.

One could argue that your sly dig against "technical people" and your claim that it was "paradigm shaking" weren't really on-topic either. if you'd not said anything, I suspect no-one else would've felt like commenting (or even bothered to listen to the audio at all).

[MysteryX]

Ugh... what's the recommended way to open h.265 in Avisynth?

It doesn't play in VLC... plays badly in MPC-HC...
I have no problem playing H265 files in MPC-HC. Do you have the latest version? You might have outdated LAV filters.

Use this to open H265 files. It works with pretty much any file format.

LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))


One could argue that your sly dig against "technical people" and your claim that it was "paradigm shaking" weren't really on-topic either. if you'd not said anything, I suspect no-one else would've felt like commenting (or even bothered to listen to the audio at all).
Groucho and The Fluff would have commented, you can be sure of it. Maybe The Fluff just hasn't got to this thread yet. Now let's stay on-topic.