Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd July 2016, 14:08   #1  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Deshaker Comparison

I've been doing some comparison between Deshaker and Mercalli ProDad v4. I wanted to share my results.

First, Deshaker is a VirtualDub plugin and is hard to use. I'm trying to program it within my own software to automate the process but am encountering a few challenges.

Mercalli ProDad is a commercial software. It used to integrate within Adobe Premiere but v3 and v4 are stand-alone software.

So here are the results, for those interested.

Deshaker
Pros:
- Cleaner image
- Better efforts at reconstructing borders and not zooming unecessarily
Cons:
- Difficult to configure and use
- Slow
- Some wobble or jello effects

Mercalli ProDad
Pros:
- Much easier to use
- Much faster
- Wobble and jello effects are still there but weaker
Cons:
- Image is blurrier
- It zooms a lot more
- Requires a lossy re-encoding of the file -- not ideal in a video processing chain

I am aware that some of the settings can be tweaked in both software to improve the results. Here I have gone mostly with default settings with just minor customization.

I would have expected Mercalli ProDad to do a much better job because it analyzes the movements in 3D and is a lot more sophisticated.

Overall, Deshaker wins. It will be worth spending more time to program its easy integration within an AviSynth script.

However, whenever I use "fill borders with previous or future frames", it fails to render the video. First the video processes, then the output jams on a frame, and after being jammed on the same frame for a while, it aborts the encoding. Will have to investigate.

There was a bug in AviSynth+ where it crashed in the destructor and I fixed it. You need this updated VDubFilter.dll.

I'll have to keep testing whether the issues I'm seeing are related to AviSynth+ or I'm doing something wrong.

Last edited by MysteryX; 2nd July 2016 at 14:17.
MysteryX is offline   Reply With Quote
Old 2nd July 2016, 14:52   #2  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Interesting subject. When it comes to testing codecs everyone have test patterns and psnr graphs, but all I can find for deshaking is "looks good".

What kind of shakiness are you trying to repair?
Supposedly strong part of Mercalli4 is 3d and cmos repair, but it is very slow and actually when I run single test it did not repair anything and created artifacts.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 2nd July 2016, 16:21   #3  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
You can't really test a deshaker with test patterns and graphs; unless you take a video of a sheet of paper having those patterns.

I have a 1080p video testimonial recorded from a compact camera. The video is quite noisy and slightly shaky (hand-held); with a part where I'm moving the camera around so it's then very shaky. For the part that is fixed, I find it works best with Deshaker when I only detect rotation and disable "detect zoom" and "detect scene change", but then when I start moving around it causes the image to skip quite a lot so "detect zoom" must be enabled.

I just gave Mercalli another try. This time I disabled Smooth Camera Path and reduced "avoid borders" from 40 to 22, and reduced "pan shot smoothing" from 40 to 29. Most importantly, I used the preset "Vibrations/Wobble". Compared to Deshaker, the first pass took quite a bit longer, while the second pass was still fast.

Result? The over-zoom was gone. I reduced the settings too much and it is still a bit shaky, but even there, Deshaker's video that is zoomed in more has a clearer picture.
- Mercalli loses details
- Most of the wobbles are now gone; but there are a few left that are even bigger (or they just stand out more?)
- *there is a slight yellow decoloration*

The workflow that is working best so far is
1. Convert video from PC colors to TV colors and run Deshaker's first pass from the source
2. Run SMDegrain(prefilter=4) on the source, then convert from PC to TV colors, then run Deshaker's second pass.

OR

1. Run SMDegrain(prefilter=4), convert from PC colors to TV colors, and save to AVI
2. Import into Mercalli, re-encode as H264 with highest quality.

I tried importing an AVS file into Mercalli. That didn't work.

Based on my source video, my goal is to have a video that fixes the slight shaking of the first part without degrading the quality, and that handles the camera movements of the second part nicely without too much wobbles. I think that's a good test video.

Last edited by MysteryX; 2nd July 2016 at 16:30.
MysteryX is offline   Reply With Quote
Old 2nd July 2016, 17:10   #4  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
For Deshaker, I had an idea. To deal with Wobbles, you can go through the clip (without SMDegrain it renders quite fast), see what specific frames have wobbles, and delete those frames from the Log file. In my software I might add an option to enter the list of frames to discard. But then, if the video is zoomed, will it "break the zoom" and completely break the video sequence?

Now I'm testing this script. Second argument is "Pass". You must first run with 2nd argument set to 1 which generates a log file, then run again with it set to 2. Only run SMDegrain on the 2nd pass.

Here I'm setting "Fill borders with previous and future frames" and it's working. It's just that when playing in MPC-HC, the first 30 frames are black, but I'm not getting the crash.

Code:
LoadVirtualDubPlugin("deshaker.vdf", "deshaker", preroll=0)
Trim(0, 500)
SMDegrain(prefilter=4, tv_range=false)
ConvertToRGB32(matrix="PC709")
deshaker("19|2|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|1|3|2|8|30|300|4|F:\Job_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|1|1|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")
Oh... I think VirtualDub informed me of the crash I was getting in my previous encodings: "Out of memory"

Last edited by MysteryX; 2nd July 2016 at 17:12.
MysteryX is offline   Reply With Quote
Old 2nd July 2016, 17:13   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
As you may know, I wrote an extensive guide to Deshaker, over ten years ago. I then wrote a series of Vegas and Virtualdub scripts which completely automate the stabilization process so you can batch stabilize individual scripts with Deshaker. Batch stabilization is something you still cannot easily do with Mercalli.

Having established my "creds," let me say that in most cases, Mercalli produces vastly better stabilization. There are exceptions, and I still use Deshaker on some footage, but not often.

The one thing that puzzles me is your report that Mercalli is creating footage that is less sharp. That has not been my experience. It makes me think that there is something wrong in your Mercalli workflow. I do not have the latest version, and I use the plugin for Vegas, so I may not be able to duplicate what you are doing.

The usual reasons for blurred footage is zooming. Even the smallest amount of zoom can end up softening detail. It sounds like you are using the Deshaker modes that use border fixing and don't zoom. This preserves detail and sharpness. It is important to make sure that every last vestige of zooming is turned off in Mercalli or you will indeed get a softer image. Since Mercalli's border fixing didn't used to be as good as Deshaker, it was often undesireable to completely eliminate zooming because the edges ended up looking too weird. I have heard that V4 has better edge correction. I have been meaning to purchase the SAL version, but haven't gotten around to it yet.

So, make sure that you really do have all zooming turned off in Mercalli.

Last edited by johnmeyer; 2nd July 2016 at 17:14. Reason: Modified last sentence
johnmeyer is offline   Reply With Quote
Old 2nd July 2016, 18:04   #6  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
For Deshaker, I can improve on memory limitations with something like this; but it does bring quite a few limitations that will be a hassle to program around for automatic script generation. But this script works. However, I can't use MT and CPU usage is only 50-60%.
Code:
P="Encoder\"
AddAutoloadDir(P, true)
MP_Pipeline("""
P="Encoder\"
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"MVTools2.dll")
Import(P+"SMDegrain.avsi")
LoadPlugin(P+"KNLMeansCL.dll")
LoadPlugin(P+"Dither.dll")
Import(P+"Dither.avsi")
file="F:\NaturalGrounding\Convert\TestimonialAndreas.MOV"
LWLibavVideoSource(file, cache=True)
Trim(0, 500)
SMDegrain(prefilter=4, tv_range=false, lsb=true)
### prefetch: 5, 5
### ###
""")
LoadVirtualDubPlugin(P+"deshaker.vdf", "deshaker", preroll=0)
ConvertToRGB32(matrix="PC709")
deshaker("19|2|30|4|1|0|1|0|640|640|1|2|1000|1000|1000|1000|4|1|3|2|8|30|300|4|F:\Job_Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|1|1|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|5000|100|20|1|0")
Here are some comparison images on the first frame of the video.

Original


AVI after SMDegrain (which served as the source for Mercalli)


Mercalli - no zoom, "Vibrations/Wobble" mode


Deshaker


In the Mercalli screenshot, the wood loses the sharpness of its grain and there is a yellow decoloration. I have run SMDegrain(prefetch=4) on both. In the case of Mercalli, the first script was exported as AVI.

Last edited by MysteryX; 2nd July 2016 at 18:10.
MysteryX is offline   Reply With Quote
Old 2nd July 2016, 18:35   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
RE: color discoloration

1) What kind of "AVI" did you export for import into Mercalli ?
2) What export settings are you using ?
3) How are you taking screenshots ?

YV12 Colorbars in UT Video 709 imported and exported come out the same in terms of color. So I'm thinking there are some issues with your workflow for mercalli
poisondeathray is offline   Reply With Quote
Old 2nd July 2016, 18:44   #8  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Source video is PC709. I degrain, convert color range PC->TV, and export as Rec709 UT Video AVI using FFMPEG.

From Mercalli, I export with H264(mp4) High Quality.

To take screenshots, I open in MPC-HC and "Save Image as PNG"

I rendered 2 videos with Mercalli. The first one with Standard preset doesn't have the same decoloration... or perhaps just very lightly. The second one with "Vibrations/Wobble" preset has much stronger decoloration (or coloration?)
MysteryX is offline   Reply With Quote
Old 2nd July 2016, 19:27   #9  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Mercalli uses something close to Rec601 when converting RGB<=>YUV , similar to vdub deshaker, when you don't control the conversion. But there is still a slight discoloration when corrected for , so it's not exactly Rec601. You can use prores if you want higher quality . It's unfortunate that they don't allow you to use lossless codecs
poisondeathray is offline   Reply With Quote
Old 2nd July 2016, 19:53   #10  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
What about MDepan with DepanStabilize? Has served me well so far.
ajp_anton is offline   Reply With Quote
Old 3rd July 2016, 01:44   #11  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by poisondeathray View Post
Mercalli uses something close to Rec601 when converting RGB<=>YUV , similar to vdub deshaker, when you don't control the conversion. But there is still a slight discoloration when corrected for , so it's not exactly Rec601. You can use prores if you want higher quality . It's unfortunate that they don't allow you to use lossless codecs
It would be *great* if Mercalli could open up AVS script and export in a lossless format. Besides that, it does have a few drawbacks; one of which is being a considerably expensive program.

But then, Deshaker also definitely has its drawbacks. Is there a way to fix Mercalli's decoloration and loss of quality? The decoloration can easily be fixed afterwards; but the loss of quality I guess it's just the way Mercalli works.

Quote:
Originally Posted by ajp_anton View Post
What about MDepan with DepanStabilize? Has served me well so far.
Had not heard of DePan.dll. Does anyone have experience with this and how it compares with Deshaker?

Features and limitations
Quote:
1. Works only in YV12 and YUY2 color formats.
2. Only pan and zoom motion (no rotation) are estimated, but it gives advance in speed and stability. Zoom estimation is not very precise.
3. The source clip must be same length as motion data clip.
4. Directly works only with progressive clips. For interlaced sources, you must use AviSynth following function SeparateFields and followed Weave
(after motion compensation and denoising), with AssumeTTF and AssumeBFF (both may be needed for odd fields offset). Plugin estimates and calculates motion from one field to neighbor (by time) field (from same or neighbor frame). For preserving fields order (dominance) and best denoising, set parameter MatchFields=true.
5. Mirror mode is unique but slightly strange :-). The blur is some workaround to hide sharp mirrored details. Note: blur is not implemented for rotation as yet.
6. Not very fast, not assembler optimized.
7. Tested with Avisynth 2.5.3, 2.5.4, 2.5.5, 2.5.6, 2.5.7, 2.5.8, 2.6.0.
8. Old versions DePanEstimate used free FFT2D code by Takuya Ooura (http://momonga.t.u-tokyo.ac.jp/~ooura/index.html)
Now DePanEstimate uses only more fast FFTW library version 3 (http://www.fftw.org)
as Windows binary DLL (compiled with gcc under MinGW by Alessio Massaro),
which support for threads and have AMD K7 (3dNow!) support in addition to SSE/SSE2.
It may be downloaded from ftp://ftp.fftw.org/pub/fftw/fftw3win32mingw.zip
For using, you must put FFTW3.DLL file from that package to some directory in path (for example, C:\WINDOWS\SYSTEM32). DePanEstimate will NOT work without it!
9. For best results, you may temporary add Info parameter, analyze info and tune some parameters (Trust, dxmax etc).
10. You may use not strictly same clips for motion estimation and compensation, for example try add some brightness-contrast adjusting, pre-filtering, masking, cropping to input clip used for motion estimation only (and use different processing for output compensated-stabilized results).
#2 may be a deal-breaker. The first part of the footage for the most part only has slight rotation; no zooming and only minor panning.

Last edited by MysteryX; 3rd July 2016 at 01:48.
MysteryX is offline   Reply With Quote
Old 3rd July 2016, 01:48   #12  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
There's also Deshaker3D.
Reel.Deel is offline   Reply With Quote
Old 3rd July 2016, 02:08   #13  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
One of the drawbacks of Deshaker is the need to manually configure the Rolling Shutter value to "God Knows What"; or perhaps this gives a quality advantage over Mercalli doing it automatically?

It appears there has been several other attempts at creating better deshakers: Deshaker3D and Lens Transform Tool. Neither of which support Rolling Shutter Correction. Both of which also use Deshaker's first pass. To process the video in 3D, wouldn't it have to better analyze the movement axis in the first pass instead of only changing the 2nd pass? Deshaker3D also misses many of the features of Deshaker. Has anyone seen good improvements with it still? It seems these are mostly "work in progress" that saw no further development. Mercalli ProDad definitely wins on this aspect.
MysteryX is offline   Reply With Quote
Old 3rd July 2016, 02:46   #14  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
On the same vein of comparing commercial products with free products, here's a comparison of Adobe Premiere CC 2015.1's new Optical Flow with SVP.

SVP


Adobe Premiere Optical Flow


The clip isn't exactly the same; the SVP clip is zoomed in a little bit more, but it's good enough for a comparison. Optical Flow creates UGLY artifacts.

SVP team wins!

I'll steal the words of McAfee when he ridiculed the FBI computer engineers: "Some people are born geniuses and these people can't be 'made'. Get a few of my people together and they'll do what hundreds of well-trained computer engineers can't ever do." (he didn't use these exact words but that was the general idea)

I'm all for developing products in a sustainable way and creating businesses, and this gives me faith in the Open Source path.

Last edited by MysteryX; 3rd July 2016 at 02:48.
MysteryX is offline   Reply With Quote
Old 3rd July 2016, 03:01   #15  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
When encoding with Deshaker and "Fill borders with previous and future frames", I get this error
Code:
Could not allocate video frame. Out of memory. memory_max = 1073741824, memory_used = 900757717 Request=6220831
Weird thing is that the memory usage is only 1.4GB. Is there another work-around than going the MP_Pipeline route?

In defense of Mercalli, the AviSynth Virtual File System may allow piping directly from AviSynth into Mercalli without first re-encoding.

Last edited by MysteryX; 3rd July 2016 at 03:29.
MysteryX is offline   Reply With Quote
Old 3rd July 2016, 03:26   #16  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
For the memory issue, is that with avs script and filters, or did you "bake" it to an intermediate? Splitting it out would reduce memory issues

For mercalli (and all stablizers), they can all soften to picture to an extent, even without zoom. x,y subpixel translation or rotations can also soften the picture, dewarping /cmos correction (if the program has it) can also soften the picture. Some of what you're seeing is going to be compression related issues if you're using h264. It's too bad that version doesn't allow lossless export.

You can indirectly use avs with anything, using avfs . I've never come across a Windows program that won't accept the "fake" AVI. But some receiving programs require RGB24 or 32 (ie. add ConvertToRGB24 or 32 to the end) , but many will accept as is

Other popular options for generic stabilization : ffmpeg has libvidstab, and there is also warp stabilizer in Adobe .

There are also other categories of stabilizers, but they are meant for tracking and compositing VFX work, not generic use hand held shots. They usually require more manual input

Last edited by poisondeathray; 3rd July 2016 at 03:35.
poisondeathray is offline   Reply With Quote
Old 3rd July 2016, 03:31   #17  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
I get best results by running in that order
1. SMDegrain
2. Deshake
3. InterFrame (SVP)

Perhaps MP_Pipeline is the best option then... with the hassles that will go with it. Or without it, it works fine without "use previous and future frames" on 1080p content.
MysteryX is offline   Reply With Quote
Old 3rd July 2016, 08:32   #18  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Lens Transform Tool is work in very slow progress (too difficult to do part time) and not in generally usable state now.
I worked on improving analysis pass. Deshaker and libvidstab search for 2d motion and therefore they find no solution with any amount of perspective. What I was doing is to instead solve for 3d rotation, and it kindof works.
__________________
VirtualDub2
shekh is offline   Reply With Quote
Old 3rd July 2016, 09:23   #19  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
I will soon release a DepanEstimate/DepanStabilize port.
64 bit
YV16 and YV24 support
Native 16 bit (yuv420p16.. yuv444p16) when used with the next (not yet released) avisynth plus.
pinterf is offline   Reply With Quote
Old 3rd July 2016, 09:58   #20  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by pinterf View Post
I will soon release a DepanEstimate/DepanStabilize port.
64 bit
YV16 and YV24 support
Native 16 bit (yuv420p16.. yuv444p16) when used with the next (not yet released) avisynth plus.
Quote:
Only pan and zoom motion (no rotation) are estimated
Will it be the same or will yours detect rotation?
MysteryX is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.