Deshaker by Gunnar Thalin [Archive]

Flexy

24th February 2004, 09:55

hi guys, i need help there :)

The problem right now is that this script crashes...what i did (as read elsewhere on a forum) generated jobs and try to load the jobs and run it...but VDubmod crashes. (I have the latest version). Maybe one of you knows more what i do wrong..

I put in some comments in the script for things i have qeustions on :) Thanks for any help !

guth: Yeah i was the one also emailing you about it. Thanks for reply ! :)

>>> snip <<<

// VirtualDub job list (Sylia script format)
// This is a program generated file -- edit at your own risk.
//
// $numjobs 2
//

// $job "Job 5"
// $input "C:\DATA\test_short.avi"
// $output "C:\DATA\dummy.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DATA\\test_short.avi","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AddComment(0x00000002,"ISFT","VirtualDubMod 1.5.10.1 (build 2366/release)");
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x67706a6d,0,9000,0);
VirtualDub.video.SetCompData(28,"TUpQRxgAAAAAAAAAAgAAAAgAAAACAAAAAQAAAA==");
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");
VirtualDub.video.filters.instance[0].Config("4|1|30|4|1|0|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0");
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,50);
VirtualDub.SaveAVI("C:\\DATA\\dummy.avi");
//
// actually, i do not WANT to save anything here, but i dont know
// how else to create a job for deshaker Pass 1 ??????
//
VirtualDub.Close();

// $endjob
//

// Now PASS 2 <------ !!!!
//--------------------------------------------------
// $job "Job 6"
// $input "C:\DATA\test_short.avi"
// $output "C:\DATA\result.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

VirtualDub.Open("C:\\DATA\\test_short.avi","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AddComment(0x00000002,"ISFT","VirtualDubMod 1.5.10.1 (build 2366/release)");
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x67706a6d,0,9000,0);
VirtualDub.video.SetCompData(28,"TUpQRxgAAAAAAAAAAgAAAAgAAAACAAAAAQAAAA==");
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");
VirtualDub.video.filters.instance[0].Config("4|2|30|4|1|0|1|0|640|480|1|2|1|400|400|400|1500|4|1|2|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0");
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,50);
VirtualDub.SaveAVI("C:\\DATA\\result.avi");
VirtualDub.Close();

// $endjob
//
//--------------------------------------------------
// $done

Flexy

24th February 2004, 10:05

It should work. If you really want to know what all the parameters are, let me know. But I recommend making the settings from within VirtualDub.

yeah..the parms are not the problem..i can just save a job/settings and have it in a script. Problem is ratehr i cannot run the script because it crashes. (See other posting)

Unfortunately it's impossible to do both passes internally. :(

Finally, I would never batch process clips with deshaker. You most probably won't get as good results as you could if you spent some time and finetuned the settings for each clip (and even parts of a clip). Also, personally I would never use the default settings. They are only good for beginners. ;) [/B]

i dont use deshaker for *that* long, so i didnt go in-depth and experimetned with all settings. The defaults are 'good' for my first testing...and i have good results with my vids. 'Tuning'...i can do that later when i have time :)

An average 'vacation' movie for me consists of about ~50 smaller clips..at elast the last one did and teh next movie might get even bigger. I think i would sit there 'til the year 3000 if i'd analyze each single clip (assuming it's 50 and more)..maybe even each *scene* in each clip....and then do a manual deshake with different params for each clip...that would just be extremely insane work.

I had nice results applying a deshakeover my final result video (which is the 50 clips joined together in Sonic Foundry Vegas)..so...the deshaking looks nice....PROBLEM is the titles....i need to apply the Deshake to the individual clips *before* i render the resulting video in vegas for the titles and everything ok.

WarpEnterprises

24th February 2004, 13:25

@guth:
another "technical" question:

How are scenechanges handled?
When do you make the filter snap back to the "0" position?
Is this connected to the value "skip frame" (more precise to the 0,0,1-results generated)?
Is this the same as if a line in the file is manually deleted (as you suggest for human guidance)?

Or summing up: when is in the results (of the motion equations) a "forced" zero crossing?

I may look very curious but although the docu is quite good, some things/settings may be completely clear for you - but not for me, and I can't sleep until I have understood it from the base line :)

guth

24th February 2004, 19:20

@Flexy:
You're right! It totally crashes! :)
I looked into it now (I never ran jobs before) and it crashes because VirtualDub calls my functions in a slighty unpredictable order (EndProc is called before StartProc and InitProc). Next version will be able to cope with this (I'll send you an unofficial version with a fix by email soon too).

@Warp:
I'm not completely sure I completely understand what you mean, but I think so...
It never snaps back to the 0 position during a single processing. You'll need to split the processing into several parts in order to do that.
Yes, a 0,0,0,1 row is the same as if the row was missing.
The only zero-crossings are the first and last frames that were processed in pass 1 (unless you use adaptive zoom, then there are none).

Flexy

25th February 2004, 06:31

Originally posted by guth
[B]@Flexy:
You're right! It totally crashes! :)
I looked into it now (I never ran jobs before) and it crashes because VirtualDub calls my functions in a slighty unpredictable order (EndProc is called before StartProc and InitProc). Next version will be able to cope with this (I'll send you an unofficial version with a fix by email soon too).

hi gunnar,

thanks !I knew that there was a problem ebcause i read about these crashes before somewhere on another forum where they tried the same and they got these crashes.

Ok..the new version works BETTER :) and the first couple of runs worked like a charm ! But i just found a minor bug: it crashes when i create a script with the Deshaker option 'no motion vectors shown'...but with the vectors displayed on it seems to work !!! Still testing around :)

thanks !

WarpEnterprises

25th February 2004, 09:00

@guth: that means there is no special treatment of scenechanges.

If I want to treat each scene separated I have to filter it separated.

Wouldn't that be a useful feature to add a scenechange marker (e.g. by manually editing the first frame entry of a scene after the first pass - which will probably already be a 0,0,0,1 row if it is a scenechange) to tell it to _result_ in a 0,0,0,1 ?

@all: it seems the preset of 400 for the smoothing values is pretty low. What are your preferred settings?

guth

25th February 2004, 18:04

@Flexy:
My guess is that it crashes because you're using a codec to encode the output of pass 1 that can't handle the extremely small video output (8x8 pixels). It works for me.

@Warp:
Yeah, I guess that could be a useful feature. But it means a lot of work, and I hate programming. :)
We'll see...

I use all kinds of smoothness values (400-100000). It depends on the clip.

mrg

26th February 2004, 19:52

Hi,
it is possibly to use Deshaker for DV without lost the DV Timecode?
Thanks

WarpEnterprises

26th February 2004, 22:06

No. Only with direct stream copy, and then you can't use a filter.

Flexy

27th February 2004, 02:16

guys, i am in heaven :)
My next vacation vid will be awesome :)

I just found a very useful freee program called 'dubman' which aids in creating vdubmod jobs....you can select wildcards for directories etc.

1) save vdubmod processing settings for deshaker pass1
2) save vdubmod processing sttings for deshaker pass2

use these two *.vcf files in 'wildcard' mode in 'dubman' and go thru a whole directory...right now i am deshaking a whole directory in the background...yay :)

BUT....i have one, related, question...maybe someone can help:

- I am using Deshaker to deshake my clips.
- I am using Dynapel's Motionperfect to convert my 15fps to 24fps. Motionperfect fills in the missing frames...and the results (even if not perfect) are FAR better than what i had with my original 15fps videos !

QUESTION:

What should i do first:

) FIRST convert my clips to a higher fps with MotionPerfect and THEN deshake them ?
) or FIRST deshake them and THEN convert them to higher framerate with MP ?

Logic tells me first to deshake and stabilize the 15fps clips and then let Motionperfect work (which i a doing now) fill in mroe frames

Any tips ?

Also...keep in mind my vids all come in 15fps originally..using MotionPerfect...would you convert them to 24 or should i go NTSC 29,9 fps ????
I figure 24 would be beter since MP would not have to fill in too many 'artificial' frames [artifacts]..but on the other hand if i am planning to make NTSC VCDs...maybe i should just let it generate 29,9 fps instead of only 24 ? (Vide wold be smoother and suited for NTSC VCD..but i'd get more artifacts)

greetings

mustardman

1st March 2004, 22:34

Why on earth are you starting with 15fps? Are you using one of those memory stick recorders, or perhaps an old film camera?

Personally, I would convert from 15fps to 30fps, and then use VDub to set the rate to 29.999fps (if you feel it is really necessary).

The reason being (apart from your destination) is that MotionPerfect will then always generate a frame exactly in between the existing frames. I would imagine that doing 15->24 would not work as well. Mind you, having only used MP once (it didn't work with my source - unusual content :) ) I can't say I have a lot of experience in this area.

Flexy

1st March 2004, 23:55

Originally posted by mustardman
Why on earth are you starting with 15fps? Are you using one of those memory stick recorders, or perhaps an old film camera?

Personally, I would convert from 15fps to 30fps, and then use VDub to set the rate to 29.999fps (if you feel it is really necessary).

The reason being (apart from your destination) is that MotionPerfect will then always generate a frame exactly in between the existing frames. I would imagine that doing 15->24 would not work as well. Mind you, having only used MP once (it didn't work with my source - unusual content :) ) I can't say I have a lot of experience in this area.

hi,

it's not by choice that i us 15fps ! I have a very tiny Canon S230 which does 640x480 vids at 15fps.

HOWEVER - i have *very* good results using deshaker now and MP....not perfect - but far better than ther original vids.
(I guess the 15fps are what you have to live with when you have a cam which would fit into the palm of your hand....still love it because of it's size :)

Regarding the fps:

I did some testing, but still need more. You're right, MP generates frames in between when i do exacly 30fps (and NOT 29.97 :)....however i have to figure out stll whether the results aer better than with 23.976 fps. Reason is a) File size...and b) the fact that twice as much fps via MP means that half of the movie is 'artifcial' while with 24fps only ~60% of frames are added by MP.

mustardman

2nd March 2004, 10:20

More getting to the issue of wheter MP does calculate a frame exactly in the middle (15 -> 30) or whether it calculates a frame that is "closer" to the preceeding one or the following one. Could give funny looking results? (Again, not used it for that, so I don't know).

If you are going to NTSC video (DVD, videotape, VCD, etc) go to 30fps for the extra little bit of space. Hard drives are pretty cheap these days!

WarpEnterprises

2nd March 2004, 11:02

MP leaves the original frame only when it is EXACTLY at the same point in time.
ALL other frames are reconstructed using an interpolation algo (based on motion vectors).

So if you have 15fps and go to 25 fps you get:

Source:
1s - 2s - 3s - 4s - 5s - 6s - 7s

Result:

1=1s - 2 - 3 - 4 - 5 - 6=4s - 7 - 8 - 9 - 10 - 11=7s

From 6 resultframes only 1 is an untouched source frame !

khmann

18th April 2005, 05:00

yea, so this is pretty ugly and not recently tested, but it used to work for me. When capturing DV, I use a utility that automatically breaks all the scenes into separate files. Then I drop the individual .AVIs onto this batch file and it makes a virtual dub job so that I can do all my motion-analysis in one place.

you shold probably change the paths, and modify the line "echo VirtualDub.video.filters.instance[0].Config" to the set|of|pipe|options apropriate to how you want the pass 1 done.

@echo off
set basename=%0
set path=c:\khmann\bin;%path%
set chain=%temp%\KHJPGTMP.BAT
set chain=d:\dvproc\joblist.jobs
del %chain%

rem cd /d %temp%

echo VDMod auto deshaker. Drag and drop .JPG files here
echo.

:loop
if !%1!==!! goto done

echo %1

echo // $job "autojob" >>%chain%
echo // $input "%1" >>%chain%
type d:\dvproc\_prcbdy1.txt >> %chain%
echo VirtualDub.Open("%1","",0); >>%chain%
type d:\dvproc\_prcbdy2.txt >> %chain%
echo VirtualDub.video.filters.instance[0].Config("4|2|24|4|1.21519|4|1.21519|4|720|480|0|1|1|900|600|0|0|1|1|0|2|1|25|350|30|%1.ds|1|0|48|48|24|24|0|0|0|0|0|0|0|1|5|4|5|5|1|1|30|15|1"); >> %chain%
type d:\dvproc\_prcbdy3.txt >> %chain%
echo VirtualDub.SaveAVI("D:\\test.avi"); >> %chain%
echo VirtualDub.Close(); >>%chain%
echo. >>%chain%
echo // $endjob >>%chain%
echo //-------------------------------------------------- >>%chain%

echo. >> %chain%

shift
goto loop

:chain

:done
echo.
@echo on
@if exist %chain% %chain%
@echo off

pause

some other text is included from these files:

_prcbdy1.txt:

// $output "d:\test.avi"
// $state 0
// $start_time 0 0
// $end_time 0 0
// $script

_prcbdy2.txt:

VirtualDub.RemoveInputStreams();
VirtualDub.stream[0].SetSource(0x73647561,0);
VirtualDub.stream[0].DeleteComments(1);
VirtualDub.stream[0].AdjustChapters(1);
VirtualDub.stream[0].SetMode(0);
VirtualDub.stream[0].SetInterleave(1,500,1,0,0);
VirtualDub.stream[0].SetClipMode(1,1);
VirtualDub.stream[0].SetConversion(0,0,0,0,0);
VirtualDub.stream[0].SetVolume();
VirtualDub.stream[0].SetCompression();
VirtualDub.stream[0].EnableFilterGraph(0);
VirtualDub.stream[0].filters.Clear();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x75796668,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v1.6");

_prcbdy3.txt:

VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(0,57);

not exactly the cleanest solution, but it has served me well.

Beave

17th January 2006, 00:59

I have a 720p hdtv source that shakes exactly by one line/pixel up or down. You can easily see this at the bottom edge in the small black part there.
How can I set it up so it only corrects this minimal shaking. Should I maybe rather look into some other filter instead?

WarpEnterprises

17th January 2006, 09:40

how often does it shake? every frame?

Beave

17th January 2006, 12:59

no, every few frames and sometimes every frame. It's irregular or at least I couldn't find a pattern yet.

WarpEnterprises

3rd August 2006, 20:42

(1) new version 1.9

(2) still some problems when using with AviSynth - look here (http://forum.doom9.org/showthread.php?p=859016#post859016)

WarpEnterprises

6th August 2006, 14:03

@guth: the percentage of discarded blocks is a quite important number, yet it can't be seen anywhere. Maybe it could be displayed in the pass1-output along with the other values?

guth

6th August 2006, 16:57

This is a little complicated. :)
There are three types of blocks:

A. Ok blocks. Have white motion vectors.
B. Blocks discarded due to bad motion vectors. Have red vectors.
C. Blocks discarded due to bad match. Have no motion vectors.

Which percentage do you want to see? The deep analysis kick-in percentage is compared to the ratio B / (A + B) and the frame discard limit is compared to the ratio A / (A + B + C). I think it makes sense to do that but it's somewhat confusing I guess, especially since it's not documented. :)

WarpEnterprises

7th August 2006, 07:07

since it's not documented ... well, now it is :)
Maybe simply print both?

I used the "skip frame" setting to adjust the sensitivity for big objects moving in the scene. Is this the right setting to tune? (when NOT using deep analysis)

Just another question: if I use scale=quarter does this imply that the found motion is only accurate to +/- 4 pixels (which seems not true) or how else can this value be understood?

guth

8th August 2006, 18:29

Hmm... I'm not sure I want to print both. We'll see.

The "skip frame" setting is basically just there because if very few motion vectors were good, Deshaker probably got everything wrong so it's probably best to skip the frame completely. There are many ways to deal with big moving objects depending on how big they are, how fast they move and for how long they're in the picture. Skipping the frame completely should only be done as a last resort, or if you're lazy :)

Quarter scale indeed means that the matching occurs on images that are reduced to 1/4 of the original width and height, but the accuracy is usually better than 4 pixels anyway for two reasons:

1. I sort of use sub pixel accuracy when calculating the best shift for a block. (I do this by fitting a 2nd degree function through the best shift and its two integer neighbors, and their "match probability values", and take the shift where this function has its max.)

2. Any impreciseness in the block vectors will be evened out when I combine them all for the total motion of the frame.

That said, I don't really recommend using quarter scale anyway. Personally I always use full scale.

guth

12th August 2006, 13:39

New version of Deshaker available for anyone who cares. Get it here (http://www.guthspot.se/video/deshaker.htm).

smok3

29th January 2007, 16:41

trying to batch version 2.0 via vdub.exe, but some (most) of the settings for 2nd pass get lost somewhere, should those be in job scripts?

for example, this seems to be 2nd pass
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|1|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15");

guth

29th January 2007, 18:15

I'm afraid I don't really understand what you mean. That scripts runs pass 1. Could you give me some more details? (You can email me if you want.)

smok3

30th January 2007, 09:00

i did a little batch (which is called from total commander):
@ECHO OFF
for /f "delims=+++++" %%i in (%1) DO vdub /i "t:\path\vdubdeshake1.script" "%%i" "%%~pi%%~ni_deshaked.avi" ¦ vdub /i "t:\path\vdubdeshake2.script" "%%i" "%%~pi%%~ni_deshaked.avi"

it depends on this 2 vdub scripts:
vdubdeshake1.script
VirtualDub.Open(VirtualDub.params[0]);
VirtualDub.audio.SetSource(1);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(0);
VirtualDub.video.SetOutputFormat(7);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|1|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|1|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15");
VirtualDub.audio.filters.Clear();
VirtualDub.project.ClearTextInfo();
VirtualDub.SaveAVI(VirtualDub.params[1]);
VirtualDub.audio.SetSource(1);
vdubdeshake2.script
VirtualDub.Open(VirtualDub.params[0]);
VirtualDub.audio.SetSource(1);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(0);
VirtualDub.video.SetOutputFormat(7);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetRange(0,0);
VirtualDub.video.SetCompression(0x64737664,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("Deshaker v2.0");
VirtualDub.video.filters.instance[0].Config("9|2|30|4|1.09402|1|1|0|640|480|1|2|1|400|400|400|1500|4|1|3|2|5|40|300|4|C:\\deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|0.7|15|15|5|1|1|1|15|15|0|0|0|0|1|1|0|10|1|15");
VirtualDub.video.filters.Add("sharpen");
VirtualDub.video.filters.instance[1].Config(5);
VirtualDub.audio.filters.Clear();
VirtualDub.project.ClearTextInfo();
VirtualDub.SaveAVI(VirtualDub.params[1]);
VirtualDub.audio.SetSource(1);
VirtualDub.Close();

but i dont get things like:
- use previous and future frames...
- edge compensation mode...
- ect...
(everything seems default)

EDIT: for some unknown reason things are working now...., weird.

------------

other than that, how would i generate a unique name for deshaker log? (like input_filename_deshaker.log) - using this method? EDIT2: solved

guth

30th January 2007, 20:28

Weird, but good :)

About unique log files, I don't know. I'm no script guru at all.
But I wouldn't really recommend running deshaker in batch mode at all. The results will rarely be perfect unless you do some supervision.

smok3

30th January 2007, 21:14

yes, the idea is to do some short segments using gui, then save those to this 2 scripts and run batch only after that. About unique log name, on 2nd thought it is almost not important since a lot seems to depend on 1st pass anyway, but i will do some renaming in original .bat like;

rename c:\deshaker.log to InputFilepath/InputFilename_deshaker.log
(at the end of the script)

texmex99

30th March 2007, 23:08

Just discovered Deshaker and it worked miracles on some handheld ski footage (bit hard to hold the camera still while skiing downhill). I am, however, having problems with another clip that I was hoping to "stabilize". This is a shot from a "fixed" camera (sitting on a deck rail - no tripod available) of the moon rising over the mountains. The clip is 15 minutes long, but will be compressed to about 7 seconds for "effect". Unfortunately, the camera was on the deck, and so was the beer, and whenever someone stepped out on the deck for a beer, the camera shook ever so slightly. I figured Deshaker would have no problems with this clip, but I can't figure out how to adjust the settings to make it work. The frame is mostly black, other than a few lights in fixed positions, car headlights moving through the frame, and then the moon rising. I assume that the easiest way for Deshaker to do its magic is to have it lock on to one of the "fixed" features in the frame, but I can't figure out how to tell it what to lock on to. I've tried isolating several parts of the frame, but it won't "lock on" - I guess it assumes that the black is background and the lights are "moving objects". Anyway, any suggestions from the pros here on how to deshake this scene? I've attached a frame with the "fixed" lights circled - those are the features I need to "lock on" to (i.e. they shouldn't move). Any suggestions would be appreciated! Thanks!

guth

31st March 2007, 08:00

If you'd like to send me a short problematic sample clip, I'll have a look at it.

texmex99

4th April 2007, 11:30

If you'd like to send me a short problematic sample clip, I'll have a look at it.
That would be great - only I'm on vacation at the moment. I'll post a clip as soon as I get back. Thanks so much for this great tool!

wonkey_monkey

4th April 2007, 11:38

I second that - deshaker is fantastic. I do have one question though...

In the documentation, it says:

The motion smoothness calculation is...based on frame count

But for the default of 400 frames, the video doesn't come out as smooth as I would expect. Is it actually measured in frames, or is it a %age (so 400% would be 4 frames)?

David

Terka

9th April 2007, 10:56

when a motion is fast (+long shutter speed) -the deshaked picture is blurred. is it able to add some motion deblur to this pictures? (like focus magic)

wonkey_monkey

11th April 2007, 15:35

Guth, are you still working on deshaker? In addition to my previous question, is there any chance that a future version might include perspective correction? :)

I ask because some stabilised videos suffer from a slight warping if the input is shakey enough (as my time lapse videos can be, and I've seen the same effect on normal video). If deshaker asked for the FOV of the images, it could correct in a more realistic way, rotating the input images in three dimensions rather than just shifting horizontally and vertically.

David

WarpEnterprises

22nd May 2007, 12:55

I just saw that there is a new Version 2.2 (http://www.guthspot.se/video/deshaker.htm).

@guth: I have two questions:
(1) You wrote "Improved handling of new scenes ... allows pass 2 to take appropriate action".
Would you mind giving some details what this action is? Is there a forced "jump" in the resulting deshaking shifts or the like?

(2) regarding the new "Remember discarded areas" option: how is this related to the "blocks OK" percentage?

guth

26th May 2007, 13:29

Sorry for not replying. I didn't get any notifications of any activity in this thread.

But for the default of 400 frames, the video doesn't come out as smooth as I would expect. Is it actually measured in frames, or is it a %age (so 400% would be 4 frames)?

It's based on frame count. It doesn't mean that it equals the number of frames. What I mean is just that if you have video with more fps (as interlaced video in a way has), you should increase the value to get the same result. If 400 isn't smooth enough for you, just increase it. :)

when a motion is fast (+long shutter speed) -the deshaked picture is blurred. is it able to add some motion deblur to this pictures? (like focus magic)

Focus magic looks pretty cool, but it doesn't seem to be able to correct motion blur automatically, you have to manually measure the blur direction and length. That's hard to do automatically. And I'm sure the actual correction isn't easy to do either :). I doubt it will happen.

...is there any chance that a future version might include perspective correction?

Hmmmmm..... I'll have to think about that one.... real hard... :)

(1) You wrote "Improved handling of new scenes ... allows pass 2 to take appropriate action".
Would you mind giving some details what this action is? Is there a forced "jump" in the resulting deshaking shifts or the like?

It does this:
1. The end of scene 1 and beginning of scene 2 are basically placed at "zero correction". And there's no smoothing over the scene cut, so if scene 1 ends by panning to the right, scene 2 can start by panning to the left, abruptly.
2. It doesn't attempt to use frames from another scene when using previous and future frames to fill in the border.

(2) regarding the new "Remember discarded areas" option: how is this related to the "blocks OK" percentage?

I'm not sure I understand your question. Basically, what I do is use the pattern of good/bad blocks for one frame as a starting point for the next frame. This usually results in that it is able to continue locking to the background when large objects enter the scene.

Terka

28th May 2007, 12:42

yes to remove the motion blur - the amount of motion is needed to know. and deshaker has this information-direction and amount. thats why im asking about removing motion blur.

guth

28th May 2007, 17:52

It's not quite that simple, I'm afraid. The motion blur occurs *within* each frame not *between* them. Matching between two motion-blurry frames will be extremely unreliable so you wouldn't be able to get any good information about direction, and specially not about the amount of motion blur, since it can be much different from the amount *between* the frames (the shutter speed doesn't always equal the "frame speed").

Fjord

7th June 2007, 12:23

Gunnar, I have a request for the Use previous and future frames to fill in borders option. Could you add an option to start pass2 output immediately without waiting for Future frames to be read, and to continue until all frames of the input file are output?

It would make life alot easier if the output file frames matched the input frames exactly, with audio in sync. I know how to resync the audio via VDub's audio interlace dialog, but having the frames shifted 30 frames later in the AVI isn't too handy.

Could you process the first n (30) input frames with the number of future/previous frames starting at 0, and incrementing to n (30) by input frame 30. By that I mean to process input frame 1 (the second frame of the input) as if the user had specified only 1 previous/future frames. At the third frame you would have 2 previous frames in the buffer, and use only those to fill borders.

Similarly, to decrement the number of future frames at the end, to enable all of the input frames to be processed, although with fewer previous/future frames for the last few frames.

I usually have at least a second of leader and tail on my videos, so I wouldn't be bothered by degraded border fill in the first few and last few frames. But having the Deshake'd video a frame-for-frame match with the original would definitely make some editing steps easier, with respect to original timecode for example.

Does that make sense to you?

Edit: Perhaps the easiest programming solution would be to initialize the the entire previous frames buffer with copies of the first input frame, and at the end to append copies of the last input frame to the future frames buffer, until the last input frame is finally output.

Fjord

7th June 2007, 14:15

I have a Canon S3 IS digicam, which records 640x480 30.000 fps progressive MJPEG video. When I Deshake these video clips, with 30 previous/future fill frames, the text message in the first output frames says "Deshaker: output is delayed 30 frames to collect future frames; Audio should be delayed 999 ms to maintain audio/video sync". I understand that message, but it should say 1000 ms rather than 999, since the input video is 30.000 fps. 999 would be right if the video was 29.97 fps. If I use a delay of 999 ms, my audio starts 45 samples before the video frame. 45/44100 = 0.001 = 1 ms. If I use a delay of 1000 ms I get perfect sync.

Does Deshaker internally use an NTSC framerate of 29.97 even if the input clip is 30.000? (VDub's file information dialog does show correctly "640x480, 30.000 fps (33333 us)" for the input file.

Terka

7th June 2007, 14:28

Guth thanks for reply

wonkey_monkey

7th June 2007, 16:02

At the third frame you would have 2 previous frames in the buffer, and use only those to fill borders.

You still wouldn't have any future frames - Deshaker can't use future frames until VirtualDub has played them, so the choice is either to delay the output, or not use any future frames.

The only solution would be to store all frames during pass 1 for use during pass 2.

David

guth

7th June 2007, 16:53

Fjord, about your first question David is absolutely correct. The video simply has to be delayed. Otherwise Deshaker won't be able to access the future frames.

As for your second question, you are correct. It should say 1000, but due to a rounding error it says 999. I'll fix it in the next release. I doubt many people will be able to detect a 1ms out of sync, though. (Btw, 29.97 will actually give you 1001ms, not 999.)

Fjord

7th June 2007, 21:47

Thanks David and Gunnar, now I understand the problem with future frames. I guess I'll have to figure out a script to pad the input video with 30 frames, and then to delete the first 30 "delay" frames from the output file. Has anybody worked out a script for that?

As for the 999 vs 1000....you're absolutely right. Nobody could hear a 1 ms lag. I frequently synchronize clips by matching waveforms on the timeline, so I notice it by eye.

By the way, Deshaker is fantastic. You're fantastic, Guth.

smok3

8th June 2007, 13:11

I'll have to figure out a script to pad the input video with 30 frames, and then to delete the first 30 "delay" frames from the output file.
both can be done with avisynth (to pad, check 'BlankClip', to trim check 'Trim' :))

wonkey_monkey

11th June 2007, 19:17

Gunnar, have you thought any more about adding true 3D correction to Deshaker? I have a sequence of shots that would look great, except I used a wide angle lens.

I've done similar stuff in the past if you think you might need any assistance :D

David

PS Have you any idea why Deshaker would have such a problem with these two images?

http://img14.imgspot.com/u/07/161/15/deshakerfails.jpg

Even if I force it to accept all blocks, or increase the search, Deshaker draws tails in the wrong direction. This pair of images doesn't show much more more movement than any other pair in the sequence.

David

guth

25th June 2007, 21:27

Yeah, I thought about it for two minutes, until my head exploded. :)
I'm afraid it would mean a lot of hard work. More than I'm up for atm. But we'll see...

As for your two images, I can't see them.

wonkey_monkey

25th June 2007, 22:51

Yeah, I thought about it for two minutes, until my head exploded. :)
I'm afraid it would mean a lot of hard work. More than I'm up for atm. But we'll see...

Fair enough :) I might give it a try myself, though I've only just figured out how to write AviSynth filters.

As for your two images, I can't see them.

Yeah, they kept getting deleted from the image hosts, presumably cos they were too popular... oh well. I'll dig them out again and upload them somewhere safe:

http://horman.net/trouble1.jpg
+
http://horman.net/trouble2.jpg
=
http://horman.net/deshaker.jpg

The originals are 4x larger in both dimensions, but the scale doesn't seem to matter as Deshaker still fails badly. I can manually rotate the first image to be closer so Deshaker succeeds, but I've seen it match much worse pairs before without any problem.

David

guth

26th June 2007, 16:31

I've never seen this behaviour before. For some reason deshaker finds a completely incorrect initial match and can't "get out of it". But if you limit the initial search range to 5% or so, it works.

wonkey_monkey

26th June 2007, 17:48

I've never seen this behaviour before. For some reason deshaker finds a completely incorrect initial match and can't "get out of it". But if you limit the initial search range to 5% or so, it works.

Phew, good to know I'm not just going crazy.

Guth, would you mind sharing the "secret" of how Deshaker smooths the pan/rotate/zoom variables it records in the log? From what I can see, they are all relative, i.e., they measure the difference between one frame and the previous frame. Do you translate these to "absolute" rotations/translations and average them over time, or am I missing a nicer way of doing it?

I'm having a tough time visualising the process in my head (or on a bit of paper).

David

WarpEnterprises

27th June 2007, 07:04

guth described it here in this post: http://forum.doom9.org/showthread.php?p=448933#post448933

guth

27th June 2007, 17:51

Indeed, but I'm not sure that post answered everything you asked.
You are correct in that I translate the values to "absolute" values, simply by adding them together (for zoom I add the logarithm of the zoom factors). And then I smooth that curve as described in my other post. Then I "move" each frame according to the difference between the smooth curve and the original curve. Simple as that... :)

wonkey_monkey

27th June 2007, 17:59

Thanks guth - just what I wanted to know! Your earlier post make it sound a little more complex...

David

wonkey_monkey

3rd July 2007, 14:07

Here's a gif showing the kind of distortion that can be fixed by using 3D correction (Deshaker on the left, my AVS plugin on the right - the difference is subtle):

http://www.channeltv.co.uk/deshakers.gif

The plugin's still in development. It relies on Deshaker's log file and won't be anywhere near as comprehensive, but it solves my problem :)

guth, a while ago I ran Deshaker on almost 40 minutes of video. As I was still experimenting, I set very high motion smoothness parameters, and the calculations at the start of pass 2 took an extremely long time, so I assume that the speed is dependent on the amount of smoothing, as well as the number of frames in the clip. How do you smooth the variables?

David

guth

3rd July 2007, 14:32

Nice, it still shakes a lot but the perspective distortion seems to be a lot better. To make it perfect I guess you have to do something about pass 1 too. It can only take panning, rotation and zoom into account today, so it will get confused by perspectives.

The smoothing process is described in the post that warp linked to above. The equation systems take longer time to solve when the smoothness is high, because each frame will then have a bigger impact on frames far away in the past or future. And since the equation systems are solved numerically, it takes a while for these low frequency changes to propagate over the equations, so to speak. Also, if you use max. correction limits, they can be very time consuming.

wonkey_monkey

3rd July 2007, 16:12

Nice, it still shakes a lot but the perspective distortion seems to be a lot better.

Yes, there is some trickiness to be solved when it comes to rotating first around one axis, then around another. I might be doing the z-axis rotation in the wrong place too...

Some residual shakiness is to be expected anyway, as the frames weren't taken from the same position (I was walking along the top of a damn with my finger held on the camera's shutter release).

The smoothing process is described in the post that warp linked to above. The equation systems take longer time to solve when the smoothness is high,

I was a little confused by that first post, and the mention of equation systems... but after your recent post, I just did this (taking rotation as an example):

* Add each rotation from the logfile sequentially so that it becomes an "absolute" figure for that frame.

* Smooth these absolute numbers out - essentially a blur. (a sequence of 0,1,1,0 might become 0.333,0.666,0.666,0.333 for example)

* Use the difference between the smoothed and absolute figures and translate the images.

But Deshaker does something a lot more complex, by the sound of things...? I get similar results though.

At the moment my "blurring" process is just a simple average over a specified number of frames, but applied multiple times (even just 3) this can approximate a Gaussian, err, thing (I want to say blur). This can be performed in a time that is only dependent on the number of frames, completely independent of the amount of smoothness. But I haven't considered how the maximum correction limits would mess things up, since I've never used them :D

David

guth

3rd July 2007, 17:21

My algorithm might sound a little complex, but it doesn't require a lot of coding. Blurring probably works pretty well too. One difference is that I try to minimize the *squared* correction amounts in order to minimize the worst cases, whereas your approach seems more linear. And you don't really have much control over the process, for example to fix max. corrections. But that might be fixable, I don't know. Most importantly, though, my approach just feels more mathematically correct to me. :)
Or maybe I'm just jealous I didn't come up with such an easy solution. :)

Fizick

3rd July 2007, 21:51

I know also about mathematically correct Kalman filtering, but I did not implement it in DePan....
:)

Undead Sega

27th February 2008, 23:10

http://www.channeltv.co.uk/deshakers.gif

why do these distortions happen?

and what do you call them also?

mustardman

28th February 2008, 00:39

My guess would be the effect is caused by the distortions present at the edges of the lens. The distortion becomes worse the further away from the center of the image the 'target' is.
For a steady lens, this is of course not noticeable. However, if the lens moves around, and then the image is stabilised later, the distortion becomes very obvious (as you can see).
Think 'pincusion' effect with wide angle still (and video) cameras.

Is is particulary bad with cheaper lens, and is even present with more expensive lens. If you really don't want it, buy a 'hollywood' movie camera lens... but be prepared to walk away 20k poorer!

MM

PS: One way to minimise this effect with a "cheap" lens is to zoom in to the subject. Less of the glass is used, and the area in use is closer to the center. Only problem, if the camera is shaking to begin with, zooming in is going to make it a whole lot worse!

scharfis_brain

28th February 2008, 00:50

AFAIK there was a solution for AVISynth to undo the distortions of wide angle lenses...
But I don't know its name anymore...

EDIT: http://forum.doom9.org/showthread.php?t=127432&highlight=barrel+distortion

One need to undo the distortion before deshaking.
If the fish-eye look is desired one may re-apply the distortion after deshaking again.

Undead Sega

28th February 2008, 01:18

well i remember using 2d3 SteadyMove and i get these weird distortions on my old Hi8 camcorder with just the stock lens with the settings are ranked up high to get the best stablized video.

guth

28th February 2008, 19:08

Actually, this effect doesn't come from the fact that wide angle lenses often are imperfect, but rather from the nature of wide angles.

Most wide angle lenses are usually either rectilinear or fisheye. I guess it's pretty obvious that fisheye lenses will give strange results when deshaking, but the fact is that even perfectly rectilinear lenses will look a bit weird (which that sequence above demonstrates). Here's my attempt at explaining why, in a hopefully easy to understand way... :)

A rectilinear lens will make straight lines in the world appear straight on the image too. Now, imagine you're taking a close wide-angle picture of a chessboard. All squares will get the same size on the image even though the outer squares are further from the camera than the squares in the middle. This essentially means that objects are magnified when they are close to the edge. When you turn the camera to point at an outer square it will get smaller on the image.

And deshaker doesn't like it when objects change size when you pan the camera... :mad:

Undead Sega

28th February 2008, 21:56

And deshaker doesn't like it when objects change size when you pan the camera...

i see, is this something you are going to or are wanting to work on?

guth

28th February 2008, 22:44

I have considered it, but to be honest I don't really think it will happen. Not anytime soon anyway. Sorry...

Undead Sega

28th February 2008, 23:18

oh, thats a real shame, it hasnt received an update for a very long time now, i do hope u can get to it one day, i will look forward to it.

and, since speaking to the actual creator of the filter :D, about the borders, i assume u have watched the video sample from YUVSoft and its comparisions to yours?

can your filter literally recover the missing edges in the black borders?

scharfis_brain

29th February 2008, 00:58

the video of YUVSoft is a LIE!
they didn't set up deshaker properly just to present their own product as the better one.

2Bdecided

29th February 2008, 13:13

Deshaker is lovely, but how can I speed it up?

I assumed this would have been asked before, but I can't see a list of which options really speed it up or slow it down in the five pages here.

Cheers,
David.

guth

29th February 2008, 21:46

i assume u have watched the video sample from YUVSoft and its comparisions to yours?

can your filter literally recover the missing edges in the black borders?
I hadn't watched that video before. How funny. :)
First they steal the name "deshaker", and now this.

Yes, deshaker can recover missing edges (by taking video data from past and future frames).

Deshaker is lovely, but how can I speed it up?
Thanks! I recommend you search for the words "fast" and "slow" here:
http://www.guthspot.se/video/deshaker.htm

Undead Sega

1st March 2008, 13:40

Deshaker is lovely, but how can I speed it up?

Thanks! I recommend you search for the words "fast" and "slow" here:
http://www.guthspot.se/video/deshaker.htm

i think this is something new you could work on, meaning adding support for Multithread and MultiCore CPUs? that would be so great if it took advantage of it.

and when it uses previous frames to recover missing edges, does it literally look for wat was suppose to be there?

possible to show an example?

scharfis_brain

1st March 2008, 13:42

Undead Sega: would you PLEASE read the authors site carefully before asking for obvious things?

guth

1st March 2008, 17:56

I think I'll have to agree with you, scharfis. :rolleyes:

It already does multithreading and as for recovering missing edges, please read my site and/or do some testing yourself.

2Bdecided

3rd March 2008, 14:30

Thanks! I recommend you search for the words "fast" and "slow" here:
http://www.guthspot.se/video/deshaker.htmThank you.

I have read it (the web page), and used it (deshaker!), but still have a "kind of" question about edges. I'm not sure how to phrase this. Obviously everything in the frame is processed with motion estimation and compensation. However, it seems that the areas out of frame are not. Here's an example of what I'm trying to say...

Let's say the area at the top of the frame is first revealed in original frame 100, but was needed for compensated frames 80-99 too. During those frames, it looks like the static top of frame 100 is copied to the top of frame 80-99. Let's say the existing top of frames 80-99 is moving left to right. It looks like there's no attempt to make the part pasted from frame 100 move left to right - it just sits there.

Now I know you don't actually know how it should be moving, because it's not there in those frames(!) - however, would it not be reasonable to guess that if the parts near it are moving left to right, then the part copied from frame 100 should move left to right too? It might not be spot on, but it might be closer to reality that not moving it. I guess it comes with its own complications...!

Not a criticism (because deshaker is amazing) - just a question/suggestion.

Cheers,
David.

scharfis_brain

3rd March 2008, 15:43

deshaker 'only' applies global motion compensation to the frames. This means that only the whole frame is being moved.
This is also done with previous and future frames.
But every movement that does not equal to the whole frames movement - I call it differential movement - will most likely produce some weird artifacts at the compensated borders.
Of course, one could write algorithms that will do local motion compensation on the frames that are being used form border filling. but that would slow down the process even more.

guth

3rd March 2008, 19:18

Exactly. Thanks scharfis!

This usually means that the background will be moved correctly when taken from another frame, but if there are objects moving in the borders, they tend to look a little weird.

And btw, I have absolutely no problem with questions/suggestions or even criticism. It's actually even slightly ok to ask things that's already on my web page.
:readguid::readguid::readguid: ;)

Undead Sega

4th March 2008, 00:51

would this be a good example of what your talking about?

guth

5th March 2008, 07:58

would this be a good example of what your talking about?
Yes. But you don't seem to be using "soft borders". You might want to try that.

Undead Sega

6th March 2008, 13:40

also, here is the actual video of using the Deshaker program.

it includes a before and after processing:

http://www.wikiupload.com/download_page.php?id=35945

P.S. i dont know if anyone has seen such shaky camera work either.

scharfis_brain

6th March 2008, 13:49

The border compensation would have looked much nicer if the video wouldn't be deinterlaced before deshaking.

Use either pure interlaced without modification for deshaking,
or bob-deinterlace the video to 50fps before deinterlacing.

The sample you linked here was deinterlaced to 25fps thus discarding 50% of valid image/motion data!

Undead Sega

6th March 2008, 13:57

actually, i ddi deinterlace it to 50fps before deshaking it, cause i thought it would use more frames to help compensate for the black borders.

afterwards, i then encoded it to an MPEG 2 at 25fps, which is what i have provided.

scharfis_brain

6th March 2008, 14:36

if you deinterlace before deshaking you should try nnedi() for the interpolation.
You'll get rid of all the stairstepping.

could you upload a non-deinterlaced source, please?

Undead Sega

6th March 2008, 19:57

well, the oringal source was recorded from a miniDVD camcorder at the highest quality possible (even though it still looked crap and etc.) and still at 50i, how bizare:

http://www.wikiupload.com/download_page.php?id=35962

i tried to deinterlace it with MCBob at 50p but th odd frmes keeps on coming every other frame, thus giving it a very strange stutter.

hope that helps.

scharfis_brain

6th March 2008, 21:54

it is NOT the fault of mcbob()

your MPEG2 File is totally b0rked.
you encoded it in progressive mode and set bottom field first fieldorder. But the video is interlaced Top field first!

This is the reason for:
- the BAD quality
- colour problems
- the 'strange shutter'

So, provide an unprocessed sample, please.
(DV.AVI if possible)

Undead Sega

6th March 2008, 21:56

i see then, this is a first, maybe the settings has changed this time, but i will correct it from the previous user.

anyways, besides that, i posted the stablized video clip example, what was your thoughts on that? before and after wise.

BBugsBunny

6th March 2008, 22:01

Most wide angle lenses are usually either rectilinear or fisheye. I guess it's pretty obvious that fisheye lenses will give strange results when deshaking, but the fact is that even perfectly rectilinear lenses will look a bit weird (which that sequence above demonstrates).
Am I right that deshaking films shot with wide angles lenses would require to measure each specific camcorder lens to get info about the distortion it causes and with such an info it could be deshaked?
For foto shots with digital cameras there are some tools that have measured different lenses and cameras and can do some correction afterwards eg:
http://www.epaperpress.com/ptlens/
They have the lens profiles included in the software.

Another topic: recently Deshaker was tested in the german computer magazine ct. It was compared with other deshakers all of them not free exept deshaker. Guess who got best results? Deshaker! I do not have this ct at home I only was quickly flying over. One problem they stated that deshaker could not correctly handle some caps from HD cameras.

Undead Sega

7th March 2008, 17:38

you encoded it in progressive mode and set bottom field first fieldorder. But the video is interlaced Top field first!

i know this has been brang up beofre, but either case, can this error be amended?

guth

7th March 2008, 18:44

Am I right that deshaking films shot with wide angles lenses would require to measure each specific camcorder lens to get info about the distortion it causes and with such an info it could be deshaked?
No, I'm afraid not. The problem occurs because deshaker moves the content of the frames around. To correct it you would have to know how much deshaker will move it. That info is (sort of) in the logfile, but I don't think the software to fix this exists, except maybe for davidhorman's attempt earlier in this thread. To make it perfect, though, deshaker would have to be modified for pass 1 too.

But to correct perspective distortion, no measuring of the lens would have to be made. I think the FOV angle is all that deshaker would need, and I could probably make it determine the FOV itself.

But to correct additional barrel/pincushion distortion of the lens, you would have to measure it and fix it yourself in advance.

Another topic: recently Deshaker was tested in the german computer magazine ct. It was compared with other deshakers all of them not free exept deshaker. Guess who got best results? Deshaker! I do not have this ct at home I only was quickly flying over. One problem they stated that deshaker could not correctly handle some caps from HD cameras.
I didn't know this. Nice! :)
If anyone knows what caps they are talking about, I would very much like to know.

BBugsBunny

7th March 2008, 19:15

This is the link to the article:
http://www.heise.de/kiosk/archiv/ct/2008/05/164_Software_zum_nachtraeglichen_Stabilisieren_verwackelter_Videos
But it's not a free download :( , it costs 0,70€

Guth, if you ask the authors I'm sure they will tell you what caps they had problems with.
You can find the general contact e-mail adress here: http://www.heise.de/ct/kontakt/

guth

7th March 2008, 19:21

Thanks! I guess it's in German? My German could be better... :)

BBugsBunny

7th March 2008, 19:27

Yes the article is in german, but if you contact the authors in english it should be no problem.

Adub

7th March 2008, 20:13

Holy Crap! That filter is godly! I know there were a few "tearing" artifacts, but the overall effect was stunning!

Undead Sega

7th March 2008, 20:27

i take it u saw my video example?

@scharfis_brain, i have adjusted my settings and i still keep getting the same results.

this is wen i make an AVI, i set the encode to Interlace, including the video source to Interlace and yet i still get a Progressive video with interlacing artifacts.

what do i do to make a lossless Interlaced AVI from a .VRO (DVD-RAM file)?

Adub

8th March 2008, 04:45

i take it u saw my video example?

Indeed.

guth

8th March 2008, 08:34

For some reason I can't see your video clips, Undead Sega. IE says it can't find the file, and I usually see lots of broken links on that page.

Undead Sega

8th March 2008, 14:05

oh no! its gone! :(

man, that another uploading again, unless it is the site's fault.

the only person i know here who has downloaded are: scharfis_brain and Merlin7777.

Undead Sega

9th March 2008, 20:10

this is wen i make an AVI, i set the encode to Interlace, including the video source to Interlace and yet i still get a Progressive video with interlacing artifacts.

what do i do to make a lossless Interlaced AVI from a .VRO (DVD-RAM file)?

can anyone help me with this one before i continue?

scharfis_brain

11th March 2008, 16:37

@guth: can you add an option to the fist pass?
I need to ignore all vectors that appear static (or almost static) to deshaker.
Imagine a bird in the blue sky.
the bird covers approx. 10 to 15% of the image area. (high zoom)
but deshaker discards all motion vectors of the shaking bird, cause it thinks the 'static' sky is the reference.
So effectively nothing is getting stabilized.

Maybe you could add options to
a) ignore nearly flat areas
b) ignore vectors that move less than xx pixels.

guth

11th March 2008, 17:50

Yes, that can be a problem. It happens because of built-in temporal smoothing in the camcorder. It makes noise look similar between the frames and causes deshaker to match on the noise. The noise is most apparent in the highest scale, so try using a lower (less precise) scale. Also, try increasing either (or both) of the first "discard motion of blocks that..."-values. And if nothing else helps, use "ignore pixels outside" to only match on the bird.
You could also try adding some nontemporal smoother before deshaker (pass 1 only) that removes the tiny noise.

WarpEnterprises

11th March 2008, 17:55

Can't you use "ignore outside", thereby telling DS to only use the bird's image (I suppose the bird is approx. in the middle of the image)?
(edit: guth was faster...)

scharfis_brain

11th March 2008, 18:12

actually this was only an example.

I have a highly compressed web video, I want to deshake.
There is no noise appearant. Also I smmothed out possible blocking before applying deshaker.
Ignoring outside is no option either, cause it shakes too much.

guth

11th March 2008, 18:24

Highly compressed video has almost the same problem. The encoder tends to leave blocks unchanged from the previous frame a lot, to increase compression. So deshaker will match on the blocks because of that.
Try increasing the 2nd "discard motion..." settings. Its only purpose is to discard blocks that get a good match value for many different shifts, such as a blue sky. The similarities to previous frame can still be a problem though...

mustardman

12th March 2008, 05:35

@sharfis_brain : Your suggestion of ignoring vectors less than a certain size is very similar to my request (which guth implemented over a year ago, but only fairly recently included in the release code). This was to ignore all vectors that had more than a certain size...

My problem was the opposite (as you could guess). The camera was mounted on a tripod pointing out the side window of a car. The vibration of the camera was small, but got extremely annoying after less than 5 minutes of viewing. Deshaker did a marvellous (truly marvellous in fact) job of stabilising the image. Nearly all the video was of the landscape (horizon), so deshaker had good data to work with. Problem was, every so often a tree would race past, deshaker would lock onto it (because it filled a lot of the frame), and the frame would very badly jerk to follow the tree, before returning to nice stable video!

The "ignore vectors greater than" solved this problem completely! Many thanks to guth!!!

I know what you mean about deshaker finding "random" motion vectors in a non-moving, but constant colour, regions of the image. It has been a long time since I used deshaker, has there been any movement in regard to addressing this issue?

Cheers all,
MM

edit: PS. At the risk of making deshaker even more complicated, perhaps an option similar to ignoring dark pixels, ignoring blocks that are matched with a threshold less than a certain amount? Like I said, I havent used it for a while, this may already exist...

guth

12th March 2008, 18:52

... ignoring blocks that are matched with a threshold less than a certain amount?...
Do you mean something like "discard motion of blocks that have match value < X"?
It has been there since version 1.0.

BBugsBunny

12th March 2008, 21:59

I solved some of these "blue sky" deshaking issues by adding my gradation curves filter for the first pass (position in filterchain before deshaker) and drawing a filter curve that simply blacked out these undesired color values, so deshaker discards them as "black". This method will not help always but somtimes can give better results with difficult footage.

scharfis_brain

12th March 2008, 22:35

hmm, that's a nice idea.

but I would prefer an option to drop (near to) zero length vectors, though ;-)

mustardman

13th March 2008, 09:04

@guth : sorry, didn't even think! doh :(

AnnaFan777

30th April 2008, 16:36

deleted

AnnaFan777

1st May 2008, 00:25

deleted

scharfis_brain

1st May 2008, 02:56

it is always a very good option to RTMF(!).
: If you enter -1 as smoothness you will get "infinite smoothness", meaning the camera will appear to be stationary at the position of the first frame that was processed in pass 1. This feature usually doesn't work perfectly, though.

Guest

6th May 2008, 05:12

@clint999

I've deleted the spam link in your signature twice now, and have PM'ed you about it. If you put it back again, you'll be struck and then banned.

Undead Sega

5th June 2008, 10:41

Hi, im back, and i just wanted to know if this is good practice that i had in mind in theory.

that being is, because the filter analyzes frames to enable to help stablize the video footage, how would it be if the footage was motion interpolated to 2-4 times the frame rate, thus creating more frames for it to analyze and as well to help fill in the black borders?

i have done such thing i admit, but it doesnt seem that steady (maybe because i raised the framerate that i also have to increase the motion smoothess? that is something i have to get back onto), mainly i think because there are motion artifacts from the interpolating and the motion vectors when analzing the footage mistaken them for a motion that leands to a different direction. could this be correct?

any ideas?

guth

5th June 2008, 16:06

Maybe I'm missing something, but I don't really see the point in doing what you're trying to do. Why would Deshaker be helped by adding interpolated frames?

fbs

5th June 2008, 16:22

Isn't deshaker multithread? whenever it's doing the first or second pass("Deshaking...") it only uses 50% of my CPU.. at least this is what task manager says about virtualdub.exe ...

any thoughts?

guth

5th June 2008, 17:09

Deshaker isn't the only thing that's working when you run video through it. There's the encoder/decoder and VirtualDub too, that might not be multi-threaded. And Deshaker isn't 100% multi-threaded. Only the most time consuming parts (except "deep analysis") are multi-threaded. So if you're using very fast settings, you might not see a big difference.
In the Deshaker settings dialog, does it say it has detected more than one processor?
Try using some really slow settings (just for testing), such as "differential search range" = 20. It really should go well above 50% then.

fbs

5th June 2008, 17:35

It says 2 processors..

And I'm on its 2nd pass and it's taking like forever to finish..I'm not even saving a new avi file, I JUST clicked on '2pass' and hit OK and this "Deshaking..." window is almost burning-in my lcd display lol.

I'm using bicubic resampling, no edge compensation, no fillborders and only *vertical* panning smoothing.. The virtualdub.exe process never goes above 50% and there are only 3 threads in it.. and there aren't other processes using more than 0% cpu...

My source is a 720x480 .avi with huffyuv vhs capture.. 1h40m long and I'm on a Athlon x2 with XPSP3..

Options on the right-bottom are:
0 0
8000 0

1 1
5 1

guth

5th June 2008, 18:16

Oh, you've got the "Deshaking..." window up. Right, that part is not multi-threaded. It usually doesn't take that long so I never bothered. But it can take a long time sometimes. Try increasing the max. correction limits to start with. Try putting 100 in all those boxes (and maybe lowering the smoothness values as well). It should be a lot quicker then (but the corrections won't be limited).

fbs

5th June 2008, 18:57

But.. I only want vertical correction.. I'm using it to stop some jumping my VCR produces.. the image is always going like 14 pixels up and then 14 pixel back down again in consecutive frames... the way I set it up, it's working peeeerfectly... if it would try to correct horizontal or zooming panning it would screw up the vid because it is kinda complex :(
I think I'll have to wait, right? lol it's taking like 3h to do the job

guth

5th June 2008, 19:05

Are you trying to accomplish that by using really low values on three of the max. correction limits? If so, no wonder it's taking forever :)
The correct way to only stabilize vertically is to set the other three *smoothness* values to 0.

fbs

5th June 2008, 22:31

I changed it... and 1h has passed since then.. :P still waiting.. hehe what exactly is it doing in this 'deshaking...' stage?

0 0
8000 0
100 100
5 100

Undead Sega

6th June 2008, 03:49

Maybe I'm missing something, but I don't really see the point in doing what you're trying to do. Why would Deshaker be helped by adding interpolated frames?

the interpolated frames are there for a more deeper analyzes on how the camera moves AND the frames help fill the black borders to help give more image of the object than having less frames.

guth

6th June 2008, 07:36

Undead Sega

6th June 2008, 20:22

it becomes harder because it has more frames to scan.

EDIT: nevermind.

fbs

8th June 2008, 18:01

fbs, that sounds very strange. What are those numbers?
In the deshaking stage, it calculates how all of the frames should be transformed (but doesn't actually transform them).
If you set all max. correction limits to 100 and leave all smoothness values below 10000, it should be rather quick. How long is your clip?
If you want, you can send me your settings and logfile (maybe privately) and I'll look at it.

I'm using 0 for all smoothness except vertical which is 8000
I'm using 100 for all max corrections except vertical which is 5..
My vid was 1h40m long...

What I'm trying to do here is just correct some jumps my VCR produces in almost all frames.. look the logfile:
1663B 0.31 -15.40 0.006 0.99989
1664A -0.65 15.50 0.030 0.99973

15 pixels up then 15 down.. it's this way all over the tape..

With these settings I could correct it.. the problem is the speed :P I'll have to do this for like 20 tapes.. and only the 'Deshaking..' stage is taking 2-3 hours...

can you create an option for it to just work with vertical? both at 1st and 2nd pass? ;p I would gladly test it.. I can be a beta tester lol my msn is fbs [at] brasnet.org

guth

8th June 2008, 18:54

fbs

15th June 2008, 17:03

It would probably be a lot faster if you split your clip into a few parts and processed them individually. The time needed for "deshaking..." unfortunately isn't linear with respect to the clip length. I'm still surprised it takes so long, though.
Again, if you want, you could send me the logfile and settings.

I could send it to you.. can you add me on msn? fbs [at] brasnet.org

I'd really like an optimized/upgraded version of your filter

:thanks:

guth

15th June 2008, 17:34

Didn't you get my reply to your private message?
Please email me the files, to gunnart [at] guthspot.se.

scharfis_brain

15th June 2008, 20:46

@Gunnar:

could you add a progress bar to the 'Deshaking...' popup-window?
Or even more: a counter for the estimated remaing time?

It would be very nice to be able to judge how long the calculation process will take...

guth

16th June 2008, 19:04

Without going into details, I'm afraid it's not quite that simple. I don't really know how long it will take.

Undead Sega

20th June 2008, 21:53

@ Gunnar, even though ow great this filter works for me and many people out there, is there any improvements u would want to consider or is there more work that can be done on your filter?

guth

21st June 2008, 07:38

I'm always open for suggestions.
(But I'm a lazy man, so it would have to be something truly awesome! :))

Undead Sega

21st June 2008, 12:20

(But I'm a lazy man, so it would have to be something truly awesome! )

Now that doesnt suprise me, hahahaha, but i praise you for admiting it rather than someone else slowly trying to get it out from you :d

anyways, wat im meaning is, the way it is extrapolating to fill the borders, the current state of it, is there any thing more u can do about it? or did u just got it to a certain stage and thought "this looks good enough" ? Not being rude of course.

guth

21st June 2008, 17:42

If you mean the extrapolation setting specifically, I think it's quite perfect looking, but very slow.
But if you mean using past and future frames to fill the borders, it could probably be improved a bit. It might happen some day, but then again it might not.

Undead Sega

21st June 2008, 18:31

Now, with you saying that it could improve abit, what would that be exactly may i ask? If u dont mind telling me :D

guth

21st June 2008, 19:06

Maybe doing some kind of local motion compensation that scharfis_brain talked about earlier. I don't know.
But it's kind of impossible to know what's outside a frame at the specific time that frame was shot. Guessing by looking at other frames will simply always look a bit strange under some circumstances.

Undead Sega

21st June 2008, 19:56

i see, so if u were to impliment motion compensation, what difference could we probably see? as opposed to filling black borders with previous frames, or even with the combination.

also, i have read somewhere the super resolution is a very good algorithm for extrapolation.

guth

21st June 2008, 20:19

I would obviously still have to take the image data from past and future frames (cause that's where the missing info is).
Seriously, you need to relax a little. I'm most probably not going to implement anything like that. Mostly because I don't think it would look that much better.
As for "super resolution"... believe me, no matter how awesome it is it will never be able to extrapolate a missing head from someone's body. :)

Undead Sega

21st June 2008, 20:25

As for "super resolution"... believe me, no matter how awesome it is it will never be able to extrapolate a missing head from someone's body.

oh im very aware of that, but i am just seeing if this filter can seriously be improved or become any better, its just i cannot find the right words at the moment to pass my exact words on. i know it can.

Dr.D

28th September 2008, 17:30

No doubt deshaker is a great tool, big thanks to the author.
Filling borders done very smart too, great feature.
But, I believe, there are people (like me) who are crazy about any artefacts, what filled borders definitely are, regardless of how smart this done. So, I can sacrifize of resolution, cut off borders and have an artefacts free picture.
Here is the problem: how to know how much I should cut off? Now I do some not easy calculations based on 2nd pass parameters, using even trigonometry :), and still not sure that no artefacts left.
So, the following feature will be very appreciated. For 2nd pass I'd like to see a mode (let say "cut off borders") with only one option - resolution for output file, and let Deshaker choose optimal parameters for that.

guth

29th September 2008, 17:20

Dr.D

30th September 2008, 08:09

If you select the edge compensation "fixed zoom", deshaker will add just enough zoom to completely eliminate the borders.
I have also made an "adaptive+fixed zoom" that starts by adding the usual adaptive zoom, and then adds a smaller fixed zoom to completely eliminate the borders. This feature hasn't been released yet, though.
Right, but it's not what I'm looking for. May be I've explained not good.
1. I don't need zoom, I need crop. Let say 720x480 -> 632x432.
2. This crop should comletely cut off borders. So, I need to be sure that there are no borders in the cropped clip.

Another explanation. Draw imaginery frame 632x432 inside a clip 720x480. Then do deshaking with no edge compensation and no filling borders.
In the output clip I'll see floating black frame. I need that this frame never touch the imaginery internal frame. Of course, with maximum deshaking possible.

Did you see the proDAD Mercalli plugin? There is parameter 'strength' there, you can slide this parameter and visually see what internal rectangle will be untouched. Not exactly, but close to what I'm looking for. I need opposite - set output size and let deshaker calculate strength, or whatever.

JK1974

30th September 2008, 09:37

Hi,

the problem is the mixture between correction limits and the motion smoothness settings. You can set the correction limits to a calculated value, e.g. if you create a border of 16x16 pixels for a PAL video (720x576), the x and y limit normally should not exceed 2%. But: This only counts for heavy movements in x or y direction. So, if just small corrections are necessary, low values like this might be enough.
But: You don´t want sudden jumps again by settings low motion smoothness settings, and then you might jump out of the border you have set above.

For getting this in a right way, I just can recommend to search for a scene with a heavy shift in one of the directions and try to set the settings accordingly - that´s how I did it, using AVISynth for cropping the image and replacing it against black borders.
BTW.: I don´t correct zooms and have a low value for rotation correction. Futhermore, I "just" edited PAL video for a long time where you have an overscan on every TV set, so even a little jump outside the 16x16 border was not visible on TV.

With my new AVCHD with 1920x1080, I have to reconsider what to do as a 32x32 border is quite to big and FullHD TVs don´t seem to have an overscan anymore.
So it would be great to be able to test this new "adaptive+fixed zoom" feature.

guth

30th September 2008, 17:25

Right, but it's not what I'm looking for. May be I've explained not good.
1. I don't need zoom, I need crop. Let say 720x480 -> 632x432.
2. This crop should comletely cut off borders. So, I need to be sure that there are no borders in the cropped clip.

If you use "fixed zoom", you can also adjust the output size so that you get practically no zoom. But it will be hard to get no zoom *at all* this way. Is that really a problem?

Undead Sega

30th March 2009, 19:52

hey, im back, how is everyone?

i have a small problem and question i would like to share and hope if someone here can help me with.

in a few shots i have made, like vox pops, interviews or just a single person on camera that occupies probably more or less 2/3 of the frame, are done handheld and is pretty shaky, therefore of course i used Deshaker, which ahs worked for me in alot of occasions however on this matter, it does stabilze the footage but not what i imagine. Because the background is quite static and the object or person is the one moving, like their body or hand hand gestures etc, it seems that Deshaker is picking the motion from the person to help stabilze the image.

i was wondering if there may be a way around this? i would really appreciate it, as i do not know exactly what to do and dont want to spend too much time experimenting on the footage as we all know the process it takes ages.

WarpEnterprises

30th March 2009, 20:46

Did you try the pass1-option "Ignore pixel Inside"?

Undead Sega

30th March 2009, 21:11

Umm no...i just looked at it now, and i see u have to fill in a few numbers, what does this option do exactly?

Undead Sega

31st March 2009, 14:13

also, how do u work it out? the video has people on either side of the screen, and because of the non moving background, the motion is picked up from the people's movement, therefore in a deshaked video clip, say if a person either moved/nodded their head, or waved their hands around, the camera movement will look like as if it was moving with it.

not to forget to mention, the person would appear on either side of the frame, and not just one side throughout.

does anyone get what i mean by this?

2Bdecided

31st March 2009, 14:26

does anyone get what i mean by this?Yes, you mean you need to buy a tripod. ;)

If there's a part of the frame where there's always background, you can force deshaker to concentrate on that.

Otherwise, I don't have a suggestion - I've not been able to solve this one other than by ignoring the centre of the frame when the problem is in the centre.

Cheers,
David.

2Bdecided

31st March 2009, 14:36

I asked a question a year ago...

http://forum.doom9.org/showthread.php?p=1107877#post1107877

And received a reply in the next two posts, including...

This usually means that the background will be moved correctly when taken from another frame, but if there are objects moving in the borders, they tend to look a little weird.Do I have to enable something somewhere to make this work?

What I'm seeing is that when there's up/down wobble that's being removed, then while the frame contents are moving left > right, the added border contents are not moving left > right. If the border contents are from a future frame, they will sit there, non-moving, until that frame is reached, and then obviously they'll start to move. This means the border contents don't pan, even though the rest of the frame does.

Does this explanation make sense? Is it what is supposed to happen?

Cheers,
David.

guth

31st March 2009, 16:23

does anyone get what i mean by this?
You can change the "ignore pixels" setting when the person moves. You don't have to run pass 1 in one go.

But if there's enough static background (must be static!) to stabilize on, you shouldn't need to use that setting. Try setting "discard motion of blocks that move > X pixels in wrong direction" to something very low, like "0.5". And also, try enabling "deep analysis". Also, make sure "remember discarded areas to next frame" is enabled.

guth

31st March 2009, 16:31

Does this explanation make sense? Is it what is supposed to happen?
No, it's not what's supposed to happen. The copied data of a frame should be moved so that static background (or the area that is matched on) is aligned correctly. If you look at the frame in this post:

http://forum.doom9.org/showthread.php?p=1108164#post1108164

You can see that the copied background is aligned as it should be. (But the moving person is not.)
If you still feel something is not as it should be, please email me an example.

Undead Sega

2nd April 2009, 21:02

it helped a tiny bit, one noticable shot i know was not the same like with the default settings, however it felt it was very much like how it was shot.

to me awareness now, i find Deshaker quite slow for a 10 minute footage, would speed be soemthing of an improvement for later installments? :D

shindou

22nd July 2009, 15:24

I kept wondering about something that was asked here a while ago - about deblurring individual frames to improve the deshaked result.

(My first idea was to use motion estimation for that, but long ago Gunnar mentioned it was too unreliable for the task.)

After a lot of googling and little success, I found this paper, High-quality Motion Deblurring from a Single Image (by Qi Shan, Jiaya Jia and Aseem Agarwala):
http://www.cse.cuhk.edu.hk/~leojia/projects/motion_deblurring/index.html

So in theory, this should be possible. The only problem is that this method (best so far it seems) takes about 40 seconds per frame... And I'm not sure if it is able to handle non-linear blur paths. Any ideas?

mustardman

5th August 2009, 10:47

I read the paper about motion deblurring, and was pretty impressed, but didn't understand much of the maths! I have yet to actually try the executable on some digital photos (shindou - you've done this on video frames?).

I don't quite see how the "non-blind" version would be much good for anything in the real world, only to show to an academic audience that the processing algorithm works. The real impressive work is the ability to go in "blind" and work back to a clear image from there.

@shindou: The paper claims the processor is capable of non-linear blur paths (page 6) - to be tested...

I know exactly what kind of problem you are experiencing, as I have deshaked some footage that was shot with a slow shutter (1/50th sec), and although the actual objects in the frame 'deshaked' to stationary, they exhibited a very weird looking (and entirely expected) variable blur halo around them (stationary objects and moving objects alike)!

My suggestion would be to batch process (overnight at 40 sec per frame!) the footage with this deblur tool (providing clear but shaky video) and then 'Gunnar' deshake from there.

I would be very interested in your results. I used to have the time to play with this stuff (about 4 years ago), but just don't any more :(

Another thing that interests me greatly is tools to perform deblurring that has been caused by focussing problems - especially doing this "blind". Any pointers?

Cheers,
MM

shindou

18th August 2009, 18:02

I haven't touched a single line of source-code, I'm just giving out ideas. :P
I said "per frame" because a frame would take the same time to process an individual image.

Just as you said, that's exactly the "problem" I've been experiencing. I'd call it a "minor issue" though, since deshaked videos are so much better than the originals. I love this plugin, now my videos have actually become intelligible. :D

It only seems to me that automated deshaking + deblurring would bring videos to the best achievable form. The non-technically-inclined user may actually expect deshaking to include deblurring since blur is also an undesired byproduct of shake.

And I agree with you, the process should be totally blind to be useful, more so for video.

I haven't tested the executable either. It may be my ignorance, but I would have done so if I knew any easy, automated way (I'm batch-processing a ton of videos) to extract frames from a video, process them (with this deblurring method or anything else, say ImageMagick (http://www.imagemagick.org)) and put the processed images back into a video stream.

NOTE: I know VirtualDub[Mod] can save a video stream as a sequence of images. Can it load a sequence of images as a video stream though?

I'd gladly try to use this method for video (even if it takes 40 sec per frame) in combination with DeShaker if possible. Like you, if I had the time, I'd work it up myself. Ideally, I think, this method of deblurring would need acceleration (probably on the GPU), otherwise any video, even short ones, would take an unbearably long time. For instance, 60 seconds of video at 30 fps would take a minimum of 20 hours to deblur.

Unfortunately I don't know any method for focal blur removal, I'm not really an expert in image/video processing. :/
The paper may contain references to works specific for this, but implementations tend to appear only years after papers are released. :/
Good luck!

mustardman

19th August 2009, 12:56

Not having actually tried the algorithm, but from the notes in the paper on how their executable runs, I get the impression it is very greedy on memory. That aside though, I would guess that smaller images would process faster. That is, the difference between a field of video (720x288 in PAL land) and a digital photo of 3000x2000 must have some bearing on happenings! It may also affect how good it performs too.

If anyone has tried this, I would love to hear all about it! (unfortunately, I won't have time for at least 6 months :( :( )

I think virtualdub [no-mod] can import pictures as frames, at least the older ones could. Never tried exporting, though I think it does that as well.

I think you mean 30spf, not 30fps!

Cheers,
MM

shindou

19th August 2009, 23:17

I mean 30 fps actually. 60 seconds of video running at 30 frames per second gives 1800 frames. If each frame takes 40 seconds to deblur, it takes a total of 72000 seconds, which is 20 hours.

mustardman

20th August 2009, 10:08

Cool. I see it now...

loekverhees

22nd September 2009, 19:07

If I apply the 'fixed zoom' option, will Deshaker apply only one zoom value for the entire video (so if some 3 seconds part of a 2 hour movie is extremely shaky, a huge zoom value will be applied for the entire movie) or will it detect scenes for example and apply different fixed zoom values for the different scenes?

guth

27th September 2009, 07:41

Version 2.4 uses different zoom values for each individual detected scene. Older versions use the same zoom value for the entire clip.

wonkey_monkey

27th September 2009, 19:02

Hi Gunnar,

Something I noticed a long time ago is that Deshaker takes longer and longer to do the actual "Deshaking..." part with higher values of motion smoothness. For example (10000 frames):

Motion smoothness=100, <1s
Motion smoothness=1000, 11s
Motion smoothness=10000, >200s

I assume these values decide how many frames' worth of parameters either side are averaged to decide how to move that frame, and that it might be a Gaussian distribution to give a smooth effect. Have you considered optimizing this step? For instance, you can approximate a Gaussian "blur" of parameters like this with multiple simple box blurs, which will only take longer as the number of frames increases, not (or not much) with the width of the "blur".

I know most people won't find this a problem but if you do want to use a high value of motion smoothness, it can take a while.

David

guth

27th September 2009, 21:27

To generate the smooth paths I solve some large equation systems numerically, and they converge slower with higher smoothness values. I haven't gotten many complaints about it, but maybe I can do it some other way. We'll see...

guth

28th September 2009, 15:35

Actually, your times seem a lot higher than what's normal. Maybe you're using too low max. correction limit values? They can have a huge effect on deshaking times.

wonkey_monkey

29th September 2009, 00:32

I ran a default first pass (with a 1% search range), then I altered all four of the motion smoothness parameters (horizontal, vertical, rotation, zoom) to 100, 1000, or 10000 in turn.

If you remember, a long time ago I was playing around with writing a perspective-correcting deshaker, which used deshaker.log as it's input. For each frame, all I did was average the values for a number of frames on either side, then subtract that from the original value for that frame to get the parameter for adjusting the image. It seemed to come out pretty similar to Deshaker (although I should say it was a bit of a guess, I didn't know if it would work), but I didn't take it very far.

The beauty of it is that a) if the average of a parameter for frame x is calculated from by the total from p[x-w] to p[x+w], divided by 2w+1, the next average (for x+1) can be calculated by taking that total (pre-division), and subtracting p[x-w] and adding p[x+w+1]. Only the calculation for the first frame is dependent on the size of w - all other frames only need one subtraction and one addition.

And b) if you perform this averaging 3 or more times, you have a good approximation to a Gaussian distribution, which makes for a nice smooth curve.

As I said, I don't doubt that no-one else has been bothered by this, because who really needs to smooth over 10000 frames (except me ;) )?

David

guth

29th September 2009, 17:26

Did the video come out looking ok with the 10000 smoothness? Cause using adaptive zoom with that high smoothness, usually doesn't look good at all. If you turn off the adaptive zoom it should be faster.
I just ran a test with 10000 frames and 10000 smoothness, and it took 10 seconds to deshake. But I don't doubt it can take very long with certain clips/settings.

As for your smoothing idea, I do remember it, and I'm still not sure I like it. I don't think I want a Gaussian distribution. My algorithm tries to minimize the squared corrections (ie tries extra hard to minimize the biggest corrections) in order to minimize the ugly borders. Your algorithm doesn't even bother looking at how big the corrections become. Maybe the difference in practice is small, but still... :)

wonkey_monkey

29th September 2009, 18:27

I didn't actually watch the video, I was just watching how look it took to deshake :)

Deshaker is obviously more sophisticated than I assumed - not that I thought it was unsophisticated! - my averaging looked enough like Deshaker's output on my test video that I assumed it was similar. That project wasn't about the smoothing anyway, but about the correction of wide-angle frames, which benefit from being rotated in three dimensions rather than panned in two.

David

WarpEnterprises

30th September 2009, 21:42

It's not so easy to use the adaptive zoom correctly.
I can't think of many situations where you have to "deshake" the real zoom, so although "edge compensation adaptive zoom" is the default, maybe "adaptive zoom only" will be the more likely choice.
Then the motion smoothness for zoom must indeed be quite high (~5000) because it should resemble a slow real zoom.
But the motion smoothness for panning and rotation can be much lower (~500), only those do the real deshaking.
The lowest "nice" values can be found by leaving the zoom value 0 and using "edge compensation none", then increasing the pan/rot values until the shake is gone.
Then use the high zoom value and "adaptive zoom only".
That way only ONE of the four values is high, which makes it much faster and leaves more "space" for soft zooming.

Maybe the caption is a little bit misleading, too:
It looks like the difference between "Adaptive zoom (some borders)" and "Adaptive zoom only" is only how much borders are in the result which in fact is NOT the difference (as the manual correctly tells).

guth

1st October 2009, 18:26

David...
I was just trying (hard) to come up with a reason for using my algorithm instead of yours :)

Warp...
"Adaptive zoom only (some borders)" is too long, it doesn't fit. And anyway, I never meant for Deshaker to be user friendly ;)
I'll think about it, though...

As for "nice" settings, what's nice for one person might not be nice for another. I, for example, almost always use "edge compensation: none", but do use zoom smoothing (unless there's no zooming in the clip). Almost the exact opposite of you :)

WarpEnterprises

2nd October 2009, 13:23

guth, I have one more question:
is your internal timescale/unit for motion smoothness the same for pan and zoom?
I mean, if there is a clip which shakes in X-direction with the same frequency as another clip which "shakes" in the zoom-value would I have to apply the same motion smoothness values to get similar results?

guth

2nd October 2009, 17:33

Panning and zooming don't share the same unit, so you can't really compare them. It's like asking if a minute is longer than a meter.
Another answer is that the numbers I use internally during smoothing to represent panning is 'number of pixels', and for zoom 'the natural logarithm of the zoom factor'. Based on this, I'd say that a zoom unit would "seem" quite a lot bigger than a pixel unit, but how much depends on the size (in pixels) of a frame. That is, if they were comparable at all :)
I haven't really thought much about this. It's an interesting question... sort of...

loekverhees

2nd October 2009, 19:10

Version 2.4 uses different zoom values for each individual detected scene. Older versions use the same zoom value for the entire clip.Ok, thanks for the answer.

WarpEnterprises

2nd October 2009, 19:45

@guth: interesting! But I was concerned of the time dimension - if I understand correctly higher smoothness values make the resulting video shake less AND shake slower (make the resulting amplitude smaller AND the frequency lower). That way pan and zoom would be more comparable, at least the frequency. But maybe I'm simplifying things the wrong way...

guth

2nd October 2009, 21:04

I'm not sure I understand what you're asking. Yes, higher smoothness values makes the video shake less and slower.
But you can't really separate the frequency from the amount during the smoothing. And since it's hard to compare the amount, it's hard to compare the overall effect (including frequency).
My algorithm doesn't really correspond to a lowpass filter thingy with a certain cutoff frequency, or at least I don't think so. :)

BabaG

21st January 2010, 21:35

shindou

22nd January 2010, 01:03

just tried the deshaker on some old digitized film i have and it worked great! i'd like to try running it as a batch. i know it's not the recommended procedure but i think i could narrow down my tasks that way since the default setting worked so well on my first jitter test.

being new to this, i'm very much unclear on how to set up a batch test using the jobs function. could someone post a step-by-step on how to use jobs to execute deshaker on several files? i'd very much appreciate it.

the thing that has me most confused is how i'd run the first pass since this seems very much a manual operation the way i tested it.

thanks,
BabaG

First of all, welcome to the wonderful world of Deshaker!

Second, and sadly, I have never heard of any practical way of queueing up Deshaker jobs in batch. I've made my own scripts which would, in fact, generate the VirtualDubMod job file directly. Writing them was a trial and error job. They surely won't work for any video - just for those I take with my cam, a Canon PowerShot 720 IS. I also needed to make a variety of similar scripts just to handle slightly different movie formats. Mine will probably not work for you unmodified, unless you have the same camera. Moreover, it makes some assumptions on file naming which may add to the hassle and solve nothing for you after all.

Third, and contrary to what I did (I just needed a quick fix for 270 files), applying Deshaker in batch might not be a good idea. Several of my movies did not deshake well, even after I carefully adjusted "my own default parameters".

If you have the time, I suggest you read up the whole thread to grab a few ideas on how to find the best parameters for each of your movies, manually. If you have even more time and expertise, or a nice programmer friend with plenty of time (not the case usually), make your own scripts.

Currently I think the problem of batch deshaking is just one of several deficiencies in Deshaker (e.g. an easy, intuitive way of telling it to disconsider a specific object or other part of the image). Nonetheless, Deshaker is the best and we'd love an open-source version, which could then prompt a solution more quickly.

guth

22nd January 2010, 18:02

could someone post a step-by-step on how to use jobs to execute deshaker on several files? i'd very much appreciate it.
Well, it's not super convenient (and not really recommended), but you can do it like this:

Setup everything as you usually do for pass 1 processing.

In the job control window, use "Edit/Process directory" to add your clips.

Setup everything as you usually do for pass 2 processing.

In the job control window, use "Edit/Process directory" to add your clips again.

In the job control window, move the clips so that you have order: clip1pass1, clip1pass2, clip2pass1, clip2pass2 etc. You need to do this since the log files will be overwritten at each pass1.

BabaG

24th January 2010, 06:46

thanks guth! i'll have a look at this.

BabaG

Diamenz

1st March 2010, 06:18

hey all. first of all guth i'd like to say how great of a plug in this is for shaky video. it really works wonders for me. i own a flip hd mino which is really bad for moving around while recording, especially when walking. unfortunately i don't know the exact rolling shutter %. the default 89% seems to work pretty well, but i emailed flip and asked so hopefully they'll reply with an answer. if they do, i'll be sure to post it here for everyone.

i have a few questions for you guth... i'm not very educated on video like a lot of the people here, but i also don't consider myself a complete newbie.

for motion smoothness, what would be general ideal settings for someone that wants to stabilize, but at the same time not lose sharpness. i don't know if it's me, but it seems the higher i set smoothness the blurrier my video becomes. or perhaps it's another setting that's causing blurriness? it's not too bad, but i'm a quality nut and it's bugging me. is the cost of stabilizing video slight loss of quality?

actually that's my only question now that i think about it. thanks!

i guess i should also mention i use no edge compensation and uncheck "ignore pixels darker that 15% brightness". everything else is left at default besides a full scale and using 'all pixels' under first pass. my source video is 1280 x 720 29.97 progressive.

guth

1st March 2010, 18:52

for motion smoothness, what would be general ideal settings for someone that wants to stabilize, but at the same time not lose sharpness. i don't know if it's me, but it seems the higher i set smoothness the blurrier my video becomes.
There's always a slight loss of sharpness due to the resampling process, but that's unavoidable and not dependent on the smoothness settings.

But I suspect that what you're seeing is motion blur in the frames. Deshaker can only remove unwanted motion between frames, not within frames. So if you have a slow shutter speed (slower than around 1/300 sec) there's a high risk you'll get motion blur. When you have motion both within and between frames, motion blur often looks nice, but when you remove the motion between frames, it looks bad. And in a way it might look slightly worse with higher smoothness settings.

Diamenz

2nd March 2010, 04:54

had to read that a couple of times get it, but i do now. thanks for the reply guth. great plugin.

Dogway

15th April 2010, 18:38

I have been struggling with the source pixel aspect ratio setting in the filter. I want to set it to 0.888 as a NTSC 4:3 PAR, will the filter let me choose this? or do I have to stick to 0.912? Another thing is: is it better to crop after the deshaking? that is leaving some black borders for Deshaker to estimate better.

btw. What does PAR have to be with deshaking? some better motion estimation?

wonkey_monkey

15th April 2010, 19:42

btw. What does PAR have to be with deshaking? some better motion estimation?

Far be it for me to speak for guth, but PAR affects how a rotation is performed.

0.912 is the closest AR for NTSC (I think it's actually 0.909), because of nominal analogue blanking (http://en.wikipedia.org/wiki/Nominal_analogue_blanking). What's your source, and how have you determined that the PAR is 0.888?

Another thing is: is it better to crop after the deshaking?

You can do, but Deshaker also has the "Ignore pixels" parameters. I'm not sure which will work better, or why...

David

guth

15th April 2010, 21:30

I have been struggling with the source pixel aspect ratio setting in the filter. I want to set it to 0.888 as a NTSC 4:3 PAR, will the filter let me choose this? or do I have to stick to 0.912?

When I looked into this a few years ago, I believe Adobe were the only ones I found who used 0.888, but they seem to have changed that lately too:
http://blogs.adobe.com/toddkopriva/2009/07/pixel-aspect-ratios-in-after-e.html

Anyway, the only way to be sure about what PAR your camcorder uses, is to measure it from the actual video (by filming a circle for example).

And you can enter any PAR value you want in Deshaker. (RTFM! ;))

And yes, the PAR is important for handling rotation and zoom correctly, and also if you use different PAR values for source and destination.

The reason Deshaker's preset is appr. 0.912 instead of appr. 0.909 is because I was convinced by that web page, but since most video applications out there seem to use 0.909 I may reconsider, even if it's not correct. :)

Another thing is: is it better to crop after the deshaking? that is leaving some black borders for Deshaker to estimate better.

Deshaker prefers not to see any borders during pass 1. They interfere with the motion estimation. You should remove them either by cropping or using 'ignore pixels'. If you use 'previous and future frames to fill in borders' for pass 2, you should definitely crop any borders before deshaking.

Dogway

15th April 2010, 21:47

This (http://forum.doom9.org/showthread.php?p=1390321#post1390321) is my source. I calculated PAR on the well known 8:9 value for 4:3 NTSC. I have also tried the 4320/4739 PAR on other similar sources (NTSC DVD with black bars on both sides) but they looked better in 8:9 (checking some circles of the footage and general appearance).

guth: Thank you, yes I tried to just input the value, and I got the same PAR I originally had in the .m2t (1.5 DAR), with 0.912 I was getting 1.4xx instead of the original 1.5, but I was unsure if it had some relation on the PAR value.

wonkey_monkey

15th April 2010, 23:15

The reason Deshaker's preset is appr. 0.912 instead of appr. 0.909 is because I was convinced by that web page, but since most video applications out there seem to use 0.909 I may reconsider, even if it's not correct.

AIUI, 702 (which would lead you to a PAR of 0.912 for NTSC video) is the "correct" width only for PAL video, whereas NTSC uses 704 pixels:

PAL: 576/(702*3/4)=1.09402
NTSC: 480/(704*3/4)=0.909091

Both the NTSC values we're discussing (0.909, 0.912) do account for NAB, but different amounts. If you don't account for NAB at all, you end up with 0.888, and only careful experimentation will tell you whether your camera is made up "correctly" (the difference is, at most, less than 3%).

David

Dogway

16th April 2010, 01:05

guth

16th April 2010, 15:55

AIUI, 702 (which would lead you to a PAR of 0.912 for NTSC video) is the "correct" width only for PAL video, whereas NTSC uses 704 pixels:

PAL: 576/(702*3/4)=1.09402
NTSC: 480/(704*3/4)=0.909091

I don't use 702 for NTSC, I use 710.85. :)
According to this page (http://lipas.uwasa.fi/~f76998/video/conversion/), which I still believe is the truth, it should be calculated like this:

PAL: 576/(702*3/4)=1.09402...
NTSC: 486/(710.85*3/4)=0.91158...

guth

16th April 2010, 16:04

Well, I dont digitalize video, just old animation so normally the content I work with is NTSC DVD. My question is whether having the black bars do I have to actually take into account NAB in the 100% of the cases or if it depends on every telecined source.
I'm afraid I don't understand what you mean. I don't even know what NAB means...

But in case it solves anything for you, the aspect ratio of a pixel (ie the PAR) doesn't change just beacuse the video has borders or not, or if you crop, or not.

Finally, I didn't know you wanted to use Deshaker for stabilizing that kind of video. Deshaker was designed to stabilize unwanted camera motion, not really other kinds of shakes, even if it can do that sometimes too. In your case, it might actually be better to include the borders during pass 1, at least when the animation background isn't supposed to move.

Dogway

16th April 2010, 19:13

NAB (http://forum.doom9.org/showthread.php?p=1392010#post1392010)

Because if I have the black bars (NAB) I would have to assume 704 width, therefore indicating my source is PAR 0.909. So my question is whether this only applies for direct digitalization (analog->digital) or it stands as well for DVD's digital content previous digitalized. Otherwise I would just need to assume PAR 0.888. I mean, I would like to know what do I have to base on to know if the manufacturer assumed 0.888 or 0.909. Mainly because when I do play any DVD (NTSC) on my laptop the object geometry is the same as using 0.888, and in practice it feels more natural than 0.909. Manufacturers must be doing it wrong? Id like to see examples on this.

Sorry this talk derived to AR, its something Ive been carrying on long time.

I cant differentiate between camera shake or other shakes, other than floating cels in case of animations(?)

guth

16th April 2010, 19:36

Because if I have the black bars (NAB) I would have to assume 704 width, therefore indicating my source is PAR 0.909. So my question is whether this only applies for direct digitalization (analog->digital) or it stands as well for DVD's digital content previous digitalized. Otherwise I would just need to assume PAR 0.888. I mean, I would like to know what do I have to base on to know if the manufacturer assumed 0.888 or 0.909. Mainly because when I do play any DVD (NTSC) on my laptop the object geometry is the same as using 0.888, and in practice it feels more natural than 0.909. Manufacturers must be doing it wrong?
I'm no expert on this subject, but I don't think the black bars tell you anything about the PAR. The table on the page I referred to above says that the sampling matrices 720x486, 720x480, 711x486, 704x486 and 704x480 all share the same PAR, namely 4320/4739. The table also mentions DV, DVB and DVD for that PAR.

If you think 0.888 looks better, maybe you're fooling your brain to think so, because you want it to be that way, or maybe you're used to looking at video like that, or... maybe the manufacturers are wrong. But I really don't think 0.888 is ever the correct PAR to use (except for correcting other's mistakes).

Dogway

16th April 2010, 20:56

I have been doing some checks between different PAR screenshots and some original sketches from storyboards, and yes it seems to be 0.912 or 0.909, rather 0.909 I think. I think that what it was fooling me was the CGI circle, wrongly implemented into the animation (PAR wise), so anyway, who knows I will stick to 10:11 and see if that fully satisfies me, at least for the hand drawn parts.

I dont know where did I get the 8:9 thing, I cant find it, I guess it was in a thread here. Until now I was using this table I made for reference, now I guess is somewhat wrong:

SAR examples: (704x576) (704x480)

4:3 = 12:11 pal = 10:11 ntsc
16:9 = 16:11 pal = 40:33 ntsc

SAR For Digital Playback: (720x576) (720x480)

4:3 = 64:60 (16:15) pal = 64:72 (8:9) ntsc
16:9 = 64:45 pal = 64:54 (32:27) ntsc

*You can also see the 8:9 PAR in the reference section of the PAR wiki (http://en.wikipedia.org/wiki/Pixel_aspect_ratio)

Boulder

3rd June 2010, 16:59

Gunnar,

some questions for you:

If I set the Deep analysis to kick in at for example <50%, it often doesn't show the "Deep analysis" text in the preview window even if the percentage of OK blocks is less than the limit. If I set it to 100% or so, it's used for every frame. Is this intentional or a bug?

Do you have any plans to improve the motion detection method, for example by introducing overlapping blocks (such as in MVTools2)?

guth

3rd June 2010, 17:52

If I set the Deep analysis to kick in at for example <50%, it often doesn't show the "Deep analysis" text in the preview window even if the percentage of OK blocks is less than the limit. If I set it to 100% or so, it's used for every frame. Is this intentional or a bug?
The deep analysis percentage is a percentage of all found vectors, while the text gives you the percentage of all blocks. If matching fails for a block, it won't get a vector at all, so there are usually fewer vectors than blocks. That's why. A little confusing, I know. :)

Do you have any plans to improve the motion detection method, for example by introducing overlapping blocks (such as in MVTools2)?
Most people seem to think deshaker is too slow. What you propose wouldn't exactly make it faster. :)
We'll see, but I'm afraid it probably won't happen any time soon anyway...

Boulder

3rd June 2010, 17:59

The deep analysis percentage is a percentage of all found vectors, while the text gives you the percentage of all blocks. If matching fails for a block, it won't get a vector at all, so there are usually fewer vectors than blocks. That's why. A little confusing, I know. :)OK, thanks for the info.

Most people seem to think deshaker is too slow. What you propose wouldn't exactly make it faster. :)
We'll see, but I'm afraid it probably won't happen any time soon anyway...The clips I use Deshaker on are very important so speed is not an issue.. The current 1st pass analysis is running at 0.9 - 1.2 fps at close to max settings so I don't know if it's that slow, though there's nothing in the AVS script but conversion to RGB24. Computers are getting faster all the time, my Core2Duo rig is from 2007 :)

Boulder

6th June 2010, 18:09

Another feature request: it would be nice to be able to use Lanczos or Spline for resampling. I assume that you use the selected resampling option for zooming in the edge compensation functionalities? Lanczos or Spline should provide sharper output than bicubic.

guth

7th June 2010, 18:06

Another feature request: it would be nice to be able to use Lanczos or Spline for resampling.
From what I've seen the difference is barely visible, except under extreme test scenarios. And not even then is it easy to see which is better, I think. So I don't think I'll bother. (And that's coming from a perfectionist :))

I assume that you use the selected resampling option for zooming in the edge compensation functionalities? Lanczos or Spline should provide sharper output than bicubic.
Deshaker needs resampling for lots of reasons, but everything is always combined into one single resampling (so no pixel is ever resampled twice between input and output). And that single resampling uses the selected algorithm.

Undead Sega

9th June 2010, 17:03

Hello guth, it has been awhile, hope all is well.

i probably mentioned this before a couple years ago but i cant find what ive type,plus im more experienced in this matter but im still having a problem. The problem im having is to stablize a shot, this contains a panning camera from right to left in an empty (static) room to only have a person sitting in the middle of the shot making large and fast movements. The camera itself is of course what i want to stabilize, however from what im gathering, the motion vections are picking the person's movement as camera shake, therefore the resulting video look would shake in accordance to the movement that the person is making. I was wondering how can i avoid this? and what parameters do i need to set accordingly?

i hope u understand what i mean, and i would really appreciate it if u can help me on this. thanks and look forward hearing back,

guth

10th June 2010, 17:54

The problem im having is to stablize a shot, this contains a panning camera from right to left in an empty (static) room to only have a person sitting in the middle of the shot making large and fast movements. The camera itself is of course what i want to stabilize, however from what im gathering, the motion vections are picking the person's movement as camera shake, therefore the resulting video look would shake in accordance to the movement that the person is making. I was wondering how can i avoid this? and what parameters do i need to set accordingly?
If the deshaked video is only slightly wavy, you should probably reduce the value for "discard motion of blocks that move > X pixels in wrong direction". Maybe to 1 or even lower. That's to avoid picking up even very slow motion from the person.

If it's very shaky after deshaking, then Deshaker probably lost track on where the stable background is and started matching partly or completely on the person. Remember, you shouldn't get any white vectors on parts of the person that are moving. The white vectors should only be on the background or on the person's still parts. This can sometimes be tricky to accomplish, especially if the person takes up a large portion of the frames. You can try decreasing block size and increasing differential search range and set "use pixels" to "all". If possible, you should definitely use "deep analysis". And if everything else fails, you can use "ignore pixels" to help Deshaker ignoring the person.
Next version of Deshaker will have more ways to help avoiding this problem. (Now that VirtualDub 1.9.9 is out and seems to include a bugfix I've been waiting for, I'll probably release it soon. Don't expect any major cool new features, though...)

Boulder

13th June 2010, 17:10

What sort of improvements are expected to be in the next version? I'll start working on the second passes of my projects and was wondering whether I should wait a while for the new release?

While I'm writing, here's another shameless feature request :) It would be nice to have limits for horizontal and vertical corrections in pixels so that I could enter a value that I know I can live with (for example I've measured that my TV has approximately 16 pixels of overscan).

guth

13th June 2010, 17:56

What sort of improvements are expected to be in the next version? I'll start working on the second passes of my projects and was wondering whether I should wait a while for the new release?
The changes for pass 1 will make it possible for that pass to become more robust.
For pass 2, you will be able to use future frames without having to delay audio, append clips etc. The actual output from pass 2 will be the same, however.

It would be nice to have limits for horizontal and vertical corrections in pixels so that I could enter a value that I know I can live with (for example I've measured that my TV has approximately 16 pixels of overscan).
The best way to control how big the borders will be (or to make them completely dissappear under overscan) is to select an "edge compensation" mode that gives no borders and then zoom out using an "extra zoom factor" below 1 ( ((width_or_height - 16) / width_or_height) to make borders dissappear under overscan in your case).
You can then use the smoothness settings and correction limits to find a good balance between motion smoothness and the amount of edge compensation zooming needed to get the amount of borders you wanted.

Undead Sega

14th June 2010, 03:22

Hello guth, thanks very much for the suggestion but during the past couple days, i have tried what you've said and ive tried everything else with no luck, therefore i was still wondering if you can help me with it:

http://www.megaupload.com/?d=G98LVF0E

was hoping if you could have a go with it and see if you get any good results. Hope its okay :)

guth

14th June 2010, 21:38

guth

14th June 2010, 21:40

Boulder, in case you care, the formula I gave above should probably be ((width_or_height - 32) / width_or_height) instead, since I guess you meant 16 pixels on each side...?

Undead Sega

14th June 2010, 23:25

I'm afraid I can't really deshake that clip either. There's too little detail in the background, and the details that are there are mostly straight lines, which are "unmatchable". A feature based matching algorithm would probably do better in this case. Deshaker uses area based matching. Sorry...

Damn, and i thought i was doing something terribly wrong. Probably a feature based matching algorithm might be your next feature for Deshaker? :D

regarding this, what image stabilization software could probably deshake my footage?

Boulder

15th June 2010, 03:41

Boulder, in case you care, the formula I gave above should probably be ((width_or_height - 32) / width_or_height) instead, since I guess you meant 16 pixels on each side...?Yes, I meant ~16 pixels on each side. Thanks, I'll see how it works when I get to run the second passes some time this week :)

guth

15th June 2010, 18:19

regarding this, what image stabilization software could probably deshake my footage?
Don't know...

Undead Sega

16th June 2010, 02:27

fair enough, however I came across Mokey by imagineer systems which uses their 2.5D planes to track footage and etc. and if i am correct, you can also export that tracking data into After Effects. May i ask, could something rom that software be used in the logfile for Deshaker?

guth

16th June 2010, 17:41

The Deshaker logfile contains the change (in pannaing, rotation and zoom) between one frame and next frame. I don't know what that software can produce.

Undead Sega

17th June 2010, 01:19

This:

http://forum.doom9.org/attachment.php?attachmentid=11172&stc=1&d=1276733925

any of it looks familiar by any chance? :)

Undead Sega

21st June 2010, 19:37

take it what you saw doesnt look familiar or useful?

poisondeathray

21st June 2010, 20:21

planar trackers (like mocha and mokey) and point trackers (like AE) work differently than deshaker. They don't report the difference between subsequent frames (one frame and the next) like deshaker does. They track difference between a reference frame or x,y,z position and rotation/scale/sheer etc.., ie. where you set the point tracker or 2d plane . The reason is they are more geared for compositing work and they need precision against a specific reference point (e.g you want to replace an object in a movie, like a poster with another poster etc...)

You could translate them I suppose, if you made a program and converted the numbers to make them differences between consecutive frames

guth

22nd June 2010, 19:20

take it what you saw doesnt look familiar or useful?
I've told you what Deshaker wants. Now it's up to you to see if you can get those values from somewhere else. (I'm definitely not saying it's impossible. I just don't want to do the work myself. :))

dedfish

2nd July 2010, 20:15

Any recommendations specifically for vehicle, motorcycle in my case, mounted videos? Are there particular settings that would work better for high speed movement?

guth

3rd July 2010, 09:20

I know a lot of people seem to be using Deshaker for those kind of videos, but it's not really designed to handle videos where the camera "moves around", at least not where it moves very fast. Deshaker can only handle panning, rotation and zoom. That said, I've seen some very good deshaked videos of that kind.

As always, make sure you're using a high enough shutter speed, if possible. There shouldn't be any motion blur in each frame.
Also, you would probably get *a lot* better results if the camera didn't have a rolling shutter. But I believe most helmet mounted cameras today have a rolling shutter. Deshaker can remove rolling shutter effects but only if the shaking isn't extreme.
I would also recommend trying to stabilize only on the most distant parts in the frames, since the "moving inwards"-effect is less there. You can do that by ignoring the other areas.
And turn off zoom smoothing by settings zoom smoothness to 0. You will also need to change the edge compensation type if you do that. Start with "None", maybe.
Finally, you should probably increase the value for "discard motion of blocks that move > X pixels in wrong direction". That's to allow the blocks to move "freely" a little, since Deshaker can't handle the "moving inwards"-effect.

But maybe you should ask someone who's actually stabilized these kind of videos. I haven't. :)

dedfish

3rd July 2010, 17:59

Thanks, I appreciate you taking the time to answer my question.

If anyone else has any tips they've found to work well for this type of video I would love to hear them.

Cheers!

:thanks:

Undead Sega

9th July 2010, 03:27

Hello once again guth, it is I :D

I am actually still having problems trying to deshake my footage, I have created a new image sequence from the original, but this time it was deinterlaced using TGMC to 50p and I removed the noise using Neat Video resulting me with a (slightly sharp but) clean image, I dont know if that is going to make any differences?

...but I have been trying it with formentioned software, but it is not as effective as your filter, especially when it comes to border filling, wouldnt you happen to know of other software or filters that can literally do what your filter does interms of borderfilling effectively?

Also, you say your filter uses area based matching, is this something that is limited in ur filter or in Avisynth? Possibly there could be an avisynth filter that uses a more sosphisticated algorithm for the searching of the footage (and possibly be used in Deshaker)?

guth

9th July 2010, 09:58

I am actually still having problems trying to deshake my footage, I have created a new image sequence from the original, but this time it was deinterlaced using TGMC to 50p and I removed the noise using Neat Video resulting me with a (slightly sharp but) clean image, I dont know if that is going to make any differences?
Me neither. But removing noise can be very bad for Deshaker if the smoother uses some kind of temporal smoothing. That kind of smoothing often leaves traces of earlier frames that Deshaker can match on, incorrectly.

wouldnt you happen to know of other software or filters that can literally do what your filter does interms of borderfilling effectively?
No, I don't. But I haven't been looking much either.

Also, you say your filter uses area based matching, is this something that is limited in ur filter or in Avisynth? Possibly there could be an avisynth filter that uses a more sosphisticated algorithm for the searching of the footage (and possibly be used in Deshaker)?
I'm not sure I understand your question. The matching algorithm is inside Deshaker. You can't change it without me doing something.
Could there exist filters with a more sophisticated search algorithm? Sure, but Deshaker can't use it without me doing something. Unless it can produce logfiles for Deshaker pass 2.

guth

19th July 2010, 12:36

Deshaker 2.5 is now released.
(New versions don't seem to come very often lately, so I thought I should mention it.)

Get it here (http://www.guthspot.se/video/deshaker.htm).

2Bdecided

12th August 2010, 12:33

Deshaker 2.5 is now released.
(New versions don't seem to come very often lately, so I thought I should mention it.)

Get it here (http://www.guthspot.se/video/deshaker.htm).Thank you!

You should post this in the news section.

Cheers,
David.

guth

12th August 2010, 17:26

Nah... I just want true Deshaker fans to know :)

airborne_video

14th August 2010, 03:37

Hi Gunnar,

thanks for the great filter!

I have mounted a keychain-camera to a model airplane and use your filter to improve the resulting videos. Although (as you have mentioned before) the filter is not designed for this purpose the results are quite impressive. One issue I am facing though is the relatively high amount of dropped frames these kinds of cameras produce, which appear as duplicate frames in the video. Deshaker perceives these duplicate frames as proper individual frames and this leads to bad results. I was thinking that it might be a good thing if you only compared the first frame of a series of duplicate frames to the next real frame following the duplicate frames and then spread the resulting adjustments evenly over the series of duplicate frames. I am guessing that identifying duplicate frames should be feasible, since they show zero motion compared to preceding images.

Just an idea.

Thanks for all the time and effort you put into this!

guth

14th August 2010, 06:51

If duplicate frames appear, Deshaker should already behave exactly the way you describe. But since it only adjusts the whole frames, and not moving objects within frames, those moving objects will still appear to "jump" a bit. The background in the video should, however, become perfectly smooth even when duplicate frames appear.
If you want me to look at it, email me a short clip.

airborne_video

15th August 2010, 17:18

If duplicate frames appear, Deshaker should already behave exactly the way you describe.

Hi Gunnar,

I don't know what I have done wrong, but you are right - it does work in the way described. I am not sure why I didn't get the results I was looking for so far, but it was definitely some mistake on my behalf.

I have a small example clip with lots of dropped frames that shows that you're right:

Source (MJPEG): http://rapidshare.com/files/413110232/001_original_scene.avi.html
Results (H264): http://rapidshare.com/files/413111567/002_deshaker_results_H264.avi.html
Settings: http://rapidshare.com/files/413115102/settings.png.html

Frames 80 to 130 of the filtered video show results as expected.

Do you have any advise regarding the minor glitches between frame 171 and 172 and between frame 211 and 212 of the filtered video?

guth

15th August 2010, 20:36

The glitches occur because of basically three things:
1. The video there basically only contains fields without much detail, and with straight lines in-between. It's hard to match on.
2. Motion blur makes details fuzzy.
3. The date and time at the bottom of the video confuses the matching.

If you decrease the value for "Discard motion of blocks that have 2nd best match > best - X" all the way to 0, this will force deshaker to generate motion vectors even when they aren't very reliable. It will work fine for this video clip to fix the glitches.
Also, I would "Ignore pixels inside" the area of the date and time.
And since the video has size 720x480, the pixels probably have pixel aspect ratio = NTSC.
Finally I would probably do something about those black borders.

airborne_video

15th August 2010, 21:59

If you decrease the value for "Discard motion of blocks that have 2nd best match > best - X" all the way to 0, this will force deshaker to generate motion vectors even when they aren't very reliable. It will work fine for this video clip to fix the glitches.

Thanks a lot, I will try that.

Also, I would "Ignore pixels inside" the area of the date and time.

About that: if I have a video sized 720x480px and want to ignore a box sized 410x30px lying in the right bottom corner, do I enter L311 R720 T451 B480 or L310 R0 T450 B0 into the "ignore pixels inside" fields?

And since the video has size 720x480, the pixels probably have pixel aspect ratio = NTSC.

It's actually a distorted 640x480px sized video. The camera does the distortion. I usually resize the video before using deshaker.

Finally I would probably do something about those black borders.

True. Normally I use deshaker's fill-in-options and crop the video a little, which gives nice results.

scharfis_brain

15th August 2010, 22:56

It's actually a distorted 640x480px sized video. The camera does the distortion. I usually resize the video before using deshaker.
There is absolutely NO need to do so.
This way you'll end up with two chained scalings.

Just setup deshaker correctly.
In analyse pass set 4x3 NTSC Aspect Ratio.
In render pass set 1:1 aspect ratio and 640x480 pixel.

This saves processing time and offers better image quality.

airborne_video

16th August 2010, 02:31

About that: if I have a video sized 720x480px and want to ignore a box sized 410x30px lying in the right bottom corner, do I enter L311 R720 T451 B480 or L310 R0 T450 B0 into the "ignore pixels inside" fields?

Solved: it's the latter. :-) Stupid question...

In analyse pass set 4x3 NTSC Aspect Ratio. In render pass set 1:1 aspect ratio and 640x480 pixel.

This saves processing time and offers better image quality.

Thanks for the advice. Very much appreciated!

Since 640/720 is 0.88 and not exactly 0.912 (Standard NTSC) would it make sense and work to manually enter "0.88" in the dropbox?

scharfis_brain

16th August 2010, 07:33

Aspect Ratio Correction is far more 'complicated' than you might imagine.
640 pixels actually translate to 702 pixels.
This also means that 720x480 after aspect ratio correction are 657x480 pixels.
If there aren't any borders left and right (each about 9 pixels in width)
you're lucky to have some additional image information which widenes the image.
So just leave the numbers as they are. They are correct.

airborne_video

16th August 2010, 13:42

Aspect Ratio Correction is far more 'complicated' than you might imagine.
640 pixels actually translate to 702 pixels.
This also means that 720x480 after aspect ratio correction are 657x480 pixels.

Thanks, based on your posting I have searched for more information and stumbled upon this website (in German):

http://www.dma.ufg.ac.at/app/link/Grundlagen%3AVideo.Postproduction/module/9039?step=1#chapter

So here is what I learned:

NTSC has 480 horizontal lines. With square pixels and an aspect ratio of 4:3 a frame with a height of 480 pixels would be 640 pixels wide. A digitized 4:3 NTSC frame however is defined to be 702 pixels wide with non-square pixels (as a result of a 13,5 MHz scanning frequency).

Thus the pixel aspect ratio is 640/702 = 0,912

A 9px buffer is added on both sides to deal with horizontal deviation that might occur during the digitalization of an analog NTSC signal. In sum these borders to each side add up to 18px. So in the end we get a 720x480px frame.

The keychain camera produces an image where the complete buffer to the left and right contains valid image information.

This also means that 720x480 after aspect ratio correction are 657x480 pixels.
If there aren't any borders left and right (each about 9 pixels in width) you're lucky to have some additional image information which widenes the image.

Got it! Thanks!

So if I don't cut the buffer area, since it indeed contains valid image information, I should actually configure the output to be 657x480px in pass 2, if I chose a 1:1 pixel aspect ratio for the output, right?

guth

16th August 2010, 17:44

It's actually a distorted 640x480px sized video. The camera does the distortion. I usually resize the video before using deshaker.
I'm not sure I understand exactly what you mean by this. Does the camera deliver video sized 640x480 or 720x480? And in what way is it distorted? And how do you want the final video to be? (Maybe I'm the only one who doesn't get it...)

airborne_video

16th August 2010, 20:01

Does the camera deliver video sized 640x480 or 720x480? And in what way is it distorted? And how do you want the final video to be?

The camera delivers a video in 720x480px. The image is stretched horizontally (objects appear to be too wide) if square pixels are assumed. I would like the final video to have square pixels while preserving the original height of 480px. The image content should not be distorted in the output.

So here's what I was thinking:

Let's say the camera records an NTSC video with a pixel aspect ratio of 0.912. As I understand it, NTSC has 702x480px frames with a 4:3 aspect ratio plus a buffer of 9px on each side.

In my case the whole buffer contains valid image information and should be preserved, so we actually have a 720x480px video with a 0.912 pixel aspect ratio and a 4.11:3 aspect ratio.

If I wanted the video to have square pixels without distortion while maintaining its original height I would resize the video to 657x480px (4.11:3 aspect ratio). If I resized this video to 640x480px (4:3 aspect ratio) I would assume to end up with a horizontally condensed output.

There is a website about the camera where they address this subject as well: http://chucklohr.com/808/#Resizing720x480.

p.s.: I'm not sure if I'm still on-topic here and if this is of any value for other readers of this thread. If not, please let me know.

um3k

16th August 2010, 20:23

If the information on that website is correct, then you should set a custom pixel aspect ratio of 0.889 (8/9) and simply resize to 640x480.

guth

16th August 2010, 20:26

Airborne, Your thinking is correct. But since the pixel aspect ratio is actually 4320/4739 (= 0,9115847...), although this has been debated for some time, the width should be 720 * 4320/4739 = 656.34... And since 656 is also a multiple of 16 (which is better for codecs), I'd use that.
Anyway, if you set the destination pixel aspect ratio to "square pixels" in Deshaker, they will become square no matter what destination size you use.

airborne_video

22nd August 2010, 17:36

There is a website about the camera where they address this subject as well: http://chucklohr.com/808/#Resizing720x480.

If the information on that website is correct, then you should set a custom pixel aspect ratio of 0.889 (8/9) and simply resize to 640x480.

To close the matter:

Yesterday I found the time to record a suitable test video to determine whether the camera actually records in NTSC format with a 0.912 pixel aspect ratio or upscales a 640x480px video to 720x480px (as described on the website (http://chucklohr.com/808/#Resizing720x480) I have linked to). It turns out that the video is indeed just upscaled.

So I'm now using a source pixel aspect ratio of 0.889 (not 0.88 as written before, bad rounding on my behalf) and 640x480px with square pixels for the output.

Thank you for all the helpful feedback and comments!

loekverhees

13th October 2010, 13:08

Is it possible to use the Deshaker plugin in an avisynth script? This way, I don't have to output an enormous video file first. Also, this way I have less intermediate conversions which is better for the picture quality I think. I tried some scripts, but it didn't work. This is the script I used:

LoadVirtualDubPlugin("C:\Deshaker.vdf","DeShaker",0)

AVISource("C:\sample.avi")

QTGMC(Preset="Slow").ConvertToRGB32()

DeShaker("13|1|30|4|1.09402|1|1|0|640|480|0|1|1|1000|1000|1000|1000|4|1|0|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15|1000|1|88|1|1")
DeShaker("13|2|30|4|1.09402|1|1|0|640|480|0|1|1|1000|1000|1000|1000|4|1|0|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1.1|4|4|5|5|0|0|30|30|0|0|0|0|1|0|0|10|1|15|1000|1|88|1|1")

guth

13th October 2010, 17:10

In that script Deshaker pass 2 will be applied to the output video of Deshaker pass 1, and that won't work. Both passes should operate on the same original video. I think you'll need to use two separate scripts, one for each pass. That should work.

loekverhees

13th October 2010, 18:41

Ah, that's right of course ;). I modified the script:

loadvirtualdubplugin("C:\Deshaker.vdf","DeShaker",0)

source = AVISource("C:\sample.avi")
deinterlaced = source.QTGMC(Preset="Slow").ConvertToRGB32()

firstpass = deinterlaced.DeShaker("13|1|30|4|1.09402|1|1|0|640|480|0|1|1|1000|1000|1000|1000|4|1|0|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1|15|1000|1|88|1|1")
secondpass = deinterlaced.DeShaker("13|2|30|4|1.09402|1|1|0|640|480|0|1|1|1000|1000|1000|1000|4|1|0|2|5|40|300|4|C:\\Deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1.1|4|4|5|5|0|0|30|30|0|0|0|0|1|0|0|10|1|15|1000|1|88|1|1")

eval("secondpass")

But now the Deshaker.log is not written in the first pass. So the output is a black video. It looks like it skips the first pass. I want to avoid running two different scripts (so one for the first pass and one for the second pass) as the deinterlace step takes quite some time. So I thought that when I used only one script, the deinterlacing has to be done only once. Do you (or someone else) know how to improve this script?

guth

13th October 2010, 19:16

Since all frames have to be processed by pass 1 before pass 2 can start, you simply have to deinterlace twice, or store all the deinterlaced frames somewhere.

loekverhees

13th October 2010, 19:46

Yes, you're right. But I think the real problem is in the way avisynth processes the frames. Ideally, avisynth should first deinterlace the entire video before proceeding to the next step (the firstpass Deshaker part). And then it should do a full first pass and after that the full second pass. But I think avisynth kind of works on a frame-by-frame basis, so it does not deinterlace the entire video before the the firstpass Deshaker part is called. And that's the real problem I think. And about the skipping of the firstpass: I think that the avisynth interpreter first looks to the entire script, and then sees that in the output (eval statement) only "secondpass" is present. So it thinks that firstpass is not explicitly needed and therefore skips that part. In other words, avisynth does not run the script line after line. Could someone confirm this behavior of avisynth? Or is there still another way to accomplish my goals?

guth

13th October 2010, 20:07

But if avisynth would deinterlace the entire video before proceeding to the next step, it too would need to store all the deinterlaced frames somewhere. What you're trying to do is theoretically impossible.

loekverhees

13th October 2010, 20:26

Gavino

13th October 2010, 20:35

Could someone confirm this behavior of avisynth?
Your interpretation is basically correct.
Or is there still another way to accomplish my goals?
As guth says, you need a separate script for each pass, doing the deinterlacing each time.

guth

13th October 2010, 20:45

And what about storing like 10 or 20 frames only? Deshaker needs to 'know' multiple frames of course in order to smooth the stabilization factor among the frames I guess. But can Deshaker also do this on the fly, so without needing all frames but just 10 or 20 instead? So in other words: how many frames does Deshaker take into account when it is smoothing the displacement corrections for every frame?
Deshaker actually takes all frames into account. Of course it would be possible to modify Deshaker and limit the number of lookahead frames. There's just one problem, though... I won't do it. :p

loekverhees

13th October 2010, 21:10

Yes, I understand. I won't ask you to rewrite Deshaker :p.

I think the firstpass is not part of the 'Filter graph' (see here: http://avisynth.org/mediawiki/The_script_execution_model/The_filter_graph , thanks Gavino). That's why it is skipped I guess. It's no option for me to deinterlace the video twice, so I think it's best to render the deinterlaced video and store it phyisically. Then open it in VirtualDub and execute the Deshaker plugin there. Some disadvantages: I have to store the big deinterlaced video and I have one more intermediated video file. I guess I have to live with that :o.

Undead Sega

25th May 2011, 04:16

Hi Guth, and everyone here, it's been awhile, I've gotten another shot which I wanted to stabilize but like my previous attempt and going with your suggestions, I cannot get it to deshake. The footage itself isnt that shaky but I want to as smooth as possible. I am ware that the footage may be abit dark and soft, but it is in 1080p50, I have brighten and sharpened it in the hopes that it might improve but no joy :(

The footage is here itself: http://www.megaupload.com/?d=N6WPMO6L

abit large but everything should be there (quality wise). I hope I can get this to be deshaked and I would really appreciate it if anyone can. Look forward hearing back guys :)

guth

25th May 2011, 21:18

The file seems to be "temporarily unavailable" every time I try to download it.

Undead Sega

25th May 2011, 22:38

That is very strange, try it now as I am able to download it myself.

EDIT: By the way, the file itself is called 'temp sequence 50p'.

guth

26th May 2011, 20:53

Yes, I was able to download it now.

That clip has a few problems:

It has very little detail in the background.
The detail that is there is mostly blurry because of the rather small depth of field.
It's pretty noisy. It's not very easy to see, but there is quite a lot of noise there, which Deshaker likes to see as detail. This wouldn't be a problem if the noise didn't look similar between consecutive frames. But it does. This is either because video cameras tend to use temporal smoothing to get rid of the noise, or because of some sensor problem. And since it does look similar, Deshaker finds matching blocks all the time, incorrectly. This can, however, often be fixed by adding the VirtualDub "smoother" filter before Deshaker pass 1. (Not the "temporal smoother" filter!) You should turn it off for Deshaker pass 2.
Deshaker can only stabilize panning, rotation and zoom. But in this clip, the camera moves too (forward, backward, up, down, left, right). That motion will remain unstable.

Here are some settings that work ok. They can probably be finetuned a bit, but I can't do *all* the work for you. :)

Boulder

3rd July 2011, 14:56

Is it possible to determine the frame with the largest borders by checking the log file? I.e. is it always the frame with the largest detected horizontal or vertical pan? This also brings up a feature request: ability to limit the correction by amount of pixels needed for borders :)

guth

3rd July 2011, 22:33

Is it possible to determine the frame with the largest borders by checking the log file?
There's no easy way of knowing, no.

This also brings up a feature request: ability to limit the correction by amount of pixels needed for borders :)
The max. correction limit settings limit the max. correction (= border size) coming from x-pan, y-pan, rotation and zoom, respectively. If you want to limit the resulting *total* max. border size instead, you can use an edge compensation setting that results in no borders, and then zoom out a little using the extra zoom factor setting. With a little math or experimentation, you can convert your max. border pixels to a zoom factor.

Boulder

5th July 2011, 20:37

The max. correction limit settings limit the max. correction (= border size) coming from x-pan, y-pan, rotation and zoom, respectively. If you want to limit the resulting *total* max. border size instead, you can use an edge compensation setting that results in no borders, and then zoom out a little using the extra zoom factor setting. With a little math or experimentation, you can convert your max. border pixels to a zoom factor.That's not exactly what I meant :) I do not wish to do any resampling in Deshaker as there will be also resizing when the video is played on my TV.

I meant that I'm willing to sacrifice "n" pixels of each side throughout the whole long clip so if I could tell Deshaker to do a maximum correction of worth "n" pixels of borders, I could crop off that many pixels from each side and have no borders in the final video.

guth

6th July 2011, 09:20

That's not exactly what I meant :) I do not wish to do any resampling in Deshaker as there will be also resizing when the video is played on my TV.

I meant that I'm willing to sacrifice "n" pixels of each side throughout the whole long clip so if I could tell Deshaker to do a maximum correction of worth "n" pixels of borders, I could crop off that many pixels from each side and have no borders in the final video.
Then simply use an edge compensation that results in no borders. And if you for some reason want a smaller video size (to get your cropping effect), just change the destination video size.
The max. correction limit settings affect how much zoom will be added do avoid the borders.

About the resampling, Deshaker will resample once no matter what settings you use. It can't stabilize without doing a resampling. But it will never ever do two resamplings. Using the extra zoom factor setting, for example, or using edge compensation zooming, won't cause another resampling.

Boulder

6th July 2011, 10:29

guth

6th July 2011, 11:04

Then I'm mistaken as I thought that without edge compensation Deshaker only shifts the image (and does not do any kind of resizing) to stabilize the video. So basically what I'm after is to limit this shift to be max n pixels in one frame horizontally or vertically. Kind of like what DePanStabilize in Avisynth has in parameters dxmax and dymax.
Deshaker stabilizes panning (shifting), rotation and zoom. The two latter definitely require resampling. And panning does too unless you always pan in whole pixels. But Deshaker pans in fractions of pixels, so resampling is actually required there too.

You are right that Deshaker doesn't resize the image when you don't use any edge compensation (and also disable zoom stabilization and extra zoom factor and use same destination video size as the source). But "resize" and "resample" aren't the same thing.

Usa1955

19th July 2011, 21:52

Deshaker works well for me, but how do I increase it's amount dramatically—so almost all camera motion is removed?

In the video I'm trying to sabilize all motion is not wanted and I target a not moving element in a static scene.

guth

20th July 2011, 07:51

You control the smoothness with the motion smoothness parameters. You can even set them to -1 to remove camera motion completely (i.e. let the position be fixed at the position of the first frame). If the video still moves after that it's not because of camera motion, but because Deshaker has problems in the matching process during pass 1.

I'm not sure why you just target a single non moving element. Normally, you get much more reliable output if you target the whole background. Otherwise, especially zoom and rotation probably won't be reliably detected, so you might want to turn off the settings "detect zoom" and "detect rotation" if you must target a single element.

Also, you might want to decrease the value of "discard motion of blocks that move > X pixels in wrong direction", to really ignore everything that moves just a tiny bit.

guth

31st July 2011, 16:26

Then I'm mistaken as I thought that without edge compensation Deshaker only shifts the image (and does not do any kind of resizing) to stabilize the video. So basically what I'm after is to limit this shift to be max n pixels in one frame horizontally or vertically. Kind of like what DePanStabilize in Avisynth has in parameters dxmax and dymax.
I forgot to mention that you obviously *can* make Deshaker only shift the image, if you set motion smoothness for rotation and zoom to 0, edge compensation to "None" and control the max shift with the max. correction limits (in percent). Then, after Deshaker, you crop the borders in VirtualDub.
And if you don't want to degrade the quality of the frames at all, you can set the resampling method to "Nearest neighbor". Then, it will pan in whole pixels. I'm not saying it's a good idea, though.

Gargamel

20th August 2011, 18:14

Deshaker v.3.0 on guth's page (2011.08.14).

Look at the list of improvements... A great version.
Thank you, Gunnar !

hiviking

1st September 2011, 08:26

A review of Deshaker 3.0 with some tips and tricks is now available at http://deshaker.blogspot.com (http://deshaker.blogspot.com/).

wonkey_monkey

1st September 2011, 16:12

Hey Gunnar, have you thought (more? I may have suggested this before) about allowing for perspective in the resampling stage? If you take a look at the videos on the blog page hiviking linked to, you can really see the wobbling that's introduced at the edges. I did some tests a long time ago, and taking account of perspective improves things a lot on wide angle videos.

David

MaxForce

16th March 2012, 14:16

The software is amazing! The default settings work perfect on every video I have ever tried. It might even work too well and the videos look unrealisticly smooth. My only tip is, right at the end, to limit the Max correction for horizontal and vertical panning to 5%. It makes videos look smooth but not weird.

Thank you so much for such a wonderful product!

guth

16th March 2012, 17:19

MaxForce, I'm glad you like it!

David, I missed your latest post. I'm afraid there's just too much work for me to want to deal with perspective. (Determining FOV (which can change within a clip), filling borders from past/future frames, ideally also take perspective into account during matching in pass 1, and all this combined with rolling shutter distortion etc...)

CruNcher

17th March 2012, 08:00

Is there any compare between AMDs 1st and 2nd Generation Hardware implementation and Deshaker ?

guth

18th March 2012, 13:37

Don't know.

DESuser

17th August 2012, 08:28

I am not quite sure is such a way of using the deshaker IS in accord with this topic, i have to mention this :
The _author_ intented it not only for a simulated camera motion within short stabilizitions like usual setting, hi did mention that filter is capable of producing a panoramas, where missing data is filled by blur or frames. Going further with a low resolution videos like 3gp sources or lowq phone videos being used to make a dvd movie there pops up a limitation of the filter. First one, which the author may not seen is when original camera motion IS slow eg from left to right and past-future data become unusable. Solution is to make two passes with odd and even frames, and then combine these two deshakes. Nasty thing. Next, the fillrate - every user in the world is DISsatisfied with the fillrate of that tricky blurry colour. Just disabling it brings speed improve, AND new passes of doing. I had a 9K+ frames video to be done within 29 hours on E2160 1800. Extremely slow fillrate of that. Restoration i did is to use surrounding frames of course
, and these are not the usual 30+-, good resullts for a 480p to 576p are 300+-, better ones are 450+-, and for 144p to 576p are 800-900+- and the 1000 i used too. It is just not enough for a usual scene duration of 2000-3000 frames.
It seems to me that algorithm used by author is not done neatly, it immediately take up to 2g ram and still may produce out of memory on some scene changes.
AND these is a so slow processing with no camera motion parts of any video. The movie ive done is all to last frame was restored andor deshaked via it.
LOTS and surely tons of videos are to be unfolded to greater resolutions depending on original camera motion, and those amateur shaky videos are the first to it, they really contain MORE data. And the final solution of such deshaking is a simulated panned zoom to avoid bad parts, there are currently no good ones ive seen for vd1911.
Does anyone have any ideas ?
Maybe the author may have some words ?
:thanks:

guth

17th August 2012, 17:01

Yes, Deshaker *is* extremely slow when it comes to creating panoramas, but that's mainly because Deshaker is a stabilizer and not a panorama maker.

I know the "extrapolate colors" feature is extremely slow, but for normal use it's rarely used, I think. (When I made that feature, I prioritized quality over speed, but to be honest it got a lot slower than I thought it would be. You're the first person to complain about this, though.)

If you use previous and future frames to fill in borders, Deshaker saves all those frames in memory, so if you use a lot of them, it will use a lot of memory. And since Deshaker, for now, is only 32-bit, it's limited to 2GB memory (including memory used by VirtualDub). You can patch the 32-bit VirtualDub to make it use up to 4GB memory, though, by running a "4GB patch". (Google it if you're interested.)

DESuser

18th August 2012, 03:24

Yes, thats great, i will try it. Thank you Guth, this yours piece of work is quite usable, there are no same level software, and you are the One. There no substitutes, and msu deshaker visually being alike is not to be available. Many people just do not give a try with the way i described, simply not being familiar with the configuration. Though there is a solution for such this limit, by inserting already calculated restores into video like simulated pan and thus the far-frames data may survive the limitation. And this should be called not a pan but a full scene view, surely depending on camera motion within a certain rectangle. Even HD video may be toss-n-played this way.
Despite that you do not want to progress this, may be there any workaround to recompile it to 64 bit and just increase the number bounds ? And yep, it seems no sources on page. Anyway, it is ok.
I think that deshaker may be used for deinterlacing and result to be completed with DGbob or ELA-2 or else, i just had not time, with the ideas still in mind.
And, is that you point is like a /LARGEADDRESSAWARE patch ive seen somewhere ?

DESuser

18th August 2012, 03:36

And yes the to-complete-a-dream bounds might be like this - alpha blended stream support, unlimited frame bound, gui for manual correction with 1.5 pass and 2.5 pass for a pan-zoom graphs timeline, revised data handling by using a 16x16 image blocking to save ram and speed, gpu rendered output with vignettes at least, built in-pre-pipeline strong sharpening and levels - and that be kiiler-stabilize open source application. Thats it. Within years to come none be able to supercede? while it already is so.

guth

18th August 2012, 07:28

And, is that you point is like a /LARGEADDRESSAWARE patch ive seen somewhere ?
Yes.

I don't have a 64-bit OS yet, but I can almost promise there will be a 64-bit version of Deshaker when I get one (not too long after Windows 8 is released).
As for other new features, I'm not sure. I'm afraid I don't have anything new planned at all... :(

DESuser

18th August 2012, 13:26

It works, takes up to 3.9 Gb within vd process, nice, deshaker takes four cores to work, much better now. Usual speed of 0.2 to 0.06 fps now greatly superceded. It is within W7x64U 8G/2500K@4.1G.
Thank you for reply, Guth.

isidroco

21st September 2012, 21:28

Hi guth, thanks for your excellent software. I found out that reducing previous/future frames to 24 (maybe 25) stopped "Out of memory" situation in FullHD videos under 32 bits windows (at least if using h264 crf). So you should recomend that setting for those cases (unless you use 4gb patch in 64bit os).
I noticed some slight back and forth zoom waving in some still videos I deshaked (camara was on a tripod), is there any way to avoid that? Thanks!

guth

22nd September 2012, 08:41

I found out that reducing previous/future frames to 24 (maybe 25) stopped "Out of memory" situation in FullHD videos under 32 bits windows (at least if using h264 crf). So you should recomend that setting for those cases (unless you use 4gb patch in 64bit os).
Nice to know.
I can use more than 30 previous/future frames for full-hd video. But if you encode with h264, I guess it probably uses a lot of memory.

Also, the 4GB patch works on 32-bit OS too. You won't get the full 4GB memory, but I can use more than 70 previous/future frames (i.e. 140 in total) for full-hd video on my 32-bit Vista if I do this.

I noticed some slight back and forth zoom waving in some still videos I deshaked (camara was on a tripod), is there any way to avoid that? Thanks!
Here's from my FAQ:

-----------
- I get unwanted zooming in the stabilized video.
Deshaker can add two types of zoom. One is for stabilizing zooming that it detects in the video. If this detected zooming isn't really camera zoom (for example, Deshaker usually detects zoom if the camera moves forward), or if you just don't want the camera zoom stabilized, you can turn it off by setting the zoom motion smoothness to 0. It's always a good idea to do this if there's no real zooming in the video.
The other kind of zoom that Deshaker can add is adaptive zoom. If you don't like it, choose another Edge compensation option.
-----------

Let me know if you still can't get rid of it.

isidroco

28th September 2012, 15:19

Memory limit was not only because of h264 vfw but also because of AVISYNTH used to provide MOVs to virtualdub. Using Mp4Cam2avi to convert MOVs to AVI without recompression solved the need of AVISYNTH. Tried without Avisynth +-45 frames with a small sample without problems (I will try to make more tests).

As for small waving in deshaked videos, using Scale Full (most precise) has improved A LOT. What I don't know if it's better using All pixels (veeery slow), every 4th (what I'm using now) or every 9th.

I made a batch file with virtualdub script for deshaking all avis in a folder without user intervention. Here's the post with it's source: http://forums.virtualdub.org/index.php?act=ST&f=5&t=17772&st=30#entry91160
Hope it helps others.

guth

28th September 2012, 17:10

As for small waving in deshaked videos, using Scale Full (most precise) has improved A LOT. What I don't know if it's better using All pixels (veeery slow), every 4th (what I'm using now) or every 9th.
If you set zoom motion smoothness to 0 and don't use an edge compensation option with the word "adaptive" in it, the zoom waving should be completely gone. But there can still be other types of waving.
I don't know the exact reason for your waving, but you might want to try this FAQ suggestion as well:

--------------
- I get a slight waving effect when objects move slightly, even when the camera is completely still.

Try decreasing the value for Discard motion of blocks that move more than X pixels in wrong direction.
--------------

Use as low value as you can without getting too many red vectors on the background areas. This should hopefully ignore objects that move even slightly. If the camera is somewhat stationary (but still panning/rotating or zooming), you can often use values as low as 1.0, or even lower.

"All pixels" is always better than "Every 4th" and "Every 9th", but the improvement is usually too small to see. If you can't see a difference, it's probably not worth using a slower setting.

isidroco

28th September 2012, 17:52

I will try your suggestions, but I do use zoom so I rather stabilize it too. A very useful option in pass2 would be: Ignore zoom corrections below: x% with a low default value of x=0.1% (0% would be current behaviour). Because much of the waving is very slight with zoom values very close to 1 in LOG, this would force to deshake only when zoom is significant.

guth

28th September 2012, 18:35

A very useful option in pass2 would be: Ignore zoom corrections below: x% with a low default value of x=0.1% (0% would be current behaviour). Because much of the waving is very slight with zoom values very close to 1 in LOG, this would force to deshake only when zoom is significant.
Do it yourself! :D
Seriously, though, you actually could do it yourself, by changing all zoom values in the logfile that are close to 1, to 1. Use Excel, for example.

isidroco

30th September 2012, 01:57

Do it yourself! :D
Seriously, though, you actually could do it yourself, by changing all zoom values in the logfile that are close to 1, to 1. Use Excel, for example.
Yes, I know that, but it's not practical, I should program it in pascal, because I need to do it in hundreds of LOGs (the whole point of the script I've done is to deshake automatically all my short videos). Still I think it could be an easy option to add in a future deshaker version.
Talking about that, LOGs numbers are in absolute pixel movement or as a multiplier?

Thanks for your answers :)

guth

30th September 2012, 08:39

I should program it in pascal, because I need to do it in hundreds of LOGs (the whole point of the script I've done is to deshake automatically all my short videos). Still I think it could be an easy option to add in a future deshaker version.
So, do it in Pascal then.
I don't really like the idea of such a setting. But it's also not the worst suggestion I've got. :)
We'll see...

Talking about that, LOGs numbers are in absolute pixel movement or as a multiplier?
If you're talking about the zoom values, it's a zoom factor (i.e. multiplier). There's a chapter "Log File Format" on my Deshaker page, btw.

fbs

7th October 2012, 04:23

any thoughts on how to deshake a VHS content with lots of noise and obviously low resolution? :D

guth

7th October 2012, 08:36

any thoughts on how to deshake a VHS content with lots of noise and obviously low resolution? :D
I've never tried it, so I'm not 100% sure.

The thing with Deshaker and noise is that it shouldn't have much trouble with "clean" noise. But if the noise starts to look similar between two frames in the video, Deshaker might find an incorrect match on it. I would guess that the original noise on the VHS is clean in this regard, but if you encode the video with a codec that analyses differences between frames (which means most codecs), or if you use temporal smoothing on the video, the noise will show similarities between frames. So, you'll probably want to use a codec that is either lossless, or encodes each frame (or field) individually, such as MJPEG or DV, when capturing the VHS video. (I'm not sure if MJPEG supports interlaced video, though.)

If you can't avoid noise that looks similar between frames, you can use the Deshaker setting "Discard motion of blocks that have max. pixel diff < X" to discard all blocks that have more noise than real detail. Or you can try using a good spatial denoiser before Deshaker pass 1. Never use a temporal denoiser before Deshaker pass 1, though.

If you can't make it work, you can email me a few frames if you like.

fbs

10th October 2012, 19:35

I'm capturing it with lagarith.. complete lossless.. and no temporal filters before (just one spatial CHROMA denoiser).. the strange thing is that despite using higher values in "max. pixel diff" and "match value" it's matching "invisible block" lol.. look at the shot:
http://i.imgur.com/J9Vee.jpg

guth

11th October 2012, 17:29

the strange thing is that despite using higher values in "max. pixel diff" and "match value" it's matching "invisible block" lol..
What do you mean by "invisible block"? All the motion vectors were found on extremely contrasty areas, so I'm not sure I see what's so strange. (At least not without seeing the previous frame.)
The red vectors are a bit weird, though, but those are because Deshaker has some problems with straight lines. Usually such vectors will be marked bad (red), though, which is the case here too.

wonkey_monkey

11th October 2012, 18:02

Maybe fbs is making the same wrong assumption that I always have, until now - that the vectors shown are from the center of a block. But they're not, are they? They're showing the corners of a block!

This is like when I realised that Superman isn't spinning the Earth backwards, he's just travelling back in time around it (though that doesn't explain why he has to spin it up again).

David

guth

11th October 2012, 18:41

Maybe fbs is making the same wrong assumption that I always have, until now - that the vectors shown are from the center of a block. But they're not, are they? They're showing the corners of a block!
No, the vectors are at the center of the blocks. But if it's the leftmost red vectors you and fbs are wondering about, those are becuase vectors always occur in groups of four when the Scale setting is "Half". Try setting it to "Full" instead.

This is like when I realised that Superman isn't spinning the Earth backwards, he's just travelling back in time around it (though that doesn't explain why he has to spin it up again).
:confused:

guth

18th November 2012, 17:54

There's now (finally) a 64-bit version of Deshaker...

http://www.guthspot.se/video/deshaker.htm

No new features, unfortunately.
Please report any problems.

isidroco

20th November 2012, 18:40

Don't have a 64bit os yet :)

I finished some scripts for deshaking all AVIs in a directory. I found out that it could be very useful that deshaker generates and use LOG files with the same name (and same folder) as AVIs. (though I don't know if filters can ask AVI filename to VDUB)...

guth

20th November 2012, 19:37

(though I don't know if filters can ask AVI filename to VDUB)...
They can't, as far as I know.

djonline

1st January 2013, 14:50

There's now (finally) a 64-bit version of Deshaker...

http://www.guthspot.se/video/deshaker.htm

Thank you!
+20% on first pass, +4% on second pass (using 30 future frames), 1080/60p, i7-860.

WarpEnterprises

1st January 2013, 16:28

I found out that it could be very useful that deshaker generates and use LOG files with the same name (and same folder) as AVIs. (though I don't know if filters can ask AVI filename to VDUB)...

I have similar needs, my solution to this to use AviSynth for at least pass2. This works very well, if the deshaker settings are the same for all files.
I do pass1 and pass2 with the same source script and switch off deshaker when the calling program is VDMod (which I use for pass1, having to enter the log file name manually), but not when the script is called from tmpgenc (which I use for pass2).
The main script is an AVSI in the AviSynth plugin dir, which does the loading of the MOVs and the deshaking:

LoadVirtualdubPlugin("C:\appl\VirtualDubMod\plugins\Deshaker33.vdf", "deshaker3", 1)

function mov43(name)
{
name = default(name, leftstr(scriptfile, strlen(scriptfile)-4))
vdm = (GetProcessName() == "virtualdubmod.exe" || GetProcessName() == "virtualdubavs.exe")
tmpg = (GetProcessName() == "tmpgenc.exe")

QTInput(scriptdir + name + ".MOV", audio=1, color=1)

log = vdm ? "" : scriptdir + name + ".log"
cmd = "18|2|30|4|1|0|1|0|640|480|1|2|500|500|500|0|4|1|6|2|15|30|300|4|" + log + "|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|0|0|10|1000|1|88|1|1|20|500|100|20|1"

vdm ? nop() : deshaker3(cmd)
tmpg ? normalize().ConvertToRgb24() : nop()
return last
}

That way for each source clip (MOVs in my case) a single AVS with a single line is necessary, e.g. file P1020634.avs for P1020634.MOV
mov43()

Needs helper plugin from here (http://forum.doom9.org/showthread.php?t=166805) and the latest AviSynth to get the name of the script file.

Hope it helps!

lansing

2nd January 2013, 03:39

hi, i have been trying to use deshaker to fix the film weave problem existed in old anime, where there's shakes inside the frame. Like in this example:
http://www.mediafire.com/?b5mh4xf32kzf1mc

i can't get the setting to detect the shake on the purple cloth.

WarpEnterprises

2nd January 2013, 09:18

It seems that the shake is not consistent through the whole frame. Deshaker won't take only "your" foreground object but the WHOLE frame (which is quite unshaky).

lansing

2nd January 2013, 17:24

thanks for the reply, seems like deshaker is not suitable for my case then

bcn_246

7th January 2013, 23:29

Just wanted to take a moment to say thanks for all the work you have done on this filter :)

guth

8th January 2013, 17:19

Just wanted to take a moment to say thanks for all the work you have done on this filter :)
Thanks for your thanks. :)

fvisagie

9th January 2013, 09:50

Just wanted to take a moment to say thanks for all the work you have done on this filter :)

Ditto. I use it fairly regularly now and I never cease to be amazed by the results :).

DESuser

18th March 2013, 06:03

Since mr. Gunnar personally mentioned about "no new features", a new look is quite enough.. But not.
If it is really 64 bit-wise memory handling, i will test it seriously, i ve bought a 32 Gb DDR3 for Z77 chipset, lets see, is this so.
At least such software as APG 303 and PS CS6 64 bit will also take advantage, but i wish to benefit this filter first.
The problems to be repeatedly avoided are still same, these are not to be called features, ... later.
1. Deshaker fails to simulate cam path in the very simple scene with slow left to right motion witn zooming out - a still picture of a distant mountain. Somehow it recognises a part of image as an object and voila - single scene breaks into unsolvable thrashing. Why is it so, see point 2.
2. Deshaker Hundred% fails to properly simulate after very slow panning motion, left and right borders filled improperly loosing all available details to a colourful trace, while in-tests by reducing a frame twice|thrice rate does okay. Also the motion detection, since the times i ve seen AutoPanoGiga 251 and later, is weak. Naturally a half of scenes to be detected with full resolution in both settings. Here comes point 3.
3. VHS videos with low contrast and colour noise, blocky mobile shots are all fall into category of full precision detection. Because scene detection is weak. and because motions detection do NOT have any settings to be changed. Few filters in VD show WHAT they see, like noise removers, here only mr. Gunnar saw all it's intermediates. THUS there needed a colour denoising, wide distance gamma correction, and sharpening. Happily all needed sharpening may be done by msu smart sharpen with mad setting, quite ok. All noises easily removed by Neat Video, it is not free, but even old versions quite enough. But it is after, mostly. Low resolution have a commercial solution called Video Enhancer, and free solution is just to twice resize before deshaker and then sharpen.
4. Low frame rates are okay if feed 25 fps to deshaker, but it FAILS to work after interpolate filter, and a few sequentially incompatible also be named here.
5. A full set of weapons to be handled with DESHAKER are - Fast CPU at least 2 Ghz multicore, next come RAM. And NO lagariths and x264 lossless modes. Usually deshaker thrashing to swap and does something of its matter without 100% load, especially with much of frames in scope, due to weak frame handling. I will not be surprised if they ARE stored in very redundant format, at least twice. Since that a good codec to be used is MJPG at q=1 or MJPG2K, or yes lagarith with size of 640x480 max IF you have SATAiii storage and or raid.
The filters to be used to achieve a good result are - increasing resolution, removing noise, increasing frame rate to NEEDED, and then deshaking. Detection to be done one way, final deshake - hardly another.
6. Last point - Deshaker really will increase resolution with a zooming factor, since most unuqie and rare videos are shot randomly without proper conditions.
7. Save your time - i found that deshaking usual boring panning scenes with nothing that moves is useless. Feed each 5th frame to panorama maker. thats it.
8. Save you nerves - feed hard to detect cuts of 50-200 frames to autopano giga or same level software - you will find it interesting when deshaking by hand the layers of resulting PSB file. A still photo may be pan-zoomed later.
9. Do NOT fear change the settings, since mr. Gunnar optimised it for defaults you may have no choice but to try everything, just read his site and know their initial purpose.
10. Area detection is to be revised. Oh yes.. Nope. None will do it himself.
11. The last solution - Adobe AE with their stabiliser - a wobbly shadow of deshaker, i still wonder - is there hand optimizing available.
I mean area detection fail. sometimes.
Thanks for reading.

fvisagie

28th March 2013, 11:36

Hi Gunnar and everyone,

Say you've analysed your whole input clip in Deshaker but after trimming etc. only some frames are rendered for output, will those frames get the full benefit of [x] Previous and future frames to fill in borders, including from frames not rendered for output? Or would one need to save the complete deshaken clip to an intermediate file to get that benefit?

Thanks,
Francois

guth

28th March 2013, 13:17

Say you've analysed your whole input clip in Deshaker but after trimming etc. only some frames are rendered for output, will those frames get the full benefit of [x] Previous and future frames to fill in borders, including from frames not rendered for output? Or would one need to save the complete deshaken clip to an intermediate file to get that benefit?

If you save frames A to B, Deshaker pass 2 won't use frames before A, but it will use frames after B. So you might want to save to an intermediate file first (preferably using a lossless codec).

But, unless frames A and B+1 are marked as new scenes, the motion of both frames before A and after B will still affect the output of just A to B. This may or may not be what you want.

In general, I would probably either make sure A and B+1 are marked with "n_scene" in the log file, to make sure frames outside A to B aren't used at all.
Or, I would output frames A-50 to B (if you use 50 previous frames in settings) to fully make use all frames. And then remove A-50 to A-1 afterwards.

Not ideal, I know...

fvisagie

28th March 2013, 18:50

Not ideal, I know

But your detailed explanation of what to expect in the various scenarios empowers the user to make informed decisions. :thanks:

In this particular instance, it will be much more straight-forward for me to losslessly save a full intermediate file (as I normally did anyway ;)).

asaleo

7th April 2013, 18:21

This post is part a tribute to the creator of Deshaker and part sharing of settings for "an impossible video"
I have a lot of windsurfing videos taken with a helm camera.
Since frames contains a mixture of shore with sky and other frames with board, sail and water I regarded these videos to be impossible to stabilize.
There is only a static background on the frames with shore line and sky.
Frames without a static background should not be stabilized since they miss static area.

Following settings made it possible to stabilize even these videos.
By restricting analyze area to the upper part of the frame which containes the sky and shoreline and chosing Full scale and All pixels together with some other changes it was possible to get stabilization on shore and sky frames and skipped frames on the other frames.

Pass 1
Block size 20
Scale Full
Use all pixels
..have match values < 133 (1/3 of block)
...move > 2 pixels....
Skip frame if < 5 or 4 % .... I check the white and red arrows in the beginning and decide from what I see.

Since analyze area is reduce you can use full scale and use all pixels without too much analyze time.

Pass 2
Here I have to lower the motion smoothness to 100 in order to reduce the needed zoom factor.
I do not want to use higher zoom factor than 1.05 to 1.1
Thus I reduce motion smoothness to avoid to many black borders.

Sometimes I have used lower values for max correction limits.
I decide how many mm movement of frame I permit on my screen since I do not want to use high zoom value.
I then calculate the percentage to put into the max correction.

Above settings works best on video with clouds on a blue sky. There must be something for Deshaker to fix on to compare frames.
I have not managed to stabilize frames with a blue sky without clouds. A grey sky is also impossible.

Do any of you have settings to share to stabilize difficult videos?
Can I refine my settings?
Regards

guth

7th April 2013, 19:32

Thanks for sharing.
I would also set zoom motion smoothness to 0. Always do that when there's no zooming in the video.
And based on your "match values < 133 (1/3 of block)", I think you misunderstood that setting slightly. It says "match value", without an 's'. It's always a value between -1000 and 1000 indicating how similar two blocks look, regardless of block size.

asaleo

8th April 2013, 10:06

Honored to get answer from you!

I now compared settings for match value < 133 and 300. Luckily for me there were no difference in written logfile.
Zoom smoothness shall be 0 of course in my case.

I have noticed that if I use Tateau quicktime plugin for importing mp4 files into VirtualDub for deshaking it will mostly go wrong when Mode 3 (ICM Decomp) is used. Mode 3 is recomended for mp4
I now use mode 2 (which suites .mov files) for mp4 files to get a proper stabilization. Values for frame number 1 in the logfile created for a mode 2 (decomp sequence) file is identical to frame number 2 in the logfile for mode 3 (ICM decomp) file.

I enter this post as a warning. When I encountered this first time I had to spend a lot of searching for what has happened.

Regards

DESuser

27th November 2013, 04:01

Hello everyone.
Me dont surely aware did anyone stated it, so the question is about using 64-bit version within 32 Gb memory configuration.
There is nothing hard to build such, just get and install. Now it is really cheaper than say 10 years before.
Ive used kingston hypex modules, four plates. Overlocking, if applied, is a matter of fun here.
Ive tried truly big frame sizes, like 6000x2000, but with static renderings @ -1 setting there.
Still, there is an bug which makes static scene to move left or right, depending on internals.
Thats why completely staqtic renders ought to be done within such huge resolutions and then redeshaked.
Rare case, but always have to, if going static. Typical RAM load up to 8-10 Gbs.
With settings i always use, a 2000 frames scope, is enough for full HD videos taking up to 28 Gbs.
Of course, the videos with complex movement may require only even -5 +5 scope to make it a recognizable vision.
It looks fun to see how frames follow real motion, it really increases perceptible vision field.
So conclusion is - it is okay. It works. All the problems same in comparison with, just do know and try.
As the days come by, experience is a key.
What another strange thing are to be met with ?
Say.. The trajectory of calculated motion may suffer from things like river flow or any secondary motion, no cure to this may be found except some corrections to logfile. But how ?
When rendering VGA sizes to HD at scale like 0.75, any camera returns to initial places may and will suffer from displacement. The combined pictutre will be somewhat distorted.
The blur algo is still one core slow-mo at go-wait-an-eternity performance.
May be there are some solutions to log file, like visual correction..
It works.

BabaG

11th January 2014, 23:12

guth

12th January 2014, 17:29

maybe this has been asked and i didn't see it but, is there a way to use deshaker to transfer movement from one clip to another? an example might be to add a hand-held feel to a static scene. it seems to me this could be a useful motion tracking feature. could one do something like run pass one on scene A, then run pass two on scene B, using the scene A tracking log? exactly how might one do something like this?
If you just want to add hand-held feel to a static scene, you should be able to do just what you said; run scene A through Deshaker pass 1, open scene B, and run through Deshaker pass 2 without changing the logfile inbetween.
If you, however, want to add exactly the same motion that scene A has to scene B, you could maybe do the following:

Run scene A through Deshaker pass 1.
Open the logfile in Excel or similar and change the values so that the motion is inversed. (You can get an approximate inverse by changing the sign on panning and rotation and do 1/x for zoom).
Open scene B and run through Deshaker pass 2, having the motion smoothness settings set to -1.

I haven't tried this myself, though, so I'm not sure how well it works. It might not work very well at all if there's a lot of rotation or zoom.

BabaG

13th January 2014, 02:02

thanks, guth. it may be a while before i get to try this myself. it's good to know that it may be possible and seems somewhat easy. would be a handy feature, i'd think, were it included in deshaker. would turn it into more of a motion tracker. might be as simple as an 'invert' button.

the thing that got me thinking about it is a sequence i have in something i'm working on that was shot on a set in a static car. i thought i might set up a mount in a moving car and record a couple of minutes of driving with markers on the windows to give good reference for movement. i thought this might be a good way to accurately record the bounces and jiggles of driving. i thought i might, then, try transferring this jiggle to the static image. i used deshaker once as a test to try removing film weave and jitter in some of my transferred material and it worked quite well. having done that, i thought this might be something else useful to try.

thanks again,
BabaG

scharfis_brain

16th July 2014, 23:07

I've a feature request for deshaker.

Let me explain my issue:

I was given a ten years old VHS-tape of a friend, which I try to stabilize.
The video is overlaid with a date/timestamp like we all know it from the VHS-era (bold and ugly)

So I need to remove this timestamp prior to video stabilization, in order to avoid a floating-around timestamp in the stabilized video.
Currently I use a mix of avsinpaint an some global motion compensation (with temporal smoothing) to fill in the area of the timestamp.
This process works only roughly in estimating how to fill the missing parts of the image and it i horribly sloooow.

Now here is my feature request:
- let the user load a mask-file for pass two. the mask file makes the video transparent (ie. black like uncompensated borders) in the parts where to remove a logo or timestamp
- (alternatively I can fill the area with a custom masking color, which I will specify in pass two)
- then the magic of deshaker will apply in pass two. previous and future frames used by the edge compensation will fill the masked area with texture. parts of the masked area which couldn't be filled by the prev/future frames will remain in their masking color(alternatively they can stay black) or they will be filled by the algorithmy deployed in 'extrapolate colors'

raffriff42

16th July 2014, 23:33

So I need to remove this timestamp prior to video stabilization, in order to avoid a floating-around timestamp in the stabilized video...Pass 1 parameters, Ignore Pixels Inside (Left, Top, Right, Bottom)

scharfis_brain

17th July 2014, 00:23

that only affects the motion analysis.
but it doesn't affect rendering of pass two. I need the masked area to be filled with the contents of the previous and future frames.

raffriff42

17th July 2014, 01:13

Oh sorry, I read that wrong. Yes, it's strange to have a stable video with moving titles or whatnot.

guth

17th July 2014, 11:04

Now here is my feature request:
- let the user load a mask-file for pass two. the mask file makes the video transparent (ie. black like uncompensated borders) in the parts where to remove a logo or timestamp
- (alternatively I can fill the area with a custom masking color, which I will specify in pass two)
- then the magic of deshaker will apply in pass two. previous and future frames used by the edge compensation will fill the masked area with texture. parts of the masked area which couldn't be filled by the prev/future frames will remain in their masking color(alternatively they can stay black) or they will be filled by the algorithmy deployed in 'extrapolate colors'

I suspect this maybe won't work quite as well as you image. Usually, there are sections of the video that don't have much shake at all. In these cases Deshaker won't be able to access the image data it needs, because it either doesn't exist at all, or it exists in frames too far away in time (Deshaker stores the frames in memory, which is usually limited). The reason why this still works rather well for borders is that in sections without much shake, there won't be a lot of borders to fill, becasue any corrections made by Deshaker will be rather small.

Have you tried any filters/plugins that remove logos? MSU Logo Remover seems to use motion analysis, for example. I haven't tried any myself, though.

guth

17th July 2014, 11:52

Btw, davidhorman, if you're still reading this thread. Do you remember 7 years ago when you asked in this thread why I didn't use something similar to Gaussian blur to calculate the smooth motion? I basically didn't have a better answer than that my solution felt more "mathematically correct", and that it seemed better to minimize the *squared* corrections. I now finally think I have a better explanation. :)

My algorithm, as said before, minimizes the squared corrections and the motion curvature. If we take an input motion path with absolute positions 10, 20, 30, 40, 50, then blurring these values might give you something similar to 28, 29, 30, 31, 32, which means corrections will be made.
Deshaker, however, will already think that this is as smooth as it gets, since the original motion has curvature (2nd derivative) zero. So it won't make any corrections, which is generally what we would want, right?
Sorry, but I just had to get this off my chest... :)

The newer adaptive zoom smoothing does, however, use (true) Gaussian blur, for various reasons.

wonkey_monkey

17th July 2014, 21:25

I am still reading this thread, and yes, I do remember asking you that 7 years ago - I hope it hasn't been keeping you awake at night all this time!

If we take an input motion path with absolute positions 10, 20, 30, 40, 50, then blurring these values might give you something similar to 28, 29, 30, 31, 32, which means corrections will be made.

I'm trying to remember what I did when I wrote an AviSynth script to test 3D correction using a Deshaker log file. Maybe I took the relative movements - 10, 10, 10, 10 in this case - and smoothed those...

Coincidentally the 3D correction thing has been on my mind again lately. I've been meaning to do more tests, as those preliminary results all those years ago showed it to be good for counteracting the corner wobble on wide angle videos.

David

scharfis_brain

17th July 2014, 21:39

I suspect this maybe won't work quite as well as you image. Usually, there are sections of the video that don't have much shake at all. In these cases Deshaker won't be able to access the image data it needs, because it either doesn't exist at all, or it exists in frames too far away in time (Deshaker stores the frames in memory, which is usually limited). The reason why this still works rather well for borders is that in sections without much shake, there won't be a lot of borders to fill, becasue any corrections made by Deshaker will be rather small.
I am aware of this. But:
1) this video is totally shaky
2) it contains lots of rapid zooms
3) I know that areas which have no camera shakiness will leave the 'hole'. But this can be masked afterwards in avisynth and filled with some inpainting. Alternatively it can be filled using the alorithm you use to extrapolate the colors to the border

Have you tried any filters/plugins that remove logos? MSU Logo Remover seems to use motion analysis, for example. I haven't tried any myself, though.

I've tried them. They won't work as advertised.
MSU just will move some blocks along its crappy motion estimation/compensa tion. It is not aware of scene changes. It looks really ugly no matter how I set the parameters.
ExInpaint will create a chaotic bocky result, cause it copies random image blocks into the masked area.
The only solution feasable was AVSInpaint. But It completely lacks image texture. So I tried DepanInterleave with AVSinpaint and some heavy temporal smoothing. This works kinda, but it is horribly slow. The more direct approach would be deshaker with a masking feature which makes parts of the image transparent.

If you've got some time it would be nice, if you could inpement something to make parts of the video transparent. eg by a mask file or by a keying color.

guth

18th July 2014, 21:11

scharfis_brain, I just sent you a PM with a test version that does color masking.

guth

18th July 2014, 21:18

Maybe I took the relative movements - 10, 10, 10, 10 in this case - and smoothed those...
Hmm... indeed, that seems to fix that little problem.
But just wait... in another 7 years I'll come up with more arguments! :)

As for 3D correction... that would be nice. But I still don't think I will attempt it. I believe I gave reasons before, such as needing FOV etc. Too many problems and too much work. :)

wonkey_monkey

16th August 2014, 23:36

Hey guth... I did it (http://forum.doom9.org/showthread.php?p=1690146#post1690146) :)

As for the smoothing problem, I got around it this time by "reflecting" values beyond the first and last frames, so your example of 10, 20, 30, 40, 50 becomes [-40], [-30], [-20], [-10], [0], 10, 20, 30, 40, [50], [60], [70], [80], [90] (or as wide as it needs to be for the amount of smoothing specified). I haven't tested this much, but it does result in uncorrected first and last frames.

David

guth

17th August 2014, 13:32

Hey guth... I did it (http://forum.doom9.org/showthread.php?p=1690146#post1690146) :)
Cool!
I seem to struggle a bit with it, though. First I had problems that the video wasn't stabilizing properly at all. It turned out there was a frame mismatch because I had a blank line first in the logfile. (I guess you ignore the frame numbers in the logfile?)
After that, it was pretty smooth, but it still didn't look very good, partly because of rolling shutter, but there appears to be something else that's strange with the geometry too. Maybe I'm doing something wrong. I just took an ordinary video camera (Panasonic HDC-SD900) and filmed at max wide angle (and shutter speed 1/250s) though my window, while waving the camera. I used fov=54 degrees, because that's what I *think* it should be after googling a bit, but I tried other fov values too. I used smoothness values 100.
I didn't have time to test everything I could think of to make it work better, though. And I'm afraid my time for testing is pretty limited the upcoming weeks. Just so you know.
I hope others will test it meanwhile.

wonkey_monkey

20th August 2014, 22:35

but there appears to be something else that's strange with the geometry too.

Hard to say really, although I neglected to mention that Deshaker3D is only really meant for rectilinear input (maybe your camera is slightly fish-eye-y at it's widest angle?). Rolling shutter is obviously an additional problem, though from reading your description of your solution for Deshaker, it shouldn't be hard to implement in Deshaker3D.

Would you be able to share the video (quality doesn't have to be good) and logfile?

Thanks,

David

guth

14th September 2014, 17:59

Deshaker 3.1 is released...
http://www.guthspot.se/video/deshaker.htm

fvisagie

15th September 2014, 12:06

Congratulations, and thanks!

Just to confirmThe ranges of some settings have been extended - this is a range extension and not a resolution increase, correct? In other words, current value x has the same effect as previous value x?

Gargamel

15th September 2014, 14:39

Thank you, Guth !

guth

15th September 2014, 16:44

Just to confirm - this is a range extension and not a resolution increase, correct? In other words, current value x has the same effect as previous value x?
Correct. x is still x. :)
(Via script you have always been able to go outside the normal range on many settings, so this is really nothing more than looser restrictions in the GUI.)

fvisagie

15th September 2014, 17:23

Correct. x is still x. :)

Thanks for the confirmation!

PhotoCat

18th October 2014, 15:44

Hi Guth!
May I say thank you to u once again! Your amazing deshaker app is absolutely essential to my video editing workflow :)
Just 1 feature request:
For infinitely smoothness "-1" for pans, currently deshaker would try to make all the frames the same as the 1st frame in the video.
Would it be possible to allow us to choose any frame in the video as a reference? (i.e. not just the 1st frame)

This would be useful in a case where there is a little bit of panning to the right for example to centre the subject during the first few seconds of the
clip. Therefore, using the 1st frame as a reference for infinitely smoothness "-1" may not be optimal.

Thanks a lot!

guth

19th October 2014, 10:47

For infinitely smoothness "-1" for pans, currently deshaker would try to make all the frames the same as the 1st frame in the video.
Would it be possible to allow us to choose any frame in the video as a reference? (i.e. not just the 1st frame)

You can get the same effect by adding an offset to the logfile for the frame before the first real frame, i.e. usually for frame 0. Like this:

0 200 100 0 1

Adding that line to the logfile should offset the whole video 200 pixels to the right and 100 pixels down when using "-1" as smoothness.
Let me know if you can't make it work.

PhotoCat

20th October 2014, 03:04

Wow Guth! You are awesome!
This simple frame "zero" worked wonders!

Don't know how u did it!!
In my case, I used:
0 120 0 1.0 1

I only needed an x-offset of 120 to perfect the
composition of the video. It worked beautifully!
I was so excited and then I also tried to correct the slightly tilted camera angle by 1.0 degree and it also worked!!
So I set H & V panning to -1, as well as rotation to -1 for
"infinite" motion smoothness.

Now not only is my video corrected for composition, but it
is also corrected for tilt!

Kudos to you Guth and thanks again!

:thanks:

PhotoCat

20th October 2014, 18:34

Hi Guth,

Greetings!
I am using fixed zoom in pass2 to get rid of borders.
I shoot with prime lens and I prefer a non-zoom look
to my videos. Therefore I am trying to avoid the adaptive zooms,
which zooms in and out repeatedly, giving an amateur look.
The fixed-zoom is great except it eats up potentially beautiful backgrounds. Furthermore, in my previous example, an offset
of 120 in the horizontal direction would give a black border on
the left for the initial 15 seconds or so.

I have also tried the "fill-in-borders" from previous/future frames with no edge compensation but it can't go too many frames out or it would be very slow or simply not working.

Here is another feature request:

Wondering if the user can specify the source of the fill-in frames. e.g. for video frames 1-6000, the fill-in frame source would come from frames 15000- 15100 only. Note the fill-in frame range is only 100, which is much less than the 6000 frames needing edge filling. The assumption is that the background is still.

Is this possible? Thanks once again!

guth

20th October 2014, 19:51

Fixing that isn't trivial. For example, Deshaker only compares each frame to the previous. So, if we used a frame 1000 frames away, any error in this comparison would be multiplied by 1000.
Also, I doubt a lot of people would use that feature. (At least I wouldn't, and since Deshaker is freeware, that counts. :)))
So, I'm afraid it probably won't happen. Sorry...

PhotoCat

20th October 2014, 20:25

" if we used a frame 1000 frames away, any error in this comparison would be multiplied by 1000"

Not necessarily. When the user picks the fill-in source frames from 15000-15100, there is a reason he did so.
i.e. From the user's visual inspection, these are the frames that will provide the closest matches to fill in the missing border in question.
At the same time, the user can specify a mask to mask out any moving subjects in the foreground. This way, the comparison should be pretty fruitful
to come up with an excellent edge fill. (I guess one can make the mask automatic too e.g. 100 pixels inwards from all left, right, top & bot edges.
Anything inside a "rectangular donut" will be ignored for comparison.)

I guess this is human guidance :)

However, fair enough Guth. It may take too much work :) Perhaps I should ask what is the best technique to fill frame edges for what we have right now in v3.1 :)
Assuming we are trying to avoid the "zooming in and out" effect by adaptive zooms, and also assuming a static background.

Thanks again!

PhotoCat

20th October 2014, 21:38

I have been reading your deshaker manual very carefully and there are a few things I don't quite get it fully, even after reading this thread completely. (Pls bear with me as I have just been playing with video editing for about 5 months as a hobby, though I have some DOS programming background from the old days)

Q1) I am confused by the following 2 settings in pass1:

A) Discard motion of blocks that have match value less than X
(X range is -1000 to 1000 and default is 300)

"If the best match value found while matching a certain block is below X this block is discarded.
Blocks that have been discarded by this setting show no motion vector in the output video of pass 1."

B) Discard motion of blocks that have 2nd best match larger than best minus X
(X range is 0-?, and default is 4)
"If the second best match value found while matching a certain block is close to the best match value, this means that the best shift found isn't very reliable. It could just as well be this second shift that's the correct one. For example a blue sky has good match values for all shifts. Setting this parameter correctly discards those unreliable blocks.
Blocks that have been discarded by this setting show no motion vector in the output video of pass 1."

Questions:

For A), seems like a 300 match value is a good value.
Under what circumstance should one increase or decrease this value? Can u give an example of when to use this?

For B), I am not 100% sure what u mean by best match value and 2nd best match value. Do u mean within the same block?

Please confirm my understandings:

It appears that within 1 block, you are trying out a few shift combo values (to make the current frame looks like the previous frame) and then sort the results by the "match value"
in an array. Correct?
Then you look at the first entry of the match value array and check if it is greater than 300 (e.g. default value). If so, you keep the shift vector, if not, u discard the shift vector. Correct?

For B), U compare the difference between the 1st entry of the match array and the 2nd entry. Right? If it is less than 4 (e.g. default value), then, discard the vector. Correct?
Your blue sky example is very illustrative.

Q2) When do u increase block size and when to decrease?
default is 30x30? Pls give an example.

Q3) "Skip frame if less than X percent of all blocks are ok"

When u say "all blocks", do u mean all blocks over the entire video area? In some cases, I need to mask out almost the entire frame only leaving about 20% of the background area for matching. In this case my max match percentage would be 20%, right?

Thank you Guth once again :)

guth

20th October 2014, 21:53

Not necessarily. When the user picks the fill-in source frames from 15000-15100, there is a reason he did so...
I never said it was impossible. I was just trying to give you an idea why it isn't trivial. It's simply too much work for a not so useful feature, imho.

Perhaps I should ask what is the best technique to fill frame edges for what we have right now in v3.1 :)
Assuming we are trying to avoid the "zooming in and out" effect by adaptive zooms, and also assuming a static background.
In that case there might not be a good solution. Personally, I would probably use "adaptive zoom full" with a big adaptive zoom smoothness value, like 100000, for this.

PhotoCat

20th October 2014, 22:05

"I would probably use "adaptive zoom full" with a big adaptive zoom smoothness value, like 100000, for this."

Thank you Guth! This is enlightening. Never thought of
a value this big... 100000! Will give it a try. :)

guth

20th October 2014, 22:09

Q1A: That's one of few settings I rarely change, personally. But I guess one reason to decrease it might be if there's a lot of noise. One reason to increase it might be if you for some reason get incorrect matches.

Q1B: Correct (except for unimportant details)

Q2: One reason to increase size is if there's noise or lack of detail. Large blocks increase the signal-to-noise ratio. One reason to use smaller blocks is if there's lots of moving objects all over the frame. Then, you'll want to squeeze in some blocks wherever you can.

Q3: Correct.

PhotoCat

20th October 2014, 22:37

Thank you once again Guth. Your insights have helped me become a better user of your marvelous Deshaker and a better video editor!
Moreover, I highly respect u as u have generously been giving so much to the video community! Not just the tool,
but also your teachings. U r truly a master of the trade!
:thanks:

guth

21st October 2014, 16:43

A bit exaggerated, maybe. :)
There are lots of people who have given a *lot* more to the video community.
Thanks anyway!

PhotoCat

26th October 2014, 21:43

Hi Guth,

My apology if this question has come up before.

First time trying to shoot video handheld without optical image
stabilization under heavy wind. U know what the result would be!
It was a vy shaky video!

Tried many deshaker settings and it got better.
However, I am getting a "jello effect" similar to what is
described in the following thread:

http://forum.videohelp.com/threads/348837-Video-stabilizing-problems-(software-stabilisation)

scroll down to post#11 - the warped house.

I was using Canon 5D Mk II with a 85mm lens and I placed
a rolling shutter value of 70% in deshaker config.

My question:
1) Is deshaker designed to handle this kind of warped video?
2) If yes, what settings can I tweak to compensate for this jello effect?
I have noticed that this jello effect is already there in the original video though.

thanks!

PhotoCat

27th October 2014, 11:26

My apology. I found the problem. Pls ignore my previous question in #358. I shot the video portrait and turned the camera 90 degrees.
Then I turned the video by 90 degrees BEFORE I ran deshake! Of course the rolling shutter was not compensated!
Now I just left the footage unturned and performed deshake as the first filter. The result is much much better with only very
little "breathing" or jello like effect. I have learned a big lesson here and I now know deshaker 3.1 works vy well with rolling shutter cameras :) Thanks!

PhotoCat

27th October 2014, 13:11

During the previous exercise, I thought the rolling shutter value of 70% for my 5D2 was not optimal. Therefore I tried different rolling shutter values trying to fine tune it. I tried 50, 68, 69, 70, 71, 72, 80. My thinking was that I will look at the last column of the log file (% block matched) and used it as an indication of how good the rolling shutter value was.

So I repeated the exercise after I found the problem but was surprised to find that the (% block matched) values in the log file didn't change much with different rolling shutter value! I tried the same 50, 68, 69, 70, 71, 72 & 80. So I guess the (% block matched) value is not a good indication of a good rolling shutter value.

Now my question is if there is a way to tell how good the rolling shutter value is by looking at the log file or anywhere else.

Thanks a lot!

guth

27th October 2014, 17:50

The rolling shutter amount isn't used in pass 1 at all. (I still put that setting under the "Parameters affecting both passes" category in the GUI, though, because the rolling shutter on/off switch *does* affect both passes, and the amount sort of belongs to it.)
The only guth-approved way to find the best rolling shutter amount is to follow the procedure on my web page. :)

PhotoCat

27th October 2014, 18:22

Thanks Guth once again for sharing your insights :) Yes, I have read your webpage about how to calculate the rolling shutter.
I was hoping that the log file would have some kind of indication of how precise the selected rolling shutter value is but shocked to
find out that it is not used at all in pass1!
I am using 5D2 and you've noted 66-72% on your webpage so I really don't need to re-invent the wheel.

I am really tying to squeeze the last bit of juice out of the rolling shutter distortion compensation here :)

Would it make sense if I do the following?
1) In pass 2, try out all the rolling shutter values: 66, 67, 68, 69, 70, 71 & 72 and generate a video for each.
2) Compare all the video outputs visually to determine which clip has the least distortion.
3) Pick the clip that has the least distortion and the rolling shutter amount associated with that clip.
4) Use that shutter amount for all future deshake purposes.

Do u think 66 and 72 will make a big difference to distortion?

Thanks!

guth

27th October 2014, 18:32

Would it make sense if I do the following?
1) In pass 2, try out all the rolling shutter values: 66, 67, 68, 69, 70, 71 & 72 and generate a video for each.
2) Compare all the video outputs visually to determine which clip has the least distortion.
3) Pick the clip that has the least distortion and the rolling shutter amount associated with that clip.
4) Use that shutter amount for all future deshake purposes.
You're asking me to approve a method that's different from the only guth-approved method? :)
Ok. If you're not a perfectionist, I guess your method should work fine.

Do u think 66 and 72 will make a big difference to distortion?
Not really.

PhotoCat

27th October 2014, 19:12

[QUOTE=guth;1697925]You're asking me to approve a method that's different from the only guth-approved method? :)
Ok. If you're not a perfectionist, I guess your method should work fine.

No... not really... Your method is golden standard. :)
I am just trying to fine-tune the value by looking at the
actual results, taken an initial range found by your method.
I guess if there is not much of a different between 66 and 72,
there isn't much point in fine-tuning LOL! Thanks agn Guth!

PhotoCat

27th October 2014, 21:23

Hi Guth, sorry to keep bugging u :)
Really hard to resist when u know a guru is online :)

I often use deshaker inside avisynth this way to avoid
generating a big intermediate file or even a big deshaked video file, to save disk space:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100|0|450|200|0|350|850|100|0|0|0|1|30|30|15|1|0|0|30|30|0|0|1|0|1|0|0|10|500|1|70|1|1|20|100000|100|20|0|0|ff00ff")

This way, all I need is the original .mov file, the above .avs file and the small .log file. So really hardly any diskspace is taken up and the deshaked .avs file is ready for use any time.

Today I dragged in about 4-5 .avs file like the one above into aviutl and it started complaining about running out of memory
for just about 2000 1080p frames in total!

So I am guessing the way I use deshaker inside avisynth might be too memory intensive. Is it deshaker or is it avisynth or both?

So my question is:
1) Am I totally out to lunch using deshaker inside avisynth??
2) If not, is there a more memory efficient way of using deshaker inside avisynth?
3) Can one use deshaker (just pass2) stand-alone without using avisynth or virtualdub? I guess I am asking if deshaker can frame serve by itself...??

Thanks once again!

PS: forgot to mention I have 10GB of RAM on a win7 x64 machine

PhotoCat

28th October 2014, 00:10

To save memory and CPU power, I have figured out a way to reduce deshaker workload:

Original avisynth script is:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100...

trim(0, 300)

The whole clip is 1000 frames long but I only need the first 300 for example. So I reorder the trim command as follows:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

trim(0, 300)

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100...

So deshaker only needs to work on 300 frames rather than 1000 frames. This should save some memory & CPU I guess...

But here is the problem:

It is easy to trim off the frames starting from frame 0, before deshaking pass2 since deshaker will read the log file starting from 0 too and everything is in sync. No problem.

What if I need frames 100 to 400? (given that deshaker pass1 has been run on the full clip frames 0-1000 and log file written accordingly.)

If I trim the video before deshaking pass2, the log file will go out of sync.

So here is the question:

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Is there a way to do that?
Thanks again Guth!

PhotoCat

28th October 2014, 03:43

Some update: it appears that it was not a memory problem as windows performance meter showed that I still have at least 4GB of RAM unused when I get the memory
error from Aviutl. So it is most likely an Aviutl problem. However, my previous question is still interesting and I still would like to know how to deshake pass2 from the middle of a clip,
assuming a full length log file. Thanks Guth!

20141028 update: Aviutl/plugins developer MaverickTse has confirmed that Aviutl can only use abt 1.6GB memory space because it is a 32bit application.

Details here:

http://forum.videohelp.com/threads/366724-Aviutl-Tips-Tricks-and-Support-thread?p=2353294#post2353294

fvisagie

28th October 2014, 14:38

I am just trying to fine-tune the value

In that case, it might (arguably) be more accurate AND less work to use the guth-approved method;).

PhotoCat

28th October 2014, 15:09

Now the 32bit application limit has got me thinking...
Deshaker actually has a 64bit version which I think I am using with virtualdub x64, to perform all the interactive deshaker parameters tweaking and vectors monitoring...

It would be real nice if deshaker64 can be used stand-alone as
a frame-server during pass2, since the official avisynth is also 32bits.

guth

28th October 2014, 18:47

In that case, it might (arguably) be more accurate AND less work to use the guth-approved method;).
Yes, it's at least a more accurate method. Especially if you take the average (or median) value of several runs.

guth

28th October 2014, 19:01

Aviutl/plugins developer MaverickTse has confirmed that Aviutl can only use abt 1.6GB memory space because it is a 32bit application.
You might want to try the 4GB Patch (http://www.ntcore.com/4gb_patch.php) mentioned earlier in this thread. Run it on the main exe-file for Aviutl. That *might* give you 2GB extra memory to work with.

It would be real nice if deshaker64 can be used stand-alone as a frame-server during pass2, since the official avisynth is also 32bits.
Anyone up to the task can build something that uses Deshaker (and other VirtualDub filters) just like VirtualDub and AviSynth does. It could do anything with the output frames.
(And no, I'm not up to the task... :))

PhotoCat

29th October 2014, 02:38

Tks Guth for the aviutl 4GB patch tips. I will try that :)

I have found another way to do it and it seems quite
stable now with 6 clips and 3 pics in the aviutl project.
Just audio playback is a touch jerky but way better than
VirtualDub.

I just used avsf (Pismo File Mount) to mount the .avs file as a folder and I did it
for all 6 .avs files. Then I pulled all the .avi files inside the virtual folder into aviutl. Seems to work fine and stable and I
still have about 4GB memory to spare :)

Doing it the avsf way seems much more stable as I
don't see the "deshaking..." pop-up windows anymore.

This is how I used VirtualDub64 with avisynth anyway.

PhotoCat

29th October 2014, 02:44

Still love to know the answer to this question:

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Is there a way to do that? (I know the simple way would be to delete all the lines in the log file from 1-99 but perhaps there
is a more elegant way to do it without touching the log file??)
Thanks agn!

Thanks!

PhotoCat

29th October 2014, 02:45

(And no, I'm not up to the task... :))

U r a humble person Guth! :)

PhotoCat

29th October 2014, 13:54

Thanks Guth! The 4GB patch actually worked for Aviutl! I can see RAM committed to Aviutl has increased to 3.6GB at one point. However, I find the avfs file mounting method much more stable.

guth

29th October 2014, 20:52

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Why not use trim(100, 400) before Deshaker pass 1 too? Or, if that's not possible, you could do the trimming after AviSynth. In VirtualDub, for example, you could simply select the section you want, and save it. Or am I misunderstanding you?

Thanks for letting me know about avfs, btw. I didn't know there was a way to open video from 32-bit AviSynth in 64-bit VirtualDub. Nice!

PhotoCat

30th October 2014, 02:52

Why not use trim(100, 400) before Deshaker pass 1 too? Or, if that's not possible, you could do the trimming after AviSynth. In VirtualDub, for example, you could simply select the section you want, and save it. Or am I misunderstanding you?

Thanks for letting me know about avfs, btw. I didn't know there was a way to open video from 32-bit AviSynth in 64-bit VirtualDub. Nice!

I think I should explain a bit more of my motivation.
I usually deshake on the full clip anyway, which is normally
about 1-2 minutes long. However, I usually would want
to pick just 45-90 secs of the clip to be a part of a music video :)

I could do trim(100, 400) b4 deshake but I would lose the other frames, which are good too. In another project, I
may want to use frames (20-150) for example. Then I will have
to redo deshake on frames 20-150, which is kind of inconvenient.

Since there is very little overhead for me to do the entire 2 minutes clips beforehand and store the log file and .avs script for future use as stock clips, I deshake the whole clip all the time. This gives me the flexibility of using any part of the clip in the future.

BTW I find this a great disk space saving method too,
as there is no intermediate files nor ProRes files to be stored after deshaking! All I need is the original .mov file, the tiny deshake log file and the simple .avs file. The original .mov file is there anyway so I am really not using much disk space to have a deshaked video handy for future use. :)

The requested feature of asking deshaker to begin to read from any line of the log file further reduces cpu & memory usage by deshaker pass2.

Glad u found avsf useful. I got the tip either in this forum or in the videohelp forum. Lots of excellent people on these 2 forums for sure! Thanks!

guth

30th October 2014, 18:20

You could still do the trimming after AviSynth instead, i.e. feed the avs for Deshaker pass 2 into VirtualDub or similar, and trim there.

PhotoCat

31st October 2014, 01:16

Yup, that's what I am doing now :) Tks!

sorin86

23rd November 2014, 18:36

Deshaker is a very useful tool, almost a must. It has worked nicely for many of my videos. However, there's still a problem that is not uncommon, yet not tackled even in version 3.1: DUPLICATE FRAMES.

Say (for the sake of an example) you have a video at 15 fps. In 1/3 of a second (0.333s) you have 5 frames:
1 2 3 4 5
If you convert that to 30 fps, in 1/3 of a second you will have duplicate frames:
1 1' 2 2' 3 3' 4 4' 5 5'
where actually 1 = 1' (etc).

If you apply Deshaker to the 15-fps video, it will work probably beautifully.
However, if you apply it to the converted video, it will be a mess.

This is what happens. If you have a portion of the video with relatively fast panning and at the same time significant movement of objects (persons) in the video, the processing of deshaking the 15-fps video will be smooth. Deshaker with generate smooth transitions from one frame to another.
For the 30-fps variant, Deshaker interprets frames 1' 2' 3' 4' 5' as distinct frames. Although no change happens between frame 1 and frame 1', Deshaker will still apply smooth transitions. That actually spoils frame 1', which - so logically! - should keep whatever deshaking processing was applied to frame 1. The result is a very annoying back-jumping strobo effect, which makes Deshaker unusable - you'd rather watch the non-deshaked version.

The same annoying effect will appear even in videos with less frequent duplicate frames, like this recurring structure: 1 1' 2 3 4 5 6.

One could say "just remove the duplicate frames", but that is hardly feasible, while that type of structure changes throughout the video. It can go to 1 2 3 3' 4 5 6.
So, instead of less advanced users trying to figure out complex scripts for hours-long videos, would it be much easier for Deshaker to just identify a duplicate frame and just keep the processing data of the previous frame? After all Deshaker performs such a complex analysis... Identifying a duplicate frame should be a piece of cake.

fvisagie

24th November 2014, 08:20

I'm just wondering, is frame duplication really the ideal way of converting from 15 fps to 30 fps in the first place? Effectively you'll still have 15 fps-based motion, just played out at 30 fps. Would interpolation (e.g. MFlowFps()/MBlockFPs() and I think also SVP) be an option to provide smoother motion (and get around this problem) for you?

fvisagie

24th November 2014, 09:00

Is anyone interested in looking into this? I've done some feasibility checking and it is definitely very much possible, at least for AR.Drone 2 videos. Although more direct methods are available, the quickest time-to-implementation at this point seems to simply convert the output of ExifTool. I just don't have the time to take it further at this point.

In case someone would like to look into this:

The AR.Drone 2 stores flight metadata in two places: the .MP4 recording, and userbox files on the drone.

.MP4 recording

Information on the location and data structure: http://forum.parrot.com/ardrone/en/viewtopic.php?pid=32346#p32346 (somewhere I also came across the actual structure definition but unfortunately didn't save it)
ExifTool extracts flight metadata from the .MP4 file withexiftool -b -ardronetelemetry <inputfile>
ExifTool output is explained at http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,6123.0.html and http://owl.phy.queensu.ca/~phil/exiftool/TagNames/QuickTime.html (see 'prrt')
Here's a short extract:

0 7 84.023 -0.0138157112523913 0.020807821303606 1.58786511421204 0 0
0 7 84.055 -0.013349455781281 0.0169658288359642 1.58657622337341 0 0
...
1 7 85.913 0.0174348670989275 -0.00090780045138672 1.564129114151 0 0.659
1 7 85.946 0.0190824717283249 -0.00098724861163646 1.56361031532288 0 0.662
0 9 85.979 0.0209523718804121 -0.00107379746623337 1.56401801109314 0 0.665
0 9 86.012 0.0216640736907721 -0.00160936382599175 1.56422352790833 0 0.661

To convert from ExifTool to Deshaker pass 1 output format should be a cynch. Alternatively, the data could be parsed directly from the video, of course.

Userbox files

These are created on the drone: http://forum.parrot.com/ardrone/en/viewtopic.php?pid=32169#p32169
Depending on remote controller software, these files may be copied/moved to the controller after the flight
Ruby examples for converting userbox files to GPX: http://www.ardrone-flyers.com/forum/viewtopic.php?p=66997#p66997 and http://www.ardrone-flyers.com/forum/viewtopic.php?p=74791#p74791

I can provide short sample videos and userbox files if required.

guth

24th November 2014, 20:11

sorin86... Yes, any moving objects in the video will jump back and forth, but the background that is stabilized on will look smoother than before.
My honest (but maybe not so nice :)) answer to this problem is that if you have duplicate frames you're either doing something wrong, or you're using a lousy camera. Garbage in, garbage out...
I don't think I will address this problem unless I get lots of requests. Sorry...

StainlessS

26th November 2014, 17:51

@fvisagie
Dont know if this of any use but take a peek about this post (from about post #14 onward, user name Filker could be used as search filter in that thread)
http://forum.doom9.org/showthread.php?p=1588194#post1588194
Avisynth based (still camera Exif).

fvisagie

27th November 2014, 07:40

@fvisagie
Dont know if this of any use but take a peek about this

I'd had in mind to Call() an external script for the conversion, but that looks like it could come in handy for conversion in Avisynth.

The problem for me though remains not so much how to do it, but to find the time. I posted in the hope of finding someone else who's interested in taking the idea further.

StainlessS

27th November 2014, 12:11

fvisagie,
See Martin53 posts (and mine) in same thread as posted and also in Call(and maybe CallCmd) threads, he wanted to do something like that
for deshaker, I did some coding for him to parse and extract data from the files, dont know if he finished the project.

fvisagie

27th November 2014, 13:23

StainlessS,
I found those two threads, but saw nothing that seems related to Deshaker or parsing text files. Thanks, anyway.

StainlessS

27th November 2014, 16:38

Well you could have searched on deshaker with user name Martin53 as here:
LINK REMOVED:

in particular this
LINK REMOVED:

however, it seems M53 abondoned the project (or at least that attack stance).

fvisagie

27th November 2014, 17:15

Indeed I didn't think that far. A slow day, sorry and thanks.

StainlessS

27th November 2014, 20:42

fvisagie, sorry, the links posted did not work (they worked when I posted them, I checked), but after search and
log-off they must disappear or something. anyway, this was the 2nd link posted again and hopefully working this time.
http://forum.doom9.org/showthread.php?p=1596064#post1596064

Previous bad links removed.

fvisagie

28th November 2014, 08:14

@StainlessS,
Strange, that link did work for me the first time also :). Thanks for your trouble.

fvisagie

12th September 2015, 17:05

It seems possible, but will it work correctly deshaking a concatenated file in VirtualDub, one input clip at a time?

Background to the question: a large 350+ file HD project is creating space problems with the saving of intermediate files such as Deshaken versions of each input clip. An advantage of Deshaking clips individually is that scene changes are not missed. In this case it seems I'd need to work on a real or hopefully frame-served concatenation of input files, and I was wondering whether it would be possible to still ensure reliable detection of clip transitions. That would save me having to manually locate and check all those boundaries afterwards.

In principle the following approach seems possible, in the sense that it can at least be scripted in VirtualDub:
For each clip
Apply Clear() & AddRange() to restrict processing to that clip's material only
Process Deshaker pass 1 (configured to save individual log files for each clip - AddRange() will cause frame numbers to start at 0 for each clip)
Apply Deshaker pass 2 (configured to use the same log file as used in pass 1)

Some questions that create doubt:
1. What happens to the results of Deshaker's processing when pass 2 is applied (i.e. the "Deshaking" message)? Will this get over-written or somehow become inaccessible when the next clip is processed?
2. What happens once all of the concatenated clip has been processed, the last AddRange() is removed and saving or frame-serving starts? Frame number offsets as seen by Deshaker of all clips but the very first one will change - how will that affect Deshaker's operation?
3. Does this regarding frame-serving from the VirtualDub online help apply to Deshaker: "Video filters that have a lag — most notably the temporal smoother. The filter will function, but there will be a delay in the video."?

I realise that I can try this all out for myself. However, I am under some time pressure and would rather hear now "this is guaranteed to fail" as opposed to wasting time on a bad idea. On the other hand, if this could work, I'd appreciate any corrections or improvements to the approach.

Thanks,
François

shekh

12th September 2015, 18:49

When you isolate range, deshaker sees original frame numbers (not starting from 0 for each range) so in general this way should work.
However,
there is no meaning to "apply pass 2" if you do not save the result (video).
And there is no normal way to apply different deshaker instance (with different log files) to each range.

But instead you could simply use one normal deshaker instance with one log file: it can deal with scene cuts.
If you are going to manually pass range parameters to AddRange() you could as well just force these frames to be scene cuts in the deshaker log file.

fvisagie

12th September 2015, 19:25

When you isolate range, deshaker sees original frame numbers (not starting from 0 for each range) so in general this way should work.

Unfortunately, not in my experience. Applying AddRange() results in a VirtualDub clip with frame numbers [0...n-1], and the frame numbers recorded in the resulting Deshaker log file are [0...n-1]. Unless you are referring to a way of isolating ranges other than AddRange()?

And there is no normal way to apply different deshaker instance (with different log files) to each range.

That would then be the one potential deal-breaker, see below.

If you are going to manually pass range parameters to AddRange() you could as well just force these frames to be scene cuts in the deshaker log file.

I realise that that possibility is available, but it's very easy to automatically isolate individual clips in the VirtualDub script, which itself is very easy to generate automatically. Avisynth can be used to catalogue clip lengths and generate the script, and below is an example with VirtualDub doing the concatenation (of .avi inputs) itself. A batch file can easily generate such a script from a list of filenames:
// Start of script
declare startframe;
startframe = 0;
declare endframe;
endframe = 0;
// ...

// Start of clip 000 processing
VirtualDub.Open(U"clip000.avi");
VirtualDub.subset.Delete();
startframe = endframe;
endframe = VirtualDub.video.length;
VirtualDub.Log("startframe="+Sylia.ToString(startframe));
VirtualDub.Log("endframe="+Sylia.ToString(endframe));
VirtualDub.Log("cliplength="+Sylia.ToString(endframe - startframe));
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(startframe,endframe - startframe);
// ...

// Start of clip 001 processing
VirtualDub.Append(U"clip001.avi");
VirtualDub.subset.Delete();
startframe = endframe;
endframe = VirtualDub.video.length;
VirtualDub.Log("startframe="+Sylia.ToString(startframe));
VirtualDub.Log("endframe="+Sylia.ToString(endframe));
VirtualDub.Log("cliplength="+Sylia.ToString(endframe - startframe));
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(startframe,endframe - startframe);
// ...

// ...

So it seems the question is whether it is possible to apply different Deshaker instances to different portions of a clip?

Additionally, it seems there may be issues with temporal filters when frame-serving; see the added question 3. above. However, that at least has the work-around of saving to disk if needed.

shekh

12th September 2015, 19:59

Unfortunately, not in my experience. Applying AddRange() results in a VirtualDub clip with frame numbers [0...n-1], and the frame numbers recorded in the resulting Deshaker log file are [0...n-1]. Unless you are referring to a way of isolating ranges other than AddRange()?

Yes sorry, but you can use SetRangeFrames() which restricts range for analyze pass without tweaking timeline.

I realise that that possibility is available, but it's very easy to automatically isolate individual clips in the VirtualDub script...

Maybe you can generate log files this way, and then concatenate them with another script?

whether it is possible to apply different Deshaker instances to different portions of a clip?

You may try with blend curves, when a value of blend curve is 0 the filter is skipped. But I am not sure if this is efficient as all instances will probably take a lot of memory (and preparing concatenated logfile looks more easy to do).

Additionally, it seems there may be issues with temporal filters when frame-serving

AFAIK there is internal delay in pass2 if you apply border filling with future frames. But I dont know if/how it is connected with frameserving.

fvisagie

12th September 2015, 21:18

you can use SetRangeFrames() which restricts range for analyze pass without tweaking timeline.

Yes, that seems workable, although the fact that the offsets are in milli-seconds makes that a somewhat shaky approach.

Maybe you can generate log files this way, and then concatenate them with another script?

At this point that seems like the safest way 1) to avoid the creation of intermediate files and 2) to ensure all scene changes are detected. Something like
Run Deshaker pass 1 on individual clips
Concatenate clips and log files, adjusting frame numbers in the latter
Set Deshaker to pass 2 and render

Thanks for your suggestions.

shekh

12th September 2015, 21:30

Yes, that seems workable, although the fact that the offsets are in milli-seconds makes that a somewhat shaky approach.

Which milliseconds? SetRangeFrames() is equivalent of "mark in/mark out" in the UI.

fvisagie

12th September 2015, 21:36

Which milliseconds? SetRangeFrames() is equivalent of "mark in/mark out" in the UI.

Yes, when done manually. However, when invoked from a script (from VirtualDub scripting language reference, v0.7 (http://www.virtualdub.org/docs/vdscript.txt)):
int GetRange(int var);
void SetRange(int startMS, int endMS);

Gets or sets a particular range value. If var=0, the start offset
is used, and if var=1, the end offset is used. The start offset is
measured in milliseconds from the beginning, and the end offset is
in milliseconds from the end. Yes, I know this is stupid.

Because the end offset is only an offset, it is not possible to
determine the length of the video stream using these functions.

shekh

12th September 2015, 21:45

Maybe this script reference is too old.

oops I was referring to SetRangeFrames not SetRange

it is saved when you do "save processing settings" and select "include selection"

fvisagie

13th September 2015, 09:51

oops I was referring to SetRangeFrames not SetRange

Thanks for pointing that out; that would simplify things a lot.

bxyhxyh

2nd January 2016, 12:42

How b and c of bicubicresize of this filter are set?

guth

2nd January 2016, 14:06

How b and c of bicubicresize of this filter are set?
It uses a = -0.6, which I believe is the same as b = 0 and c = 0.6.
You can't change this value.

sorin86

6th January 2016, 13:14

I have some shots taken in the night, aiming at the moon. Zoom is big and so is the shaking (no tripod at that time). How can that be "de-shaken", since there is no real "background"? I only have the bright moon circle against the black background. I tried many settings for Pass 1, but none worked. Any recommendations, please?

StainlessS

6th January 2016, 13:29

Sorin86, search PlanetCrop.

http://forum.doom9.org/showthread.php?p=1606032#post1606032

http://forum.doom9.org/showthread.php?p=1606025#post1606025

Updated to the above posted script, avs in RT_Stats plug.

EDIT: Expects images rather than video.
EDIT: Oops, Avisynth plugin, not Vdub.

raffriff42

6th January 2016, 23:04

I tried many settings for Pass 1, but none worked. Any recommendations, please?See the screenshot below. It's partly guesswork.

Most importantly, tune the borders to the area the moon is. To get these numbers, use the "crop" function built into every VirtualDub filter. Copy Crop's "offset" numbers into Dehaker's "ignore border."

https://www.dropbox.com/s/1wgz13k4qf8nup7/deshaker-moon1.png?raw=1

EDIT whoops, max horizontal & vertical panning should both be about 20.

sorin86

25th February 2016, 10:46

See the screenshot below. It's partly guesswork.

Thank you for your try. Yet nothing works. It seems to be beyond the power of Deshaker. A variant ot the video is here https://www.youtube.com/watch?v=lqyCdM8JJBI

bxyhxyh

1st March 2016, 11:03

https://dl.dropboxusercontent.com/u/58215671/dddd.png
with this settings before deshaker, it detected edges.
Then you can play with deshaker settings.
But finding good values for deshaker wasn't easy task for me.

bxyhxyh

2nd March 2016, 15:37

I have a question.
Is there any settings that chooses third, fourth or Nth frame as the first frame of the scene and stabilize previous frames compared to that frame?
Setting to add big black bars on those "falling" frames, and less black bars on other frames, instead of first frame is unaffected and the frames that shouldn't be shifted are shifted too much which current version works this way.
For some films I worked on, the first two or three frames were very off compared to the rest of the scene due to camera movement or something.

Sorry if there are some English difficulties.

guth

2nd March 2016, 17:07

Is there any settings that chooses third, fourth or Nth frame as the first frame of the scene and stabilize previous frames compared to that frame?
Not really. But you can edit the logfile after Deshaker pass 1 to offset the first frame by a certain number of pixels. It's not as convenient, but you'll get the exact same effect.

Add a line to the logfile for the frame number of the first processed frame. For example, if you started running Deshaker pass 1 at the beginning of the clip, and then add the following line...

0 100 0 0 1

...the first frame will be shifted to the right by 100 pixels. The first 0 is the frame number, and then follows x-shift, y-shift, rotation and zoom factor.

sorin86

16th March 2016, 08:10

Thank you for your suggestion. It seemed to work for a lot of frames, but there were also some terrible jerks. As one can't spend hours and hours to detect the proper Deshaker settings (if there are any), the conclusion is that this software just isn't able to deal with such a situation which is not so uncommon.

https://dl.dropboxusercontent.com/u/58215671/dddd.png
with this settings before deshaker, it detected edges.
Then you can play with deshaker settings.
But finding good values for deshaker wasn't easy task for me.

affter333

20th November 2016, 05:51

thanks for the de-shaker, it's so useful !!

I'm still using it.

dipje

20th November 2016, 14:09

@sorin86: Programs like Mocha are for that kind of stuff. Plane and image-trackers, which track a certain object in the frame (instead of all the motion detected).
If you still have huge jerks there is little any kind of video-stabilizer can do with that. Huge jerk-movements are just that. You can even stabilize it 100% but you will end up with almost no frame left, or leave them in since they would ruin the overal 'cropfactor' of the stabilization. Cutting those huge jerks out before entering any kind of stabilization process helps a lot.

StainlessS

6th July 2017, 18:06

Has anyone ever seen DeShaker source code published anywhere ? (any version, but later better)

I have a requirement for spatial alignment/zooming, for sequence matching, I have a bit of an idea for rapid detection of spatial alignment and zoom,
but it may not work at all. Would like to see how someone who has some clue what he is doing, does it.

guth

6th July 2017, 19:08

It's not published (I hope). But if you describe your idea (privately if you prefer), I'll tell you what I think about it. I promise I won't steal your idea... :)

StainlessS

6th July 2017, 22:45

Hi Guth,

Was thinking that maybe use something like Zebra() Avisynth plugin:- https://forum.doom9.org/showthread.php?t=167663&highlight=zebra
(Arh damn, my images have gone from there too. [EDIT: Fixed images])

and scan on one or several of the (resized) bands(maybe minmaxdifference band), perhaps using something like RT_Stats RT_LumaCorrelation,
maybe in 2 parts vertical and 2 parts horizontal, to gauge shift and zoom (H and V separately).

EDIT: Then once rapid/rough estimate is done, and approx shift/zoom gotten then exhaustive subpixel shift/zoom
to get final crop/resize.
We dont actually have to do it at every frame, only per clip, but is complicated by fact that we have two different clips with
perhaps differing color/gamma.

guth

7th July 2017, 11:11

I don't know the exact problem you're trying to solve. "Sequence matching"... "two different clips"... you mean you have two clips of the same scene and you want to "time match" them? And you also want to match them spatially? (Does this have something to do with 3D stereo video?)

Anyway, things that I guess could be problematic with your idea is if objects move differently in the two clips, or if rotation is different. Also, if it's not a rectilinear lens (maybe you have 180 degree video with two cameras, for VR??) the "distorted" geometry might be a problem too.

And to find shift and zoom, are you going to guess values for these and see which set gets the best match? Guessing two unknowns at the same time can be time consuming, but maybe you'll do it hierarchically, at different scales?
You might want to look into something called "least squares matching" (not to be confused with "least squares fit") that can find a shift without guessing. I'm not using this in Deshaker, though, and I don't remember why.

In short, I believe your idea might work, but depending on the exact intended use it may not be very robust.

StainlessS

7th July 2017, 12:10

And you also want to match them spatially?
Yep, thats it, need to match spatially so can then match sequences, for eg replace bad quality sequence with similar good quality sequence of different clip.
Perhaps of different input clip size, up to about 32 pixels difference H or V. (3D, of no concern).
The two clips would be spatially aligned on a single in-sync frame, rotation does not come into it.

Was thinking, something like, split vertical into top and bottom, process top via shift/resize until find match (single vertical pixel column from zebra),
same again for bottom, calc shift/resize from two results. repeat for horizontal.
Then with rough estimates, go 2D subpixel to find and return best shift/resize, and intersection coords.

The Zebra thing is intended just to do away with the gazillions of 2D shift/resize/correlation ops, do it individually in two 1D passes
for primary estimate. [selecting best Zebra band via RT_YPlaneStdev() ].

I'll have a look for "least squares matching" , thanks very much for your time, I think I have enough for now. :thanks:

EDIT: For the particular immediate use, I have a good quality, good color clip with massive opaque logo, and a low res bad color clip
without logo and of different size and cropping. I want to match spatially, then do color matching with GamMatch (from GamMac thread)
and replace logo with resized, shifted, recolored, low quality area.
The recoloring part already works better than anticipated.
EDIT: We also have to spatially match with logo area chopped off and extrapolate crop and resize stuff.

it may not be very robust
Yep, I figured that :)

guth

7th July 2017, 12:37

I was wondering what kind of strange use this could be, but your edit explained it.

StainlessS

2nd August 2017, 02:01

Hi Guth,

Just had a glance at your docs for DeShaker, the Rolling Shutter docs Motorola Defy as 848x480 widescreen: 52%,
Actual screen resolution is 854x480, but captures video highest rez at 640x480 4:3 (dont know if of any interest).

General
Complete name : C:\Z\2017-08-02_01-09-46_141.3gp
Format : MPEG-4
Format profile : 3GPP Media Release 4
Codec ID : 3gp4 (3gp4/mp41/3gp6)
File size : 5.10 MiB
Duration : 16 s 789 ms
Overall bit rate : 2 550 kb/s
Law rating : 0x00000000
Released date : 0
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10
Classification : 0x00000000

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Baseline@L3
Format settings, CABAC : No
Format settings, ReFrames : 1 frame
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 16 s 789 ms
Bit rate : 2 483 kb/s
Maximum bit rate : 2 000 kb/s
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Variable
Frame rate : 29.841 FPS
Minimum frame rate : 8.197 FPS
Maximum frame rate : 34.483 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.271
Stream size : 4.97 MiB (97%)
Language : English
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10

Audio
ID : 1
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 16 s 768 ms
Bit rate mode : Constant
Bit rate : 64.0 kb/s
Channel(s) : 1 channel
Channel positions : Front: C
Sampling rate : 16.0 kHz
Frame rate : 15.625 FPS (1024 spf)
Compression mode : Lossy
Stream size : 131 KiB (3%)
Language : English
Encoded date : UTC 2017-08-02 00:10:10
Tagged date : UTC 2017-08-02 00:10:10

From Aida64

SYSTEM:

Manufacturer Motorola
Model MB525
Brand moto
Board jordan
Device umts_jordan
Platform omap3
Product MB525_RTGB
Installed RAM 512 MB

Device Features
android.hardware.camera
android.hardware.camera.autofocus
android.hardware.camera.flash
android.hardware.location
android.hardware.location.gps
android.hardware.location.network
android.hardware.sensor.accelerometer
android.hardware.sensor.compass
android.hardware.sensor.light
android.hardware.sensor.proximity
android.hardware.telephony
android.hardware.telephony.gsm
android.hardware.touchascreen
android.hardware.touchascreen.multitouch

CPU:

Core ARM Cortex-A8@800
Architecture MHz
Instruction Set 32-bit ARMv7
CPU Revision r3p2
CPU Cores 1
CPU Clock Range 300-800 MHz
Core 1 clock 300 MHz
CPU Utilization 7%
Scaling Governor ondemand
CPU ABI armeabi-v7a
AES Not Supported
NEON Supported

DISPLAY:

Screen Resolution 480 x 854
xdpi / ydpi 96 / 96 dpi
GPU Vendor Imagination Technologies
GPU Rednderer PowerVR SGX 530
Refresh Rate 68Hz
Default Orientation Portrait
OpenGL ES Version 2.0
GPU Version OpenGL ES-CM 1.1

ANDROID:
Android Version 2.1-update1 (Eclair)
API Level 7
Rooted Device Yes
Android ID xxxxxxx
Build ID JRDNEM_U3_2.21.0

DEVICES:

Camera
Resolution 5.0 MP (2592 x 1936)
Focus Modes auto, Macro, infinity, off
Flash Supported

USB Device - Motorola Inc Motorola Phone ()
Manufacturer Motoroloa Inc
Product Motoroloa Phone

SENSORS:
KXTF9 3-axis Accelerometer ????
AK8973 3-axis Magnetic field sensor ????
AK8973 Temperature Sensor 29.0 degrees C
ISL29030 Proximity sensor ????
Orientation Sensor Azimuth:???? / Pitch: ???? / Roll: ????
ISL29030 Ambient Light Sensor 2.0 lux

The Defy+ (with a software upgrade,) is supposed to support up to 720p, but I dont know if 16:9 or what, nor if update was released.
Only difference with ordinary Defy was the plus had a Bayer lens(red, some later Defy also had red), whereas plain Defy had SOC (green) lens.
Also, Defy+ had bigger battery and the (pretty much) same processor was clocked at 1GHz instead of 800MHz as in the plain Defy.

Plain Defy:- http://www.gsmarena.com/motorola_defy-3514.php
Defy+:- http://www.gsmarena.com/motorola_defy+-4098.php

Picked myself one up for £8.00 a couple of months back :) :- https://uk.webuy.com/search/index.php?sortOn=sellprice_asc&stext=phones+android+motorola+defy&section=
(Gonna try Jelly Bean on it when I get time, great little phone, me loves it).

guth

2nd August 2017, 18:01

Do you mean that my rolling shutter listing for this mobile is misleading?
I just add the rolling shutter info that people send me. I think it's better if the end user uses that info as best as he can instead of me trying to correct it, without knowing what the guy who sent it to me might have done.

StainlessS

2nd August 2017, 19:29

Dont know much about rolling shutter or whether affected by wrong screen size info (and cap size also different), just posted as I stumbled across your
doc (again), and had recently gotten a Defy.
Incidentally, in the last link above, there are a number of Defy's @ £5.00 and £8.00 and above, but many out of stock.
If you click on "By Availability - In Stock Only", you will see first on list at £8.00 is out of stock, but if you click
on it, they currently have one in Bromley and one "In-Stock On-Line".

My latest party trick is to play a movie then accidentally drop my defy in my beer :)
EDIT: It's an IP67 phone, water resistant to 1 Meter for 30 mins (with covers in place).

trondmm

19th August 2017, 15:32

Hi.

Is it possible to stabilize the image based on the position of a channel logo?

I have this video, that seems to have been the victim of a botched stabilization, and I want to reverse the damage as best as possible.

https://drive.google.com/open?id=0B1VlH492frJrQmhnWjl0RGFGLWc

In the first frame, the logo is in its proper position, and I'd like to "lock" the logo in this position, and move the video around to wherever the logo position takes it. I've tried to ignore the pixels outside the logo area, without success.

guth

19th August 2017, 18:08

I'm afraid I don't think you'll have much success in using Deshaker for this. And I actually doubt any tool can fix it in a good way. If it had been stabilized only vertically and horizontally it would have been easier, but this clip has rotation and zoom altered too, and I think the logo is too small to measure rotation and zoom accurately enough.

trondmm

20th August 2017, 23:41

Thank you for taking the time to look at this. I kinda expected the answer to be negative, but it was worth a shot.

bxyhxyh

11th March 2018, 21:27

Deshaker seems to ignore skipped n_scene when motion smoothness is set to -1.
Black bars don't get "resetted" after new scene n_scene changes.
Is it made that way or is it a bug?

guth

12th March 2018, 18:56

A bug. :o
To my defense, smoothness -1 is a pretty experimental feature.
But if you really, really want a fix, I can send you one.

bxyhxyh

12th March 2018, 19:29

But if you really, really want a fix, I can send you one.

Yes please, I would like to. Any other smoothness values don't work quite good on my source.

guth

12th March 2018, 20:04

(bxyhxyh, I sent it in a private message.)

MysteryX

17th May 2018, 04:39

This is the best deshaker out of the various options I tried. The downside is that it is only a VirtualDubFilter, and needs RGB32 instead of YV12 input -- although it can be used in Avisynth with VirtualDubFilter. Also the source code isn't available.

Would it be possible to have a version for Avisynth and VapourSynth? That would be very useful.

guth

17th May 2018, 18:01

"Best deshaker"? It should hopefully be the only deshaker! When I made Deshaker I googled "deshaker" and got one single hit; some guy with that name. It's funny that some people seem to be using it as a general term for video stabilization now. :)
Anyway, thanks.

I understand your wishes completely. I've considered making a native AviSynth plugin and I've looked into how much work it would be to support more color spaces. The problem is that I've become a bit lazy, or at least prefer doing other things on my free time. Also, I'm not quite ready to release the source code. Sorry... :(

guth

17th May 2018, 18:24

shekh

17th May 2018, 19:16

I could add this fake RGB in vd2, sounds funny. Would it actually be useful? :) Btw, black borders will turn green borders, and what about color mask detection.

shekh

17th May 2018, 19:21

Just noticed a bug:
add filter->deshaker
enable color mask, open dropdown
click cancel in filter, cancel in filters list -> crash
No input file needed, tested in 1.10.5

guth

17th May 2018, 19:32

Yes, you would have to select a different color for mask detection. As for black borders, maybe you could use a lossless conversion where black is still black?

And the crash... Yes, indeed, I can reproduce it. (But not when using a Deshaker built with a newer compiler version. Probably just luck, though.)
We'll see if I care enough to fix it. It's not a procedure you'd do often. Thanks for letting me know, though.

StainlessS

17th May 2018, 20:41

Guth, we appreciate everything that you have done for us, you are the best, please, let the new guys have a go at making it better,
we look forward to your decision, thanks ether way, we does love you. :)

MysteryX

18th May 2018, 17:34

Btw, about color spaces. Maybe someone could write a simple plugin that converts YV24 into RGB32 by simply assigning R=Y, G=U and B=V, or similar. After that, you could deshake (or other things) RGB32 video that's really YUV, and then do the reverse "conversion". It's not an ideal solution, because the colors would look like crap in Deshaker, but I can't see why it shouldn't work. You would get no quality loss due to color space conversions that way.
Just a thought...
The problem is that YV12 has U and V planes at half resolution, while RBC32 has 4 planes at full resolution. Even a fake conversion can't be done without rescaling U and V planes with rounding errors. And then you lose performance by processing more data than necessary.

Any specific reason why you're not comfortable releasing the source code?

btw I'm glad to see you're still active and responding to messages. The authors of lots of old plugins are long gone.

guth

18th May 2018, 18:02

The problem is that YV12 has U and V planes at half resolution, while RBC32 has 4 planes at full resolution. Even a fake conversion can't be done without rescaling U and V planes with rounding errors.
Yes, but I think those errors are negligible. Deshaker still has to resample all pixels, which destroys quality a lot more. If it's doing it in 4:4:4 instead of 4:2:0 mostly just means it's doing it in a higher resolution than needed.
Not being able to process YUV video at all could however result in banding problems, which I think could actually be seen in some cases. I haven't really examined this, though.

And then you lose performance by processing more data than necessary.
A little, yes.

Any specific reason why you're not comfortable releasing the source code?
Partly because I've sold Deshaker source code licenses to a few companies in the past, and that might be harder to do if the source was "more open". And partly I guess because it somehow feels better to have full control over it.

isidroco

20th May 2018, 00:24

I've perfectioned my batch which automatically [generates LOGs, deshakes] and compress A/V of all video files in folder with plenty of options. For anyone interested it's here: https://sourceforge.net/projects/virtualdub-batch-video-deshake

MysteryX

20th May 2018, 16:30

I'm also working on implementing it in the Yin Media Encoder -- but it's implementation is more complicated than other filters due to requiring a pre-scan, and it's not so simple to do full preview for only a portion of the clip, and the fact that the second pass (in x86) requires MP_Pipeline to split into 2 processes to avoid running out of memory; adding to that the RBG32 conversion and a couple of other technical details.

I was also considering whether I could switch to Vapoursynth to avoid all kinds of MT and memory issues, but there are filters that don't work in VapourSynth:
- Deshaker
- my own filters (AvisynthShader, FrameRateConverter)
- Audio filters

MysteryX

24th May 2018, 15:41

Converting YV12 to fake RBG32 is very easy.

1. Convert to YUVA444P8
2. Use dummy filter to change video format to RGBAP8 without changing content
3. Convert to RGB32

RGB32 is flipped up-side-down, but is RGBAP8 up-side-up? If so, even the flipping will be handled by this method. If it's inserted within a script that reverse it again after, then even the previews will look good.

As for VapourSynth, someone pointed out that VDubFilter source code isn't complicated at all, it would be very easy to port to VapourSynth for native support of VirtualDub filters.
https://github.com/pinterf/AviSynthPlus/blob/master/plugins/VDubFilter/VDubFilter.cpp

redfordxx

6th January 2020, 23:57

Hi.
Thank you for wonderful work, in a first place.

I am just curious:
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...

nji

7th January 2020, 14:09

Hi.
The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

This is what I see a lot, when having handheld smartphone...

That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.
It is as your movie shows only flat surfaces,
or as if you're using an extreme telephoto lens.
DeShaker does very good results for that.
Rotation on x and y axis (called panning and tilt)
is simply x and y shift then.

But the less the condition holds
(i.e. you have 3D objects or using a lens that's not telephoto)
the operation on the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).

So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there are at least two serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146
https://forum.doom9.org/showthread.php?p=1747268#post1747268

redfordxx

8th January 2020, 00:18

That seems a quite nice idea of yours.

However, as far as I (just another thankful user) know about
DeShaker operates in 2D only.

The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...

the 2D "model" (of the 3D reality) will
lead to worse results.
(You'll find a note about that in DeShaker's FAQ).

I don't know about that...
Especially if there would be option to limit x and y rotation.

So the improvement would be to do DeShaking in 3D.
(As I know, DeShaker isn't modified anymore).
But there is at least one serious attempts I know about:
https://forum.doom9.org/showthread.php?p=1690146

(https://forum.doom9.org/showthread.php?p=1747268#post1747268
just uses DeShaker's analysis data and does some lens correction.
So: no 3D at all.)
But they still use deshakers "2D" data.

guth

8th January 2020, 08:12

The Deshaker compensates movement against x,y,z axis and rotation z axis.
How about rotation x and y axis?

To be honest, I'm not sure which movements you mean. If we talk movements of the video, basically only one kind of rotation exists. And if we talk movements of the camera, Deshaker handles rotation of the camera along all axes (+zoom), but nothing else. Moving the (center of the) camera forward, backward, right, left, up, down often works somewhat well too, though.

Anyway, nji is right. Deshaker both analyses and transforms the video in 2D only, which means wide angle video will look ugly when there is heavy shaking.

nji

8th January 2020, 11:40

The 3D movement projects into 2D and when we have hundreds of vectors, it might be possible...

I don't know about that...
Especially if there would be option to limit x and y rotation.

But they still use deshakers "2D" data.

First of all:
You are right, I did a mistake in my last post (#442):
"Deshaker3D" uses DeShaker pass1 data, too.
(So maybe both tools do some kind of "3D-magic"?
And then "Lens transform" will be probably more sophisticated
with alpha consideration and rolling shutter/ "skew"?)

Second:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.
It does only 2D-operations:
left/right, up/down, resize and (2D-) rotation.
These 2D-operations do ("by chance") match the effects that
the corresponding 3D movements would have on the 2D projection (= frame).

Thinking it over ... DeShaker very well possibly could
take into account the 3D rotation at x- and y-axis
(or even an arbitrary axis) too.
(Would be a kind of magnifying "above" and reducing "below"
the axis I guess).
Seems possible, but isn't done (and won't be ... believe me :))
But maybe/ probably the other mentioned tools will?

nji

8th January 2020, 13:38

I do hope it's not too much OT if I post a summary of my (small)
experiences of the filter?

DeShaker (VD)
Pros:
- excellent results on all but wide angle scenes (no 3D/FOV accounting)
- optional rolling shutter accounting
- optional progressive/ interlaced source
- accounting sophisticated border and logo handling (by neighbour frames)
- many options to handle difficult source/ get optimal results
- excellent description of all that
- multithreaded (? nevertheless takes the most times of all filters)
Cons:
- Bad results on wide angle scenes (even worse than the shaky sources)

Lens Transform (VD)
Pros:
- accounting of field of view (FOV)
- accounting of simple border handling
Cons:
- no accounting of logo handling (have to delogo before)
- no interlaces sources (have to convert before)
- Rolling shutter accounting ("skew") "not implemented completely" (although active check-box)
- Unsufficient description (at least for me)

Deshaker3D (AviSynth)
Pros:
- Accounting of FOV
Cons:
- no border/ logo handling
- no rolling shutter accounting
- no interlaces sources (have to convert before)
- needs format conversion (= quality loss)
- no multithreading (?)

Deshaking (Shotcut)
Does about the same as DeShaker but with no options and worse results.

Conclusion:
The best (for non-wide angle scenes) does DeShaker.
If results are unacceptable in DeShaker (due to wide angle)
the best choice is Lens Transform (if rolling shutter source: try Deshaker before with all smoothness = 0).

4 runners at the launch, each performing good on their
respective distance, but some targets aren't reached by only one.

nji

8th January 2020, 21:01

https://forum.doom9.org/showthread.php?p=1895181#post1895181

redfordxx

8th January 2020, 23:58

First of all:
"(DeShaker option to) limit x and y rotation".
No, you get DeShaker's GUI wrong.

No, I didn't.
I thought if there will be 3D added, there could be these limits added so that it will not lead to worse results ( which I still don't know what you refer to).

(Would be a kind of magnifying "above" and reducing "below"
the axis I guess)

Exactly.

Seems possible, but isn't done (and won't be ... believe me :))
But maybe/ probably the other mentioned tools will?
You mean even if I will find some time and make a proof of concept that it could work, there is no one who will continue the deshaker?
Because other projects will not help without better 1st pass.

nji

9th January 2020, 00:54

Deshaker3D and Lens Transform do something like that.

But of course you're free to try whatever you want to.

I'm just a user and not the one to answer about the availabilty of source code.

redfordxx

9th January 2020, 01:09

Maybe I test something in Avisynth.
But first, can someone tell me, how can I get rid of the black frames in the beginning of the video, when I
"Use previous and future frames to fill in borders"?
Only Avisynth does that, VDub is ok...

redfordxx

10th January 2020, 23:19

I hope I understand optics roughly correctly.
I played a little with the idea of compensating x and y rotation.

I took one frame of my video and tried to find warp with best improvement of OK blocks.

Original motion estimation: 28.7%
After warp: 62.9%

See the pictures. I used avs codepoints_f1="
0,0 0,0
640,0 1,0
640,480 15,8
0,480 0,3
"
warp(points_f1, interleaved = true, relative = true)
I know there is also partial z rotation and x and y shift but it does not matter to the demonstration.

Anyone thinks it has a potential?

BTW:
"19|1|20|4|1|0|1|0|640|480|0|1|1000|1000|1000|1000|4|1|0|2|8|30|350|4|deshaker.log|0|0|0|0|0|0|0|0|0|0|0|0|0|1|15|15|5|15|0|0|30|30|0|0|0|0|1|1|0|10|1000|0|88|1|1|20|5000|100|20|1|0|ff00ff"

wonkey_monkey

10th January 2020, 23:41

Do you get a better result with Quad in place of Warp?

nji

11th January 2020, 01:04

I would like to mention I've been told
that the equations to determine the 4 parameters
are kind of "ill-posed".
If you reduce the number of parameters the situation
gets better (That's the reason for being able to disable
calculate for rotation and zoom in Deshaker's GUI).
If you enter two more paramters the problem probably
will get way worse?

Second: I'm not sure if a better matching of the OK blocks
in general means better matching of the actual movements.
Don't forget there is no accounting of optical conditions (FOV etc.)

redfordxx

11th January 2020, 09:47

Do you get a better result with Quad in place of Warp?

I read somewhere warp is successor of quad.

Ok, I try again with quad...

redfordxx

11th January 2020, 10:21

I would like to mention I've been told
that the equations to determine the 4 parameters
are kind of "ill-posed".
Meaning complicated or what?
Second: I'm not sure if a better matching of the OK blocks
in general means better matching of the actual movements.
But I think it is good estimate as long as I keep all the discard parameters pretty strict.
Don't forget there is no accounting of optical conditions (FOV etc.)
Meaning what?

nji

11th January 2020, 10:21

...Don't forget there is no accounting of optical conditions (FOV etc.)

To be more specific about what I mean by that.

To my "math intuition" analyzing the 2D data without taking into account
FOV (for larger values) is like doing the first step, very accurate,
more and more sophisticated ... but into the wrong direction.

That's why I was heading for doing something with FOV before analyzing
https://forum.doom9.org/showthread.php?p=1895012

Also ... if you have a look on the example video that the author of
Lens Transform did
https://forum.doom9.org/showthread.php?p=1895255#post1895255
and have a closer look on the difference between
Deshaker original and Lens Transform
you see that the "perspective distortion from FOV" in the original 2D data
are so massive that it doesn't seem to make sense to do sophisticated
analysis on them.

But ... as said ... just intuition from me, ... maybe completely nonsense.

EDIT:
Our posts where at the same time .... so I didn't answered on your last.
Concerning "ill-posed": Please see the def of it, can't explain better.

redfordxx

11th January 2020, 10:29

I tried Lens Transform and even created aan adobe profile. But probably did something wrong, since I did not observe any change on my video.

nji

11th January 2020, 10:32

Did you import Deshaker.log? (Right click in the graphic)
Did you set the smooth value to say 50++ ?

EDIT
And it suffices to edit in the lens profiles something like:

[lens]
name = "24 mm (mFt)"
hfov = 40

redfordxx

11th January 2020, 11:17

Loaded the lens profile. There was hero (extremely visible) and my profile not perceivable.
Could be maybe downloaded profiles for smartphones? I have Samsung A520F.

nji

11th January 2020, 11:59

Did you checked all points from my last post?

BTW
Maybe specific discussion on Lens Transform
should be in
https://forum.doom9.org/showthread.php?p=1747268
as this thread is about Deshaker related.

wonkey_monkey

11th January 2020, 12:57

Quad and Warp produce different results. Quad used to be a separate plugin but is now part of Warp.

redfordxx

11th January 2020, 21:29

That's why I was heading for doing something with FOV
My guess is to fix the lens distorsion first and project it to sphere so that movement of camera won't change the image?
and then run both deshaker passes
and project it to back whatever I want...?

redfordxx

12th January 2020, 01:18

I will try to figure out defish with a, b, c parameters from PTGui and then run deshaker.

nji

12th January 2020, 10:08

Once more I would like to point to the thread
https://forum.doom9.org/showthread.php?p=1895012#post1895012
and the statements wonkey_monkey did on that.

Also in the thread about shekh's Lens Transform at the end
there are some thoughts about it.

As I said before I doubt this thread is the right place for
discussion the matter, as as far as I know
orig Deshaker won't be modified, and source is (and probably will never?)
be available.

However I personally think your 2 ideas could lead to a great
improvement - but only for movies with large FOV.
Good luck! :)

redfordxx

15th January 2020, 23:10

Please help about settings 1st pass. I suspect this behaviour:
When there is some problem finding vector for certain spot, then there is whole big square of missing vectors around.

Which parameter to tweak to help with this?

redfordxx

17th January 2020, 00:43

Can I setup somehow, that I want:

first 2000 frames --- infinite smoothness
rest of video --- normal smoothing

StainlessS

17th January 2020, 07:22

Can only suggest to do in two chunks.
1st 2000, in two passes, save result.
remainder, in two passes, save result.
Join together.

(There could be a way to do the lot in single script using TWriteAvi:- https://forum.doom9.org/showthread.php?t=172837&highlight=TWriteAVI
but would need to do a pair of double passes, with first half filename for FirstHalf_pass1, FirstHalf_pass2, and second half filename for SecondHalf_pass1, SecondHalf_pass2.
Also need two sets of args for Deshaker.
TWriteAVI, cannot not use avs MT and x86 only - TwriteAVI contains old Vdub code incl a little bit of assembler, I dont do x86/x64 assembler)

EDIT:
Basic two pass script (only for normal single use)

WhateverSource(...)
A_Two_Pass_Function(Pass=1) # Create some kind of deshaker log or whatever.
ForceProcessAVI() # Force Pass 1, creating log (A function From TWriteAVI plug)
A_Two_Pass_Function(Pass=2) # Use created log
return Last # 2nd pass when you play/save clip.

First suggestion will probably be less hassle.

EDIT: Double chunks something like

WhateverSource(...)
Clip1 = Trim(0,-2000) # 2000 frames
File1="D:\File1.txt"
Clip2 = Trim(2000,0) # remainder
File2="D:\File2.txt"
Clip1.A_Two_Pass_Function(Pass=1,LogFile=File1)
ForceProcessAVI() # Clip1, Force Pass 1
Clip2.A_Two_Pass_Function(Pass=1,LogFile=File2)
ForceProcessAVI() # Clip2, Force Pass 1
Result1 = Clip1.A_Two_Pass_Function(Pass=2,LogFile=File1) # Clip 1 Pass 2
Result2 = Clip2.A_Two_Pass_Function(Pass=2,LogFile=File2) # Clip 2 Pass 2
return Result1++Result2

redfordxx

17th January 2020, 08:50

Can only suggest to do in two chunks.
1st 2000, in two passes, save result.
remainder, in two passes, save result.
Join together.
Something like this was first thing I was thinking of...
However...
I thought the last frame of the first part could be little off because the clip is fixed to first frame and there would be a little movement.

In your example would not it be the case?

If so, I was thinking: reverse the first part, then do the job with one frame in the middle overlapping, reverse back, get rid of the overlapping frame and join...

StainlessS

17th January 2020, 09:57

because the clip is fixed to first frame
Not sure what you mean there, also not sure why you want to reverse the clip. [I have not read the entire thread]
For duplicate, maybe DuplicateFrame:- http://avisynth.nl/index.php/DuplicateFrame
for delete DeleteFrame:- http://avisynth.nl/index.php/DeleteFrame

If you are going to reverse, then you better be using a totally frame accurate source, nowt better than AVI Lossless, convert to AVI Lossless with FFMPEG if not.

General purpose demo.bat

REM We DO NOT LIKE SPACES IN FILE NAMES (REM == REMark ie comment)

setlocal

REM Where to Find ffmpeg
set FFMPEG="C:\BIN\ffmpeg.exe"

REM Where to get input file, No terminating Backslash, "." = current directory (ie same as dir .bat file)
set INDIR="."

REM Where to place output file, No terminating Backslash. "." would be same as .bat file, ".\\OUT" = OUT folder in same directory as bat file.
set OUTDIR="D:"

REM Below, can add extensionas as eg *.WMV (SPACE separated)
FOR %%A IN (*.mp4 *.vob *.mpg *.TS *.MOV) DO (

REM ****** Un-REM ONLY one of below lines *******.
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec copy -acodec copy "%OUTDIR%\%%~nxA.MKV"
REM %FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec copy "%OUTDIR%\%%~nxA.MKV"
%FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"
REM *********************************************.

)

REM ... Above UN-REM'ed lines :
REM (1) Remux, copy both video and audio (output MKV).
REM (2) UtVideo lossless video, copy audio (output MKV).
REM (3) UtVideo lossless video, PCM audio (output AVI).

Pause

redfordxx

17th January 2020, 12:15

Not sure what you mean there
Because afaik with the infinite smoothness the deshaker fixes the first frame and then the view can go little off.

So, at the and of the first part the view would not be not aligned with the beginning of the second part.

Therefore reverse... instead moving away the camera should converge to the desired position...starting point of the second part.

I have one more idea right now: copy the first frame of the second part before the first part. That should position the clip correctly in the beginning ... and hold ok ... except of computational error.

But now...just more thinking...I cannot use zoom for borders restoring because in the first and second part I will end up with different zoom...and they micht not fit together.

StainlessS

17th January 2020, 15:57

I guess you must wait for advise from somebody that knows what is what, I cant remember the last time I used Deshaker.

redfordxx

3rd February 2020, 09:32

I use Deshaker to process avs script. I that script, I added some black borders (with AddBorders and Quad filters). I set up Deshaker to ignore the black.

When I process the clip then, sometimes Deshaker fails to analyse the vectors properly. When I go back to the bad frame and manually step though them, the results are ok.

Anyone experienced this kind of behaviour?

Bernardd

3rd February 2020, 09:52

I am not sut that resolve your problem.
But to choose black color is not a good idea, black is everywherre.
Choose color used in incrustion video work, like green.

redfordxx

4th February 2020, 13:49

I am not sut that resolve your problem.
But to choose black color is not a good idea, black is everywherre.
Choose color used in incrustion video work, like green.

Thank you. I used magenta and helped.

But still, clip playback has different results that stepping with arrouw.
Really frustrating.

redfordxx

6th February 2020, 10:09

Enabled rolling shutter. Gives better results.
However, what is the meaning of the percentage?
I compared visually the analysis of vectors 1% and 99% and looks same...

redfordxx

6th February 2020, 14:24

I am processing again once processed video. I have added borders not to loose the whole frame.
I use borders $FF00FF not to be processed.

I have found another behaviour.
Deshaker does not process fixed area at the border image, in certain cases. See the picture.
https://i.postimg.cc/bDWFgP8n/20180513-171911-pass2-115.jpg (https://postimg.cc/bDWFgP8n)

(1)The processing area seems to be always rectangular
(2)The rectangle is sometimes smaller than necessary

Changing Initial search range has influence on that, but is not solution.

I don't know whether it is possible to fix (1) but maybe (2)?
Any settings?

redfordxx

19th February 2020, 08:48

Hi.
Is that possible, that the rolling shutter depends on the shutter speed? If the camera supports setting it fixed.

If so, I would say
RS for 30FPS with 1/33sec equals 60FPS with 1/66sec.
RS for 30FPS with 1/66sec is half of 60FPS with 1/66sec.
RS for 30FPS with 1/33sec is double of 30FPS with 1/66sec.

When I take an example record from the Deshaker RS documentation I think all was taken at the same fixed shutter speed.
Canon EOS 80D
- 1080p60: 90%
- 1080p30: 45%
- 1080p24: 36%
- 1080p30 with 3x digital zoom: 90%

MysteryX

23rd January 2022, 02:43

Using this plugin under Vapoursynth could be possible by porting the VirtualDub compatibility plugin....

But will there ever be a possibility of using it on Linux? Since the source code isn't available and the author hasn't been heard from in 8 years, I suppose not a chance?

poisondeathray

23rd January 2022, 02:46

Using this plugin under Vapoursynth could be possible by porting the VirtualDub compatibility plugin....

I haven't tested it in vapoursynth, but apparently you can load it with avs.LoadPlugin

Note that if you are really insane you can load Avisynth’s VirtualDub plugin loader and use VirtualDub plugins as well.

http://www.vapoursynth.com/doc/functions/general/loadpluginavs.html

StainlessS

23rd January 2022, 11:04

the author hasn't been heard from in 8 years
Author (guth, ie Gunnar Thalin, first post in this thread @ post #5),
last sign-in Dec 2020.

EDIT: I think that last time I posted in a Gunnar Thalin thread, he popped up a day or two later.

guth

19th June 2022, 07:15

Author (guth, ie Gunnar Thalin, first post in this thread @ post #5),
last sign-in Dec 2020.

EDIT: I think that last time I posted in a Gunnar Thalin thread, he popped up a day or two later.

I'm still alive! :)
I don't seem to get notification emails?

Using this plugin under Vapoursynth could be possible by porting the VirtualDub compatibility plugin....

But will there ever be a possibility of using it on Linux? Since the source code isn't available and the author hasn't been heard from in 8 years, I suppose not a chance?

Funny, the reason I'm here is to post the next post... see below.

guth

19th June 2022, 07:15

I don't know if there's any interest still, but I (finally) think it might be a good idea to make the Deshaker source code public. :)

Any recommendation on the best way to do this?
Do I just put a zip on my Deshaker page? Or should I put it on GitHub, or something, and how much work is that? Maybe someone else would like to "run" it??
And what license? I don't really care if people make money on it (anymore), but is it still a bad idea to make it completely free? (Excuse my lack of knowledge and interest in legal stuff.)
Also, please keep in mind I've become rather lazy. :)

Boulder

19th June 2022, 11:03

The notification emails have been broken since a long time ago :devil:

I vote for putting the source code on GitHub. I do not believe that it's a big job but someone else can confirm that - or just add it to their repo :D I'm sure there's something useful for someone in the code sooner or later.

StainlessS

19th June 2022, 11:38

I dont do GitHub and so cannot comment on how easy that would be.
I don't really care if people make money on it (anymore)
But I'm guessing that you would prefer not to have someone take totally Freeware source, change a few labels
to say that they are the authors, and make a packet of cash from the people that they deceive.
Open source with GPL would maybe be best.

Nice to see you again Guth :)

VoodooFX

20th June 2022, 11:50

Any recommendation on the best way to do this?
Do I just put a zip on my Deshaker page? Or should I put it on GitHub, or something, and how much work is that?
On GitHub it would be there ~forever, and it's easy, few clicks if your source is one-few files, do it through the website there, not through Git.

And what license? I don't really care if people make money on it (anymore), but is it still a bad idea to make it completely free?
I would say, good luck if anyone by some magic manages to make money out of it. :D

This is do whatever you want license:

The Unlicense

A license with no conditions whatsoever which dedicates works to the public domain. Unlicensed works, modifications, and larger works may be distributed under different terms and without source code.

This is simplest preserving a credit to your name, and preserving the open source if it's not incorporated into some MicrosoftDub:

MIT License

A short and simple permissive license with conditions only requiring preservation of copyright and license notices. Licensed works, modifications, and larger works may be distributed under different terms and without source code.

guth

21st June 2022, 14:36

I would say, good luck if anyone by some magic manages to make money out of it. :D

I have sold it to a few companies in the past, at least. Without even trying. :)

Anyway, thanks for the input, guys. If nothing unexpected happens I'll put it on GitHub within a couple of weeks.

guth

28th June 2022, 19:43

The Deshaker source code is now on GitHub:

https://github.com/gu-t/Deshaker

Please let me know if you see anything weird. I've used Git before, but I'm new to GitHub.

StainlessS

28th June 2022, 20:19

Most gracious of you, cheers Guth.

kedautinh12

29th June 2022, 00:22

Most gracious of you, cheers Guth.

I think you need release binary :D

guth

29th June 2022, 08:19

I think you need release binary :D

Binaries exist on my Deshaker page, that I link to. Or, what do you mean?

kedautinh12

29th June 2022, 09:08

Binaries exist on my Deshaker page, that I link to. Or, what do you mean?

Oh thanks

Rob105

30th May 2023, 11:59

Can i somehow set a horizon across different control frames so Deshaker would stabilize rotation to match this horizon or any way to set horizon at all?

Here's sample video https://www.upload.ee/files/15288523/00007_original_clip.zip.html

What approach would you use to make horizon straight with Deshaker?

stephen22

27th June 2023, 14:26

Find a level frame and process from there with rotation smoothness -1, max correction some large number. Reverse the first section and process similarly. Unreverse and join the 2 together.

isidroco

17th October 2024, 23:39

Hi, I made a PowerShell script [+batch caller] which will pick up all .LOGs from current [or specified] folder and generate fileName_OGG.txt chapter info per LOG (ie: to use with MKVmerge). Minimum duration can be specified (default 75sec). FPS must be known (default NTSC). Has chapter name skeleton as used in mkvtoolnix-gui.exe.
Here's Log2chaps.ps1 script:

# PARAM must be first command
param ( [int]$gi_minSec=75, [float]$gr_fps=29.970, [string]$gs_chapSkel='Chap <NUM:2>',
[switch]$pal, [switch]$ntsc, [switch]$yes )

$debugging= $false

write-host -ForegroundColor Green ''
write-host -ForegroundColor Green 'Log2Chaps v24.1016'
write-host -ForegroundColor Green ''
write-host -ForegroundColor Green ' Extract chapter list from each deShake .LOG files'
write-host -ForegroundColor Green ' Generates: originalfilename_OGG.txt'
write-host -ForegroundColor Green ''
write-host -ForegroundColor Green 'Usage: Log2Chaps [[[minSec] fps] chapSkeleton] [-ntsc|-pal] [-yes]'
write-host -ForegroundColor Green ' minSec: Minimun Chapter length in seconds (default 75)'
write-host -ForegroundColor Green ' fps: Frames Per Second (default 29.970) or -pal / -ntsc'
write-host -ForegroundColor Green ' chapSkeleton: (default "Chap <NUM:2>" where NUM is chapter # using :N digits)'
write-host -ForegroundColor Green ' -ntsc: 29.970 takes precedence over fps'
write-host -ForegroundColor Green ' -pal: 25 takes precedence over ntsc'
write-host -ForegroundColor Green " -yes: unnatended"
write-host -ForegroundColor Green ''

# param ( $G_ARCHIVE = $(throw 'Need file to upload!'),
# $G_LOGFILE = $(throw 'Need logfile!') )
#
# Run as ADMIN
# Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Force
# 24.1016 better dispay info
# 23.0513 fps between [3..240]; only inform skipped >10 frames
# 23.0510 escaped ´: in write-host, avoids error in win10
# 22.0918 minSec first argument, newline after chapterlist
# 22.0916 switches: -yes -pal -ntsc
# 22.0914 success/ already done messages
# 22.0831 range check
# 22.0829 vars: XY_name: {X: Local, Global} {Y: Integer Long Real(float) String Fileinfo Timespan Date ...}
# Array: XAY_name (ie local string array: las_nameOfVariable)
# 22.0829 do all LOGs, ask before
# 22.0824 implement digits
# 22.0823 chapNames as MKVGui default: 'Chap <NUM:2>'
# 22.0822 ogg chapters format
# 22.0821 chapters time hh:mm:ss.ddd
# 22.0806 initial idea
#

function parse_log {
param ( [ref]$lf_inFile, [long]$li_minSec=90, `
[float]$lr_fps=29.970, `
[string]$ls_chapName='Chap <NUM>', `
[string]$ls_chapNDig='00' )
write-host -ForegroundColor Green 'Params:', $lf_inFile.value.name, `
"; chapNm:$ls_chapName; fps:$lr_fps; minSec:$li_minSec; `
chapDigStr:$ls_chapNDig" -separator '' # no separator
# parse file
###
$las_nscenes= @( get-content $lf_inFile.value | Where-Object { $_ -match "n_scene"} )
[long]$ll_minimo= $lr_fps * $li_minSec
[long]$ll_lastChapN= 0
[long]$ll_lastSkipN= 0
write-host -ForegroundColor Green "`n Chap 0 " -nonewline # `n is newLine only in double quotes
$lal_chapterFrames= @( [long]0 ) +
@( foreach( $ls_line in $las_nscenes) {
# $ll_n= [long]( $ls_line -split '[ \t]+',3)[1] # 1st method: Take 2nd element, must have first empty
$null= $ls_line -match '\D+(?<digitos>\d+)' # OR 2nd method: Seek first digits, faster and +reliable
[long]$ll_n= [long]$matches[ 'digitos']
if ( $ll_n - $ll_lastChapN -lt $ll_minimo) { # skip if less than min seconds
if ( $ll_n - $ll_lastSkipN -gt 50) { #only inform if more than 50 frames
$ll_lastSkipN= $ll_n
write-host -ForegroundColor Green " skp$ll_n" -nonewline
}
} else { # new chapter
write-host -ForegroundColor Green "`n Chap $ll_n " -nonewline
$ll_lastChapN= $ll_n
$ll_n # Returns chapter number of frames
}
} ) # end foreach $las_nscenes
if ( $ll_lastChapN -eq 0) {
write-host -ForegroundColor Red "*** no chapters ***`n"; return }
write-host -ForegroundColor Green "`n"
# write-host $lal_chapterFrames
[float]$lr_1frame_ms= 1000/ $lr_fps
[int]$li_chapNum= 1
foreach ( $ll_frames in $lal_chapterFrames) {
# *NO* $lt_chap_ms= new-timespan -seconds $lal_chapterFrames[ 2]
# [string]$ls_chapMs= new-timespan -seconds ( [float]$ll_frames* $lr_1frame_ms* 1000)
$lt_chap_ms= [timespan]::frommilliseconds( [float]$ll_frames* $lr_1frame_ms)
# $ls_chapMs= "{0}" -f $lt_chap_ms; [string]$ls_chapMs= $lt_chap_ms; $ls_chapMs= $lt_chap_ms.tostring();
[string]$ls_chapMs= $lt_chap_ms.tostring()
if ( $ls_chapMs.length -eq 8) {
$ls_chapMs+= ".0000000" } #add missing ms in case of exact seconds
$ls_chapMs= $ls_chapMs.subString( 0, 12) # keep first 3 digits of ms leaving: hh:mm:ss.fff
# write-host -ForegroundColor Green ( '{0:hh\:mm\:ss\,fff}' -f $ls_chapMs) # not for powershell v2
$ls_outOdd= "CHAPTER{0:00}={1}" -f $li_chapNum, $ls_chapMs
write-output $ls_outOdd
$chapStr= $ls_chapName -replace '<num>', $li_chapNum.tostring( "$ls_chapNDig")
$ls_outEven= ( "CHAPTER{0:00}NAME={1}" -f $li_chapNum, $chapStr)
write-output $ls_outEven
$li_chapNum+= 1
}
# @{ frame= $_.Split( ' ')[0] }
# | foreach-object { $_ -spilt " ", 2) }
# select-string -path $lf_inFile -pattern "n_scene"
# write-output $chapters
}

#
# MAIN
#
[int]$li_chapDigits=2
if ( $gs_chapSkel -match '<NUM:*(?<xnumero>\d+)>') {
[int]$li_chapDigits= $matches.xnumero }
if ( $li_chapDigits -lt 1 -or $li_chapDigits -gt 9 ) {
$li_chapDigits= 2 }
$gs_chapNDig= '0' * $li_chapDigits # number of digits to print '00'
$gs_chapSkel= $gs_chapSkel -replace '<NUM:*(?<numero>\d+)>', '<NUM>'
if ( $gr_fps -lt 3 -or $gr_fps -gt 240 -or $ntsc ) {
[float]$gr_fps= 29.970 }
if ( $pal ) {
[float]$gr_fps= 25 }
if ( $gi_minSec -lt 1 -or $gi_minSec -gt 960 ) { #not more than 16min
$gi_minSec= 75 }
write-host -ForegroundColor Yellow "Params: minSec:$gi_minSec; ",
"fps:$gr_fps; chapSkel:$gs_chapSkel; digits:$li_chapDigits"
write-host ''
if ( $debugging) { b:; cd \v } # use B:\V tor testing
if ( -not $debugging -and -not $yes
# -and $gs_chapSkel -eq 'Chap <NUM>' #no params
) {
do {
$ls_confirm= Read-Host -Prompt 'Press (Y)es to continue, (N)o to abort'
if ( $ls_confirm -match 'n') { return ' ABORT '}
} until ( $ls_confirm -match '[ys]')
}

$li_outCount= 0
foreach ( $lf_logfile in get-childitem *.log) { #all .LOGs
if ( $lf_logfile.mode -match 'd') { continue } #skip dirs
write-host -ForegroundColor Green $lf_logfile
$ls_oggFileName= $lf_logfile.basename+'_OGG.txt'
if ( -not ( test-path -path $ls_oggFileName -pathType leaf )) {
# parse
$gas_oggFileOut= parse_log ([ref]$lf_logfile) $gi_minSec $gr_fps $gs_chapSkel $gs_chapNDig
write-host -ForegroundColor blue $gas_oggFileOut
if ( $gas_oggFileOut.count -gt 1) {
out-file $ls_oggFileName -inputobject $gas_oggFileOut -encoding ASCII -Width 132 }
if ( test-path -path $ls_oggFileName -pathType leaf ) {
$li_outCount+= 1
write-host "** Generated $li_outCount`: $ls_oggFileName"
write-host
}
if ( $debugging) { break } # debug: stop after first .LOG
} else {
write-host -- Already present: $ls_oggFileName
}
} # foreach .log
write-host
if ( $li_outCount -gt 0 ) {
write-host *** TOTAL Generated Files: $li_outCount ***
} else { write-host *** Nothing to do *** }
write-host

isidroco

17th October 2024, 23:40

And here's the log2chaps.bat batch caller (directory may be specified), help included:

@echo off
set log2chapsVersion=log2chaps v24.1016, isidrococo @ gmail

setlocal enableextensions enabledelayedexpansion & REM to get var value at exec with !varNm!

rem Log2Chaps [PATH] [minSec] [chapSkeleton] [fps]|[-pal]|[-ntsc] [-yes]
rem 75 "Chap <NUM:2>" 29.970 -yes

title %isiVersion%
echo/
echo/%log2chapsVersion%
echo/
echo/ Extract chapter list from each deShake .LOG files based in n_scene marks
echo/ Generates: originalfilename_OGG.txt
echo/
echo/Usage: Log2Chaps [logsFullPath] [[[minSec] fps] chapSkeleton]
echo/ [-ntsc^|-pal] [-yes] [-noPause]
echo/
echo/ logsFullPath: Path of all deshaker .LOG
echo/ minSec: Minimum Chapter length in seconds (default 75)
echo/ fps: Frames Per Second (default 29.970) or -pal / -ntsc
echo/ chapSkeleton: ^(default "Chap <NUM:2>" where NUM is chapter # using :N digits^)
echo/ -ntsc: 29.970 takes precedence over fps
echo/ -pal: 25 takes precedence over ntsc
echo/ -yes: unnatended
echo/
echo/

rem 24.1016 added pathDir and elevate command sending current dir, change parameters order, noPause
rem 22.0918 simple call to powershell

set selfDir="%~dp0"
set selfFullPath="%~f0"
echo Current dir "%cd%", batch dir %selfDir%

rem If 1st param has :\ try changing to it and remove it from PARAM
rem ~ remove and [re-]add quotes
set param1="%~1"
rem needs to be % instead of !, to avoid :: comment out
if not %param1% == %param1::\=% (
cd /d %param1% 2>nul& rem make current dir
rem remove first param
shift
)
echo Current dir %cd%

rem
set PARAM=
:getParam
rem Include trailing space
set paramN=%1 &rem dummy
set "PARAM=!PARAM!!paramN!"
shift
if not "!paramN!"==" " goto getParam

REM Check privileges; https://stackoverflow.com/questions/7044985/how-can-i-auto-elevate-my-batch-file-so-that-it-requests-from-uac-administrator/12264592#12264592
net file 1>NUL 2>NUL
if not '%errorlevel%' == '0' (
REM ELEVATE AND RERUN -WorkingDirectory "%cd%"
rem echo powershell -NoProfile -Exec Bypass Start-Process -FilePath %selfFullPath% -ArgumentList '"%cd%" %PARAM%' -verb runas
powershell -NoProfile -Exec Bypass Start-Process -FilePath %selfFullPath% -ArgumentList '"%cd%" %PARAM%' -verb runas
rem >NUL 2>NUL
exit /b
)

REM ensure powershell privileges to run scripts {not recommended, better use -ExecutionPolicy ByPass on run}
rem powershell Set-ExecutionPolicy RemoteSigned -Scope LocalMachine
REM Change directory to current script. "runas" starts in C:\Windows\System32
rem cd /d %~dp0

net file 1>NUL 2>NUL
if not '%errorlevel%' == '0' (
echo Must execute as adminstrator !!!
pause
exit
)

echo powershell -noprofile -exec ByPass -file "%selfDir:"=%log2chaps.ps1" %PARAM%
powershell -noprofile -exec ByPass -file "%selfDir:"=%log2chaps.ps1" %PARAM%

IF "%PARAM:nop=%"=="%PARAM%" pause

ReinerSchweinlin

28th November 2024, 07:31

The Deshaker source code is now on GitHub:

https://github.com/gu-t/Deshaker

By accident, I cam across this thread...

Thank you !!

Your plugins have been a great support for me over the years, thanx for the contribution and all the work!

tormento

5th December 2024, 16:18

By accident, I cam across this thread
We just need someone who provides native AVS+ plugin now. :D

ReinerSchweinlin

16th December 2024, 12:16

We just need someone who provides native AVS+ plugin now. :D

Indeed - or Vapoursynth :) Would be lovely :) I have been dreaming about learning to code stuff like that for yearts - but sadly I don`t have the time or am too stupid...

nji

5th June 2025, 09:58

I would like to annotate that DeShaker doesn't account
- field of view (FoV)
- movements within a frame.

As consequence it produces good results only for parallel projection
(telephoto lenses) that have no movement within the scene;
like a photocopier where you move the paper while scanning ;).

But already for a "standard view" objective (50 degrees),
DeShaker will produce kind of "wobbling" (you get sea-sick),
the more the farther away from the centre.
The wrong transformation also effects sub-optimal filling of borders
and "colour masked areas".

Movements within the frames:
DeShaker analyses the movements between frames,
but then doesn't use that vector field (dense optical flow),
but takes only a "main" movement of the frame; global transformation.
For particular movements this leads to another sub-optimal reconstruction
of filling borders and "colour masked areas".

As the author thankfully provides the source code,
it could be improved by someone who's able to...
(FoV might be easier than particular movements).

I know of 2 other stabilising filters accounting FoV around here:

There is "lens transform" from @shekh.
https://forum.doom9.org/showthread.php?t=172886
But it uses only a simple interpolation (info lost) and border filling
and doesn't account rolling shutter. No code provided.
Still its results are much better FoV-wise than DeShaker.

Then there is something for AviSynth (of course there is :) ) from @wonkey_monkey:
https://forum.doom9.org/showthread.php?p=1690146
But it is only for FoV; stuck in its very first stage/ proof of concept,
and lacks for accounting all other aspects. No code anyway.

So, sadly, there is no real good deshaking so far.
We've the choice between shaken or sea-sicked ;)