Deshaker by Gunnar Thalin

[PhotoCat]

" if we used a frame 1000 frames away, any error in this comparison would be multiplied by 1000"

Not necessarily. When the user picks the fill-in source frames from 15000-15100, there is a reason he did so.
i.e. From the user's visual inspection, these are the frames that will provide the closest matches to fill in the missing border in question.
At the same time, the user can specify a mask to mask out any moving subjects in the foreground. This way, the comparison should be pretty fruitful
to come up with an excellent edge fill. (I guess one can make the mask automatic too e.g. 100 pixels inwards from all left, right, top & bot edges.
Anything inside a "rectangular donut" will be ignored for comparison.)

I guess this is human guidance :)

However, fair enough Guth. It may take too much work :) Perhaps I should ask what is the best technique to fill frame edges for what we have right now in v3.1 :)
Assuming we are trying to avoid the "zooming in and out" effect by adaptive zooms, and also assuming a static background.

Thanks again!

[PhotoCat]

I have been reading your deshaker manual very carefully and there are a few things I don't quite get it fully, even after reading this thread completely. (Pls bear with me as I have just been playing with video editing for about 5 months as a hobby, though I have some DOS programming background from the old days)

Q1) I am confused by the following 2 settings in pass1:

A) Discard motion of blocks that have match value less than X
(X range is -1000 to 1000 and default is 300)

"If the best match value found while matching a certain block is below X this block is discarded.
Blocks that have been discarded by this setting show no motion vector in the output video of pass 1."

B) Discard motion of blocks that have 2nd best match larger than best minus X
(X range is 0-?, and default is 4)
"If the second best match value found while matching a certain block is close to the best match value, this means that the best shift found isn't very reliable. It could just as well be this second shift that's the correct one. For example a blue sky has good match values for all shifts. Setting this parameter correctly discards those unreliable blocks.
Blocks that have been discarded by this setting show no motion vector in the output video of pass 1."

Questions:

For A), seems like a 300 match value is a good value.
Under what circumstance should one increase or decrease this value? Can u give an example of when to use this?

For B), I am not 100% sure what u mean by best match value and 2nd best match value. Do u mean within the same block?

Please confirm my understandings:

It appears that within 1 block, you are trying out a few shift combo values (to make the current frame looks like the previous frame) and then sort the results by the "match value"
in an array. Correct?
Then you look at the first entry of the match value array and check if it is greater than 300 (e.g. default value). If so, you keep the shift vector, if not, u discard the shift vector. Correct?

For B), U compare the difference between the 1st entry of the match array and the 2nd entry. Right? If it is less than 4 (e.g. default value), then, discard the vector. Correct?
Your blue sky example is very illustrative.

Q2) When do u increase block size and when to decrease?
default is 30x30? Pls give an example.

Q3) "Skip frame if less than X percent of all blocks are ok"

When u say "all blocks", do u mean all blocks over the entire video area? In some cases, I need to mask out almost the entire frame only leaving about 20% of the background area for matching. In this case my max match percentage would be 20%, right?

Thank you Guth once again :)

[guth]

Not necessarily. When the user picks the fill-in source frames from 15000-15100, there is a reason he did so...
I never said it was impossible. I was just trying to give you an idea why it isn't trivial. It's simply too much work for a not so useful feature, imho.

Perhaps I should ask what is the best technique to fill frame edges for what we have right now in v3.1 :)
Assuming we are trying to avoid the "zooming in and out" effect by adaptive zooms, and also assuming a static background.
In that case there might not be a good solution. Personally, I would probably use "adaptive zoom full" with a big adaptive zoom smoothness value, like 100000, for this.

[PhotoCat]

"I would probably use "adaptive zoom full" with a big adaptive zoom smoothness value, like 100000, for this."

Thank you Guth! This is enlightening. Never thought of
a value this big... 100000! Will give it a try. :)

[guth]

Q1A: That's one of few settings I rarely change, personally. But I guess one reason to decrease it might be if there's a lot of noise. One reason to increase it might be if you for some reason get incorrect matches.

Q1B: Correct (except for unimportant details)

Q2: One reason to increase size is if there's noise or lack of detail. Large blocks increase the signal-to-noise ratio. One reason to use smaller blocks is if there's lots of moving objects all over the frame. Then, you'll want to squeeze in some blocks wherever you can.

Q3: Correct.

[PhotoCat]

Thank you once again Guth. Your insights have helped me become a better user of your marvelous Deshaker and a better video editor!
Moreover, I highly respect u as u have generously been giving so much to the video community! Not just the tool,
but also your teachings. U r truly a master of the trade!
:thanks:

[guth]

A bit exaggerated, maybe. :)
There are lots of people who have given a *lot* more to the video community.
Thanks anyway!

[PhotoCat]

Hi Guth,

My apology if this question has come up before.

First time trying to shoot video handheld without optical image
stabilization under heavy wind. U know what the result would be!
It was a vy shaky video!

Tried many deshaker settings and it got better.
However, I am getting a "jello effect" similar to what is
described in the following thread:

http://forum.videohelp.com/threads/348837-Video-stabilizing-problems-(software-stabilisation)

scroll down to post#11 - the warped house.

I was using Canon 5D Mk II with a 85mm lens and I placed
a rolling shutter value of 70% in deshaker config.

My question:
1) Is deshaker designed to handle this kind of warped video?
2) If yes, what settings can I tweak to compensate for this jello effect?
I have noticed that this jello effect is already there in the original video though.

thanks!

[PhotoCat]

My apology. I found the problem. Pls ignore my previous question in #358. I shot the video portrait and turned the camera 90 degrees.
Then I turned the video by 90 degrees BEFORE I ran deshake! Of course the rolling shutter was not compensated!
Now I just left the footage unturned and performed deshake as the first filter. The result is much much better with only very
little "breathing" or jello like effect. I have learned a big lesson here and I now know deshaker 3.1 works vy well with rolling shutter cameras :) Thanks!

[PhotoCat]

During the previous exercise, I thought the rolling shutter value of 70% for my 5D2 was not optimal. Therefore I tried different rolling shutter values trying to fine tune it. I tried 50, 68, 69, 70, 71, 72, 80. My thinking was that I will look at the last column of the log file (% block matched) and used it as an indication of how good the rolling shutter value was.

So I repeated the exercise after I found the problem but was surprised to find that the (% block matched) values in the log file didn't change much with different rolling shutter value! I tried the same 50, 68, 69, 70, 71, 72 & 80. So I guess the (% block matched) value is not a good indication of a good rolling shutter value.

Now my question is if there is a way to tell how good the rolling shutter value is by looking at the log file or anywhere else.

Thanks a lot!

[guth]

The rolling shutter amount isn't used in pass 1 at all. (I still put that setting under the "Parameters affecting both passes" category in the GUI, though, because the rolling shutter on/off switch *does* affect both passes, and the amount sort of belongs to it.)
The only guth-approved way to find the best rolling shutter amount is to follow the procedure on my web page. :)

[PhotoCat]

Thanks Guth once again for sharing your insights :) Yes, I have read your webpage about how to calculate the rolling shutter.
I was hoping that the log file would have some kind of indication of how precise the selected rolling shutter value is but shocked to
find out that it is not used at all in pass1!
I am using 5D2 and you've noted 66-72% on your webpage so I really don't need to re-invent the wheel.

I am really tying to squeeze the last bit of juice out of the rolling shutter distortion compensation here :)

Would it make sense if I do the following?
1) In pass 2, try out all the rolling shutter values: 66, 67, 68, 69, 70, 71 & 72 and generate a video for each.
2) Compare all the video outputs visually to determine which clip has the least distortion.
3) Pick the clip that has the least distortion and the rolling shutter amount associated with that clip.
4) Use that shutter amount for all future deshake purposes.

Do u think 66 and 72 will make a big difference to distortion?

Thanks!

[guth]

Would it make sense if I do the following?
1) In pass 2, try out all the rolling shutter values: 66, 67, 68, 69, 70, 71 & 72 and generate a video for each.
2) Compare all the video outputs visually to determine which clip has the least distortion.
3) Pick the clip that has the least distortion and the rolling shutter amount associated with that clip.
4) Use that shutter amount for all future deshake purposes.
You're asking me to approve a method that's different from the only guth-approved method? :)
Ok. If you're not a perfectionist, I guess your method should work fine.

Do u think 66 and 72 will make a big difference to distortion?
Not really.

[PhotoCat]

[QUOTE=guth;1697925]You're asking me to approve a method that's different from the only guth-approved method? :)
Ok. If you're not a perfectionist, I guess your method should work fine.

No... not really... Your method is golden standard. :)
I am just trying to fine-tune the value by looking at the
actual results, taken an initial range found by your method.
I guess if there is not much of a different between 66 and 72,
there isn't much point in fine-tuning LOL! Thanks agn Guth!

[PhotoCat]

Hi Guth, sorry to keep bugging u :)
Really hard to resist when u know a guru is online :)

I often use deshaker inside avisynth this way to avoid
generating a big intermediate file or even a big deshaked video file, to save disk space:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100|0|450|200|0|350|850|100|0|0|0|1|30|30|15|1|0|0|30|30|0|0|1|0|1|0|0|10|500|1|70|1|1|20|100000|100|20|0|0|ff00ff")

This way, all I need is the original .mov file, the above .avs file and the small .log file. So really hardly any diskspace is taken up and the deshaked .avs file is ready for use any time.

Today I dragged in about 4-5 .avs file like the one above into aviutl and it started complaining about running out of memory
for just about 2000 1080p frames in total!

So I am guessing the way I use deshaker inside avisynth might be too memory intensive. Is it deshaker or is it avisynth or both?

So my question is:
1) Am I totally out to lunch using deshaker inside avisynth??
2) If not, is there a more memory efficient way of using deshaker inside avisynth?
3) Can one use deshaker (just pass2) stand-alone without using avisynth or virtualdub? I guess I am asking if deshaker can frame serve by itself...??

Thanks once again!

PS: forgot to mention I have 10GB of RAM on a win7 x64 machine

[PhotoCat]

To save memory and CPU power, I have figured out a way to reduce deshaker workload:

Original avisynth script is:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100...

trim(0, 300)

The whole clip is 1000 frames long but I only need the first 300 for example. So I reorder the trim command as follows:

LoadVirtualDubPlugin ("C:\Virtualdub32\plugins32\Deshaker.vdf", "deshaker", preroll=0)
LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_7720.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

trim(0, 300)

deshaker("19|2|30|8|1|0|1|0|640|480|2|2|-1|-1|-1|0|2|1|6|2|0.1|15|300|4|W:\\Canon5D2_2014\\201407th\\Video\\7720\\MVI_7720_deshake01.log|0|0|1100...

So deshaker only needs to work on 300 frames rather than 1000 frames. This should save some memory & CPU I guess...

But here is the problem:

It is easy to trim off the frames starting from frame 0, before deshaking pass2 since deshaker will read the log file starting from 0 too and everything is in sync. No problem.

What if I need frames 100 to 400? (given that deshaker pass1 has been run on the full clip frames 0-1000 and log file written accordingly.)

If I trim the video before deshaking pass2, the log file will go out of sync.

So here is the question:

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Is there a way to do that?
Thanks again Guth!

[PhotoCat]

Some update: it appears that it was not a memory problem as windows performance meter showed that I still have at least 4GB of RAM unused when I get the memory
error from Aviutl. So it is most likely an Aviutl problem. However, my previous question is still interesting and I still would like to know how to deshake pass2 from the middle of a clip,
assuming a full length log file. Thanks Guth!

20141028 update: Aviutl/plugins developer MaverickTse has confirmed that Aviutl can only use abt 1.6GB memory space because it is a 32bit application.

Details here:

http://forum.videohelp.com/threads/366724-Aviutl-Tips-Tricks-and-Support-thread?p=2353294#post2353294

[fvisagie]

I am just trying to fine-tune the value

In that case, it might (arguably) be more accurate AND less work to use the guth-approved method;).

[PhotoCat]

Now the 32bit application limit has got me thinking...
Deshaker actually has a 64bit version which I think I am using with virtualdub x64, to perform all the interactive deshaker parameters tweaking and vectors monitoring...

It would be real nice if deshaker64 can be used stand-alone as
a frame-server during pass2, since the official avisynth is also 32bits.

[guth]

In that case, it might (arguably) be more accurate AND less work to use the guth-approved method;).
Yes, it's at least a more accurate method. Especially if you take the average (or median) value of several runs.

[guth]

Aviutl/plugins developer MaverickTse has confirmed that Aviutl can only use abt 1.6GB memory space because it is a 32bit application.
You might want to try the 4GB Patch (http://www.ntcore.com/4gb_patch.php) mentioned earlier in this thread. Run it on the main exe-file for Aviutl. That *might* give you 2GB extra memory to work with.

It would be real nice if deshaker64 can be used stand-alone as a frame-server during pass2, since the official avisynth is also 32bits.
Anyone up to the task can build something that uses Deshaker (and other VirtualDub filters) just like VirtualDub and AviSynth does. It could do anything with the output frames.
(And no, I'm not up to the task... :))

[PhotoCat]

Tks Guth for the aviutl 4GB patch tips. I will try that :)

I have found another way to do it and it seems quite
stable now with 6 clips and 3 pics in the aviutl project.
Just audio playback is a touch jerky but way better than
VirtualDub.

I just used avsf (Pismo File Mount) to mount the .avs file as a folder and I did it
for all 6 .avs files. Then I pulled all the .avi files inside the virtual folder into aviutl. Seems to work fine and stable and I
still have about 4GB memory to spare :)

Doing it the avsf way seems much more stable as I
don't see the "deshaking..." pop-up windows anymore.

This is how I used VirtualDub64 with avisynth anyway.

[PhotoCat]

Still love to know the answer to this question:

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Is there a way to do that? (I know the simple way would be to delete all the lines in the log file from 1-99 but perhaps there
is a more elegant way to do it without touching the log file??)
Thanks agn!

Thanks!

[PhotoCat]

(And no, I'm not up to the task... :))

U r a humble person Guth! :)

[PhotoCat]

Thanks Guth! The 4GB patch actually worked for Aviutl! I can see RAM committed to Aviutl has increased to 3.6GB at one point. However, I find the avfs file mounting method much more stable.

[guth]

Is there a way to ask deshaker pass2 to start reading the log file from a frame other than frame 1?
For the above example, only frames 100-400 are needed out of 1000 total frames so one can trim(100, 400)
before deshaker pass2. However, we need to somehow tell deshaker to start reading the log file from frame 100 as well,
skipping 1-99, to keep everything in sync.
Why not use trim(100, 400) before Deshaker pass 1 too? Or, if that's not possible, you could do the trimming after AviSynth. In VirtualDub, for example, you could simply select the section you want, and save it. Or am I misunderstanding you?

Thanks for letting me know about avfs, btw. I didn't know there was a way to open video from 32-bit AviSynth in 64-bit VirtualDub. Nice!

[PhotoCat]

Why not use trim(100, 400) before Deshaker pass 1 too? Or, if that's not possible, you could do the trimming after AviSynth. In VirtualDub, for example, you could simply select the section you want, and save it. Or am I misunderstanding you?

Thanks for letting me know about avfs, btw. I didn't know there was a way to open video from 32-bit AviSynth in 64-bit VirtualDub. Nice!

I think I should explain a bit more of my motivation.
I usually deshake on the full clip anyway, which is normally
about 1-2 minutes long. However, I usually would want
to pick just 45-90 secs of the clip to be a part of a music video :)

I could do trim(100, 400) b4 deshake but I would lose the other frames, which are good too. In another project, I
may want to use frames (20-150) for example. Then I will have
to redo deshake on frames 20-150, which is kind of inconvenient.

Since there is very little overhead for me to do the entire 2 minutes clips beforehand and store the log file and .avs script for future use as stock clips, I deshake the whole clip all the time. This gives me the flexibility of using any part of the clip in the future.

BTW I find this a great disk space saving method too,
as there is no intermediate files nor ProRes files to be stored after deshaking! All I need is the original .mov file, the tiny deshake log file and the simple .avs file. The original .mov file is there anyway so I am really not using much disk space to have a deshaked video handy for future use. :)

The requested feature of asking deshaker to begin to read from any line of the log file further reduces cpu & memory usage by deshaker pass2.

Glad u found avsf useful. I got the tip either in this forum or in the videohelp forum. Lots of excellent people on these 2 forums for sure! Thanks!

[guth]

You could still do the trimming after AviSynth instead, i.e. feed the avs for Deshaker pass 2 into VirtualDub or similar, and trim there.

[PhotoCat]

Yup, that's what I am doing now :) Tks!

[sorin86]

Deshaker is a very useful tool, almost a must. It has worked nicely for many of my videos. However, there's still a problem that is not uncommon, yet not tackled even in version 3.1: DUPLICATE FRAMES.

Say (for the sake of an example) you have a video at 15 fps. In 1/3 of a second (0.333s) you have 5 frames:
1 2 3 4 5
If you convert that to 30 fps, in 1/3 of a second you will have duplicate frames:
1 1' 2 2' 3 3' 4 4' 5 5'
where actually 1 = 1' (etc).

If you apply Deshaker to the 15-fps video, it will work probably beautifully.
However, if you apply it to the converted video, it will be a mess.

This is what happens. If you have a portion of the video with relatively fast panning and at the same time significant movement of objects (persons) in the video, the processing of deshaking the 15-fps video will be smooth. Deshaker with generate smooth transitions from one frame to another.
For the 30-fps variant, Deshaker interprets frames 1' 2' 3' 4' 5' as distinct frames. Although no change happens between frame 1 and frame 1', Deshaker will still apply smooth transitions. That actually spoils frame 1', which - so logically! - should keep whatever deshaking processing was applied to frame 1. The result is a very annoying back-jumping strobo effect, which makes Deshaker unusable - you'd rather watch the non-deshaked version.

The same annoying effect will appear even in videos with less frequent duplicate frames, like this recurring structure: 1 1' 2 3 4 5 6.

One could say "just remove the duplicate frames", but that is hardly feasible, while that type of structure changes throughout the video. It can go to 1 2 3 3' 4 5 6.
So, instead of less advanced users trying to figure out complex scripts for hours-long videos, would it be much easier for Deshaker to just identify a duplicate frame and just keep the processing data of the previous frame? After all Deshaker performs such a complex analysis... Identifying a duplicate frame should be a piece of cake.

[fvisagie]

I'm just wondering, is frame duplication really the ideal way of converting from 15 fps to 30 fps in the first place? Effectively you'll still have 15 fps-based motion, just played out at 30 fps. Would interpolation (e.g. MFlowFps()/MBlockFPs() and I think also SVP) be an option to provide smoother motion (and get around this problem) for you?

[fvisagie]

Is anyone interested in looking into this? I've done some feasibility checking and it is definitely very much possible, at least for AR.Drone 2 videos. Although more direct methods are available, the quickest time-to-implementation at this point seems to simply convert the output of ExifTool. I just don't have the time to take it further at this point.

In case someone would like to look into this:

The AR.Drone 2 stores flight metadata in two places: the .MP4 recording, and userbox files on the drone.

.MP4 recording

Information on the location and data structure: http://forum.parrot.com/ardrone/en/viewtopic.php?pid=32346#p32346 (somewhere I also came across the actual structure definition but unfortunately didn't save it)
ExifTool extracts flight metadata from the .MP4 file withexiftool -b -ardronetelemetry <inputfile>
ExifTool output is explained at http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,6123.0.html and http://owl.phy.queensu.ca/~phil/exiftool/TagNames/QuickTime.html (see 'prrt')
Here's a short extract:

0 7 84.023 -0.0138157112523913 0.020807821303606 1.58786511421204 0 0
0 7 84.055 -0.013349455781281 0.0169658288359642 1.58657622337341 0 0
...
1 7 85.913 0.0174348670989275 -0.00090780045138672 1.564129114151 0 0.659
1 7 85.946 0.0190824717283249 -0.00098724861163646 1.56361031532288 0 0.662
0 9 85.979 0.0209523718804121 -0.00107379746623337 1.56401801109314 0 0.665
0 9 86.012 0.0216640736907721 -0.00160936382599175 1.56422352790833 0 0.661


To convert from ExifTool to Deshaker pass 1 output format should be a cynch. Alternatively, the data could be parsed directly from the video, of course.

Userbox files

These are created on the drone: http://forum.parrot.com/ardrone/en/viewtopic.php?pid=32169#p32169
Depending on remote controller software, these files may be copied/moved to the controller after the flight
Ruby examples for converting userbox files to GPX: http://www.ardrone-flyers.com/forum/viewtopic.php?p=66997#p66997 and http://www.ardrone-flyers.com/forum/viewtopic.php?p=74791#p74791


I can provide short sample videos and userbox files if required.

[guth]

sorin86... Yes, any moving objects in the video will jump back and forth, but the background that is stabilized on will look smoother than before.
My honest (but maybe not so nice :)) answer to this problem is that if you have duplicate frames you're either doing something wrong, or you're using a lousy camera. Garbage in, garbage out...
I don't think I will address this problem unless I get lots of requests. Sorry...

[StainlessS]

@fvisagie
Dont know if this of any use but take a peek about this post (from about post #14 onward, user name Filker could be used as search filter in that thread)
http://forum.doom9.org/showthread.php?p=1588194#post1588194
Avisynth based (still camera Exif).

[fvisagie]

@fvisagie
Dont know if this of any use but take a peek about this

I'd had in mind to Call() an external script for the conversion, but that looks like it could come in handy for conversion in Avisynth.

The problem for me though remains not so much how to do it, but to find the time. I posted in the hope of finding someone else who's interested in taking the idea further.

[StainlessS]

fvisagie,
See Martin53 posts (and mine) in same thread as posted and also in Call(and maybe CallCmd) threads, he wanted to do something like that
for deshaker, I did some coding for him to parse and extract data from the files, dont know if he finished the project.

[fvisagie]

StainlessS,
I found those two threads, but saw nothing that seems related to Deshaker or parsing text files. Thanks, anyway.

[StainlessS]

Well you could have searched on deshaker with user name Martin53 as here:
LINK REMOVED:

in particular this
LINK REMOVED:

however, it seems M53 abondoned the project (or at least that attack stance).

[fvisagie]

Indeed I didn't think that far. A slow day, sorry and thanks.

[StainlessS]

fvisagie, sorry, the links posted did not work (they worked when I posted them, I checked), but after search and
log-off they must disappear or something. anyway, this was the 2nd link posted again and hopefully working this time.
http://forum.doom9.org/showthread.php?p=1596064#post1596064

Previous bad links removed.

[fvisagie]

@StainlessS,
Strange, that link did work for me the first time also :). Thanks for your trouble.

[fvisagie]

It seems possible, but will it work correctly deshaking a concatenated file in VirtualDub, one input clip at a time?

Background to the question: a large 350+ file HD project is creating space problems with the saving of intermediate files such as Deshaken versions of each input clip. An advantage of Deshaking clips individually is that scene changes are not missed. In this case it seems I'd need to work on a real or hopefully frame-served concatenation of input files, and I was wondering whether it would be possible to still ensure reliable detection of clip transitions. That would save me having to manually locate and check all those boundaries afterwards.

In principle the following approach seems possible, in the sense that it can at least be scripted in VirtualDub:
For each clip
Apply Clear() & AddRange() to restrict processing to that clip's material only
Process Deshaker pass 1 (configured to save individual log files for each clip - AddRange() will cause frame numbers to start at 0 for each clip)
Apply Deshaker pass 2 (configured to use the same log file as used in pass 1)

Some questions that create doubt:
1. What happens to the results of Deshaker's processing when pass 2 is applied (i.e. the "Deshaking" message)? Will this get over-written or somehow become inaccessible when the next clip is processed?
2. What happens once all of the concatenated clip has been processed, the last AddRange() is removed and saving or frame-serving starts? Frame number offsets as seen by Deshaker of all clips but the very first one will change - how will that affect Deshaker's operation?
3. Does this regarding frame-serving from the VirtualDub online help apply to Deshaker: "Video filters that have a lag — most notably the temporal smoother. The filter will function, but there will be a delay in the video."?

I realise that I can try this all out for myself. However, I am under some time pressure and would rather hear now "this is guaranteed to fail" as opposed to wasting time on a bad idea. On the other hand, if this could work, I'd appreciate any corrections or improvements to the approach.

Thanks,
François

[shekh]

When you isolate range, deshaker sees original frame numbers (not starting from 0 for each range) so in general this way should work.
However,
there is no meaning to "apply pass 2" if you do not save the result (video).
And there is no normal way to apply different deshaker instance (with different log files) to each range.

But instead you could simply use one normal deshaker instance with one log file: it can deal with scene cuts.
If you are going to manually pass range parameters to AddRange() you could as well just force these frames to be scene cuts in the deshaker log file.

[fvisagie]

When you isolate range, deshaker sees original frame numbers (not starting from 0 for each range) so in general this way should work.

Unfortunately, not in my experience. Applying AddRange() results in a VirtualDub clip with frame numbers [0...n-1], and the frame numbers recorded in the resulting Deshaker log file are [0...n-1]. Unless you are referring to a way of isolating ranges other than AddRange()?

And there is no normal way to apply different deshaker instance (with different log files) to each range.

That would then be the one potential deal-breaker, see below.

If you are going to manually pass range parameters to AddRange() you could as well just force these frames to be scene cuts in the deshaker log file.

I realise that that possibility is available, but it's very easy to automatically isolate individual clips in the VirtualDub script, which itself is very easy to generate automatically. Avisynth can be used to catalogue clip lengths and generate the script, and below is an example with VirtualDub doing the concatenation (of .avi inputs) itself. A batch file can easily generate such a script from a list of filenames:
// Start of script
declare startframe;
startframe = 0;
declare endframe;
endframe = 0;
// ...

// Start of clip 000 processing
VirtualDub.Open(U"clip000.avi");
VirtualDub.subset.Delete();
startframe = endframe;
endframe = VirtualDub.video.length;
VirtualDub.Log("startframe="+Sylia.ToString(startframe));
VirtualDub.Log("endframe="+Sylia.ToString(endframe));
VirtualDub.Log("cliplength="+Sylia.ToString(endframe - startframe));
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(startframe,endframe - startframe);
// ...

// Start of clip 001 processing
VirtualDub.Append(U"clip001.avi");
VirtualDub.subset.Delete();
startframe = endframe;
endframe = VirtualDub.video.length;
VirtualDub.Log("startframe="+Sylia.ToString(startframe));
VirtualDub.Log("endframe="+Sylia.ToString(endframe));
VirtualDub.Log("cliplength="+Sylia.ToString(endframe - startframe));
VirtualDub.subset.Clear();
VirtualDub.subset.AddRange(startframe,endframe - startframe);
// ...

// ...

So it seems the question is whether it is possible to apply different Deshaker instances to different portions of a clip?

Additionally, it seems there may be issues with temporal filters when frame-serving; see the added question 3. above. However, that at least has the work-around of saving to disk if needed.

[shekh]

Unfortunately, not in my experience. Applying AddRange() results in a VirtualDub clip with frame numbers [0...n-1], and the frame numbers recorded in the resulting Deshaker log file are [0...n-1]. Unless you are referring to a way of isolating ranges other than AddRange()?


Yes sorry, but you can use SetRangeFrames() which restricts range for analyze pass without tweaking timeline.


I realise that that possibility is available, but it's very easy to automatically isolate individual clips in the VirtualDub script...


Maybe you can generate log files this way, and then concatenate them with another script?


whether it is possible to apply different Deshaker instances to different portions of a clip?

You may try with blend curves, when a value of blend curve is 0 the filter is skipped. But I am not sure if this is efficient as all instances will probably take a lot of memory (and preparing concatenated logfile looks more easy to do).


Additionally, it seems there may be issues with temporal filters when frame-serving

AFAIK there is internal delay in pass2 if you apply border filling with future frames. But I dont know if/how it is connected with frameserving.

[fvisagie]

you can use SetRangeFrames() which restricts range for analyze pass without tweaking timeline.

Yes, that seems workable, although the fact that the offsets are in milli-seconds makes that a somewhat shaky approach.

Maybe you can generate log files this way, and then concatenate them with another script?

At this point that seems like the safest way 1) to avoid the creation of intermediate files and 2) to ensure all scene changes are detected. Something like
Run Deshaker pass 1 on individual clips
Concatenate clips and log files, adjusting frame numbers in the latter
Set Deshaker to pass 2 and render

Thanks for your suggestions.

[shekh]

Yes, that seems workable, although the fact that the offsets are in milli-seconds makes that a somewhat shaky approach.


Which milliseconds? SetRangeFrames() is equivalent of "mark in/mark out" in the UI.

[fvisagie]

Which milliseconds? SetRangeFrames() is equivalent of "mark in/mark out" in the UI.

Yes, when done manually. However, when invoked from a script (from VirtualDub scripting language reference, v0.7 (http://www.virtualdub.org/docs/vdscript.txt)):
int GetRange(int var);
void SetRange(int startMS, int endMS);

Gets or sets a particular range value. If var=0, the start offset
is used, and if var=1, the end offset is used. The start offset is
measured in milliseconds from the beginning, and the end offset is
in milliseconds from the end. Yes, I know this is stupid.

Because the end offset is only an offset, it is not possible to
determine the length of the video stream using these functions.

[shekh]

Maybe this script reference is too old.

oops I was referring to SetRangeFrames not SetRange

it is saved when you do "save processing settings" and select "include selection"

[fvisagie]

oops I was referring to SetRangeFrames not SetRange

Thanks for pointing that out; that would simplify things a lot.