New Filter: Stabiliser (like DePanStabilise, DeShaker)

[mg262]

Demonstration (http://people.pwf.cam.ac.uk/mg262/posts/Background/boysaved_stabilised_abs15_comparison.avi)

You will need both the filter -- which only takes YV12 and requires SSE2:
Cel Background, 23 September 2005 (http://people.pwf.cam.ac.uk/mg262/posts/Background/CelBackground_23Sep05B.dll)

...and a script file:
Cel Stabilise, 23 September 2005 (http://people.pwf.cam.ac.uk/mg262/posts/Background/CelStabilise.avs)

Usage:
LoadPlugin("CelBackground_23Sep05.dll")
Import("CelStabilise.avs")

#load clip here
Stabilise(capabsolutemetric(15))
#crop(4, 4, -4, -4) #or as appropriate to remove borders created by stabilisation

It's fairly slow at present, although speeding it up is on my to do list; if a frame is requested non-sequentially, it will motion-compensate all the intermediate frames (if they haven't been calculated already) -- so it will freeze for a few seconds.

This filter has lots of options but I have found the hard way that no one will look at a filter with too much documentation, especially if there aren't any pretty pictures! So I shall give lots of little examples slowly over time, or if people express interest. Next will probably be the built-in scene detection, and then after that automatic selection of still scenes to stabilise.

Edit: I should have said that this is primarily meant for animation, especially old animation, although as far as I can see it should work on live material if the background is reasonably still; if you have trouble with this, post a clip.

[Guest]

This filter has lots of options but I have found the hard way that no one will look at a filter with too much documentation! Sounds a bit like an excuse that you can't be bothered to properly document it. No offense intended.

My suggestion: Provide a quick start guide for beginners and a full reference manual for power users. That's how you maximize your user base.

Thanks for the new filter. Source code?

[mg262]

Sounds a bit like an excuse that you can't be bothered to properly document it.It's not that... it's that I haven't had a single piece of feedback suggesting that anyone has actually used the corresponding Foreground filter, which has been quite thoroughly documented. (Edit: including a split into basic and advanced material.) I should have put up the HTML at the start, but even allowing for that, I felt there was too much material and it overwhelmed people.

Edit: hence also putting the script in a .avs file so users don't have to look at/understand it if they don't want to. Source is still very much incomplete and in flux. Stabilisation is really a side-effect of the filter, but I decided it was useful enough in itself to release at this stage.

Edit: I have just discovered that (for some reason which I don't understand), pixels with luma below 16 (and some chroma values) are rendered as white rather than black on my system. So I have modified the filter to produce only luma values of 16 and above.

[Tommy B.]

Looked at the example, interesting.

Tried it my self, well, didn't work. There is no function named "capabsolutemetric".

What's wrong? :confused:

[mg262]

I just re-downloaded the plug-in and checked with the following script:

LoadPlugin("CelBackground_23Sep05.dll")
capabsolutemetric(blankclip().converttoyv12(),15)

and it doesn't produce an error message... would you mind posting your script? Also please try putting the full path to the DLL in the LoadPlugin command.

(And if anyone else is getting the same error message, please let me know...)than a

Edit: My version of AVISynth is not case sensitive, but perhaps yours is for some reason... so please try changing the capitalisation to CapAbsoluteMetric

[Tommy B.]

Import("C:\Programme\AviSynth 2.5\plugins\CelStabilise.avs")

LoadPlugin("CelBackground_23Sep05.dll")
capabsolutemetric(avisource("sta.avi").converttoyv12(),15)

I'm using avisynth 2.5.5 and put the "CelBackground_23Sep05.dll" into the plugins folder (so it is loaded automaticly).

When I try loading the dll manualy (using LoadPlugin("C:\Programme\AviSynth 2.5\plugins\CelBackground_23Sep05.dll"), it reports with an error (Unable to load "CelBackground_23Sep05.dll").

BTW, when using your code, I get some strange cryptic sings (see attachment).

And I also tried the case sensitive thing, didn't work either.

[mg262]

I just tried putting it in the plug-ins directory and that also seems to work for me on 2.5.5. The attachment isn't approved yet, but I think I know the error message you mean -- it happens when you try and LoadPlugin with an invalid path.

This is very strange... whatever it is, I'm sorry. One more thing to try if you haven't run out of patience:

Remove it from the autoload directory, put it in the same directory as both CelStabilise.avs and your script file, rename it to something shorter without any underscores or numbers (e.g. Background.dll), and load it manually.

Would someone else be kind enough to try downloading the filter and telling me if they get an error?

EDIT: Are you loading the plugin after you import the script? If so, please switch them round.

[Tommy B.]

I put anything into one folder. deleted the dll and avs from the plugins directory. The script:

LoadPlugin("s.dll")
Import("s.avs")

blankclip = avisource("sta.avi")

capabsolutemetric(blankclip().converttoyv12(),15)

Hmmm... say, I just had some thoughts about sse2 on Athlon XP... and after some research on google it seems as if only Pentium 4 supports it. Great. That's it for me then. Back to VirtualDub and deshaker :rolleyes:

[mg262]

I am sorry... I was using particular quirks of sse2 to pull out speed; it would be horrendously slow without. In this case adapting it to MMX isn't entirely straightforward but I will have a go at it when I have time.

Edit: I have also fixed a problem occurring when the framework couldn't support allocation of a very thin frame. (Thanks, Myrsloik!)

[Tommy B.]

No problem. There are some alternatives to stabilize video.

Keep it up, you're creating the first avisynth filter of this kind ;)

[krieger2005]

@Tommy B.: And what's about the depan-filter?

[Tommy B.]

I'll try that. Thank you.

[mg262]

Yes. For what it's worth, these are the distinctive features of this filter:

1. It takes masks specifying regions to ignore.
2. It works on a scene by scene basis and tries to stabilise still scenes completely rather than applying a filter to reduce the motion.*
3. The sensitivity to different luma-differences is specifiable.

*This isn't "better" than the alternatives -- they're suitable for different tasks.

3. is a bit cryptic, so here is some explanation. When you try to stabilise the motion, you are roughly doing this: take frame n+1, put it on top of frame n, and slide it around until you find a good match. (Think of n+1 as semitransparent.) The standard ways to judge how good a match is are a) to add up the differences of the luma values* and b) to add up the squares of the differences of the luma values. So you slide around n+1 to try and make this number as small as possible.

*i.e. of each pair of pixels that lie on top of each other.

Both methods say that a white object "matches" a black background much worse than a grey object does -- whereas in fact, they are equally bad. Above a certain level, depending on noise but always relatively small, bigger differences don't mean anything... so the filter lets you specify the "penalty" for each luma difference. (+this has subtler uses which I won't go into now.)

[trolltuning]

(edit)
*This isn't "better" than the alternatives -- they're suitable for different tasks.


Would some one please explain when you would use DePan and when you would use this?

[mg262]

I'm a bit reluctant to start a full-out comparison because it will be difficult for me to be unbiased. For the tasks I was working on, I couldn't obtain sufficiently stable motion using DePanStabilise or DeShaker -- but that might reflect my inability to use those tools correctly rather than any limitation built into them. So, I will write something, but give me a bit to try and phrase it sensibly. The other thing is, if you happen to have a clip that you're trying to stabilise, I could use a section of it!

Life is pretty hectic at the moment but when things calm down I will put out an iSSE version.

[trolltuning]

@mg262
Thanks I'll try to upload one over the weekend. (This one came out perfect using DePan (much to my surprise as I'm pretty sure I hadn't followed all the instructions right.)
How many frames do you want? (29.97 FPS)

[mg262]

At least one complete scene, if you can manage it -- more if it's convenient. But I have other material to test the filter on, so don't worry if you want to leave it now you have it sorted out! If you do upload some, could I also have the DePan parameters (and, if relevant, inverse telecine settings), please?

Thanks, M.

[Derailed]

Is there any developement going on with this filter? I used it on an old anime sample that I had, but for some reason, a quarter of the original frame remained, and the other three quarters of the frame were white. So only a corner of the original frame remained.

Here is the script I used:

loadplugin("CelBackground_23Sep05B.dll")
loadplugin("CelBackground_22Oct05.dll")
loadplugin("Cel_Background_06Jan06.dll")
import("CelStabilise.avs")
avifilesource("O:\Anime\test.avi").converttoyv12()
Stabilise(capabsolutemetric(15))
crop(6,6,-6,-6).lanczosresize(720,480)

[EDIT] I would also like to let you know, that I did try depanstabilizer and deshake, but I wasn't getting any of the frames with those filters.

[Boulder]

I think you should load only the latest plugin, not all of them.

[Derailed]

I tried that, but then I got an error.

"There is no function called AccumulateMotion"

[Boulder]

What is the error message?

[Derailed]

I already showed the error message:

I tried that, but then I got an error.

"There is no function called AccumulateMotion"

[3ngel]

I'm interested in this filter, but as i've understood the CelBackground_23Sep05B.dll is for P4 only, so it gives error on me. There are any other non P4 ports of the dll so i can test it?
Thanks

EDIT : Putting the dll in the autoloading directory of AviSynth i got the error :
"There is no function named CapAbsoluteMetric" same error with capabsolutemetric

[Derailed]

It seems to me, that everytime there is a scene with low motion in front of a scene with high motion or medium motion (pans), it tries to adjust the high or medium motion scenes to the low motion scene. However, when I split the scenes apart, no problems like the one I described before have occured.

Some errors that still occur even after seperating the scenes are:
1.) Sometimes there are scene that fade in from black, and those scenes always mess up like I described before.
2.) Sometimes the scene changes in the show are really bad, and it still gives the same problem.
3.) [EDIT] This problem also occurred in pans. Also, seemed to adjust pans to a low motion scene, when the pan is "connected" to the low motion scene. I tried to enable the Expand(900, 640)) in the Cell Stabilise script, but avisynth reports it as an error. Am I missing a plugin?

Would scene change detection fix some of these problems?

I noticed that the pan of the mountains in one of mg262's examples didn't have this problem and was cropped perfectly. The one with the original source and denoised sample right next to each other. The link for that is here:

http://people.pwf.cam.ac.uk/mg262/posts/Background/mountainhalf_d40_sidebyside_xvid.avi

I will quote what I said before to explain the problem I am talking about.

Is there any developement going on with this filter? I used it on an old anime sample that I had, but for some reason, a quarter of the original frame remained, and the other three quarters of the frame were white. So only a corner of the original frame remained.

Here is the script I used:

loadplugin("CelBackground_23Sep05B.dll")
loadplugin("CelBackground_22Oct05.dll")
loadplugin("Cel_Background_06Jan06.dll")
import("CelStabilise.avs")
avifilesource("O:\Anime\test.avi").converttoyv12() Stabilise(capabsolutemetric(15))
crop(6,6,-6,-6).lanczosresize(720,480)


[EDIT] I would also like to let you know, that I did try depanstabilizer and deshake, but I wasn't getting any of the frames with those filters.

[djspy]

Anyone knows where to get the CelStabilise.avs now?

When I tried the link of the first post I get a 403.

Thanks in advance!

Dj

[lansing]

is this CelStabilise.avs still avaliable, i can't find it anywhere on the internet

[Fizick]

here is content of CelStabilise.avs file downloaded 26.09.2005:

function Stabilise(clip o, clip metric)
{
Selectall(o)
SelectFirstOfRun()
scene = Breakon(last)

FindIntegerPan (o, metric)
UnmaskedRefinePan(o, metric, last)
motion = last.ScaleMotion(-1.0)
totalmotion = AccumulateMotion(scene, motion)

# Stabilise
ApplyUnmaskedPowerTranslate(o, totalmotion, o)#.Expand(900, 640))

return Paper()
}


Please add it to the wiki
http://avisynth.org/mediawiki/External_filters#Stabilization

[lansing]

thanks for the repost.

This filter is very slow and I got some problem with the line Stabilise(capabsolutemetric(15)) , I got something like this with a panning from the sky to ground:

http://img12.imageshack.us/img12/9624/errorgmi.jpg

this sequel happen in the first few seconds of the anime, and eventually the whole video became white. What is wrong?

[pvann]

The links in Post 1 seem to be dead.

Does anybody have a copy of the dll and script?

Thanks
Peter

[Dogway]

I was looking for the dll too. You can find it here (http://avisynth.org/warpenterprises/). Pitty I also get the all white panning like lansing.
Edit:I got it working. Yes, indeed you have to avoid pans, tilts, fade ins and stuff like that. But oh my friend, it works wonders! I think you can use it with MT multithreading and if you need to trim, do it before the stabilise(). Trick must be to work in chunks. Here is a test video I just made:

1Mb mp4
http://www.sendspace.com/file/rpjuv5

This filter does need an update, its pure magic.

[18fps]

From what I understand, the concept of this filter is very interesting: to search the supposedly fixed background and rock-steady stabilize it: since I frequently work with silent movies where pans or tilts are rare, I think I can well use this filter.

I didn't found a link to download. Could someone share it?

[Jenyok]

Dogway

Old links are to DLL files...

http://avisynth.org/warpenterprises/files/celbackground_20050923.zip
http://avisynth.org/warpenterprises/files/celforeground_25_dll_20050909.zip

Where do download new DLL, like below, I don't know and finding process does not have results...

loadplugin("CelBackground_22Oct05.dll")
loadplugin("Cel_Background_06Jan06.dll")

Guys,
Where I can download CelBackground_22Oct05.dll and Cel_Background_06Jan06.dll ?

[kypec]

Dogway, would you mind to post your complete script for CelStabilise including the parameters used?
I'm trying to get it work but it's incredibly slow and all I get is a picture insanely shifted with 3 quarters of the frame clean white :(

[Dogway]

Its not THAT slow if you do it correctly. Actually its pretty fast, but it slows down when the output is gonna be wrong ->picture insanely shifted with 3 quarters of the frame clean white :(

Keep in mind that in order for it to work you must always have a fixed element on screen to stack to. Otherwise screen is gonna shift to white in some way. I used the script as it is, post a clip of your video if you keep having the problem.

[Gerry62]

Sorry for dredging this up again, But i'd like to fool about with the CelBackground_22Oct05.dll and Cel_Background_06Jan06.dll files, Ive just spent a couple of weeks trying to sort out some film that all the current stabilization techniques don't quite work.
I'm working on material like 18fps is, so no pans, Anyone help?

[videoFred]

Ive just spent a couple of weeks trying to sort out some film that all the current stabilization techniques don't quite work.
I'm working on material like 18fps is, so no pans, Anyone help?

Please upload an example clip. I have pretty good stabilizing results with my script, based on DepanStabilize(), but used in a special way.

PS: 18fps? No pans? Panning has nothing to do with frame rate.
Panning means moving the camera horizontal to film a landscape for example.

Fred.

[Gerry62]

Please upload an example clip. I have pretty good stabilizing results with my script, based on DepanStabilize(), but used in a special way.

PS: 18fps? No pans? Panning has nothing to do with frame rate.
Panning means moving the camera horizontal to film a landscape for example.

Fred.

I was referring to the user with the name 18fps! not the framerate!

Here's a clip showing a fairly decent example of the 'gate weavy' Kine, Now, Ive tried every 'stab' type code or filter That's out there, What I want to do is stabilize the material first, and then clean it, as when Ive played about with cleaning first then stabilizing, sometimes the 'cleaning' creates matrix like artifacts of the 'flats' of the sets (I don't like to clean till everyone looks 'plastic' so the less you use the more of these mpg2 grid(ish) type patters Emerge).

Now it looks fine with this 'effect', but as soon as you then stabilize the cleaned vision these 'artifacts' become all too evident, as of course they jump about the place.

I have tried your script Fred! (and is the best of the bunch) and Ive broken the clip into 'shots' and used your stab on each 'shot' selecting a nice flat section in the background that's 'still' and then playing with the est_cont, and for the main part it works well, and was even thinking of using your stab script Pre and Post cleaning, as the grain and crud effect the stab ever so slightly.

Now as you will see the program material does indeed have the odd pan every now and then, but for those shots I was going to leave em pretty much un-stabilized (but still cleaned), even if its a few frames every now and then. I mean the program itself broke up to 271 separate files, a few more would not kill me!
(I am recovering from sickness so have all the time in the world!)

Cruddy 1963 16mm Telerecording (raw)

http://www.megaupload.com/?d=W83CLSX3

any advice would be very welcome, am also looking for a better way to clean it without the movement getting all grainy, It's one bug bear so far, the cleaning script is pretty much MVDegrainMulti, much like your version, Works well, but anything moving, not so well!

[videoFred]

I was referring to the user with the name 18fps! not the framerate!

Ah! I see now :p


I have tried your script Fred!

I have changed the stabilizing part after many, many tests on all kinds of footage. I will see what I can do on your clip but I have no time right now, tomorrow I will post the result.

Fred.

[jmac698]

www.avisynth.org/mediawiki/CelStabilize

[videoFred]

Cruddy 1963 16mm Telerecording (raw)

http://www.megaupload.com/?d=W83CLSX3

any advice would be very welcome, am also looking for a better way to clean it without the movement getting all grainy, It's one bug bear so far, the cleaning script is pretty much MVDegrainMulti, much like your version, Works well, but anything moving, not so well!

It looks like this clip is post-processed already. (deinterlacing? degraining? ) The objects are moving in a strange way. The bottles for example. The general shaking can be removed, the moving objects is another matter. This can not be fixed.

Fred.

[jim_henry]

I used CelStabilize to stabilize a 10 minute animated "intermission clock". It was very static with an elf eating snacks in about 25% of the frame and advancing a clock hand every minute. I got the effect of the frame moving up and left from time to time. It seems like these jumps occurred where there was a sudden shift in the average luma. I found that I was able to eliminate the jumps by changing the value of i passed for CapAbsoluteMetric(i) from 15, given in the example, to 63. My guess is that is the parameter that "lets you specify the "penalty" for each luma difference."

It is a shame that CelStabilize and the supporting CelBackground functions weren't better developed and documented. It does provide some useful capabilities.