Using Avisynth to perform a crude "time average" similar to blended fields

[tlevine]

I previously posted here (http://neuron2.net/board/viewtopic.php?t=1197).

I've been working on a study analyzing average mousing location as a function of keyboard size. I've videotaped 54 people using two different keyboards and was planning on analyzing the video by manually measuring the location of the frames taken at intervals.

During the trials, the camera was placed directly above the desk, so I realized that I could use motion tracking or "time averaging" (That's the term that Cinelerra uses for laying all of the frames in a clip on top of each other. It seems similar to "blended fields.") to do determine the location of mousing automatically. I've been playing with these effects on Cinelerra, but I haven't been able to get them to work too well as the software is somewhat buggy.

I should be able to use some still image processing tool to analyze the average frames to determine the distribution of mouse locations. It won't be perfect, but it'll be much easier than motion tracking, which wouldn't be perfect either.

A crude version of this process seems rather simple. If I have 600000 frames, I divide the saturation of each one by 600000 and overlay all of the frames. The only more complex feature that I think may be useful is a way of emphasizing certain colors, but I want to try a very crude version first.

I think Avisynth would be appropriate for this, but is there a better way to go about it? Does such a script already exist, or does anyone want to write one? If nobody else does, I suppose I'll learn how to use Avisynth and do it myself, but I've never done any work with video before, so I'd appreciate any suggestions anyone has.

I've attached a paper that explains the project in case anyone's interested. I couldn't get all of the equipment that I wanted, so the methods changed a bit from that paper, but it's mostly what I did. And I'll be sure to mention that Avisynth is awesome when I publish this.

[stickboy]

A crude version of this process seems rather simple. If I have 600000 frames, I divide the saturation of each one by 600000 and overlay all of the frames. The only more complex feature that I think may be useful is a way of emphasizing certain colors, but I want to try a very crude version first.I don't think that's correct. Assuming you mean you're adjusting the opacity of each (unless you want grayscale video?), then you really want to weight the opacities as:

1/1, 1/2, 1/3, 1/4, 1/5, ..., 1/600000.

if you want them to end up even.

But you probably don't want to do that either. I don't think AviSynth has that kind of precision for alpha-blending. You could mitigate that by blending small groups, and then blending the small groups together. (But again, you have to be careful to pick the right opacities if you want equal weights.)

If you want to blend a bunch of individual frames together, you could use TemporalSoften with a very large radius and very low thresholds. If, however, you have N video clips and want to blend them into a single clip (such that the zeroth frames of each get blended together, and the first frames of each get blended together, and so on), then you'll probably have to use Overlay. You probably could write a recursive function to do much of the work for you.

[Guest]

I don't think that's correct. Assuming you mean you're adjusting the opacity of each (unless you want grayscale video?), then you really want to weight the opacities as:

1/1, 1/2, 1/3, 1/4, 1/5, ..., 1/600000. He just wants to create a frame that is the average of all the frames. For a given pixel position and 5 frames:

(F1 + F2 + F3 + F4 + F5) / 5

He could have a frame sized array of floats and just accumulate the sums and then divide by the number of frames at the end. Your diminishing weights idea is needed only if he has to maintain a running average. But he doesn't need that.

It would be fairly simple to write a filter to do this. I referred the poster to this web site in the hope that someone might like to make it for him (as I'm pretty busy right now).

[stickboy]

He could have a frame sized array of floats and just accumulate the sums and then divide by the number of frames at the end. Your diminishing weights idea is needed only if he has to maintain a running average. But he doesn't need that.If he's writing his own filter, sure, he doesn't need that. I was assuming he was trying to use existing filters (specifically Overlay).

[foxyshadis]

Actually, clouded has an Average() plugin that does what you describe, very quickly, although you have to calculate the weights manually. (A task for a wrapper function, sounds like.) I don't think avisynth's floats are precise enough for 1/60000th though, even if you could make an expression that long.

[Guest]

You don't need an expression that long!

You just keep a running sum and then divide it at the end by the number of frames. You do know the dynamic range of a float, right?

[foxyshadis]

I was just talking about the Average plugin, a plugin that weighted automatically wouldn't have that problem.

I guess if you used 1./60000 it would work fine, I was just thinking the script parser would run into problems if you tried to use 0.00001666666667 or something along those lines. Maybe not, it's pretty short now that I look at it. >.>

[Guest]

I see. I was thinking just in terms of the algorithm internals.

[2Bdecided]

Idiot suggestion: can't you just use temporal soften, and then select every 600000 to get one output frame?

Not that I can make it work, though I don't understand why not.

Cheers,
David.

[tlevine]

I analyzed the video in a different way, but now I still want to do it this way. I'm not managing to get myself to learn Avisynth. But maybe I'll be able to get myself to start just because I'm thinking about it again now.

I want to ask this anyway: Would anyone be able to do write this for me and release the code freely if I payed him?