Average plug-in : weighted average of any number of clips; fast (500 FPS on 2 clips)

[mg262]

Average, 21 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/blend/Average_21Oct05.dll)

Usage is

Average(clip, weight, clip, weight, clip, weight, ... ) #YV12 clips

For example, a simple average of clips alpha and beta is given by

Average(alpha, 0.5, beta, 0.5)

And for three clips

Average(alpha, 1.0/3, beta, 1.0/3, gamma, 1.0/3)

In action:
http://people.pwf.cam.ac.uk/mg262/posts/blend/average_example.jpg

It uses (unoptimised) iSSE if present and otherwise falls back to C. On 720x576 source, the iSSE implementation runs at 500 FPS (2 clips), 270 FPS (4 clips) and 190 FPS (6 clips) on my Pentium 4/2400. (Measured, of course, by AVSTimer (http://www.avstimer.de.tf/).)

It is probably best to make sure the weights sum to one; the plug-in will work fine if they don't but the average may not make sense. Audio is always taken from the first clip.

This can be used as a temporal average by using arguments like:

Alpha.trim(1, 0) #next
Alpha
Alpha.trim(0, -1) + Alpha #previous

With a little script work, this can be combined with the scene detection built into Cel Background to avoid averaging over scene boundaries; I'll explain how if one of the scripters out there is interested.

This release was brought to you by the Association for the Prevention of Cruelty to TemporalSoften.

Edit: iSSE added -- read next posts.

[Didée]

Hah!

Idea and realisation: :) :) :)

No speed rush for non-64bit Athlons: :( :( :(


This can be used as a temporal average ...

[...]

With a little script work, this can be combined [...] to avoid averaging over scene boundaries
... and probably some other tasks ;)

[mg262]

Why, Didée, you know I can't bear to see you unhappy ;)

Since it was unoptimised assembly it was pretty straightforward to change to iSSE -- and for reasons I don't follow it is also four times faster. (Not that I'm complaining! I will analyse it when I have more time.) I assume Athlons can cope with this? In any case, please let me know if it doesn't seem to be working. Also, change for MMX would be pretty straightforward if needed.

Now a request for you, Didée: please consider making a filter wish list, for anything you want but especially for simple operations like this one...


Edit: the reason it was four times faster was as follows:

DVDepisode("1")#.crop(8, 2, -12, -2)average(last,1.0/6,Trim(1,0),1.0/6,Trim(2,0),1.0/6,Trim(3,0),1.0/6,Trim(4,0),1.0/6,Trim(5,0),1.0/6)
reduceby2()
o= last

a=o.Trim(3650, 3650+100)
b= o.Trim(4392, 4527) #Bridge
c=o.Trim(700, 800) #
s4(a.Subtitle("A"),b.Subtitle("B"),c.Subtitle("C"),Average(a, 0.2,b,0.4,c,0.4).Subtitle("Average(A,0.2, B,0.4, C,0.4)"))
#reduceby2()

#Average(a, 0.2,b,0.4,c,0.4)
#
o
AvsTimer(frames=1000, name="mpeg2dec", quiet=true)
#average(last,1.0/6,Trim(1,0),1.0/6,Trim(2,0),1.0/6,Trim(3,0),1.0/6,Trim(4,0),1.0/6,Trim(5,0),1.0/6)
#average(last,0.5,Trim(1,0), 0.5)
average(last,1.0/4,Trim(1,0),1.0/4,Trim(2,0),1.0/4,Trim(3,0),1.0/4)
AvsTimer(frames=1000, name="average", difference=1)


:stupid:

[Didée]

Thank you. Oh, and a nice method to squeeze 400% speed increase, indeed. Could work for other filters, too. :)

Okay, I consider. Problem is, when you shall think of something, then usually nothings comes into mind ...

BTW, above link starts with "http://http://..." - that's probably not correct.

[mg262]

Oops -- link fixed. Thank you.

Don't worry about compiling something right away! Just as/if things occur to you while you are playing with scripts.

Edit: the " test for relevant instruction set" may still reporting an error as "sse2 not present"... but it's just that I forgot to change the other message; it really is testing for integer sse. Will fix in next version.

[Mug Funky]

hey, cool. just 2 days ago i was trying to average 3 clips together to get one uber-clip... i was using ye olde interleave(1,2,3).temporalsoften(1,255,255).selectevery(3,1) to do it.

i like your method better :)

[mg262]

:)

average 3 clips together to get one uber-clip... i was using ye olde interleave(1,2,3).temporalsoften(1,255,255).selectevery(3,1) to do it.Doesn't the scene detection mess that up?

@Mug Funky + @Didée (plus anyone else who feels like chipping in),
Do you have preferred methods of scene detection?

[Mug Funky]

not really... i set that to 255 as well. so long as you're selectevery-ing the correct frame, there's no bleeding between frames.

as far as preferred scene detection methods go... haven't really needed it, as filters that require it (MVtools stuff) tend to be too slow to use practically anyway.

[mg262]

Right. Suppose you're taking a temporal average, radius two, like this:

A|B C D E

with a scene break between A and B.

Is it all right to take the five-point average

B B C D E

Or do you think it's necessary to take the four-point average

B C D E

? (The former is straightforward with the scene detection I have, I guess it's straightforward with ConditionalFilter as well, whereas the latter is possible but more fiddly and harder to generalised higher numbers.)

[MOmonster]

I used your Average filter that way:
Average(source.duplicateframe(0), -0.5, source, 1.0, source.trim(1,0), 1.0, source.trim(2,0), -0.5)Now, with the avisynth 2.56 final it runs stable on my Pc, but I get a green stripe on the right side, see attachment. Do I something wrong?

I made some more test and I have still stability problems with the new avisynth version. I have a athlon XP 2000+ and a simple 256mp DDR.
I get this message when I use it in the function (have also problems with stand alone):
Evaluate: System exception - Access Violation

[mg262]

Do I something wrong?
No, I did! When I switched from sse2 to isse, the code changed to process four pixels at a time rather than eight... but there was a stray 8 sitting in the code that I didn't catch. (As it happens, that would have caused an access violation under some circumstances too...) As usual, problem caused by copy/paste... thank you for spotting it. Fixed version above... please let me know what the results of using it are.

[MOmonster]

@mg262
Thanks, now everything is running fine.
Really nice fast filter.

[foxyshadis]

Hey, I was wondering where all those Access Violations/crashes were coming from, I thought they were something I was doing wrong. I'll pick up this new version right away and start testing!

[mg262]

Sorry about that!

[AVIL]

Hi,

I've used this filter (flawessly, by the way) and i love the possibilty of merge any number of clips i need. Moreover, i think it could be extended, to another colourspaces, and another modes of merge (calculation of median, instead of avegerage). Maybe will be necessary to use another name for the hipotetic new filters, but the central idea will be the same.

[Mug Funky]

median is a great idea! that'd sort out stuff like overlaying multiple captures. would also be good for mocomped interleaving and stuff.

[mg262]

More colour spaces should be quick. Median is possible but will be slow compared to the (assembly optimised) average -- unless you only want it for three clips?

[mg262]

Average, 24 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/blend/Average_24Oct05.dll)

Should cope with any colourspace.

[Mug Funky]

hmm. good point about the assembly and median stuff. i guess for median filtering weights would behave counter-intuitively, and it would be better handled with interleave(1,2,3).clense().selectevery(3,1)

[edit]

btw, thanks for the quick update :)

[mg262]

If it's for a small, fixed, number, I think I might have a way of doing it reasonably fast in assembly. But I don't understand how you would even put weights in for median filtering? Unless you wanted to take the weighted average of minimum, median, and maximum (for 3 clips).

[Mug Funky]

that was my point... median filters by their nature can't really take weights (or else they cease being medians...)

[mg262]

Got it! Would median of 3 be useful?

[AVIL]

Hi,

For three clips there are the clense alternative (even with worse performance). I think that to use variable number of clips is a must for a proyect. In my case, I use two, trhee or four captures of the same VHS, plus motion compensated generated clips. The median value is the reference to choose the best candidate pixels.

The weight isn't essential. Indeed is not applicable before the median is calculate but may be usable por selective modification of the output. At the moment I don't know a useful application for it but maybe exists.

[Mug Funky]

@ AVIL: do you have a TBC? 3 caps of the same tape might still contain timebase errors after averaging...

[AVIL]

@Mug Funky

Yes, often I capture from a camcorder with TBC. If I capture without TBC (from my cheapo VCR) I must visually align all the clips. I use virtualdub's plugin FrameTweaker in my avisynth scripts to scroll the frame 1 line up or down for the frames misaligned. Fortunately my VCR don't misaling too much frames.

[Seed]

Hi,

For three clips there are the clense alternative (even with worse performance). I think that to use variable number of clips is a must for a proyect. In my case, I use two, trhee or four captures of the same VHS, plus motion compensated generated clips. The median value is the reference to choose the best candidate pixels.

Not exactly what you describe here, but something similiar, E-Male's "Two out of Three" (do a search for "toot") filter might be useful in eliminating the worst out of three sources. Perhaps you are aware of it already, anyway just a suggestion.

[AVIL]

@Seed

Thanks for the tip, but I need operate on more than three clips. Also, I think is preferable general purpose plugins to specific ones.

[mg262]

AVIL,

What would you want done in the case of an even number of clips? (Mean) average of the middle two? Or would it be sufficient to just have a filter that required an odd number of clips?

[AVIL]

@mg262

Averaging the two middle points, I think is the best response. We can see the even case like an odd case with the central value unknown. Probablity theory tell us that on unknown cases (with no previous knowledge of real probabilities), the average is a good choice.

Thanks.

[redfordxx]

Hi
I have a question. Would it be possible to remake this Average function to some WeightedAverage fucnction? Like
WeightedAverage(clip, weightclip, weight, clip, weightclip, weight, clip, weightclip, weight, ... )
so the function will be something like
Average(mt_lutxy(clip,weightclip,"x y * 255 /"), weight, mt_lutxy(clip,weightclip,"x y * 255 /"), weight,mt_lutxy(clip,weightclip,"x y * 255 /"), weight, ... ),
only faster and much PRECISER.


I don't know how difficult it would be to change the existing Average code and whether somebody else finds it useful but I would appreciate it.

Please refer to this (http://forum.doom9.org/showthread.php?t=111312) thread.

BTW:Now I am weighting clips with MaskTools followed by Average and I am facing big rounding errors problems with increasing number of clips.

[redfordxx]

Hi,

I have almost working solution:
cFloat1=mt_lutxy(cMovie1,cWeight1,"x y * 255 %",y=3,u=3,v=3)
cInt1=mt_lutxy(cMovie1,cWeight1,"x y * x y * 255 % - 255 /",y=3,u=3,v=3)
cFloat2=mt_lutxy(cMovie2,cWeight2,"x y * 255 %",y=3,u=3,v=3)
cInt2=mt_lutxy(cMovie2,cWeight2,"x y * x y * 255 % - 255 /",y=3,u=3,v=3)
.
.
.
Average(cInt1, weight1, cFloat1, weight1/255, cInt2, weight2, cFloat2, weight2/255,...)
But, as soon as the weight passed to Average is<1/511 (it means weight?/255<1/511 ... weight?<=1/2 in my example), there is another rounding error...

Is that Avisynth's fault or Average's fault or I'm missing something?

[Zarxrax]

Does anyone know where I can find this plugin? I can't seem to find a working download anywhere.

[krieger2005]

All available Plugins are here: http://forum.doom9.org/showthread.php?t=118430

but i could not find this plugin so i uploaded the plugin here:
http://rapidshare.com/files/9865411/Average_21Oct05.dll.html

[redfordxx]

Hi may I have also one wish?
Is the source code available? Can I have it?

[Wilbert]

Hi may I have also one wish?
Is the source code available? Can I have it?
No, i didn't ask for that one. I will ask him about it, but it can take a long while before he responds ...

[Blue_MiSfit]

I'm also looking for Average.dll, the "mg262's plugins with source" doesn't have it...

Does anyone know where it is?

~Misfit

[Leak]

I'm also looking for Average.dll, the "mg262's plugins with source" doesn't have it...

Does anyone know where it is?
Take a very close look at krieger2005's post above, then click on his second link... ;)

(Yeah, automatic abbreviation through the forum hasn't quite helped that link... :D)

np: Star You Star Me - Sweet Things (Simple Things)

[Blue_MiSfit]

Thanks Leak, I dont know what happened. I need to learn to use these wierd things called eyeballs.. :)

[redfordxx]

One year later:
Is the source somewhere available?

[Wilbert]

I've sent him an e-mail asking about it. I'm not sure whether the e-mail address that i have is still valid though. We will see ...

[Wilbert]

He sent me the source of his plugin (the plugin itself is included):

http://www.wilbertdijkhof.com/mg262/Average_v11.zip

I didn't test it myself, so, test it as much as your can.

He also sent me some other things, but i will post about that in an other thread.

[redfordxx]

Thanx

[redfordxx]

The old 2005 version (73kB dll) works on my AthlonXP.

The new one (102kB dll) crashes.

In debug mode it writes:
Unhandled exception at 0x01e97151 (Average.dll) in VirtualDub.exe: 0xC0000005: Access violation reading location 0x00000000.

It crashes somewhere after leaving averageplane_P3
called by WeightedAverage
in averageplane_P3(PLANAR_Y, env);
when coming back to WeightedAverage...it crashed somewhere around the bold spot:...end of averageplane_P3
99: }
100: }
01E9712F push edx
01E97130 mov ecx,ebp
01E97132 push eax
01E97133 lea edx,[ (1E97158h)]
01E97139 call @ILT+2455(@_RTC_CheckStackVars@8) (1E7C99Ch)
01E9713E pop eax
01E9713F pop edx
01E97140 pop edi
01E97141 pop esi
01E97142 mov ecx,dword ptr [ebp-4]
01E97145 xor ecx,ebp
01E97147 call @ILT+640(@__security_check_cookie@4) (1E7C285h)
01E9714C mov esp,ebp
01E9714E pop ebp
01E9714F mov esp,ebx
01E97151 pop ebx
01E97152 ret 8
I have four warnings like this, if it helps:
warning C4731: 'WeightedAverage::averageplane_P4' : frame pointer register 'ebx' modified by inline assembly code
When I comment the P3 version, the C version goes well.

[redfordxx]

...pushad and popad helped

[Fizick]

well, Wilbert asked me to compile it, but he forget to ask me to test it :)
i confirm the bug. I tested it right now only :)

push ebx
...
pop ebx
will be enough

I also provided him the version compiled by VC7.1 (VC 2003 tollkit),
it works fine with same code (without the patch).


http://forum.doom9.org/showthread.php?t=100374

[redfordxx]

I made function AverageM(clip, maskclip, clip, maskclip....) based on Average.
However I have one problem (it is similar as in MaskedMerge in MaskTools): I cannot deliver full range of mask values [0,256] but only [0,255] in maskclips. I have folowing options and I don't know what to do:
1) divide by 256 at the end...cons:cannot have full scale in one maskclip
2) divide by 255 at the end...cons:cannot do fifty-fifty for example (averaging pixels from two clips weighted both by 128 results in x=(a*128+b*128)/255=(a+b)/1.99 instead of (a+b)/2
3) make the weightscale [0,128]... cons: reduced precision

What is your opinion on this?

[redfordxx]

BTW:despite there are only constants used in Average, AverageM is by 10%-20% faster;-)

[nonsens112]

redfordxx

can you or anybody else post a link to this wonderful AverageM plugin? :)
I really need such thing.

[nonsens112]

can anyone at least post a link to Average's source code?
link above is broken.

[thetoof]

Is this what you're looking for?