Average plug-in : weighted average of any number of clips; fast (500 FPS on 2 clips)

[mg262]

Average, 21 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/blend/Average_21Oct05.dll)

Usage is

Average(clip, weight, clip, weight, clip, weight, ... ) #YV12 clips

For example, a simple average of clips alpha and beta is given by

Average(alpha, 0.5, beta, 0.5)

And for three clips

Average(alpha, 1.0/3, beta, 1.0/3, gamma, 1.0/3)

In action:
http://people.pwf.cam.ac.uk/mg262/posts/blend/average_example.jpg

It uses (unoptimised) iSSE if present and otherwise falls back to C. On 720x576 source, the iSSE implementation runs at 500 FPS (2 clips), 270 FPS (4 clips) and 190 FPS (6 clips) on my Pentium 4/2400. (Measured, of course, by AVSTimer (http://www.avstimer.de.tf/).)

It is probably best to make sure the weights sum to one; the plug-in will work fine if they don't but the average may not make sense. Audio is always taken from the first clip.

This can be used as a temporal average by using arguments like:

Alpha.trim(1, 0) #next
Alpha
Alpha.trim(0, -1) + Alpha #previous

With a little script work, this can be combined with the scene detection built into Cel Background to avoid averaging over scene boundaries; I'll explain how if one of the scripters out there is interested.

This release was brought to you by the Association for the Prevention of Cruelty to TemporalSoften.

Edit: iSSE added -- read next posts.

[Didée]

Hah!

Idea and realisation: :) :) :)

No speed rush for non-64bit Athlons: :( :( :(


This can be used as a temporal average ...

[...]

With a little script work, this can be combined [...] to avoid averaging over scene boundaries
... and probably some other tasks ;)

[mg262]

Why, Didée, you know I can't bear to see you unhappy ;)

Since it was unoptimised assembly it was pretty straightforward to change to iSSE -- and for reasons I don't follow it is also four times faster. (Not that I'm complaining! I will analyse it when I have more time.) I assume Athlons can cope with this? In any case, please let me know if it doesn't seem to be working. Also, change for MMX would be pretty straightforward if needed.

Now a request for you, Didée: please consider making a filter wish list, for anything you want but especially for simple operations like this one...


Edit: the reason it was four times faster was as follows:

DVDepisode("1")#.crop(8, 2, -12, -2)average(last,1.0/6,Trim(1,0),1.0/6,Trim(2,0),1.0/6,Trim(3,0),1.0/6,Trim(4,0),1.0/6,Trim(5,0),1.0/6)
reduceby2()
o= last

a=o.Trim(3650, 3650+100)
b= o.Trim(4392, 4527) #Bridge
c=o.Trim(700, 800) #
s4(a.Subtitle("A"),b.Subtitle("B"),c.Subtitle("C"),Average(a, 0.2,b,0.4,c,0.4).Subtitle("Average(A,0.2, B,0.4, C,0.4)"))
#reduceby2()

#Average(a, 0.2,b,0.4,c,0.4)
#
o
AvsTimer(frames=1000, name="mpeg2dec", quiet=true)
#average(last,1.0/6,Trim(1,0),1.0/6,Trim(2,0),1.0/6,Trim(3,0),1.0/6,Trim(4,0),1.0/6,Trim(5,0),1.0/6)
#average(last,0.5,Trim(1,0), 0.5)
average(last,1.0/4,Trim(1,0),1.0/4,Trim(2,0),1.0/4,Trim(3,0),1.0/4)
AvsTimer(frames=1000, name="average", difference=1)


:stupid:

[Didée]

Thank you. Oh, and a nice method to squeeze 400% speed increase, indeed. Could work for other filters, too. :)

Okay, I consider. Problem is, when you shall think of something, then usually nothings comes into mind ...

BTW, above link starts with "http://http://..." - that's probably not correct.

[mg262]

Oops -- link fixed. Thank you.

Don't worry about compiling something right away! Just as/if things occur to you while you are playing with scripts.

Edit: the " test for relevant instruction set" may still reporting an error as "sse2 not present"... but it's just that I forgot to change the other message; it really is testing for integer sse. Will fix in next version.

[Mug Funky]

hey, cool. just 2 days ago i was trying to average 3 clips together to get one uber-clip... i was using ye olde interleave(1,2,3).temporalsoften(1,255,255).selectevery(3,1) to do it.

i like your method better :)

[mg262]

:)

average 3 clips together to get one uber-clip... i was using ye olde interleave(1,2,3).temporalsoften(1,255,255).selectevery(3,1) to do it.Doesn't the scene detection mess that up?

@Mug Funky + @Didée (plus anyone else who feels like chipping in),
Do you have preferred methods of scene detection?

[Mug Funky]

not really... i set that to 255 as well. so long as you're selectevery-ing the correct frame, there's no bleeding between frames.

as far as preferred scene detection methods go... haven't really needed it, as filters that require it (MVtools stuff) tend to be too slow to use practically anyway.

[mg262]

Right. Suppose you're taking a temporal average, radius two, like this:

A|B C D E

with a scene break between A and B.

Is it all right to take the five-point average

B B C D E

Or do you think it's necessary to take the four-point average

B C D E

? (The former is straightforward with the scene detection I have, I guess it's straightforward with ConditionalFilter as well, whereas the latter is possible but more fiddly and harder to generalised higher numbers.)

[MOmonster]

I used your Average filter that way:
Average(source.duplicateframe(0), -0.5, source, 1.0, source.trim(1,0), 1.0, source.trim(2,0), -0.5)Now, with the avisynth 2.56 final it runs stable on my Pc, but I get a green stripe on the right side, see attachment. Do I something wrong?

I made some more test and I have still stability problems with the new avisynth version. I have a athlon XP 2000+ and a simple 256mp DDR.
I get this message when I use it in the function (have also problems with stand alone):
Evaluate: System exception - Access Violation

[mg262]

Do I something wrong?
No, I did! When I switched from sse2 to isse, the code changed to process four pixels at a time rather than eight... but there was a stray 8 sitting in the code that I didn't catch. (As it happens, that would have caused an access violation under some circumstances too...) As usual, problem caused by copy/paste... thank you for spotting it. Fixed version above... please let me know what the results of using it are.

[MOmonster]

@mg262
Thanks, now everything is running fine.
Really nice fast filter.

[foxyshadis]

Hey, I was wondering where all those Access Violations/crashes were coming from, I thought they were something I was doing wrong. I'll pick up this new version right away and start testing!

[mg262]

Sorry about that!

[AVIL]

Hi,

I've used this filter (flawessly, by the way) and i love the possibilty of merge any number of clips i need. Moreover, i think it could be extended, to another colourspaces, and another modes of merge (calculation of median, instead of avegerage). Maybe will be necessary to use another name for the hipotetic new filters, but the central idea will be the same.

[Mug Funky]

median is a great idea! that'd sort out stuff like overlaying multiple captures. would also be good for mocomped interleaving and stuff.

[mg262]

More colour spaces should be quick. Median is possible but will be slow compared to the (assembly optimised) average -- unless you only want it for three clips?

[mg262]

Average, 24 October 2005 (http://people.pwf.cam.ac.uk/mg262/posts/blend/Average_24Oct05.dll)

Should cope with any colourspace.

[Mug Funky]

hmm. good point about the assembly and median stuff. i guess for median filtering weights would behave counter-intuitively, and it would be better handled with interleave(1,2,3).clense().selectevery(3,1)

[edit]

btw, thanks for the quick update :)

[mg262]

If it's for a small, fixed, number, I think I might have a way of doing it reasonably fast in assembly. But I don't understand how you would even put weights in for median filtering? Unless you wanted to take the weighted average of minimum, median, and maximum (for 3 clips).

[Mug Funky]

that was my point... median filters by their nature can't really take weights (or else they cease being medians...)

[mg262]

Got it! Would median of 3 be useful?

[AVIL]

Hi,

For three clips there are the clense alternative (even with worse performance). I think that to use variable number of clips is a must for a proyect. In my case, I use two, trhee or four captures of the same VHS, plus motion compensated generated clips. The median value is the reference to choose the best candidate pixels.

The weight isn't essential. Indeed is not applicable before the median is calculate but may be usable por selective modification of the output. At the moment I don't know a useful application for it but maybe exists.

[Mug Funky]

@ AVIL: do you have a TBC? 3 caps of the same tape might still contain timebase errors after averaging...

[AVIL]

@Mug Funky

Yes, often I capture from a camcorder with TBC. If I capture without TBC (from my cheapo VCR) I must visually align all the clips. I use virtualdub's plugin FrameTweaker in my avisynth scripts to scroll the frame 1 line up or down for the frames misaligned. Fortunately my VCR don't misaling too much frames.

[Seed]

Hi,

For three clips there are the clense alternative (even with worse performance). I think that to use variable number of clips is a must for a proyect. In my case, I use two, trhee or four captures of the same VHS, plus motion compensated generated clips. The median value is the reference to choose the best candidate pixels.

Not exactly what you describe here, but something similiar, E-Male's "Two out of Three" (do a search for "toot") filter might be useful in eliminating the worst out of three sources. Perhaps you are aware of it already, anyway just a suggestion.

[AVIL]

@Seed

Thanks for the tip, but I need operate on more than three clips. Also, I think is preferable general purpose plugins to specific ones.

[mg262]

AVIL,

What would you want done in the case of an even number of clips? (Mean) average of the middle two? Or would it be sufficient to just have a filter that required an odd number of clips?

[AVIL]

@mg262

Averaging the two middle points, I think is the best response. We can see the even case like an odd case with the central value unknown. Probablity theory tell us that on unknown cases (with no previous knowledge of real probabilities), the average is a good choice.

Thanks.

[redfordxx]

Hi
I have a question. Would it be possible to remake this Average function to some WeightedAverage fucnction? Like
WeightedAverage(clip, weightclip, weight, clip, weightclip, weight, clip, weightclip, weight, ... )
so the function will be something like
Average(mt_lutxy(clip,weightclip,"x y * 255 /"), weight, mt_lutxy(clip,weightclip,"x y * 255 /"), weight,mt_lutxy(clip,weightclip,"x y * 255 /"), weight, ... ),
only faster and much PRECISER.


I don't know how difficult it would be to change the existing Average code and whether somebody else finds it useful but I would appreciate it.

Please refer to this (http://forum.doom9.org/showthread.php?t=111312) thread.

BTW:Now I am weighting clips with MaskTools followed by Average and I am facing big rounding errors problems with increasing number of clips.

[redfordxx]

Hi,

I have almost working solution:
cFloat1=mt_lutxy(cMovie1,cWeight1,"x y * 255 %",y=3,u=3,v=3)
cInt1=mt_lutxy(cMovie1,cWeight1,"x y * x y * 255 % - 255 /",y=3,u=3,v=3)
cFloat2=mt_lutxy(cMovie2,cWeight2,"x y * 255 %",y=3,u=3,v=3)
cInt2=mt_lutxy(cMovie2,cWeight2,"x y * x y * 255 % - 255 /",y=3,u=3,v=3)
.
.
.
Average(cInt1, weight1, cFloat1, weight1/255, cInt2, weight2, cFloat2, weight2/255,...)
But, as soon as the weight passed to Average is<1/511 (it means weight?/255<1/511 ... weight?<=1/2 in my example), there is another rounding error...

Is that Avisynth's fault or Average's fault or I'm missing something?

[Zarxrax]

Does anyone know where I can find this plugin? I can't seem to find a working download anywhere.

[krieger2005]

All available Plugins are here: http://forum.doom9.org/showthread.php?t=118430

but i could not find this plugin so i uploaded the plugin here:
http://rapidshare.com/files/9865411/Average_21Oct05.dll.html

[redfordxx]

Hi may I have also one wish?
Is the source code available? Can I have it?

[Wilbert]

Hi may I have also one wish?
Is the source code available? Can I have it?
No, i didn't ask for that one. I will ask him about it, but it can take a long while before he responds ...

[Blue_MiSfit]

I'm also looking for Average.dll, the "mg262's plugins with source" doesn't have it...

Does anyone know where it is?

~Misfit

[Leak]

I'm also looking for Average.dll, the "mg262's plugins with source" doesn't have it...

Does anyone know where it is?
Take a very close look at krieger2005's post above, then click on his second link... ;)

(Yeah, automatic abbreviation through the forum hasn't quite helped that link... :D)

np: Star You Star Me - Sweet Things (Simple Things)

[Blue_MiSfit]

Thanks Leak, I dont know what happened. I need to learn to use these wierd things called eyeballs.. :)

[redfordxx]

One year later:
Is the source somewhere available?

[Wilbert]

I've sent him an e-mail asking about it. I'm not sure whether the e-mail address that i have is still valid though. We will see ...

[Wilbert]

He sent me the source of his plugin (the plugin itself is included):

http://www.wilbertdijkhof.com/mg262/Average_v11.zip

I didn't test it myself, so, test it as much as your can.

He also sent me some other things, but i will post about that in an other thread.

[redfordxx]

Thanx

[redfordxx]

The old 2005 version (73kB dll) works on my AthlonXP.

The new one (102kB dll) crashes.

In debug mode it writes:
Unhandled exception at 0x01e97151 (Average.dll) in VirtualDub.exe: 0xC0000005: Access violation reading location 0x00000000.

It crashes somewhere after leaving averageplane_P3
called by WeightedAverage
in averageplane_P3(PLANAR_Y, env);
when coming back to WeightedAverage...it crashed somewhere around the bold spot:...end of averageplane_P3
99: }
100: }
01E9712F push edx
01E97130 mov ecx,ebp
01E97132 push eax
01E97133 lea edx,[ (1E97158h)]
01E97139 call @ILT+2455(@_RTC_CheckStackVars@8) (1E7C99Ch)
01E9713E pop eax
01E9713F pop edx
01E97140 pop edi
01E97141 pop esi
01E97142 mov ecx,dword ptr [ebp-4]
01E97145 xor ecx,ebp
01E97147 call @ILT+640(@__security_check_cookie@4) (1E7C285h)
01E9714C mov esp,ebp
01E9714E pop ebp
01E9714F mov esp,ebx
01E97151 pop ebx
01E97152 ret 8
I have four warnings like this, if it helps:
warning C4731: 'WeightedAverage::averageplane_P4' : frame pointer register 'ebx' modified by inline assembly code
When I comment the P3 version, the C version goes well.

[redfordxx]

...pushad and popad helped

[Fizick]

well, Wilbert asked me to compile it, but he forget to ask me to test it :)
i confirm the bug. I tested it right now only :)

push ebx
...
pop ebx
will be enough

I also provided him the version compiled by VC7.1 (VC 2003 tollkit),
it works fine with same code (without the patch).


http://forum.doom9.org/showthread.php?t=100374

[redfordxx]

I made function AverageM(clip, maskclip, clip, maskclip....) based on Average.
However I have one problem (it is similar as in MaskedMerge in MaskTools): I cannot deliver full range of mask values [0,256] but only [0,255] in maskclips. I have folowing options and I don't know what to do:
1) divide by 256 at the end...cons:cannot have full scale in one maskclip
2) divide by 255 at the end...cons:cannot do fifty-fifty for example (averaging pixels from two clips weighted both by 128 results in x=(a*128+b*128)/255=(a+b)/1.99 instead of (a+b)/2
3) make the weightscale [0,128]... cons: reduced precision

What is your opinion on this?

[redfordxx]

BTW:despite there are only constants used in Average, AverageM is by 10%-20% faster;-)

[nonsens112]

redfordxx

can you or anybody else post a link to this wonderful AverageM plugin? :)
I really need such thing.

[nonsens112]

can anyone at least post a link to Average's source code?
link above is broken.

[thetoof]

Is this what you're looking for?

[Nikos]

Here: (http://users.chan.sch.gr/nikos_ch/files/bwgt/Average_24Oct05.rar) Average_24Oct05

[Wilbert]

@Fizick,
I also provided him the version compiled by VC7.1 (VC 2003 tollkit),
Do you still have this? Then i will include it in the package above.

[nonsens112]

thetoof, Nikos

I am looking for an AverageM plugin or source code (in C++) of Average :(

[gzarkadas]

thetoof, Nikos

I am looking for an AverageM plugin or source code (in C++) of Average :(

Look at the bottom of the 1st post of the "mg262's plugins with source (http://forum.doom9.org/showthread.php?t=118430)" thread.

[nonsens112]

gzarkadas

big thanks! :)

[Fizick]

@Fizick,

Do you still have this? Then i will include it in the package above.

Yes, I have.
You missed it? How I can send it to you?

[Fizick]

ttttt

[Wilbert]

Yes, I have.
You missed it?
I was to lazy to copy it to my hard drive :) I named it v1.1 (see above).

[rfmmars]

ttttt

Still waiting for this to be approved

Richard

[pbristow]

Wilbert's links appear to be dead now. :(

Please, oh gods of AVIsynth, do not allow this wonderful, really useful filter to die out. Could it not be incorporated into the AVIsynth core filter set? It really is that useful!

[Wilbert]

Wilbert's links appear to be dead now.

Please, oh gods of AVIsynth, do not allow this wonderful, really useful filter to die out. Could it not be incorporated into the AVIsynth core filter set? It really is that useful!
I updated the link.

[redfordxx]

Hi, something I found on my HDD:
usage:
AverageM(clip1,mask1,clip2,mask2....)

Years ago I learned C and even asm because of avisynth filters and I forgot it again. So I don't know what exactly is the averaging formula...probably equally ugly as other merges in avs...i.e. somewhere, where should be 0 is 1 etc...because mask (0;255) is normalized by 256...I honestly can't remember how I solved...I know how I would solve now...I would added adjusted version of this filter where would beAdjustedMask=Mask+Max(0;Mask*2-256)/256and user would be able to choose the method...but I am no developer anymore...so if someone wants to implement it somewhere, let me know;-)
Cheers
R.

[redfordxx]

Hi, I have reworked the pluging and here it is...some early version.
Find description in the first post of this (http://forum.doom9.org/showthread.php?p=1537311#post1537311) thread. Keep non-development user comments here.

[Bloax]

Hmm, your filters (RAverageW/RMerge) seem to refuse to receive any frames beyond the first one.

That is, they output everything just fine, except that they refuse to process anything but the first frame. (And thus the whole video is frozen at the first frame.)

So unless they're strictly for still images (why would speed be a concern then :P), I'd see this as a bug. Though it may just be my system.
Edit: Tested it on another one, still same issue.

[redfordxx]

OMG...big fat thank you...fixed in version 1.4.3...
I hope there is nothing else...

[travolter]

Im not able to run it.
Im trying to run RedAverage.dll 1.4.3 with this script
msk=RAverageW(last,64,last.RemoveGrain(20,-1),-64,mode=4,u=0,v=0,bias=-196).Mt_Invert().Mt_Deflate()
RMerge(last,Sharpen(0.6),msk,mode=255)

"there is no function named RAverageW..."
The dll is placed in avisynth/plugins directory. Any other dlls are required to run it?. Something into the system32? MT compatible? (setmtmode (2) here).

[redfordxx]

I just tested this script:

w=32
h=16
ColorBars(width=w*20, height=h*20, pixel_type="YV12")
msk=RAverageW(last,64,last.RemoveGrain(20,-1),-64,mode=4,u=0,v=0,bias=-196).Mt_Invert().Mt_Deflate()
RMerge(last,Sharpen(0.6),msk,mode=255)
With C:\Program Files (x86)\AviSynth 2.5\plugins\RedAverage.dll it works and when I remove it it doesnot work, so I don't know.
Here I am on Win7 x64 runnig 32bit Avisynth.

I don't know what else can matter...

btw, how I see 28 downloads of the last version, and only you reported this problem

[redfordxx]

When you got it working, I think invert is redundand, if iirc what it does.
msk=RAverageW(last,64,last.RemoveGrain(20,-1),-64,mode=4,u=0,v=0,bias=-196).Mt_Invert().Mt_Deflate()
msk=RAverageW(last,-64,last.RemoveGrain(20,-1),64,mode=4,u=0,v=0,bias=255+196).Mt_Deflate()
But not tested

[Bloax]

It's some weird AviSynth bug where it refuses to acknowledge the plugin.
I've had it happen some time in the past, really annoying bugga.

As for the inversion, you know where it's from. ;)

[redfordxx]

As for the inversion, you know where it's from. ;)Yep that was before I knew...

And this bugga, does it go away? Or Loadplugin helps?

[travolter]

I was not able to load RedAverage.dll yet. Loadplugin does not help.
Im using XPsp1 with C++2005/2008 Dlls..

I dont have installed .NET installed.
I dont have any problem running other avisynth dlls in this machine but Redaverage.

You guys are using windows7? Maybe this Dll is not compatible with XP? What depencencies/other dlls are required to run Redaverage.dll?

[jmac698]

Again, the original Average(clip clip1, int weight1, ...) is archived at Wilbert's site, http://forum.doom9.org/showthread.php?t=118430

Also Average.dll here
http://www.sendspace.com/file/bjd6k0
average 1.1
2007