General Convolution

[Zarxrax]

Is it possible to create a spatial smoothing effect with general convolution?

[Richard Berg]

If by "smooth" you mean blur, then yes. (The filter gurus here usually mean something slightly different when they talk about smoothers). The main advantage of GeneralConvolution is that you can set up blurs (or sharpens) that work in one direction at a time.

[LigH]

Could you please update the examples of GeneralConvolution to the new syntax? I wonder why the examples from the official documentation don't work with AviSynth 2.5.5, because you added a "divisor" parameter, but didn't document how and where to use it - so the result is:
Avisynth open failure:
Script error: Invalid arguments to function "GeneralConvolution"
...

[WarpEnterprises]

it's there already in convolution.htm.
where exactly is the old version?
There may be old versions left on the HD since the docs directory structure has changed.

[LigH]

http://www.avisynth.org/index.php?page=GeneralConvolution

A parameter "divisor" is mentioned in the table, but the examples are still the old ones.

And, well... of course, the german version of the documentation which is extracted by the installer. There, the english doc is up to date.

[Wilbert]

Is it better now :)

The english doc which comes with the installation should also be corrected.

[LigH]

Beautiful!

Now I just have to find out why the following examples (emboss) produce different results: The first works fine, the second makes bad colors (looks like clipping):

GeneralConvolution(4,128,"-2 -1 0 -1 0 1 0 1 2")

GeneralConvolution(16,128,"-8 -4 0 -4 0 4 0 4 8")

Is the second one not just equal to the first one, "extended" by 4 (or how is it called in english, when numerator and denominator are multiplied by the same factor)?

[Wilbert]

Is the second one not just equal to the first one, "extended" by 4 (or how is it called in english, when numerator and denominator are multiplied by the same factor)?

I don't know how that is called :) But, the clips should be equal. I will look at it this weekend.

[IanB]

Originally posted by LigH
Beautiful!

Now I just have to find out why the following examples (emboss) produce different results: The first works fine, the second makes bad colors (looks like clipping):

GeneralConvolution(4,128,"-2 -1 0 -1 0 1 0 1 2")

GeneralConvolution(16,128,"-8 -4 0 -4 0 4 0 4 8")

Is the second one not just equal to the first one, "extended" by 4 (or how is it called in english, when numerator and denominator are multiplied by the same factor)?

Perhaps "scaled" or "normalized".

Any now to your problem :-

GeneralConvolution annoyingly attempts to auto scale the results by summing the matrix coefficients to get a scale factor, if the sum is zero(0) (the case here) then one(1) is used instead. From history to be efficient and avoid doing lots of divisions, a reciprocal, scaled by 0x100000, is used then a ">> 20" is performed to undo scaling.

The relevant code :-int iR, iCountT, iCountDiv;
...
iCountDiv = 0x100000 / (iCountT == 0 ? 1 : iCountT);
...
iR = (iR * iCountDiv) >> 20 / divisor + nBias;Of course recent addition of the divisor option has been a little slack and we have inherited 3 slow division per pixel. If correctly integrated into the iCountDiv code and this problem would not be here.

Worst case analysis values of iR for your 2 statements are +/-1020(0x3FC) and +/-4080(0xFF0) which have intermediate values of +/-1069547520(0x3FC00000) and +/-4278190080(0xFF000000) respectively.

But oops +4278190080 in signed 32bits is -16777216 which after the ">> 20" is -16 divided by your 16 divisor plus your 128 bias gives 127.

Incidently the first statement has clipping problems anyway (+/-1069547520 >> 20) / 4 + 128 = +383/-127 which will be clamped at 255/0. Unless you want the clamping effect the divisor should have been 8.

I'll put up a modified source in the sourceforge bug tracker soon.

IanB

P.S. In the process of analysing this I found another bug, the matrix string is getting poisoned during parsing, this makes using GeneralConvolution in Animate or user functions a little tricky, a work-around is to use string concatination to force the parser to allocate unique memory for all occurance of the matrix string.

i.e. GeneralConvolution(4,128,"-2 " + "-1 0 -1 0 1 0 1 2")

[Wilbert]

I'll put up a modified source in the sourceforge bug tracker soon.
Ok, thanks!

P.S. In the process of analysing this I found another bug, the matrix string is getting poisoned during parsing,
I'm just this is clear for programmers, but could you elaborate?

[IanB]

Originally posted by Wilbert
Ok, thanks!

I'm just this is clear for programmers, but could you elaborate?

If you try to use GeneralConvolution in Animate you get "...sez: matrix to small" errors for no reason.BlankClip(20)
Animate(0, 19, "GeneralConvolution", 1, 0, "1 2 3 4 5 6 7 8 9", 20, 0, "1 2 3 4 5 6 7 8 9")orFunction xyzzy(Clip clip) {
GeneralConvolution(clip, 1, 0, "1 2 3 4 5 6 7 8 9")
}

BlankClip(20)
xyzzy()
xyzzy()Problem is the matrix string internal storage gets scribbled on between the 1st and 2nd function call geting parsed.

See http://sourceforge.net/tracker/index.php?func=detail&aid=978383&group_id=57023&atid=482673 for source fix.

IanB