LanczosResize vs. SplineResize

[foxyshadis]

Talking about image resamplers with pathological cases is not a useful discussion for practical uses. If you hand a resizer a known pathological case, you get a pathological result, that should be obvious. I assume that's why mp4 guy doesn't care to discuss it.

Nonetheless, it does come up in photography, as digital cameras can often have discrete transitions from smooth to a 1 or 2 pix solid line back to smooth, with no transition period. Naturally, it actually does ring when you attempt to resize, but how else are you going to do it?

It still isn't a terribly useful question, since the result is already known.

[MfA]

The sinc function is sin(x)/x, when used as a filter the only thing which can vary is it's period (ie. x is scaled). That period is uniquely determined by the distance between samples if you want to get a critically sampled image (ie. not aliased, and not unnecessarily low pass filtered). There is no variation possible. You can of course window it however you like ... but the result will never be a sinc filter.

Basically it's really simple, if you apply a linear filter with an impulse response which rings to a signal you get ringing around edges. It's that simple, ringing in -> ringing out. It's an unavoidable property of linear filtering.

Foxyshadis, how else? Non-linear filtering obviously.

[DeathTheSheep]

Some interesting (if irrelevant) filesize results with a CQ encode in AVC:
spline16w: 8162KB (!!!)
spline36w: 7825KB
spline64w: 7938KB

spline16: 7808KB
spline36: 7949KB
spline64: 7952KB

As expected, the sharper the image, the larger the encode at constant quantizer (no B-frames, exhaustive motion search).
However, there is a very strange outlier in this data: spline16w. I re-encoded all clips just to be sure of this phenomenon.

[*.mp4 guy]

The sinc function is sin(x)/x, when used as a filter the only thing which can vary is it's period (ie. x is scaled). That period is uniquely determined by the distance between samples if you want to get a critically sampled image (ie. not aliased, and not unnecessarily low pass filtered). There is no variation possible. You can of course window it however you like ... but the result will never be a sinc filter.

Basically it's really simple, if you apply a linear filter with an impulse response which rings to a signal you get ringing around edges. It's that simple, ringing in -> ringing out. It's an unavoidable property of linear filtering.

Foxyshadis, how else? Non-linear filtering obviously.
Before you can perform the sinc filtering you have to separate the frequencies, which is what I was getting at. The fourier series is commonly used to separate frequencies before performing a sinc function on them, but they are unoptimal and often cause lots of ringing, there are better ways to perform the seperation, the DCT performs mildly better then the fourier series, but is also much slower. The simple sinc function you refer to is not actually the optimal one (it is only optimal when fed perfect imput data), the optimal one is achieved by performing a KLT transform on every datapoint individually, adjusting the size of the transform to match the available data points, then performing a brickwall filter on the frequency coeficients of the KLT transform of each pixel, then performing the inverse KLT. Because the KLT transform has perfect frequency separation there will be no ringing, just a gradual loss of sharpness near the edges as the available transform size decreases. However the KLT transform is an NP class problem, so performing it at full accuracy is for all purposes impossible, instead you have to perform it at sufficient accuracy for your destination colorspace, which will still be insanely slow.

Talking about image resamplers with pathological cases is not a useful discussion for practical uses. If you hand a resizer a known pathological case, you get a pathological result, that should be obvious. I assume that's why mp4 guy doesn't care to discuss it.

Nonetheless, it does come up in photography, as digital cameras can often have discrete transitions from smooth to a 1 or 2 pix solid line back to smooth, with no transition period. Naturally, it actually does ring when you attempt to resize, but how else are you going to do it?

It still isn't a terribly useful question, since the result is already known.
Yep, but I am interested in pathalogical cases, just only when they are actuall images, not specialty data sets.

[MfA]

The fourier series is commonly used to separate frequencies before performing a sinc function on them
I have no idea what this even means ... but as long as it's common you should be able to give me a couple of citations to explain it.
Because the KLT transform has perfect frequency separation
No it doesn't. It has perfect decorrelation, but the basis functions it uses are not even necessarily all periodic and of increasing frequency.

Why are you doing this? Nothing good will come from just making stuff up on the spot.

[DeathTheSheep]

Why is this discussion even happening? Who cares about the sinc-function (or KLT transform) in this thread anyway? Take it to another thread, my friends :).

[*.mp4 guy]

Why is this discussion even happening? Who cares about the sinc-function (or KLT transform) in this thread anyway? Take it to another thread, my friends :).

Ok, fine by me.

Mfa, this isn't personal, don't try to change that.

[Wilbert]

Which is a great explanation of how to join cubic splines for C2 smoothness, but doesn't explain how you chose your curves.

So how did you derive/where did you get the equation?
Ok, here's the derivation from http://mathworld.wolfram.com/CubicSpline.html :
For now i just compute the numbers, but if you want a derivation of equation 18, i will give it to you.

I put (a0,a1,a2,a3) = (1,0,0,0). I'm not sure whether that's correct, but ok ...

1) Equation 18 says A*b = y with

A = [[2, 1, 0, 0], [1, 4, 1, 0], [0, 1, 4, 1], [0, 0, 1, 2]]

b = [[D0],[D1],[D2],[D3]] (with Di = bi, see equation 7)

and

y = 3*[[a1-a0],[a2-a0],[a3-a1],[a3-a2]] = 3*[[-1],[-1],[0],[0]]

Solving for Di in equation 18 gives (with yi = ai, see equation 6):

A^(-1) = ([[26, -7, 2, -1], [-7, 14, -4, 2], [2, -4, 14, -7], [-1, 2, -7, 26]])/45

b = A^(-1) * y = [[-19/15], [-7/15], [2/15], [-1/15]]

Equation 8 says:

c = y2 + C*b

with

C = [[-2, -1, 0, 0], [0, -2, -1, 0], [0, 0, -2, -1], [-1, 0, 0, -2]]

y2 = 3*[[a1-a0],[a2-a1],[a3-a2],[a0-a3]] = 3*[[-1], [0], [0], [1]]

and

c = [[c0],[c1],[c2],[c3]]

Substituting everything gives:

c = [[0], [4/5], [-1/5], [22/5]]

Equation 9 says:

d = - 2/3*y2 + C2*b

with

C2 = [[1, 1, 0, 0], [0, 1, 1, 0], [0, 0, 1, 1], [1, 0, 0, 1]]

and

d = [[d0],[d1],[d2],[d3]]

Substituting everything gives:

d = [[4/15], [-1/3], [1/15], [-10/3]]

Ok, so above we solved for (ai,bi,ci,di). Thus

Yi(t) = ai + bi*t + ci*t^2 + di*t^3

and thus

Y0(t) = 1 -19/15*t + 4/15*t^3
Y1(t) = -7/15*t + 4/5*t^2 - 1/3*t^3
Y2(t) = 2/15*t - 1/5*t^2 + 1/15*t^3
Y3(t) = -1/15*t + 22/5*t^2 - 10/3*t^3

Btw, in the code i use wi=Yi and x=t.

I hope it's a bit clear :)

[DeathTheSheep]

Spline16w produces--by far-- the sharpest image of all the included splines visually. It also produces the largest files under compression (see previous post).

All this leads me to ask whether spline16w is actually a 4-point spline at all, and if so, what makes it so obscenely different from the much-weaker normal spline16resize()?

[MfA]

Why does Wilbert's spline16w have support 2 when it can take values upto 4? (I know it was bugged with support upto 4, but it doesn't make much sense to keep the higher lobes in there now.)

Also it's rather strange that the 2nd lobes of the original spline16 and spline16w are identical.

[wonkey_monkey]

Why does Wilbert's spline16w have support 2 when it can take values upto 4? (I know it was bugged with support upto 4, but it doesn't make much sense to keep the higher lobes in there now.)

Those extra equations are now redundant - I'm sure Wilbert will get around to removing them from the source eventually :)

Also it's rather strange that the 2nd lobes of the original spline16 and spline16w are identical.

This makes me wonder if the "W" splines were only constructed as one-sided and then reflected instead of taking into account the necessary zeroes on the other (negative) side, which is why it meets at a point instead of flattening out at [0,1]. The straight lines make it practically bilinear for the nearest pixels.

As for why spline16 is sharper, I think it follows from these facts: the nearest two pixels will always have positive weights - this causes some softening. The furthest two pixels will have negative (though smaller) weights which causes sharpening. When you go up to spline64, you're adding another set of positively weighted pixels, and another pair of negatively weighted pixels. The new positively weighted pixels are weighted higher than the negatively weighted pixels, which overall will lead to more smoothing.

Don't forget, though, that "sharper" doesn't always objectively mean "better". Spline64 will still keep more detail through more transformations despite not looking as sharp.

David

[DeathTheSheep]

I understand this--but the data should be as follows:
http://forum.doom9.org/showpost.php?p=965947&postcount=153

According to this pattern, spline64 > spline64w > spline36 > spline36w > spline16 > (should be) spline16w. Meaning that the 64 is bigger than 64w, and all spline64 are bigger than spline36, of which 36 is bigger than 36w, and so on predictably. However, this is not the case with the anomalous spline16w, which is significantly bigger than everything else instead of smaller than everything else.

Also noteworthy is that in the compressed material, the spline16w kept--by far-- the most amount of detail as well, more than both spline64 implementations. This is something of a shock to me, as my testing indicated a patterned increase in spline# :: detail : filesize
with the huge exception of spline16w, which instead of producing the smallest filesize (like expected given the trend), produced not only the largest filesize, but also the most detailed encode!
So my implementation is correct, but what i called Spline16w is actually Spline64 (but with different coefficients)?
This might explain something.

[wonkey_monkey]

So my implementation is correct, but what i called Spline16w is actually Spline64 (but with different coefficients)?

This might explain something.

That was before Wilbert fixed his support values. Now that spline16w has support=2, it's definitely a spline16.

Also noteworthy is that in the compressed material, the spline16w kept--by far-- the most amount of detail as well

How are you judging that? Do you really mean that it kept detail, or that it's subjectively sharper?

I'm in the middle of running some tests, but I can tell you now that spline16w is oversharpening and losing a lot of fine detail.

David

[wonkey_monkey]

Here are some images to peruse. I resized up to 200%, then back down to the original size multiple times in order to emphasise the differences between the filters. On the left is the result of 8 pairs of resizes, on the right is the result of 32 pairs of resizes:

http://img15.imgspot.com/u/07/65/14/splinetest1173294903.jpg

As I think you'll agree, spline16w looks oversharp (though there's not much difference in the amount of fine detail between it and spline16 after all).

Spline64 seems worse than spline36. Spline36w looks positively broken. Spline64w doesn't look as sharp as spline64, which itself looks less sharp than AVS's spline36.

In order of best to worst, I'd place them like this:

spline36 > spline64 > spline64w > spline 36 > spline16w > spline36w

David

[Wilbert]

I don't know what to say. I'm not sure how to implement the equations which i listed in the previous page correctly. Perhaps IanB can help us out :)

[wonkey_monkey]

I don't know what to say. I'm not sure how to implement the equations which i listed in the previous page correctly. Perhaps IanB can help us out :)

Perhaps, but they may also be the wrong equations. I'm going to do some more graphing and see what I can determine.

David

[MfA]

The existing Spline16w is not C1 continuous around x = 0 BTW.

Anyway, lets do the math ... again :)

|x| < 1 : A1*x^3 + B1*x^2 + C1*X + D1
|x| < 2 : A2*(x-1)^3 + B2*(x-1)^2 + C2*(x-1) + D2

f1(0) = 1 : D1 = 1
f1(1) = 0 : A1 + B1 + C1 + D1 = 0
f2(1) = 0 : D2 = 0
f2(2) = 0 : A2 + B2 + C2 + D2 = 0
f1'(0) = 0 : C1 = 0
f1'(1) = f2'(1) : 3*A1 + 2*B1 + C1 = C2
f2'(2) = 0 : 3*A2 + 2*B2 + C2 = 0
f1''(1) = f2''(1) : 6*A1 + 2*B1 = 2*B2

Throw it all into Maple :) And you get :
A1 = 5/4, B1 = -9/4, C1 = 0, D1 = 1, A2 = -3/4, B2 = 3/2, C2 = -3/4, D2 = 0

So Spline16W which had been intended to be C2 continuous should be (simplified and using horner) :
|x| < 1 : (5/4*x -9/4)*x^2 + 1
|x| < 2 : ((-3/4*x + 15/4)*x - 6)*x + 3

PS. oops, it's only partly C2 continuous of course ... only at the 0 crossing, but that's the best you can do with cubics.

PPS. I also put Keys paper up (http://karton.student.utwente.nl/01163711.pdf) if you are interested, it gives the exact coefficients for spline16/36 optimized to match the first 3/4 terms of the Taylor expansion of the signal. It's simply the Catmull Rom spline, but the spline36 version might interest you (though I personally don't believe interpolation kernels with more than 2 lobes are a good idea).

[IanB]

@Wilbert,

The resizer generator evaluates the "f" function with [-support <= value <= +support]. Under normal circumstances the "f" function should be symetrical, hence most start with an value=fabs(value); and only provide the positive half of the relevant curve.

I think for your approach of Spline16W you should be starting with the following 7 control points [(-3,0), (-2,0), (-1,0), (0,1), (1,0), (2,0), (3,0)] and then using 2 of the resulting spline equations for [(0,1), (1,0)] and [(1,0), (2,0)].

The resultanting curve should be symetrical. Adding the [-3, -2]=0 and [2, 3]=0 segments to the ends might not be necessary as I think the end conditions already imply such.

[DeathTheSheep]

As I think you'll agree, spline16w looks oversharp
Yes, correctamundo David [Tennent]! That was the word I was looking for: "oversharp!" My guess is that this is precisely how it forced x264 to keep an abnormal amount of detail: not because it actually preserved more, but because it oversharpened the detail it actually preserved.

It seems to work obscenely well on low-bitrate/low-resolution anime (for encoding). Of course, this definitely depends on how you define " works well." From a solidly neutral perspective, it works positively terribly! :)

[DeathTheSheep]

On a completely different (summarizing) note:

Spline16w is way too sharp, spline36w is way too soft, spline36 beats both spline64 implementations, and my 64 vs 64w test conclusion earlier in this thread (Spline64w doesn't look as sharp as spline64) is reversed.

What amazing results... Are you sure that upsizing 2:1 then downsizing 1:2 repetitively is the best way to test interpolator quality? The old "rotation test" was hyped in the past...

PS: davidhorman, can you upload that test picture and your test script please? I'm really interested in this. :o

[MfA]

The old "rotation test" was hyped in the past...They have this amazing algorithm called the identity transform which completely blows that benchmark away (no, I don't think it's a particularly good benchmark ... it's more a test of reversibility than accuracy).

For upsizing the test aruzinsky (http://www.general-cathexis.com/interpolation.html) uses is okay, although I don't quite agree with using a box filter. For downsizing coming up with a good test is a little harder, you should probably start with a high rez image ... downsize it for some integer factors (say 16&17) with gaussian blur+decimation (most physically correct downsizer for integer factors) and then downsize from one those intermediate resolutions to another.

BTW. higher order continuity seems to not (http://lear.inrialpes.fr/people/triggs/pubs/Triggs-iccv01-subpix.pdf) be a very desirable property in piecewise cubic interpolators.

[wonkey_monkey]

What amazing results... Are you sure that upsizing 2:1 then downsizing 1:2 repetitively is the best way to test interpolator quality? The old "rotation test" was hyped in the past...

I couldn't find a spline16rotate function in AVS ;)

it's more a test of reversibility than accuracy

Doesn't one (at least somewhat) imply the other?

David

[MfA]

Doesn't one (at least somewhat) imply the other?
Only one way ... good accuracy means good results after iterative rotation to 360 degrees, good results after iterative rotation to 360 degrees however says very little about accuracy in the intermediate steps.

Exactly not the way needed to make it a good benchmark.

[Seraphic-]

It is understood that when you up-size, the image will most likely need to be sharpened back up. But how would LanczosResize compare with SplineResize for up-sizing? Also, how does the anti-aliasing of Spline and Lanczos compare?

(my source, if it will help, is a 16:9 ratio resolution captured at 720x480 - so I'm attempting to upscale to a wide-screen resolution)

Thanks

[IanB]

@MfA,

Thanks for the Robert G. Keys paper (http://karton.student.utwente.nl/01163711.pdf) and the Bill Triggs paper (http://lear.inrialpes.fr/people/triggs/pubs/Triggs-iccv01-subpix.pdf), very interesting reading.

So it would seem all this mathemagical tail chasing has been for naught. The current Spline16 and Spline36 implementations have been carefully chosen for a particular look. And I guess the Spline64 I dug up is also likely to have a somewhat arbitary lineage. Lets all keep digging, hopefully someone will come up with a paper or reference that will provide some grounding.

Wilbert's experimenting indicate almost anything will have an effect that at least some people will like.


It is understood that when you up-size, the image will most likely need to be sharpened back up.If that is your personel preference. However choosing a resizer that provides an appropriate amount of sharpening may save you an extra sharpening step.
But how would LanczosResize compare with SplineResize for up-sizing? Also, how does the anti-aliasing of Spline and Lanczos compare?Strictly, aliasing is not an issue with upsizing, the Nyquist frequency increases with upsizing. As for handling existing jaggies in the source image I personally think Spline16 does fairly a reasonable job.

(my source, if it will help, is a 16:9 ratio resolution captured at 720x480 - so I'm attempting to upscale to a wide-screen resolution)So you want to do an xxxResize(854, 480), use StackVertical or Interleave to compare the various offerings.

[*.mp4 guy]

Intersting test image, lots of aliasing and lots of stuff above the nyquist limit, this is what it looked like after 8 iterations with the 2x up/down sizer I'm working on.

http://img102.imageshack.us/img102/1733/testwa3.png

[Seraphic-]

So you want to do an xxxResize(854, 480), use StackVertical or Interleave to compare the various offerings.

Wouldn't the resolution be 853x480 for wide-screen? But since that is odd, and has some resize issues, would make 852x480 the better choice, no? I know it's only a few pixels, but wouldn't 854x480 be too wide?

Also, what are your thoughts on the final resolution of the video? Should it be 852x480 after cropping, be an up-size from something like 636x444 directly to 852x444, or be an up-size from 720x480 to 852x480 and then use the final size after cropping out the black-bars (around 758x444)? I was thinking #3, but wanted some other opinions.

[525/60]

but wouldn't 854x480 be too wide?872x480 would be too narrow.

[foxyshadis]

A tenth of a percent never killed anyone, come on, guys. You're overanalysing this, they're both equivalently good, and you can go up any way that looks good to you - although almost everyone uses crop-then-resize, because it makes the output size simple and deterministic. But hey, it's your script and you get to choose the workflow that makes the most sense for you, especially when it comes to dealing with AR.

mp4 guy, is that the same as the current wdresize, or is it a newer iteration?

[525/60]

A tenth of a percent never killed anyone,The difference between 853 and 872 is 2.2%.

[Bankis]

spline36 > bilinear > Lanczos > bicubic for downsizing with the current spline36 implementation? is this so?

[*.mp4 guy]

Its the current version, but with anti-aliasing/ringing and a bit of sharpening after upsizing, all done by dcte.

[jamieuk]

very cool thread

[techreactor]

yeah! but its confusing too, so whats the final verdict for downsizing and upsizing with respect to quality and which resizers should be preferred ??

[canuckerfan]

^I think spline36 for downsizing and maybe still lanczos for upsizing

[*.mp4 guy]

The lighthouse image with presharpening and my latest pet interpolator script (http://img476.imageshack.us/img476/8403/lighthouselzpgj0.png). As always, ideas for improvement are welcome.

[edit] Clown image (http://img297.imageshack.us/img297/8371/clownlzpeb0.png)

[DeathTheSheep]

And now the alpha adds support for a spline64. Is this the same spline64 as posted/tested earlier in this thread?

[manoj4986]

where can i find blackman resize filter

[buzzqw]

in avisynth 2.5.7

BHH

[IanB]

No, in 2.5.8

The is also a freestanding plugin version somewhere earlier in this thread.

[microchip8]

I found this thread which compares Lanczos vs Bicubic. It's a bit old, but the posts by user "dmunsil" are very interesting

http://archive2.avsforum.com/avs-vb/showthread.php?t=460922&page=2&pp=30

[DeathTheSheep]

Thanks for the link; that's quite a read. - Catmull-Rom plus sharpening is great at 1, very good at 2, and great at 3.
I'm assuming a filter for this exists since it was posted in the avs board, but I can't seem to put my finger on it. Anyone?

[manoj4986]

No, in 2.5.8

The is also a freestanding plugin version somewhere earlier in this thread.


thanq i didnt knew it was a core filter

[IanB]

IIRC Catmull-Rom isBicubicResize(... , b=0.0, c=0.5)

[*.mp4 guy]

That guy was... exagerating about how good Catmull-Rom is. Spline36 is better, as are a lot of others (for upsizing, for downsizing spline36 is about the best you can get)

IIRC Catmull-Rom isBicubicResize(... , b=0.0, c=0.5)
As far as I am aware, thats correct.

[foxyshadis]

It's just another one of those perceptual things. Though frankly I have a very difficult time discerning Catmull-Rom from Spline36... or I'm just too drunk to....

[microchip8]

ummm is there a difference between Spline36 and the Natural Bicubic Spline scaler or are both the one and same scaler, just with different names?

[DeathTheSheep]

And is the spline64 included in the newest alphas the same one linked to in this thread? If so, I'm afraid it might have problems since the coefficients haven't been verified, correct?

[IanB]

I'm afraid it might have problems since the coefficients haven't been verifiedWell yes, the red car is a firetruck. It won't have problems because the coefficients haven't been verified, it may have problems if they are wrong.

It resizes, it's sharp, it rings more than expected, do you have some technical insight or verification references to contribute.

[_ck_]

This thread is very close to a question I have though it's for ffvdub and not necessarily avisynth so I hope you'll tolerate me.

Does anyone know how many taps the Spline resize routine in ffvdub uses? Is it 16? 36? Unlike it's Lanczos which has a slider for taps up to 10, it does not have a scale.

I noticed that using Lanczos for some videos captures was creating a finely focused overscan line near the top of the result, vs using Spline was much more subdued while the rest of the image seems just as high quality as a 10-tap Lanczos.