AviSynth smoothers - In proportion to frame size, or is it?

[Gew]

This has been on my mind for a while now.

Check this out.
I have two clips of (the same) footage, but with different frame sizes. To make this example simple, assume that one is D1 (704x576px) and the other one is Half-D1 (352x576px), PAL ofc.

Now..

Would:

avisource("352x576.avi").fluxsmoothst(7,7)

..be equal to..

avisource("704x576.avi").fluxsmoothst(14,14)

..in the terms of "softening ratio", so to speak?
This by looking at the fact that the first clip is only half the frame size, meaning that any motion change in the picture would be lifted twice the amount, or like I expressed myself on another board: "each pixel would contain less change on motion by play in the larger frame size", which is quite logical.

So, how is it? When come to think about it, it feels sort of natural that any smoother (temporalsoften,fluxsmooth,temporalcleaner,ff3dfilter...) would have some part of coding that makes them perform some GetFrameSize-like function, and then doing all the "softening math" in proportion to this, so that avisource("352x576.avi").fluxsmoothst(7,7) would after all come to have an amount of smoothering equal to avisource("704x576.avi").fluxsmoothst(7,7).

Elite guys out there, don't laugh me out, just give me the chorus on this one, would ya?! ;)

In short
Does temporal/spatial smoothers (in general) take such a simple base fact as the actual frame size into account when deciding ratio for it's thresholds?

[2Bdecided]

It depends on the filter, but in general, no.

They're pixel based, not frame size based.

Specifically, I'm not sure about all of the examples you gave - e.g. frequency based ones could be cycles per pixel or cycles per frame. It's in the fft based filter's docs.

As for how frame size effects temporal smooth...!

Cheers,
David.

[Gew]

I see, I see. Well, actually I'm still a bit in the dark. I'm sort of novice into the whole spectrum of video processing, really. However, I think I may have come up to a reasonable slash logical answer to my own question. Now I need your opinion on my thinking.

First off, the mathematics that are involved in smoother plugins are way out of my league to understand, I suffer ADD from scratch and as for math I'm barely past the multiplication table <x12.

However, I read the manual for some of the more common smoother plugins. There was a [to my advance] quite simplified explanation on how temporal versus spatial smoothers work. Simply like this, temporal smoothers compare the luma/chroma in the same pixel within ie. prev/curr/next frame, and depending on threshold setting it blurs them together; whileas spatial smoothers compare luma/chroma in the surrounding pixels in one and the same frame. I'm sure this is alot(!) simplified, but the vague explanation on the principle was to my great satisfaction anyways.

Now, when I laid down and thought about all this, I figured that I was only causing greater confusion (make a mountain out of a molehill *) by thinking on how smoothers act differently on different frame sizes, 'cuz they're really NOT. And still not wrecking your explanation. Simply 'cuz smoothers is indeed (like you wisely explained) pixel-based. But then again, the threshold is set and it's logical that the smoother will keep on doing it's math cuz.. well.. damn my English is so sloppy I need to think on how to put this so I make myself understood.. frame size is really unessential in this, 'cuz the smoother will cause the same blending with ie. default threshold on 352x376 such as with 704x376, simply cause it's the same footage we're processing, only put in a larger (or smaller) box (frame size), and the math processing on luma/chroma contrast will be the same, only that it would prolly require more CPU to handle a larger frame size, cuz the tiny pixel changes would then be "stretched".

I can see my own way of logics in this (new) hypothesis of mine so clear, I hope you see my vision as I see it, and hopefully verify it ;)


Cheers!

[Gew]

I've now been running a few tests, and I seem to get wind in my sails for my hypothesis.

I process the very same clip with the following two script lines and try to compare the output result.

First variant:
bicubicresize(512,288).temporalSoften(5,5,8,15,2)

Second variant:
bicubicresize(1280,720).temporalSoften(5,5,8,15,2)

I cannot see how one is reducing more (or less) noise than the other. It should however be mentioned that my old Celeron CPU is having great difficulties on the playback of the HD-alike resize output, so the comparison is hard to judge, but I've tried to focus on one particular spot, and I cannot see any difference in how it blends out the noise.



(Skip the paragraph below, test's complete, this down here is only my own inner thoughts put into words)

I've started thinking that I may have made myself one big disservice (causing greater confusion than expert knowledge) by dragging the aspect of different frame sizes into the way smoothers play. I know for a fact that smoothers documentation in general advices users to do any possible resizing after(!) softening, which makes sense due to the fact that it's raw logics - optimizing results. Resizing (upwards) before smoothing may probably not only be inefficient ie. on CPU, but also cause loss of compression by ..err.. sort of +/- 1% or some tiny figure like that, but that's just fragment loss; smoothers rule is still the same - mind over matter - it won't stop blazing 'em blends on pixels, not until it reaches it's presuppose threshold limit, whereas this will cause blend over 3 or 30 pixels is only a matter of general optimization.

[Gew]

Just read the Soften documentation, and it clearly explains that the term "radius" is indeed involved in the smoothening process. You set a radius value, the higher - the more cpu demanding. Simple, really. The reason I couldn't tell a difference in my test was either 'cuz the fuzz in the clip was really just those tiny "mosquito dots" (often seen in raw MPEG2 captures), and my radius value "5" [which I assume equals to raidius=5px] would eliminate these dots regardless on drawn on 1280x720 or 512x288. OR(!) it was my eyes that just did not see the actual difference, I'm not always so good on that.

So, now I've cleared that one out. Now, there's a few other smoothers out there that are still quite a mess to me. FF3DFilter is the most powerful standalone .dll smoother (been too feeble to sync myself with the MVTools package, since it seems to be divided into so many pats = confuzion) I've tried out, but the documentation is heavy so the standard values ff3dfilter() is what I've been doing and it's never failed.

Anyways, there are two more smoothers.

TemporalCleaner() & FluxSmoothst()

There are no information on how these really work in their documentation, other than the actual threshold setting. Would that make it's radius spectrum "dynamic" (in condition to the frame size, like I first thought all filters were), or is it something else that I'm unaware of?

Feedback, corrections, everything of value here, please! ;)

[Gavino]

my radius value "5" [which I assume equals to raidius=5px] ...
In your example, using TemporalSoften, the radius represents 5 frames. As the doc says, "all frames no more than radius away are examined".

The reason TemporalSoften takes longer for 1280x720 than for 512x288 is that it has to process more pixels, but the work done for each pixel is the same.

[Gew]

In your example, using TemporalSoften, the radius represents 5 frames. As the doc says, "all frames no more than radius away are examined".

The reason TemporalSoften takes longer for 1280x720 than for 512x288 is that it has to process more pixels, but the work done for each pixel is the same.

Aaaah, I'm catching on.

Of cooourse, I was so blinded by thoughts so that I overlooked the fact that I was running a temporal(!) smoother, which blends pixels over frames(!) - not pixel to pixel in one and the same frame.

I guess this leaves my question at issue on to Spatial smoothing. What's the drill here? Also the SpatialSoften command has a radius parameter, does the radius value represent pixels there?

[Gavino]

Also the SpatialSoften command has a radius parameter, does the radius value represent pixels there?
Yes, it does.
Actually, the term 'radius' is possibly misleading. As I understand it, it looks at pixels not within a circular radius, but within a square area using the given radius both horizontally and vertically.

So radius 2 uses
x x x x x
x x x x x
x x o x x
x x x x x
x x x x x
where x marks additional pixels potentially (depending on threshold) contributing to output pixel at o.
Each output pixel is thus a blend of up to 25 (5x5) input pixels.

[Gew]

Awsome explanation, got to me completely.

Now..

Anyways, there are two more smoothers.

TemporalCleaner() & FluxSmoothst()

There are no information on how these really work in their documentation, other than the actual threshold setting. Would that make it's radius spectrum "dynamic" (in condition to the frame size, like I first thought all filters were), or is it something else that I'm unaware of?

Feedback, corrections, everything of value here, please! ;)

Could that be the case then? Okey so TemporalC~ is temporal so we leave that one out, but FluxSmooth does indeed have a spatial mode (FluxsmoothST()*) that does not leave the user with a float/int on radius. As I get it there are a few more look-alike-smoothers out there, without possibility of setting value on radius.

Now, does it seem conceivable to you that developers of such smoothers have made the radius into sort of a "crib sheet auto", by assuming that a frame would likely be burdened with traditional noise in such-many-percent-of-the-frame, and thus making a calculation so that it's spatial processing will [simply said] be using radius in similar proportion to the frame size being processed (ie. let's say it makes the value 0.5% of total w/h).

Could I be on the right track?

Ty in advance.


PS/ Don't ask me why I'm asking all these [apparently incoherent for the actual use] questions, I'm just sort of a control freak, who has this urge on knowing "what happens under the hood", always! ^^ /DS

[Gavino]

Now, does it seem conceivable to you that developers of such smoothers have made the radius into sort of a "crib sheet auto", by assuming that a frame would likely be burdened with traditional noise in such-many-percent-of-the-frame, and thus making a calculation so that it's spatial processing will [simply said] be using radius in similar proportion to the frame size being processed (ie. let's say it makes the value 0.5% of total w/h).
I'm not aware of any smoothers that work that way. If a smoother doesn't have a radius parameter, it's usually because it uses a fixed radius. FluxSmoothST, for example, has a (spatial and temporal) radius of 1, using the 8 immediate spatial neighbours within the frame and the 2 temporal neighbours at the same point in adjacent frames. I don't see any theoretical reason for taking the frame size into account either.

[Gew]

Aaah, I see, I see.

Then you always have to consider frame sizes manually.
Sorry to rewind, but that brings me back on top.

bicubicresize(704,576).fluxsmoothst(14,14)

bicubicresize(352,576).fluxsmoothst(7,14)

These two lines (targeting the same clip) would result in just about the same amount of spatial smoothening?

[Gavino]

bicubicresize(704,576).fluxsmoothst(14,14)
bicubicresize(352,576).fluxsmoothst(7,14)
These two lines (targeting the same clip) would result in just about the same amount of spatial smoothening?
Ah, I think I understand what you're trying to say now, but you've chosen a bad example because:
a) FluxSmoothST, as I said, has a fixed radius - the parameters affect only the threshold for deciding if one pixel is close in value to another;
b) in any case, the first param is the temporal threshold, the spatial one is the second.

However, if we compare
bicubicresize(704,576).spatialsoften(8, 5, 5) and
bicubicresize(352,288).spatialsoften(4, 5, 5) [note vertical dimension also halved]
you might expect the smoothing to be roughly equivalent in its effect on corresponding parts of each clip - not because the filter uses the frame size, but simply because the second is (approximately) a scaled-down version of the first.

[Gew]

Nice work! It all made sense now. And yes, I may have come up with a poor example. You're contra-example with SpatialSmoothen made it all very clear. I'll not change the threshold-params, but the radius-param, to have (~approx.) equivalence to scaled frame size. I think this case can now be closed, and I will print out this whole ping-pong-conversation and add it to my files I go back to whenever I stumble into the cloud of forgetfulness (which happens alot). Ty alot! Cheers!

E O T