FluxSmooth

[SansGrip]

Here's a new spatio-temporal smoother I wrote today based on ideas that came to me developing NoMoSmooth.

The source code is available. Here's the description from the documentation (http://www.indeus.com/sansgrip/avisynth/FluxSmooth-readme.html):

One of the fundamental properties of noise is that it's random. One of the fundamental properties of motion is that it's not. This is the premise behind FluxSmooth, which examines each pixel and compares it to the corresponding pixel in the previous and last frame. Smoothing occurs if both the previous frame's value and the next frame's value are greater, or if both are less, than the value in the current frame.

I like to call this a "fluctuating" pixel, then I like to wipe that pixel from existence by averaging it with its neighbours. This is (by default) done in a spatio-temporal manner, in that for each fluctuating pixel its 8 immediate spatial neighbours as well as its 2 temporal neighbours (the abovementioned corresponding pixel from the previous and next frames) are considered for inclusion in the average. If the value of each pixel is within the specified threshold, it is included. If not, it isn't.

This filter seems to remove almost all noise from low-noise sources (such as DVD) and a lot of noise from high-noise sources (such as cable TV captures), while maintaining a good amount of detail.

The speed at which it operates depends upon the amount of noise in the clip.

As usual your mileage may vary. This filter is unrelated to flux capacitors, which enable one to travel temporally.

source: http://www.kvcd.net/sansgrip/avisynth/

latest version: http://bengal.missouri.edu/~kes25c/FluxSmooth-1.1b.zip

[jorel]

Originally posted by SansGrip
Here's (http://www.jungleweb.net/~sansgrip/avisynth/FluxSmooth-0.1.zip) a new spatio-temporal smoother I wrote today based on ideas that came to me developing NoMoSmooth.

The source code is available (http://www.jungleweb.net/~sansgrip/avisynth/FluxSmooth-0.1_src.zip). Here's the description from the documentation (http://www.jungleweb.net/~sansgrip/avisynth/FluxSmooth-readme.html):



As usual your mileage may vary. This filter is unrelated to flux capacitors, which enable one to travel temporally.

thanks!........
the download works here,but can't open the page.;)

[High Speed Dubb]

The noise detection sounds similar to a temporal median filter, though your averaging method is different. You might want to check Tom’s STMedianFilter at
http://www.trbarry.com/STMedianFilter.zip
with docs at
http://www.trbarry.com/Readme_STMedianFilter.txt

[JuanC]

Originally posted by SansGrip
... One of the fundamental properties of motion is that it's not. This is the premise behind FluxSmooth, which examines each pixel and compares it to the corresponding pixel in the previous and last ¿next? frame. ... It looks like your web page is unavailable.:(

[SansGrip]

All links working fine for me. Must have been just a temporary thing...

[SansGrip]

The noise detection sounds similar to a temporal median filter, though your averaging method is different. You might want to check Tom’s STMedianFilter

From what I can gather reading the docs and source, STMedianFilter doesn't specifically target "fluctuating" pixels but instead clips pixels to its nearest neighbours.

I think what makes FluxSmooth different (though I've not checked every smoother out there :)) is that the temporal sequence 2, 0, -1 will not get averaged, but 2, 0, 1 will.

Perhaps this has already been implemented. What we need is some central filter algorithm repository we can check before beginning coding ;).

[SansGrip]

Here's verion 0.2, incorporating a minor bugfix, some optimizations, and an Avisynth 2.5/YV12 version.

Testing appreciated :).

Edit: Removed links to old version.

[Rrrough]

hi, I'd love to test this filter as well as the new version of NoMoSmooth, but unfortunately I can't load your page... :(
troublesome provider ? can you attach it to a post ?

cheers

[jorel]

Originally posted by Rrrough
hi, I'd love to test this filter as well as the new version of NoMoSmooth, but unfortunately I can't load your page... :(
troublesome provider ? can you attach it to a post ?

cheers

....but unfortunately I can't load your page too... :(
troublesome provider ? can you attach it to a post ?
thanks!

[High Speed Dubb]

Sansgrip,

That’s how a median works, too. With 2,0, -1, the median is 0, so the middle pixel’s value isn’t changed. With 2, 0, 1, the median is 1, so the middle pixel is changed.

[SansGrip]

hi, I'd love to test this filter as well as the new version of NoMoSmooth, but unfortunately I can't load your page... :(

Very odd. The site works fine for me. Can you PM me a traceroute?

can you attach it to a post ?

I did. It's still waiting to be modded.

[SansGrip]

That’s how a median works, too.

I'm beginning to think I should pick up a Statistics 101 textbook instead of reinventing it myself :D.

Ah well. Say, what would be your response to the idea of a spatio-temporal smoother with skin detection?

[High Speed Dubb]

I wouldn’t toss the idea just because there’s a related filter out there. (Well, actually there are two related filters. Simon’s Sandwich filter at
http://forum.doom9.org/showthread.php?s=&threadid=33817
also has some similarities.)

Medians are a good way to go for noise reduction — They’re usually a little inefficient, but they can be very robust. In other words, they don’t make the best possible use of the data, but the also don’t get messed up by unexpected stuff like interference. That’s because a median will mostly ignore the tails of a distribution, and just look at its center. (There are some sporting events which get judged with medianish filters — I think diving always tosses the low and high ranking before averaging the rest.) Medians will also do a good job with edge problems (i.e., when Y = black), while means will be biased.

I haven’t tried FluxSmooth yet, but the averaging you’re using when fluctuation is detected sounds like a good idea — It’ll probably avoid most of the usual median artifacts, like problems with very fast motion.

Skin detection sounds like it’s worth a try. Are you planning to look for specific colors, and avoid smoothing those too much? Just be sure not to claim that it “discriminates based on skin color” in the docs. ;)

[SansGrip]

Medians are a good way to go for noise reduction — They’re usually a little inefficient, but they can be very robust.

I'd love to find a more efficient way of implementing the "fluctuation detection" than what I'm using right now. Basically it's:


int prev_diff = prev_luma - curr_luma,
next_diff = next_luma - curr_luma;
if((prev_diff < 0 && next_diff < 0) ||
(prev_diff > 0 && next_diff > 0))
{
// Do the smoothing
}


I've optimized everything else about as far as it'll go (algorithmically that is), and this is the one remaining "if" in the main loop. I'd like to make it as efficient as possible, since I'm not able to remove it.

That’s because a median will mostly ignore the tails of a distribution, and just look at its center.

Interesting, I've never thought about averages in terms of distributions before. The more I get into this stuff the more I want to visit one of the local universities and see about a math course...

I haven’t tried FluxSmooth yet, but the averaging you’re using when fluctuation is detected sounds like a good idea — It’ll probably avoid most of the usual median artifacts, like problems with very fast motion.

I'm ashamed to admit I've not had time to try your fruity smoothers yet either. I'll try yours if you'll try mine... :D

Skin detection sounds like it’s worth a try. Are you planning to look for specific colors, and avoid smoothing those too much?

Yes, since I find that temporal smoothing, while overall very successful, is particularly destructive to skin tones and the slight details that make it look "real". It seems sensible to me to try to reduce this effect since skin is what we spend a great deal of time looking at in movies (some movies more than others, he added predictably ;)).

Inspection of colour range is one of the approaches I'd use. A little reading on the web reveals a lot of material relating to skin detection. Most of it is above my head (a significant amount was wrt neural nets), but I did pick up that skin tones, regardless of race, fall within a fairly specific colour range. Not only that, but by checking the overall variance of a 3x3 block it's possible to get a good idea of whether we're dealing with smooth skin or some other kind of similarly coloured texture.

I'm kinda worried it would pick up things like smooth pink walls. I'm still not sure how to differentiate between the two. Any ideas?

Just be sure not to claim that it “discriminates based on skin color” in the docs. ;)

heheh yes, it would need to be worded delicately ;).

[OUTPinged_]

SansGrip, you can't do much about skin smoothing unless you will update your motion engine so it would pick pixels according to motion vectors.

All these "smart" algorithms don't help a big deal and they get real slow very fast.

NoMoSmooth is about as far as it would get with current motion engine. You may see some improvement with that algo but it wont be drastical.

I know i sound like a whiner, but maybe try to concentrate on new ME engine. This will solve more problems than just "blended faces".

..Did a quick check and it works about same as nomosmooth, but a bit slower :-(

[SansGrip]

SansGrip, you can't do much about skin smoothing unless you will update your motion engine so it would pick pixels according to motion vectors.

Well, that's not quite true. It should be possible to detect skin even without motion vectors, though how much else that is similarly coloured and textured you'd also detect is another question.

NoMoSmooth is about as far as it would get with current motion engine. You may see some improvement with that algo but it wont be drastical.

The reason I'm toying with the idea of skin detection is because I find skin to be where temporal smoothing is most noticible. If one could avoid over-smoothing skin then more aggressive settings could be used on the rest of the frame.

I know i sound like a whiner, but maybe try to concentrate on new ME engine. This will solve more problems than just "blended faces".

You don't sound like a whiner, you sound like someone giving valuable input :). I'm currently looking around for a simple explanation/C(++) implementation of one of the fast motion estimation algorithms. If I find one I'll try to code it up and see what kind of results I get.

..Did a quick check and it works about same as nomosmooth, but a bit slower :-(

Now that is surprising, since in Flux the averaging is totally table-based while in NoMo it's arithmetical. I'll have to do some benchmarks/profiling and check out what's going on.

Thanks for your continued testing, it really is appreciated :).

[slk001]

Seems to me that the LAST thing you want to do with a noise filer for television is to AVERAGE it with its SPATIAL neighbors. Let me explain. First, since TV is a serially transmitted signal, any noise "blip" is not likely to be only one pixel long, but many pixels in a horizontal line. Assume, for arguments sake, that it is 5 pixels long horizontally, and only one pixel high vertically. Then, a tiny snapshot of the viewing screen would look something like this:

55555555555
55567842555
55555555555.

In this example, the "5" is the "real" signal, while anything else is noise. If we spatially average this noise, we will still have the noise - just averaged. But since we have identified this part as noise, why don't we try to eliminate it altogether? PROBLEM: What do we replace the noise with? Well, the most LIKELY candidates are its TEMPORAL neighbors. As an example, here is the same "noise" section with its two temporal neighbors:

55555555555
55555555555
55555555555 (PREVIOUS FRAME)

55555555555
55567842555
55555555555 (CURRENT FRAME, WITH NOISE)

55555555555
55555555555
55555555555 (NEXT FRAME)

In this scenario, if we endeavour to REPLACE the noise with a suitable substitute, then a very good approximation is the value of the pixels either before, or after the noise - or, the average of these two values NOTE: The noise itself is NOT included in the average, since we have assumed it to be an INVALID value, so, in this case, we are replacing a value based on a calculation in which the original value plays no part!!

As you can see in this simplistic scenario, the noise present in the current frame would be processed out entirely.

Now, progress to the NEXT FRAME. To prevent GHOSTING, the PREVIOUS FRAME is the frame that we PROCESSED OUT THE NOISE EARLIER. As a matter of fact, the ONLY frames that will be passed through unmodified are the very first frame in a video (since it has no "previous neighbor") and the very last frame (since it has no "next neighbor").

PROBLEMS:
1) Scene changes. This can be resolve by setting a WINDOW that the PREVIOUS pixel and the NEXT pixel have to be in before the filter kicks in.
2) Rapid, one frame movements (like a sword slashing through a scene). This can be resolved by setting a WINDOW based on the SPATIAL information of pixels ABOVE and BELOW the current test pixel (if the text pixel is within the window of EITHER the pixel ABOVE or BELOW, then filter ignores the current pixel under test - ie, doesn't change).

Now here are the assumptions that I am making regarding the noise:
1) 98% of the noise "blips" will not be over or under another separate noise "blip". Or, they will not do this (SPATIALLY):

55555555555
55567842555
55555567842
55555555555.

2) The noise will never appear temporally "sync'ed". (Ie, two noise "blips" will never occur at the same spot in two adjacent frames.)

For assumption 1), a WIDTH parameter could be added, where, instead of testing spatially one line above, testing two or three lines above or below the current pixel under test. For obvious reasons, the WIDTH parameter should be limited to a maximum of two or three.

For assumption 2), the filter should ignore the noise as signal, since any algorithm devised to correct this would most likely cause ghosting.

[High Speed Dubb]

SansGrip,

By saying that medians are inefficient I just meant that they don&rsquo;t make use of all the available information. I didn&rsquo;t mean to imply anything about speed.

Beyond your idea of checking the local variance, I can&rsquo;t think of any way to prevent the occasional smooth skin-toned feature from being identified as skin. As mistakes go, though, that might not look too bad &mdash; It&rsquo;ll just mean an occasional extra amount of noise. Considering that it will be mostly in smooth areas, it might even improve the picture.

There&rsquo;s also the usual threshold problem to watch out for. The boundary between &ldquo;skin&rdquo; and &ldquo;not skin&rdquo; might show some artifacts. But if the reduction of smoothing is subtle enough, the boundaries might be hard to spot.

slk001,

It looks like you&rsquo;re assuming that noise consists of small horizontal noise patches on a noiseless background (aka &ldquo;salt and pepper&rdquo; noise). That&rsquo;s a fair description of certain kinds of interference. But it isn&rsquo;t what I&rsquo;ve seen from broadcast video. Instead, there is noise everywhere. Because everything has noise, combining a whole bunch of pixels can improve the accuracy.

But you are right that noise is horizontally correlated (i.e., &ldquo;red&rdquo; noise) in broadcast material. So far I&rsquo;ve found that this can be ignored, but it is worth keeping in mind for the noise filters.

[SansGrip]

PROBLEM: What do we replace the noise with? Well, the most LIKELY candidates are its TEMPORAL neighbors.

Assuming they aren't noise too ;). Rather than thinking of noise as a completely invalid value, I tend to think of it as simply a corrupted one. Just because the pixel was originally a 5 and is now an 8 doesn't make that pixel's value useless -- we just need to correct it, and that's the hard part. Averaging, of course, isn't perfect.

But you're right in that there's an argument to be made over whether spatial smoothing is even a good idea. That's why I allowed both temporal and spatial smoothing to be separately disabled by setting the threshold to zero.


As an example, here is the same "noise" section with its two temporal neighbors:


55555555555
55555555555
55555555555 (PREVIOUS FRAME)

55555555555
55567842555
55555555555 (CURRENT FRAME, WITH NOISE)

55555555555
55555555555
55555555555 (NEXT FRAME)


If noise really looked like this, it would be very easy to detect and remove :). Unfortunately it's usually very very hard (if not impossible) to make the distinction between noise and fine details. What we're all trying to achieve is an algorithm that is, as it were, tough on dirt yet gentle on your hands :).

Now, progress to the NEXT FRAME. To prevent GHOSTING, the PREVIOUS FRAME is the frame that we PROCESSED OUT THE NOISE EARLIER.

This is one approach, but it has the disadvantage that it tends not only to reduce more noise but also destroy more details.

2) Rapid, one frame movements (like a sword slashing through a scene).

I briefly considered this when playing with algorithms, but decided that any movement that lasts for a single frame is so quick that the amount of processing applied to it barely matters. In fact, such a fast-moving thing would be a candidate for more smoothing, not less.

2) The noise will never appear temporally "sync'ed". (Ie, two noise "blips" will never occur at the same spot in two adjacent frames.)

Unfortunately since noise is inherently random, we can't be sure of anything like that.

[SansGrip]

Beyond your idea of checking the local variance, I can&rsquo;t think of any way to prevent the occasional smooth skin-toned feature from being identified as skin. As mistakes go, though, that might not look too bad &mdash; It&rsquo;ll just mean an occasional extra amount of noise. Considering that it will be mostly in smooth areas, it might even improve the picture.

I had a brief play with it and using the chroma range I settled on, along with a very simple variance measure, there were a lot of false hits in my source material -- including things like static walls that you really want aggressive temporal smoothing on. What might work better is a chroma histogram instead of a simple range, fed with test data from a bunch of different scenes.

But you are right that noise is horizontally correlated

You mean distributed over several horizontal pixels?

[slk001]

Assuming they aren't noise too .

True, so true... in fact, this is one assumption that you have to make when you are going to try to replace the noise with what you think is "good" data.

This is one approach, but it has the disadvantage that it tends not only to reduce more noise but also destroy more details.

Well, not really. If your pixel under test value is "8", and its temporal neighbors are both "5", then the probability that this pixel is noise and not detail data is quite high (remember, my vision of this filter would be to NOT FILTER if the temporal neighbors exceeded a threshold, and detail is NOT LIKELY to be only one frame long). But this technique would also almost completely remove single frame movements (like the slashing sword). That is why I believe an additional test of the pixels SPATIAL neighbors ABOVE and BELOW (although maybe not DIRECTLY) would be needed.

Unfortunately since noise is inherently random, we can't be sure of anything like that.

Again, this is so true. Unfortunately, we have to put a limit on how much we expect our filter to do. If the source is getting too noisey, then we have to direct our filter to shut down.

Just because the pixel was originally a 5 and is now an 8 doesn't make that pixel's value useless -- we just need to correct it, and that's the hard part.

When noise corrupts a signal, it tends to not just slightly modify the value, but instead tends to send the value to one of the luminance rails (16 to 235). This is where, of course, averaging is less than perfect.

You mean distributed over several horizontal pixels?

For TV, the answer is yes. This is because TV is sent out serially and noise, when it occurs, affects a "serial string" of data. This means that determining an average with a pixel's 8 spatial neighbors is likely to contain corrupted data in 3 of them. However, if this spatial averaging is done with the top 3 and bottom 3 neighbors, only 1 is likely to be corrupted.

The type of noise that I am interested in filtering, is the random (but sometimes heavy) IMPULSE noise, like from ignition noise or lightning strikes. These are perfect candidates for my replacement "theory" of filtering as opposed to simply trying to average it out.

The filter I envision would probably fail miserably against a picture with heavy picture snow (as would most noise filters). Here, the only alternative would be the averaging techique that you propose.

I can send you a clip of the type of noise that I am talking about, if you are interested.

[SansGrip]

This is because TV is sent out serially and noise, when it occurs, affects a "serial string" of data. This means that determining an average with a pixel's 8 spatial neighbors is likely to contain corrupted data in 3 of them. However, if this spatial averaging is done with the top 3 and bottom 3 neighbors, only 1 is likely to be corrupted.

This is interesting, I hadn't considered it that way. I'm going to let it percolate through my brain and see if I can work it into some kind of algorithm :).

I can send you a clip of the type of noise that I am talking about, if you are interested.

Sure, maybe we can spot some kind of pattern that could be useful in working out a filtering technique.

[SansGrip]

Here's 0.3 which fixes a very bad bug that significantly changed the algorithm. Since fixing this I've also been able to put the defaults back up to 7, 7. I was tempted to put them higher but I decided to wait for some feedback on that first.

It includes both a 2.0/YUY2 version and a 2.5/YUY2/YV12 version.

I also made some optimizations, and for me at least it runs fairly quickly (roughly 6-7fps faster than C3D movieHQ on 720x480 material). Though bear in mind that its performance depends on the amount of noise in the clip, and is still pure C++ with no assembler.

For very noisy sources try jacking up the thresholds. You'll be surprised how far you can go with them before you see artifacts.

I'm pretty pleased with this filter as it seems to remove a great deal of the most noticible noise with very little impact on details (depending on the threshold, of course).

Feedback is, as always, very much appreciated. I'd be especially interested in hearing its effect on compression, as well as suggested defaults for various kinds of sources (clean DVD, noisy DVD, cable captures, digital captures, etc. etc.).

Let me know :).

[Boulder]

Hi SansGrip, it seems that your site is not reachable..it's been like this for two days now:confused:

[SansGrip]

Hi SansGrip, it seems that your site is not reachable..it's been like this for two days now:confused:

It seems to be a problem coming from Europe, for some reason. Would someone unable to get to my site please do a traceroute to www.jungleweb.net and PM me the result? I administer the server so I'm very interested to know why it's not working :).

[ajp]

DNS lookup failed, can't find the host...

[slk001]

The site works fine for me.

[slk001]

The site works fine for me.

[vlad59]

It works fine for me too (I'm from France)

[rocker60]

Not for me:scared:
I'm from Portugal-(Europe)

[ajp]

Works for me now...weird... DNS server problems probably... :)

[SansGrip]

heheh thanks for the feedback. Based on a traceroute from Columbia we figured out something was wrong in the firewall (the senior administrator had been a little over-zealous fighting spammers and accidentally blocked most of the world outside North America :D).

Should be accessible to everyone now, though it might take a little while for DNS caches to sort themselves out.

Thanks for the reports and my apologies for the downtime :).

[Asmodian]

It works great for me (but I am not in Europe).

[cult]

works here,europe

[High Speed Dubb]

slk001,

When noise corrupts a signal, it tends to not just slightly modify the value, but instead tends to send the value to one of the luminance rails (16 to 235). This is where, of course, averaging is less than perfect.

You’ve definitely got heavy “salt and pepper” interference. I see something similar on a couple of channels when a certain fluorescent light with a dimmer switch is turned on. My solution has been to turn off the light. ;)

If you can’t find the source of the interference, it would be better to use a filter designed for getting rid of dots rather than noise. A simple temporal/vertical median should work very well for that. You might want to try Tom’s Undot, from

http://www.trbarry.com/UnDot.zip

SansGrip,

Yep, noise (not just interference) does tend to be horizontally correlated. I don’t know how many pixels you have to go before you approach independence — By eyeball, I would guess about 10 to 15 (with NTSC broadcasts and a pixel width of 720).

With respect to slk001’s problem, I think it would make sense to design the filter for use with noise or with salt and pepper interference, but not both. The two problems are different enough that it would make more sense to use separate filters for them.

About skin identification &mdash; I take it that skipping filtering on pinkish walls doesn’t look so good. Maybe the thing to do with skin color stuff is to use only slightly less filtering on it? That would reduce blurring, and might be subtle enough not to be noticed elsewhere.

[SansGrip]

You’ve definitely got heavy “salt and pepper” interference. I see something similar on a couple of channels when a certain fluorescent light with a dimmer switch is turned on. My solution has been to turn off the light. ;)

Very efficient, but does it work with YV12? ;)

Yep, noise (not just interference) does tend to be horizontally correlated. I don’t know how many pixels you have to go before you approach independence — By eyeball, I would guess about 10 to 15 (with NTSC broadcasts and a pixel width of 720).

Interesting. You don't think it would help to take that into account in a smoother designed for captures?

About skin identification &mdash; I take it that skipping filtering on pinkish walls doesn’t look so good.

It looks pretty bad :).

Maybe the thing to do with skin color stuff is to use only slightly less filtering on it? That would reduce blurring, and might be subtle enough not to be noticed elsewhere.

That might work. I'll have to play with the code and see what I can do with it.

[High Speed Dubb]

I had a “red noise” reducer in an unreleased early version of Peach Smoother. It did help, but the same technique hasn’t worked for the more advanced versions of the filter. It’s somewhere on my list for future things to try — I think there is a way to make good use of horizontal correlation even for the current algorithm.

The general approach I’ve tried has been to use inferred noise to the left as a predictor for the current noise. In other words, if the pixel to the left has an inferred noise of 4, then you can predict before even looking at the next pixel that it will have an expected noise of 4*k. This follows from a model called a (buzzword coming...) Ornstein-Uhlenbeck process.

[SansGrip]

This follows from a model called a (buzzword coming...) Ornstein-Uhlenbeck process.

Good Lord. Something as simple as that has a name so complex? ;) I suppose the tricky part is deciding on a value for k...

[High Speed Dubb]

It&rsquo;s a continuous random walk model, with an extra term which causes the result to move toward the &ldquo;true&rdquo; value.

You can call it &ldquo;red noise&rdquo; instead, since that sounds less intimidating. But then people think it has something to do with color, which it doesn&rsquo;t.

[SansGrip]

You can call it &ldquo;red noise&rdquo; instead, since that sounds less intimidating. But then people think it has something to do with color, which it doesn&rsquo;t.

I'm still fuzzy (no pun intended ;)) on the whole coloured noise thing. It's something else to add to my to-do list. Now I'm never going to get round to learning Esperanto... :D

[High Speed Dubb]

&ldquo;Colored&rdquo; noise is just a bunch of different noise distributions (and relationships between noise at nearby times). Audio engineers apparently dipped a little too deeply into the Cool-Aid and decided to name those distributions after colors.

White is the important one, and is just another name for independent, normally distributed noise.

[vinouz]

AFAIK, white noise is named so relatively to the white spectral distribution of this signal. E.g if you do an FFT on white noise, all frequencies have the same power.
I don't know for wideo, but for sound signals, it's what it means. And I don't think it must be different on video (just 2D spatial instead of 1D tmporal - btw, as spatial signal used to be transmitted sequentially, 2D spatial means practically 1D temporal, which for example on a tv signal would give the exact same function.)

for red and coloured, is it due to certain parts of the signal spectrum being stronger than others (e.g. for red : lo freq noise).

Cheers, Vince

[High Speed Dubb]

White light doesn&rsquo;t have the same power for all frequencies, or even all visible ones. Red noise does have lower frequencies, but &ldquo;red&rdquo; is a misleading term to use to describe low frequencies, at least when talking about video. (If we need something easier to remember than Ornstein-Uhlenbeck, we can call it Mean Reversion, or just correlated noise.)

I think the colors are just a metaphor which was stretched too far.

PS I&rsquo;ll switch to private messages for further stuff on noise nomenclature.

[vinouz]

I'm ok about the metaphor being pushed too far here. As red is quite closer to red color when talking about video ;).

About white light. When you put all spectral parts of light together, didn't we learn at school it makes white ?
I Agree you can make other whites lights with unequal spectral distribution. But this is due to the sensibility of the eye being itself unlinear. The best example of one of these lights is your CRT's white (Here let's consider it's perfectly well calibrated, so as not to confuse problems). blue phosphores make a certain blue, distributed unequally around the blue freq, and so on for red and green. But as your eyes only discern some frequencies of blue, of red and of green, if the sum of blue, reb and green percepted energy is the same in these three respective sensibility frequency domains, we see it white. But it isn't.
A well known example of this fact is known by photographs, who knows that fluorescent and incandescent light render far different on photographs, because argentic coloured crystals don't have the exact same frequency domains for R, G and B sensitivity than the eye. (we see them different too, but not as much as when rendered on photographs)
So even if white is not often constantly spectral distributed, that doesn't mean a constant spectral distribution of all the frequency doesn't give white.

(Not talking .6/.3/.1 coefficients for eye response to primary colours here.)

I'm not a specialist, so tell me where I missed something if that's the case.

Vince

[OUTPinged_]

Very hard case for temporal filters are a very dark scenes.


And scenes with deep red/blue color, too (have no idea, why, though).

[High Speed Dubb]

OUTPinged_,

Yep, dark scenes have a different noise distribution, and also a different signal distribution.

Some problems with saturated chroma (deep red/blue) can come from color crosstalk, which tends to make any extreme chroma areas appear to be in motion. Other problems can come from color calibration (which I find significantly more difficult than luma calibration).

A third possibility is that the filter handling of chroma isn&rsquo;t so good. The suggested thresholds for chroma differences on this forum are sometimes very inflated &mdash; From what I&rsquo;ve seen, YUY2 chroma actually varies much less than luma does. It&rsquo;s also possible for separate handling of chroma and luma to cause problems.

[slk001]

@SansGrip

I have a three second noisey video that is ~1.15MB long. If you would like to look at it, where do I send it?

[SansGrip]

I have a three second noisey video that is ~1.15MB long. If you would like to look at it, where do I send it?

Replied in PM.

[iago]

@SansGrip

A very short feedback, but I just wanted to inform you that I still cannot lose details using FluxSmooth(15,15)! ;)

regards,
iago

[SansGrip]

A very short feedback, but I just wanted to inform you that I still cannot lose details using FluxSmooth(15,15)! ;)

Glad to hear you're experimenting :).

You probably are losing a little with thresholds that high, but because it's only selecting fluctuating pixels it won't be too noticible.

Try doing a StackVertical with FluxSmooth(15, 15) above and FluxSmooth(15, 0) below. I think you'll see that a high spatial threshold does cause some compression-friendly softening -- that's what it's there for :).

[lamer_de]

I tried it on The Simpsons TV Caps, and the results were very promising. I have those SVCD Rips and re-encode them to XVID, but the original MPGs have dot crawl all over which is ultra-annoying. I can't use GuaveComb cause of the source format, so i used wnr till now. Thing is, that it smoothed the whole image, cause i had to use rather agressive settings an still couldn't get rid of all the craling. So warpsharp was applied afterwards, which has the drawback of "sharpening away" fine, small lines. Your description of the filter looked like something which could be usefull in this issue, so i used the "insane" settings of

FluxSmooth(7,80)

which gave pretty good resluts. I haven't done a full encode yet, only small clips, but the crawling was nearly gone and the rest of the picture wasn't altered as much as with wnr.

So, pretty usefull filter for me :)

http://members.tripod.co.uk/forumpic/homer.jpg

It's not that visible, cause it's not moving, but it should be visible at the Telephone letters, especially the P,L,O. The unsharpness comes from convolution3d and PP settings of 5 (fast motion scenes were rather blocky plus mosquito noise at edges).

Thank you,
lamer_de

[SansGrip]

So, pretty usefull filter for me :)

Well, you certainly found a use for it I didn't intend :).

i used the "insane" settings of

FluxSmooth(7,80)

which gave pretty good resluts.

Is a spatial_threshold of 80 really necessary? Theoretically, since the dots are close in brightness to the surrounding pixels, you shouldn't need such a high setting.

I'm glad you find it useful. I'm working on 0.4 at the moment, which also does chroma smoothing. I'll be interested to hear if that helps reduce the dot crawl too :).

[SansGrip]

Just to let you all know that I've not been idle over the past week or so, but have been attempting to reimplement Flux with multimedia extensions. It was hard going for a while, but then something seemed to click and this morning I fixed the last bug in the new iSSE version of Flux-for-YUY2 (I'm going to start the YV12 version this afternoon, and it should go quicker because I won't have to monkey around with pixel order like I do in the YUY2 version).

While the code is not optimized wrt pairing and stalls and all that good stuff, it still runs at twice the speed of the old C++ version on my system :D. I'm predicting the YV12 version will see an even bigger increase because of the lack of aforementioned monkeying around.

I'm hoping to finish it and release today, but I'm not promising anything. This is my first real C-to-MMX port and I'm still pretty slow at the moment.

[iago]

@SansGrip

I'm impatiently waiting for your new optimized YV12 version of FluxSmooth. I have just finished a 1CD encode which came out really good with a combination of "FluxSmooth(7,25).LanczosResize(576,240).UnDot()" and I haven't noticed any artifacts introduced by the filter.

Btw, I want to mention two more points:

Some terrible blockiness in the source itself which was really annoying especially in some scenes were eliminated totally with strong BlockBuster noise! ;)

Also, the encoded movie contained both real-life and anime scenes, and the anime parts were encoded with FluxSmooth(7,25).BilinearResize(576,240).Convolution3D(preset="animeHQ"), which came out without any problems but only a lovely smoothness, just as I exactly aimed to get for those scenes ;).

And finally, FluxSmooth is already fast enough imho, but even more speed keeping the quality intact, why not, is of course welcome! ;)

best regards,
iago


LOL! That must be exactly what they call "heavy filtering"! ;)

[SansGrip]

iago: FluxSmooth(7,25)

Wow, that's some serious spatial smoothing! There's no visible artifacts with a threshold that high?

Some terrible blockiness in the source itself which was really annoying especially in some scenes were eliminated totally with strong BlockBuster noise! ;)

Interesting that it can fix existing blockiness as well as prevent it occurring in the encode... I'm going to have to try that on one of the 90-minute 400mb screeners I downloaded ;).

And finally, FluxSmooth is already fast enough imho

It's pretty speedy with YV12, but 0.4 did slow down a little when I added chroma smoothing. However, I'm getting about 100fps in YV12 with the optimized code :D.

The code's all there but I need to do some more testing as I'm seeing some corruption on frames 0 and 1. This is particularly weird since frame 0 is just passed straight back in GetFrame. I'm wondering if it's not a 2.5 bug, so I need to get the latest build and do some more tests.

I'm expecting to release it today.

LOL! That must be exactly what they call "heavy filtering"! ;)

You're not kidding!

[FuPP]

@SansGrip



Using Fluxsmooth 0.3 in the following script causes a crash (access violation at 0x08ccb826 attempting to read from 0x0f0b9000 (This last address can change)

rem : When fluxsmooth is placed before cropping, then everything is right.

LoadPlugin("C:\video\avsfilters\MPEG2Dec3.dll")
LoadPlugin("C:\video\avsfilters\Fluxsmooth.dll")
mpeg2source("C:\video\tests\vts_01.d2v",cpu=0,idct=2)
LumaFilter(-2, 1.02)
Crop(30,80,660,416)
Fluxsmooth(15,8)
BicubicResize(448,320,0,0.5)
AddBorders(16,128,16,128)
YV12toYUY2(interlaced=false)

[MaTTeR]

Originally posted by SansGrip
While the code is not optimized wrt pairing and stalls and all that good stuff, it still runs at twice the speed of the old C++ version on my system :D. I'm predicting the YV12 version will see an even bigger increase because of the lack of aforementioned monkeying around. Wow, that's certainly something for me to get anxious about. I'm sold on this filter now:D It has now replaced TemporalSoften(x,x) in all my movies, very impressive. Keep up the great work SansGrip!

[SansGrip]

Using Fluxsmooth 0.3 in the following script causes a crash (access violation at 0x08ccb826 attempting to read from 0x0f0b9000 (This last address can change)

rem : When fluxsmooth is placed before cropping, then everything is right.

Thanks for the info -- hopefully this bug has been fixed in 0.4, which is almost a rewrite. I'll double-check before releasing.

[SansGrip]

Originally posted by MaTTeR
Wow, that's certainly something for me to get anxious about. I'm sold on this filter now:D It has now replaced TemporalSoften(x,x) in all my movies, very impressive. Keep up the great work SansGrip!

Glad to hear it's useful for you :).

Sorry for the delay in releasing 0.4, I've had some important things come up that, if you can believe it, don't involve Avisynth ;). I'm hoping to get done with what I'm working on now in a few days (or at least get enough of it done so I can return to 0.4, which is nearly ready to go).

[rocker60]

For at least 24h. Can you please see what's appening?
Thx.

[morsa]

Hi, I suppose many of you know about Restoreinpaint project.

Sourceforge.net/projects/restoreinpaint

It has many, many advanced filtering techniques.I would be nice to have some portings of them.They also can be an inspiration.

[morsa]

Is going to be someday a Vdub version of this filter?

[iago]

@SansGrip

Long time no see, man! ;) I'm looking forward to FluxSmooth 0.4 and I hope the package will come with both YV12 and YUY2 versions together again! ;)

regards,
iago

[Bulletproof]

Originally posted by SansGrip
I'm working on 0.4 at the moment, which also does chroma smoothing. I'll be interested to hear if that helps reduce the dot crawl too :).

That's going to be optional right? :)

[gigah72]

hi,

I read here someone had an accessviolation if fluxsmooth is placed before resize. i have sometimes the opposit problem, but it depends on the crop command:

LoadPlugin("C:\PROGRA~1\DVD2SVCD\MPEG2Dec\mpeg2dec.dll")
mpeg2source("H:\dvd2svcd\VAIACO~1\DVD2AV~1.D2V")
LoadPlugin("C:\Programme\DVD2SVCD\avs-dll\FluxSmooth.dll")
Crop(0, 10, 720, 554) <------ ok
Crop(8, 11, 704, 552) <------ crash
FluxSmooth()
AutoResize()
AutoAddBorders()
#SVCD
function AutoAddBorders(Clip c) {
aab_top = (Floor(((576 - c.height) / 2) / 16))*16
aab_bottom = (576 - aab_top) - c.height
return addBorders(c, 0, aab_top, 0, aab_bottom)
}
function AutoResize(Clip c) {
factor = (72 == 72) ? 0.75 : 1.0
height = Round((720.0 / c.width) * c.height * factor)
return BilinearResize(c, 480, height)
}

both after the resize is ok. any idea ?

greets,
g.

[SansGrip]

Attached is 0.4. Sorry about the delay -- darn real life ;).

The biggest change in this version is that I implemented ISSE-optimized algorithms for both the YUY2 and YV12 versions. From my quick benchmarks it seems to run about twice the speed of the C++ version. I also added chroma smoothing.

As usual you can get the latest version (and the source) from the web site in my sig. (By the way, I apologise for the downtime: the web server it's on is located in North Carolina, and they had a killer storm which took power down for days. It should be back up now.)

Many thanks to trbarry, sh0dan, dividee, and everyone else who helped me figure out assembler and MMX by putting up with and patiently answering my stupid questions :).

Edit: Removed links to old version.

[SansGrip]

Originally posted by Bulletproof
[Chroma smoothing]'s going to be optional right? :)

It would be simple to make optional for the YV12 version, but quite hard for the YUY2 version. Which do you use? :D

[SansGrip]

Originally posted by gigah72
I read here someone had an accessviolation if fluxsmooth is placed before resize. i have sometimes the opposit problem, but it depends on the crop command

Sorry to hear you're having problems.

I'd be grateful if you could try the same thing on 0.4, since I made significant changes from 0.3 and could have eliminated the bug accidentally.

[onesoul]

Hi. First of all humbly I must say I admire all the work you and all programmers been doing with avisynth filters.
I would like to have your opinion about the usage of fluxsmooth.

I've seen people in this thread using both fluxsmooth and convolution3d at the same time, is it wise to do that?

I've read the "readme.txt" of both nomosmooth and fluxsmooth but still not being confortable at this field, I apologize for the following question. Is nomosmooth made obsolet by fluxsmooth? If not when one should use it?

Could you recomend some settings, like convolution3d has, but including also for dv source.

I hope I am not asking too much... (oops)
Thanks

[Bulletproof]

Originally posted by SansGrip
It would be simple to make optional for the YV12 version, but quite hard for the YUY2 version. Which do you use? :D

YV12 ;) , Thanks.

[SansGrip]

Originally posted by onesoul
I've seen people in this thread using both fluxsmooth and convolution3d at the same time, is it wise to do that? I see no reason why not, if it produces superior results. If using both together, though, I would recommend using very light settings on each.

Originally posted by onesoul
Is nomosmooth made obsolet by fluxsmooth? If not when one should use it? While both NoMo and Flux are spatio-temporal smoothers, they are otherwise not very similiar.

NoMo tries to differentiate between static areas and those in motion, with the hope of reducing motion-related artifacts. It is reasonably successful, but can introduce other problems when similarly coloured objects are moving over each other.

Flux, on the other hand, specifically targets what I like to call "fluctuating" pixels. Since a video frame is really just a bunch of numbers, imagine that you are currently processing the pixel at x=50, y=50. If the previous frame's value at that location is 70, the current frame's 72, and the next frame's 71, then the pixel in the current frame is a fluctuating pixel because it's greater than the value of the corresponding pixel in both the previous and next frames. This makes it unlikely to be a product of motion -- therefore likely to be noise -- and it is smoothed. This seems to work very well, though theoretically fine textures in motion could be mistaken for a fluctuation.

So Flux hasn't really "superceded" NoMo in a theoretical sense. That said, I use Flux almost exclusively now (when I find time to actually encode some video, that is ;)).

Originally posted by onesoul
Could you recomend some settings, like convolution3d has, but including also for dv source. Once Flux has received more testing I'm hoping we can compile sets of settings for exactly this purpose. Personally I tend to use a temporal threshold of 5-10 (depending on how noisy the source is) and a spatial of 0-5.

But others have used this filter far more often than me, so can likely provide more valuable feedback on good settings :).

[SansGrip]

Originally posted by Bulletproof
YV12 ;) , Thanks. Ok, I shall implement enabling/disabling chroma smoothing for the YV12 version in the next release. TBH I don't think I'll ever implement it for the YUY2 version unless I see a compelling reason to do so...

[Boulder]

Hi SansGrip, good to see you again. Thanks for the optimized filter, I've been using FluxSmooth in all my (X)VCD conversions these days:)

[SansGrip]

Originally posted by Boulder
Hi SansGrip, good to see you again. Thanks for the optimized filter, I've been using FluxSmooth in all my (X)VCD conversions these days:) Glad to hear that :). Do you have any observations regarding good settings for various sources? I'm considering implementing something similar to C3D's "preset" in the next release.

[SansGrip]

Originally posted by morsa
Is going to be someday a Vdub version of this filter? I have no current plans to port any of my filters to VDub (mainly because I never use it for filtering ;)).

[hakko504]

I can give you one more reason not to convert the temporal smoothers to VD. VD cannot access frames randomly like AviSynth, so you need to build and maintain a cache of the frames you want to use later. This results in a delayed picture, usually by one frame.
Virtual Dub is supposed to be a completely linear solution, a one frame input will result in one frame output (encoded). There is one large exception to this, and that concerns b-frames in DivX5.02 and XviD.

[FuPP]

hi SanSgrip

Thx again for your great filter! I still have though a problem when putting it after crop function, except using some filters after yours
(don't know if I am clear :D)

ie :

LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\asharpyv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
LoadPlugin("C:\video\avsfilters\undotyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
colorYUV(levels="PC->TV")
fluxsmooth(15,8)
BicubicResize(448,320,0,0.5).AddBorders(16,128,16,128)
converttoyuy2()

DOES NOT WORK -> weird image



LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\asharpyv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
LoadPlugin("C:\video\avsfilters\undotyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
colorYUV(levels="PC->TV")
fluxsmooth(15,8).asharp(1,4.5).undot()
BicubicResize(448,320,0,0.5).AddBorders(16,128,16,128)
converttoyuy2()

WORKS !

[SansGrip]

@FuPP

Could you try:


LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
fluxsmooth(15,8)
and so on, to eliminate some of the other filters from the equation?

[SansGrip]

Originally posted by hakko504
VD cannot access frames randomly like AviSynth, so you need to build and maintain a cache of the frames you want to use later. If I wasn't sure before, I am now ;).

[onesoul]

Originally posted by SansGrip
So Flux hasn't really "superceded" NoMo in a theoretical sense. That said, I use Flux almost exclusively now (when I find time to actually encode some video, that is ;)).


:) I've been learning so I've encoded only to test (a few cds thrown away), I can say this forum has taught me everything I know about encoding, everyday is a lesson learned.

Thank you for your enlightened explanation, really apreciated :)

I'll stay tuned for any recomended presets. Maybe one day I can contribute too with valuable info such as settings.

[SansGrip]

Originally posted by onesoul
I'll stay tuned for any recomended presets. Maybe one day I can contribute too with valuable info such as settings. Read, read, read, test, test, test. Then read and test some more. That's how everyone here learned, unless there are Avisynth classes that I'm unaware of ;).

The very best way to learn Avisynth thoroughly is a) read the source and b) write your own filter. Even if it does something very simple such as invert the image you'll learn a lot about how filters operate which you can then apply to actually using them to their fullest.

[Guest]

Originally posted by hakko504
I can give you one more reason not to convert the temporal smoothers to VD. VD cannot access frames randomly like AviSynth, so you need to build and maintain a cache of the frames you want to use later. This results in a delayed picture, usually by one frame.
Virtual Dub is supposed to be a completely linear solution, a one frame input will result in one frame output (encoded). There is one large exception to this, and that concerns b-frames in DivX5.02 and XviD. You can implement a frame cache in VirtualDub that does not delay the stream. You just save your own copy of the previous frame(s) in your filter data. This requires special handling for the first frame(s) but that is usually not a problem.

As proof, SmartDeinterlacer (among others) does this. :)

[SansGrip]

Originally posted by neuron2
You can implement a frame cache in VirtualDub that does not delay the stream. You just save your own copy of the previous frame(s) in your filter data. This requires special handling for the first frame(s) but that is usually not a problem. It might be worth trying once as a programming exercise, but what with maintaining two different (YUY2 and YV12) versions of my filters already I think I'll stick with the one API ;).

Edit: Correction! Four versions -- YUY2 C++, YUY2 MMX, YV12 C++ and YV12 MMX :scared:

[Boulder]

I can't say that there are settings that are the best in certain situations, I just try them out and compare the filtered output to the original one (with two VDubMods running).

For example, Hitchcock's The Birds lost a lot of noise but kept the details with the default settings. Without the filter the movie would have had a lot of noise in the background.

In my analog TV captures I've mainly used FluxSmooth(15,15). I know it could remove details as well but I think that the extra smoothing is worth that. The values depend on the captured source, of course. It seems that the Finnish Broadcasting Company has a slightly better signal than the commercial ones.

The thing about the settings is truly trial-and-error:devil:

[SansGrip]

Originally posted by Boulder
For example, Hitchcock's The Birds lost a lot of noise but kept the details with the default settings. Without the filter the movie would have had a lot of noise in the background. I found the defaults to be quite good -- that's why I picked them :D.

In my analog TV captures I've mainly used FluxSmooth(15,15). I know it could remove details as well but I think that the extra smoothing is worth that. Yes, I use pretty strong values for caps too (more for analog sources than digital of course).

By the sound of it we pick similar settings, except you use a lot more spatial smoothing than I...

The thing about the settings is truly trial-and-error :devil: Maybe having a "preset" parameter is a bad idea, then...? It would be great for lazy people such as myself, but might discourage someone from playing with the parameters and coming up with their own values :).

[MaTTeR]

Great to see you back in action here SansGrip. Thx for the new build, been testing it and indeed it's quite speedy and found no bugs:)

In regards to presets, for a high quality encode of a DVD source the defaults might be a little to strong. My settings have been lingering around 3,3 or at the most 5,5 for clean DVD sources. Something in this range seems to be a nice trade off when fine details need to be kept IMO. Perhaps I'm being a little conservative here but I do like sharpness and detail;)

[SansGrip]

Originally posted by MaTTeR
Thx for the new build, been testing it and indeed it's quite speedy and found no bugs:) Quite speedy? QUITE speedy?? :angry: ;) heheh :D

I'm sure there are a few optimizations I can squeeze out yet, wrt pairing and stalls and all that stuff I don't quite understand fully yet (though I did just get in the mail an Intel book on software optimization). Actually what would be great is if someone more experience with MMX were to take a look at the source and give me any suggestions... Hint, hint... :D

In regards to presets, for a high quality encode of a DVD source the defaults might be a little to strong. Yep, I agree. I found them good for "average" DVDs like American Pie, but for clean sources it's too much.

My settings have been lingering around 3,3 or at the most 5,5 for clean DVD sources. Something in this range seems to be a nice trade off when fine details need to be kept IMO. Perhaps I'm being a little conservative here but I do like sharpness and detail;) If so then I'm on the conservative side too. In fact, I often lower the spatial threshold or even turn it off completely unless I see a real need for it.

[iago]

@SansGrip

Welcome back man! Nice to see you around again ;). Now, it's time to give a shot to the speedy new release (with "Fight Club" as usual, as a personal preference for testing filters with a good DVD source ;)).

regards,
iago

[SansGrip]

Originally posted by iago
Welcome back man! Nice to see you around again ;). Now, it's time to give a shot to the speedy new release Thanks :). Let me know how you find it...

[calvore]

I am planning to test out fluxsmooth over the Christmas break while I convert many of my home videos from VHS to CVDs. So, analog interlaced NTSC source. Does fluxsmooth work with an interlaced source or do I need to separate the fields, apply the filter, then join them again?

Thanks, it looks like these filters will work excellent for cleaning up the video.

While I am asking, how much of a border should I place around the CVD image to actually aid in compression without moving outside the overscan area? Chunks of 8x8 blocks, or is it 16x8 (or 8x16)?

[SansGrip]

Originally posted by calvore
Does fluxsmooth work with an interlaced source or do I need to separate the fields, apply the filter, then join them again? Flux doesn't specifically handle interlaced source (yet, anyway), so I think you'll need to separate and rejoin.

While I am asking, how much of a border should I place around the CVD image to actually aid in compression without moving outside the overscan area? Chunks of 8x8 blocks, or is it 16x8 (or 8x16)? It depends on your TV. For me I can add 8-pixel borders top and bototm without seeing them on the TV screen, but left and right I can't add any (well, up to 6 pixels, but that doesn't help compressibility any). Most TVs allow 8-16 pixels each side, and some even more, so the best thing to do is experiment.

[onesoul]

I've done some reading and some testing. I find fluxmooth to be running at more 50% of the speed of the convolution3d or nomosmooth (these 2 filters gave me similar speeds).

Fluxsmooth (at the default values) has made me sastified with the results it achieved so I will use it for now on.

I'm still not very confortable with changing values but I guess I will try some. I must say it is the spatial smoother setting which gives me more dificulties.

Keep up the good work SansGrip

[SansGrip]

Originally posted by onesoul
I've done some reading and some testing. I find fluxmooth to be running at more 50% of the speed of the convolution3d or nomosmooth (these 2 filters gave me similar speeds). NoMo is not yet MMX-optimized, but is fairly high on my todo list.

Fluxsmooth (at the default values) has made me sastified with the results it achieved so I will use it for now on. Glad it's working for you :).

I'm still not very confortable with changing values but I guess I will try some. I must say it is the spatial smoother setting which gives me more dificulties. For what it's worth, I tend either to use the default values (fast to type :D) or I'll set spatial lower than default, especially for very clean sources.

[MaTTeR]

SansGrip,

I forgot to post a shot of ringing artifacts for you. Have a look at the 2 PNG files I've attached. The 2nd pic of a sunset might demonstrate the problem a little better. You can adjust the color properties to exagerate the problem a little more if need be. Notice the rings around the sun?

This movie (US Navy Seals) is an absolute ringing nightmare due to the overall darkness and underwater scenes:devil:

[MaTTeR]

2nd ringing artifact shot...

[SansGrip]

Originally posted by MaTTeR
Notice the rings around the sun? Yep, though I've never seen Flux produce anything like this and really can't think what mechanism might be causing it. Flux only ever uses a spatial radius of 1, so shouldn't be able to cause artifacts beyond that range.

Unless I'm missing something...?

[MaTTeR]

Sorry, I should have been more clear earlier. The shots above aren't related to flux in anyway. These ringing artifacts will occur even without any filtering on the source, they just happen to be worse when encoding to YV12 IMHO. I doubt MPEG-2 has this problem though, I've never noticed it on any KVCDs.

[kilg0r3]

@matter

i always thought this was called blocking or color stepping and that ringing was the same as moskito noise.

are thes screen shots from encoded material or from avisynth output? if this is xvid encoded material, please post about this problem also in the xvid forum. might be an xvid issue.

my guess is that this problem is due to heavily compressed chroma information, because it mostly shows in color gradients. furthermore it always looks like that in some areas the codec or whatever, uses the wrong colors. i am just a n00b though.

anyway to get to it by using blockbuster?

[MaTTeR]

Originally posted by kilg0r3
anyway to get to it by using blockbuster? This is my next test actually, I'm hoping it might help.

Maybe I'm confusing the term a bit but I always assumed since I seen ringing(circle rings) artifacts around objects or on walls that "deringing" post processing would deminish them. I do like the color blocking term though, havent heard that one before:)

[SansGrip]

Originally posted by MaTTeR
The shots above aren't related to flux in anyway. Man... Don't DO that :D ;) heheh

[iago]

@SansGrip

I just wanted to thank you one more time for having provided us with such an amazing smoother. I have tried the latest 0.4 version (not exceeding the range of 1-10 for both temporal and spatial thresholds) on a couple of sources, some of which contain a considerable amount of source noise (such as U-Turn) and some of which are pretty clean (such as Fight Club). So far I have not noticed any artifacts in my test encodes particularly introduced by FluxSmooth, and I'm also pleased with the detail level preserved within the above mentioned ranges; still the parameters choice depending on the noise level of the source of course ;).

I used BicubicResize(576,320,0,0.5) for 1.85:1 AR and BicubicResize(640,272,0,0.5) for 2.35:1 AR sources, and XviD and (don't tell Koepi! ;)) Nandub SBC for encoding.

Best regards,
iago


Edit: Ah, I forgot to mention that I have been working especially with the YUY2 version, and all the above said words are related to the YUY2 version of the filter ;).

[SansGrip]

I just wanted to thank you one more time for having provided us with such an amazing smoother. I'm very glad you like it :). Personally I use it on almost everything now. Do you like the speed?? :)

and I'm also pleased with the detail level preserved within the above mentioned ranges; still the parameters choice depending on the noise level of the source of course ;). Yes, while it's possible to go very high with the thresholds without introducing artifacts, I've found the smoothing can become too strong above 10. For me the spatial_threshold is the most sensitive wrt over-smoothing.

XviD and (don't tell Koepi! ;)) Nandub SBC for encoding. Your secret's safe with me :D.

Edit: Ah, I forgot to mention that I have been working especially with the YUY2 version, and all the above said words are related to the YUY2 version of the filter ;). hehe ok. Do you notice much of a speed difference between YUY2 and YV12 modes?

[wing1]

12,3 here for capture source, and the yv12 version is a speed demon on my 1900+ AMDxp. Great filter.

[iago]

@SansGrip

Well, frankly, of course I have also tried the YV12 version, and it's definitely as successful as the YUY2 version in terms of denoising/smoothing and details retaining.

However, on my Celeron900/256mb system (MaTTeR, how are you man!? :D) and when I put FluxSmooth before the resizer, I can't notice a drastic speed gain actually. Well, of course I gain a few more fps especially when using "-p 3" with the fantastic AVS2AVI command line utility, but that's not much of a difference on my end due to limitations of my system specs ;).

And as a result, in my YUY2 encodes, when I feel the need to use a smoother, now I absolutely prefer FluxSmooth since it's still a lot faster than Convolution3D YUY2 version, but certainly as successful too.

As for the YV12 encodes, I sometimes use my all time favourite Convolution3D (since its YV12 version is also pretty fast imho), but I sometimes use my new favourite FluxSmooth too. It's really difficult to make a choice between the two in YV12! ;)

And that's all for now :)...

best regards
and a happy, peaceful new year to everybody,

iago

[MaTTeR]

@iago

I haven't for got you man, I'll send you a PM;)

@Bach
Many thx for the description. It's strange that not many people are seeing this artifact in YV12 encodes. The artifacts seems worse than an encode in YUY2, maybe it's just my eyes. Anyways, I'm still testing Blockbuster with _lots_ of different settinsg to see if it helps. I can't think of any other filter that might help me in this situation. To be honest the problem is so bad that it might warrant me going back to YUY2 encodes. FWIW, the artifact appears with all MPG4 codecs even at Quant 2:rolleyes:

[kilg0r3]

@matter

how about starting a new thread in the avisyth forum? i think this would be a good idea. i think i have also noticed this problem, yet, i am not entirely sure.

[MaTTeR]

Good idea kilg0r3. Perhaps others have seen the problem as well and can offer some suggestions.

Edit- Sorry to take this thread slightly OT SansGrip.

[SansGrip]

Originally posted by iago
[B]Well, of course I gain a few more fps especially when using "-p 3" with the fantastic AVS2AVI command line utility, but that's not much of a difference on my end due to limitations of my system specs ;). Through my brief benchmarks with VDub I noticed about a 15-20fps increase in the speed of the YV12 version, but then I'm on a more powerful machine.

As far as AVS2AVI goes, it does sound excellent. Wish there was something similar for MPEG-1 :).

[onesoul]

AviSource("file.avi")
FixBrokenChromaUpsampling()
separatefields()
vid_e=selecteven()
vid_o=selectodd()
vid_e=convolution3d(vid_e,0,4,4,4,4,2.8,0)
vid_o=convolution3d(vid_o,0,4,4,4,4,2.8,0)
interleave(vid_e,vid_o)
weave()


This is a script suggested by bira using convolution3d with dv pal source.
the usage of fluxsmooth will be similar as the usage of convolution3d? or could it be something like:
...
separatefields()
fluxsmooth()
weave()

I don't really understand why to selecteven and odd and applying the filter separately.

Thanks for the patience

[FuPP]

Originally posted by SansGrip
@FuPP

Could you try:


LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
fluxsmooth(15,8)
and so on, to eliminate some of the other filters from the equation?

Sorry for that delay :( but I've been quite busy these last weeks...

LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
fluxsmooth(15,8)

-> Works

LoadPlugin("C:\video\avsfilters\MPEG2Dec3yv12.dll")
LoadPlugin("C:\video\avsfilters\fluxsmoothyv12.dll")
mpeg2source("F:\test\vts_01.d2v",cpu=0,idct=2)
Crop(32,80,656,416)
fluxsmooth(15,8)
BicubicResize(448,320,0,0.6)
AddBorders(16,128,16,128)

-> doesn't work

Seems to be same kind of problem than the one I describe in the Dup thread.

Regards,

FuPP

avysynth 2.5 03/01
xp 1800+

[Asmodian]

@onesoul
I think using selecteven and odd and applying the filter separately is a good idea because convolution3d uses temporal information and each field bobs up and down by .5 (or 1?) pixels and one doesn't want to use temporal information from the wrong pixel.

[onesoul]

@Asmodian
Thank you for the reply. I was told this by sh0dan at the "ntsc to pal conversion again" thread. It is going on there some questions about the reason for the second field being lower 0.5 line.

[SansGrip]

Originally posted by FuPP
Seems to be same kind of problem than the one I describe in the Dup thread.

Just to narrow it down even more, could you try a ConvertToYUY2 before running Flux? That way I know if it's specific to a colourspace or if it's a more general problem.

[FuPP]

Did it.

Get now an access violation and no image at all :cool::D

FuPP.

[Guest]

Make a local version of MakeWritable() and comment out the part that tests IsWritable() and returns. That's what fixed the problem for Dup. I've asked sh0dan to comment.

[SansGrip]

Originally posted by FuPP
Get now an access violation and no image at all :cool::D Ouch! New release later today ;).

[SansGrip]

Originally posted by neuron2
Make a local version of MakeWritable() and comment out the part that tests IsWritable() and returns. Thanks for the suggestion and code, but I'm not using MakeWritable in Flux -- I just create a new frame. But if both YUY2 and YV12 versions cause the problem then the bug must be in the startup code somewhere, which shouldn't take too long to find assuming I can duplicate it :).

[FuPP]

Originally posted by SansGrip
But if both YUY2 and YV12 versions cause the problem then the bug must be in the startup code somewhere

My tests were only with yv12 version. But you probably mean that putting converttoyuy2 switch on same code than yuy2 version ?

[SansGrip]

Originally posted by FuPP
My tests were only with yv12 version. But you probably mean that putting converttoyuy2 switch on same code than yuy2 version ? Yep, the same code is used for the YUY2 processing in both 2.0 and 2.5 versions of FluxSmooth.

I'm having a hard time tracking this problem down. It also occurs in the 2.0 build, but only when Flux is in-between a crop and a resize. This (and the video corruption) would suggest a row size/pitch error somewhere, but I can't for the life of me find it.

Either that or I'm hitting the stack somewhere and it's causing the resize filter to misbehave...

[sh0dan]

My same question goes to you, as to Donald - are you reading pitches _before_ you make your image writable - if yes, then we've got the cause.

[SansGrip]

I've commented out basically everything except the frame fetching and allocation, and I still receive an access violation. Here's the code that's running:


PVideoFrame __stdcall FluxSmooth_YUY2::GetFrame(int n, IScriptEnvironment* env)
{
assert(n >= 0 && n < vi.num_frames);
assert(env);

PVideoFrame currf = child->GetFrame(n, env);
assert(currf);
if(n == 0 || n == vi.num_frames - 1)
return currf;
const BYTE* currp = currf->GetReadPtr();

const int src_pitch = currf->GetPitch(), row_size = currf->GetRowSize(),
height = currf->GetHeight();

PVideoFrame destf = env->NewVideoFrame(vi);
assert(destf);
BYTE* destp = destf->GetWritePtr();
assert(destp);

const int dst_pitch = destf->GetPitch();

return destf;
}


No assertions fail. When I return currf instead of destf, the access violation is gone :confused:.

[SansGrip]

Forgot to mention, I'm testing with 2.07.

[sh0dan]

And this happends in both debug and release builds?

Edit: What happends with:

PVideoFrame __stdcall FluxSmooth_YUY2::GetFrame(int n, IScriptEnvironment* env)
{
PVideoFrame destf = env->NewVideoFrame(vi);
assert(destf);
BYTE* destp = destf->GetWritePtr();
assert(destp);
const int dst_pitch = destf->GetPitch();
return destf;
}

Edit2: Are you using avisynth.h from the 2.07 release (shouldn't matter, but worth a shot).

[SansGrip]

Originally posted by sh0dan
And this happends in both debug and release builds? Yep.

Edit: What happends with:

PVideoFrame __stdcall FluxSmooth_YUY2::GetFrame(int n, IScriptEnvironment* env)
{
PVideoFrame destf = env->NewVideoFrame(vi);
assert(destf);
BYTE* destp = destf->GetWritePtr();
assert(destp);
const int dst_pitch = destf->GetPitch();
return destf;
}
With the above code, access violation is gone. Green image, of course.

Edit2: Are you using avisynth.h from the 2.07 release (shouldn't matter, but worth a shot). I'm 99% sure. I'll double-check.

[SansGrip]

I just recompiled with the avisynth.h from 2.07 and the result is the same: access violation if I fetch the current frame and return the newly created frame, no access violation if I return the current frame.

[sh0dan]

Do you have a complete source (If you can't put it on a web-page, mail it to bingo (at) bongo.zz

[SansGrip]

Originally posted by sh0dan
Do you have a complete source (If you can't put it on a web-page, mail it to sh0dan (at) stofanet.dk The source code for 0.4 is available here. All I did to test was comment out everything in the FluxSmooth_YUY2::GetFrame method except for what I posted above.

Let me know if you need anything else (or happen to spot any gross errors/inefficiencies ;)).

[wing1]

@sansGrip

fluxsmooth is simply grand :D No mblock and fast! Great work.

[SansGrip]

Originally posted by wing1
fluxsmooth is simply grand :D No mblock and fast! Great work. Thanks :).

[sh0dan]

Originally posted by SansGrip
I've commented out basically everything except the frame fetching and allocation, and I still receive an access violation. Here's the code that's running:
[...]


It compiles and runs without any problems here, when I insert the code.

Could this be a VC7 issue (which I can see you use)? Is it possible for you to test on a VC6 SP5?

[SansGrip]

Originally posted by sh0dan
It compiles and runs without any problems here, when I insert the code. Strange.

Could this be a VC7 issue (which I can see you use)? Is it possible for you to test on a VC6 SP5? I suppose it could be an issue with VC7, but it strikes me as unlikely -- we're not really using any esoteric features here :). I shall install VC6 on my laptop and try it there.

[sh0dan]

Just attached the compiled version with the code above for you to test.
No crashes on WinXP here.

I also included the sources I modified and project/workspace for VC6. Give me a sound when you have tried it, so I can remove the binary



The script I used: (with latest 2.07 binary)
----
loadplugin("debug\fluxsmooth.dll")
colorbars(512,512)
converttoyuy2()
crop(48,8,-12,-20)
FluxSmooth()
----


Edit: Attachment deleted - obsolete!

[SansGrip]

Originally posted by sh0dan
The script I used I too get no crashes with that script. But try replacing ColorBars with an Mpeg2Source, and adding a resize after Flux. That's what does it for me.

Doesn't crash:


ColorBars(512, 512)
ConvertToYUY2()
Crop(48, 8, -12, -20)
FluxSmooth()

-- or --

ColorBars(512, 512)
ConvertToYUY2()
Crop(48, 8, -12, -20)
FluxSmooth()
BilinearResize(256, 256)

-- or --

Mpeg2Source("blah.d2v")
Crop(48, 8, -12, -20)
FluxSmooth()
Does crash:


Mpeg2Source("blah.d2v")
Crop(48, 8, -12, -20)
FluxSmooth()
BilinearResize(256, 256)
Note this is with my binary -- your attachment hasn't appeared yet.

[sh0dan]

Found it - it crashes somewhere in the vertical resizer. I'll investigate. bilinearresize(512,256) works fine (It resizes horizontal before vertical).

[sh0dan]

Bingo! Found it!

if(n == 0 || n == vi.num_frames - 1)
return currf;

is the offender!

The problem is, that you return a frame with different pitch. FilteredResizeV relies on pitch being the same for all frames, and when your filter hands down a frame with a new pitch, it reads out-of-bounds.

I'd actually say this is the fault of resize, but I could imagine that this isn't the only place the error could occur. Shouldn't we both fix the error?

It's incredible how many 1.0 bugs still surface!

[FuPP]

Is it serious doctor ? ;)

[sh0dan]

It has existed for a long time, and probably caused a lot of weird crashes. Don't know how many filters provoke the fault - probably not many, but it's nice to have it fixed!

[SansGrip]

Originally posted by sh0dan
Bingo! Found it! Good work! I would still be looking for that one next Christmas :D.


if(n == 0 || n == vi.num_frames - 1)
return currf;
So the correct fix in FluxSmooth, then, is to test n after allocating the new frame, and if it's the first or last frame then do a memcpy and return the new frame?

Shouldn't we both fix the error? I think that would be sensible -- it's quite likely that another temporal filter might do the same thing. It's an extremely subtle bug.

[SansGrip]

Mmmm... Does 2.5's BitBlt work with YV12?

[SansGrip]

Here's a new release with the bugfix and a couple of other minor changes (mainly Avisynth 2.5-related). This one's been tested pretty good so I decided to bump the version up to 1.0, making this the first "stable" release :).

As usual, let me know if there are any problems.

[SansGrip]

@sh0dan

Boulder found yet another strangeness with Flux and the latest 2.5 (Jan 9). Apparently this:


LoadPlugin("c:\avs25\fluxsmooth-2.5.dll")
AVISource("c:\temp\leffat\labyrinth\labyrinth.avi")
FluxSmooth()
causes an access violation when seeking, which is fixed by converting to YUY2 before FluxSmooth.

This would indicate a problem in the YV12 code, but the strange thing is that I can't duplicate this, and the problem goes away for him when using the Jan 3 release.

Do you have any ideas?

[sh0dan]

Download latest - I included a far too old cache version, causing crashes. Jan 10.th should fix this. Jan 9 is EVIL! :devil:

[Boulder]

Funny that it only affected FluxSmooth - for example UnDot, Blockbuster and DCTFilter were not affected. Is this because SansGrip compiled Flux v1.0 with the new AVS2.5a-recommended thingies? Those three others filters are somewhat older releases.

[sh0dan]

Many other (also internal) filters were also affected

[wing1]

@SansGrip

I am currently using Jan.3 version of 2.5 along with your v1.0; I am also experiencing the seek problem which is causing access error. You can produce this problem by quickly seek forward by a huge jump and immediately seek backward by the same jump. v0.4 does not exhibit this problem. Furthermore, I am seeing smearing with the same settings that I am using with v0.4.

avisource("e:\capture001.avi",false).converttoyv12()
coloryuv(1,1,1,1, 1,1,1,1, 1,1,1,1, "TV->PC", "","")
unfilter(-10,-10)
cnr2()
lumafilter(-5,1.02)
fluxsmooth(9,3)
bicubicresize(576,432,-0.5,0.75)
asharp()
dctfilter(1,1,1,1,1,0.7,0.5,0)

[SansGrip]

Originally posted by wing1
[B]I am also experiencing the seek problem which is causing access error. You can produce this problem by quickly seek forward by a huge jump and immediately seek backward by the same jump. I tried seeking all over the file (big jumps and small jumps) and can't get it to crash, even with the evil Jan 9 build :confused:. Can you post a barebones script with as few filters as possible, i.e. just enough to get Flux to crash?

Furthermore, I am seeing smearing with the same settings that I am using with v0.4. Would you be able to post (or email to me -- ross@grunfunuty.com -- change all U's to I's) two grabs of the same frame, one from 0.4 and one from 1.0?

I don't understand why this would be the case, because none of the processing code was changed between 0.4 and 1.0. The only change was to fix the access violation when used in combination with a resizer, and I made a couple of changes to the support code to come into line with the latest 2.5 requirements.

I hate bugs I can't duplicate :(.

[SansGrip]

Originally posted by Boulder
[B]Funny that it only affected FluxSmooth - for example UnDot, Blockbuster and DCTFilter were not affected. Could well be because I was calling SetCacheHints(CACHE_RANGE, 1). Those other filters do not (AFAIK) request frames other than the current one.

[wing1]

@sansgrip

Here is the compare between the two fluxsmooth versions running on Jan. 03, 2002 Avisynth 2.5 with the following script:

import("c:\avisynth2\yv12\fluxsmooth-0.4.avs")
#import("c:\avisynth2\yv12\fluxsmooth-1.0.avs")
AviSource("e:\capture\capture002.avi",false).converttoyv12()
colorYUV(1,18,0.7,-54,4,2,-10,-30,-4,4,-10,-54,"PC->TV","coring","")
unfilter(-5,-5).Cnr2("xxx",7,29,192,47,255,47,255,false)
lumafilter(-5,1.02).fluxsmooth(9,3).undot()
bicubicresize(640,448,-0.5,0.75).asharp(2.25,5.6,-1,true)
dctfilter(1,1,1,0.98,0.9,0.44,0.18,0)

The source is captured from Network TV using mpeg4vki. Pictures are taken from vdub1.4.13 (5 frames) at the same location. Look at the cane and you will see slight detail loss due to smearing.

btw..I can't seem to reproduce the seek failure either...this is strange???

[bond]

Yup I know i am very late for beginning to look at fluxsmooth but today i did it :D

first i used avscompare to look which fluxsmooth settings will look like c3d's hq preset -> for me it seems that this will be between 5,4 and 4,3 (ok i just compared a small sample ~7000frames)

then i went on to compare the speed between c3d(hq) and flux(4,3) [together with xvid] -> surprisingly this came out:

FluxSmooth(4,3)_28:09_4.2fps
Convolution3D("movieHQ")_29:11_4,1fps

so i thought that there has to be a fault anywhere because everybody recommended fluxsmooth because of its speed gain compared to c3d!

my .avs script:

LoadPlugin("C:\PROGRA~1\...\PLUGINS\mpegdecoder.dll")
mpegsource("C:\...\movie.d2v")
trim(136757,143873)
crop(2,80,716,418)
i put the noise filter here...
BicubicResize(640,256,0,0.5)

i always used the latest plugin available!
i have a p-3, 866mhz; 128mb ram (yeah i know...)

[Boulder]

IIRC, if the source clip is noisy, FluxSmooth will run slower..so there's a noise <-> speed -relation.

[bond]

Originally posted by Boulder
IIRC, if the source clip is noisy, FluxSmooth will run slower..so there's a noise <-> speed -relation. thanks for your answer!

my source was the matrix dvd which shouldnt have much noise i think (i always use the same sample as doom9 in his codec comparison for testing) :(

[Boulder]

Actually, Flux is not *that* fast compared to C3D:

http://forum.doom9.org/showthread.php?s=&threadid=51181

Just remembered this good old thread. Forget what I said earlier, I'm not nearly 100% sure it was Flux that slowed down with more noise:scared:

[bond]

:(
as c3d is really slow i thought about a speed increase of 33% or so :D

as i am in filter testing mood at the moment any more filters which can be recommended for increasing compressibility whereas retaining as much quality as possible (yes i hate such questions too ;) )?

[Boulder]

TemporalCleaner (by Vlad59) is a must. The defaults give a huge compression increase and it's very fast too. It's a regular one in my scripts:)

Edit: UnDot is a good one too. Doesn't increase encoding time almost at all and yet gives more compression without losing details. I don't see why TemporalCleaner didn't score better in the tests in that thread. I often get more than 10% off the size of the sample clip encode (analog captures).

[bond]

will give it a try (i already tested undot, seems to be very nice)

thanks a lot for your help!

[JohnMK]

Boulder,

Which UnDot() parameters do you use with recent, clean hollywood DVDs such as say, Matrix?

[Boulder]

Originally posted by JohnMK
Boulder,

Which UnDot() parameters do you use with recent, clean hollywood DVDs such as say, Matrix?

The nice thing about UnDot is that it doesn't have any parameters! ;)

[bilu]

About FluxSmooth and C3D: Flux is faster on AVS 2.0x and C3D is faster on AVS 2.5x on my system. Haven't tried AVS 2.5x with YUY2 though.

I only use Deen("a3d",1,10,12) now.

Bilu

[bond]

Originally posted by bilu
I only use Deen("a3d",1,10,12) now.
hm these settings seem to blur too much...

i think i will stay with TemporalCleaner(3,6) and UnDot (before TC and used only once, doesnt seem to make a difference if used twice)

hm pretty much the same as boulder suggested :)

[JReiginsei]

In the C3d Readme it says this for the Avisynth 2.5 version:

Know problem :
- works only with YV12
- Temporal influence currently disabled.

Since the temporal influence is disabled, thats why its faster than the C3d for Avisynth 2.0x, right?

[Boulder]

Originally posted by bond
i think i will stay with TemporalCleaner(3,6) and UnDot (before TC and used only once, doesnt seem to make a difference if used twice)


Lately I've used FluxSmooth with temporal processing disabled and then TC after that for my analog TV caps. I haven't tested how much difference it would make to use TemporalCleaner alone but I believe that slight spatial processing won't hurt, especially when SansGrip told that the filter keeps the details quite well with low thresholds.

This is the filtering I do:


UnDot()
FluxSmooth(-1,7)
TemporalCleaner()


I compared the compressibility a while ago and found out that a spatial Flux with TemporalCleaner compress *a lot* better than FluxSmooth with high temporal thresholds and with no TemporalCleaner. FluxSmooth(15,7) resulted in ~5% larger filesize than that filtering I wrote above. I'm also sure that a temporal threshold of 15 could cause some ghosting easily.

[bond]

Originally posted by Boulder
Lately I've used FluxSmooth with temporal processing disabled and then TC after that for my analog TV caps. I haven't tested how much difference it would make to use TemporalCleaner alone but I believe that slight spatial processing won't hurt, especially when SansGrip told that the filter keeps the details quite well with low thresholds.that would cause a drop in speed again, i think?
can it cause problems if i dont do spatial filtering (on clean dvd sources) only temporalcleaner?
is it possible to say which one compresses better, spatial or temporal? and which one is better to keep details? (of course that also depends on the settings)

[bilu]

I used TemporalCleaner before Deen("a3d") but not anymore, on quick panning scenes TemporalCleaner did ghosting even with settings like

TemporalCleaner (ythresh=5, cthresh=5)

which are lower than defaults.

The movie that made me stop using it was The Abyss. I was making tests over the second chapter, when a sub gets on fire after hitting a rock.
It has some scenes with fire plus a earth-quake like panning. And some parts of the scene had very few colors, like a guy screaming something in the dark behind a sort of shower device during the fire. Those parts got a lot damaged even using TC alone.

Deen("a3d",1,10,12) is the default of Deen("a3d"), with the same spatial and temporal settings as Deen("c3d"). But it seems a lot more cleaned, but not blurred, almost like being seen through some sort of mask.

The main reason I stopped using Flux in 2.5 was the speed, it was slower than C3D or Deen, while in 2.07 was faster.


Bilu

[Boulder]

Originally posted by bond
that would cause a drop in speed again, i think?
can it cause problems if i dont do spatial filtering (on clean dvd sources) only temporalcleaner?
is it possible to say which one compresses better, spatial or temporal? and which one is better to keep details? (of course that also depends on the settings)

The speed decrease is small but noticable. However, I like the extra compression so I don't mind;)

Usually temporal filtering compresses better IMO and keeps details better too. Excessive spatial filtering will kill details and excessive temporal filtering causes ghosting. FluxSmooth's default parameters should keep the details intact, I think SansGrip had that in mind when he decided them.

@bilu: I don't mind slight ghosting as I do MPEG-1 encodes and view them on my TV. I can't notice any ghosting with the default TC parameters myself.:)

[bond]

hm i searched the forum for a defintion of "ghosting" but didnt find a good one (seems that everyone who suffers ghosting knows that it is ghosting when he sees it)
how does ghosting look like? can i find a screenshot somewhere?

[Boulder]

You can usually notice ghosting when there's a dark background and someone's moving in front of it. Encode such a scene with a high temporal threshold and see if any ghosting appears.

[High Speed Dubb]

“Ghosting” (as used on this forum) means that part of an earlier or later frame shows up in the current frame. It’s usually caused by a temporal smoother with overly permissive parameters.

You do need to be careful using the term, though. “Ghosting” also has a different meaning with broadcast video -- It refers to a spatially displaced image superimposed on the main signal.

[SansGrip]

I just threw together a new version of FluxSmooth, in case anyone other than me is still using it ;).

You can get it here (http://www.indeus.com/sansgrip/avisynth/). The source code is available, as usual.

Major changes:

* Improved noise reduction.
* Split into two different filters, FluxSmoothST and FluxSmoothT. The former, as with previous versions, does (by default) both temporal and spatial filtering. The latter does only temporal filtering, and is about 50% faster.
* Removed Avisynth 2.0x version.

I would be grateful for feedback on the (slightly) improved noise reduction and the (greatly) enhanced speed of FluxSmoothT.

Here's a snippet from the readme:

Changed the averaging code so that the current pixel is excluded, which produces better noise reduction. Also split the code into two different filters, FluxSmoothT and FluxSmoothST. The former does temporal-only smoothing (equivalent to setting "spatial_threshold=-1" in FluxSmoothST) and is about 50% faster. Removed Avisynth 2.0x version to tidy up the code base. Does anyone actually use it any more? My thanks to fabrice and sh0dan for the 1.01 release during my extended absence :).

[Wilbert]

Sorry for the OT post, but great to see you back!

[SansGrip]

Thanks :).

I have a couple of ideas for filters, hopefully I'll have the time to implement them. I'm very intrigued by the motion estimation toolkit. Need to read more about it, though.

[yaz]

heya sansgrip ! welcome back !! :-)))

Originally posted by SansGrip
I just threw together a new version of FluxSmooth, in case anyone other than me is still using it ;)i wouldn't take even a single step without :-))Originally posted by SansGrip
... FluxSmoothST and FluxSmoothT ... The latter does only temporal filtering, and is about 50% faster.whoops ... it'd never been so slow in 'temp-only' mode. btw, are you aware of sh0dan's improvements made on 'flux' in the meantime ? are that changes kept or left ?Originally posted by SansGrip
I would be grateful for feedback on the (slightly) improved noise reduction and the (greatly) enhanced speed of FluxSmoothT. i'm about it :-)
thx
y

[SansGrip]

Originally posted by yaz
heya sansgrip ! welcome back !! :-)))

Thanks :).

it'd never been so slow in 'temp-only' mode.
I'm not sure what you mean here. Flux was never really slow (I get about 55fps on my 2100+ XP), but since I always disable spatial smoothing anyway I thought I'd try commenting it out of the code. It ran at over 75fps, so I figured it was worth keeping a temporal-only version :).

btw, are you aware of sh0dan's improvements made on 'flux' in the meantime ? are that changes kept or left ?
fabrice's memory leak fix and sh0dan's speed improvements are still in this new version.

[krieger2005]

Hi

can FluxSmooth only one time used in a script? I used this Script:

a=FluxSmoothST(7,22)
b=FluxSmoothST(7,7)
return subtract(a,b).colorYUV(autogain=true)


and get a grey screen. When i use a or b in the script the look different.

greets
krieger

[SansGrip]

Um... I see no reason why it couldn't be used twice. What does that script meant to do, out of interest?

And are you running it in YUY2 mode? If so, that's definitely suboptimal. Flux is twice the speed in YV12.

[krieger2005]

I used the Script in YV12. First i used the Filter twice in the script to see the Difference of spatial smoothing:
Interleave(FluxSmoothST,FluxSmoothST(7,22))

But i does not see any differences. I tured 22 to 53 and there were still no diferences (with the script above). Then i used FluxSmoothST only one time: with parameter (7,7) and (7,22) and i could notice differences. The i tried to use the both again and had the Result of one of them (the first of them).

Then i posted this problem here...

When no one has this problem, that maybe i does something wrong and must search...

krieger

[SansGrip]

That's odd. I have no idea what could be causing that problem. I'll try to duplicate it.

[ARDA]

First of all welcome back.
quote:
----------------------------------------------------------------------------------------
That's odd. I have no idea what could be causing that problem. I'll try to duplicate it.
-----------------------------------------------------------------------------------------
I must confess I did't do any test; but I think the following static variables
are the problem .You should change them for just constant. But you won't be able use ebx
register (at least with c++ 6.0 )in your DoFilter_MMX assembler code.

__declspec(align(16)) static const __int64
#ifdef DO_SPATIAL
spat_thresh = ((__int64)spatial_threshold << 48) |
((__int64)spatial_threshold << 32) |
((__int64)spatial_threshold << 16) |
(__int64)spatial_threshold,
#endif
temp_thresh = ((__int64)temporal_threshold << 48) |
((__int64)temporal_threshold << 32) |
((__int64)temporal_threshold << 16) |
(__int64)temporal_threshold,

I hope that can be usefull .ARDA

[Leak]

Originally posted by ARDA
But you won't be able use ebx register (at least with c++ 6.0 ) in your DoFilter_MMX assembler code.

That's strange - the VC++ 6.0 docs state:

When using __asm to write assembly language in C/C++ functions, you don't need to preserve the EAX, EBX, ECX, EDX, ESI, or EDI registers.

so why wouldn't you be able to use ebx?

Works for me...

np: T.Raumschmiere - Substrom (Anti)

[ARDA]

Maybe my visual c++ has a problem; but if I declare a non static variable in a void function (as DoFilter_MMX of fluxsmooth is)I always get a warning when ebx is used in assembler code inside this function; and in all cases I get a crash.
Forgive in advance if that is not a general rule; but it always happens to me.

Thanks ARDA.

[Leak]

Originally posted by ARDA
Maybe my visual c++ has a problem; but if I declare a non static variable in a void function (as DoFilter_MMX of fluxsmooth is)I always get a warning when ebx is used in assembler code inside this function; and in all cases I get a crash.

That's strange... could you try compiling my BlendBob plugin (it's just one source file + 2 header files and the AviSynth.h) on your system? The function LumaDifferenceHistogram_MMX makes extensive use of ebx and works just fine. Also, if you push ebx at the beginning of your asm blocks and pop it at the end it really shouldn't be a problem; after all, the compiler can't make use of it inside your asm block... :confused:

It's probably a compile setting or something that causes your problems - I must confess that I've stolen the compiler settings I use from Donald's Decomb... :D

[ARDA]

You're right; no problem at all to compile you plugin.By the way it looks a good job at first impression,still didn't test.
There is no difference in settings,the problem is probably in the structure of some of my tests;in many cases I have all classes functions in same cpp without headers and extern definitions.
I'll study that more and post maybe in another moment not to be off topic.
Coming back to fluxsmooth still think the static variables (compilation time) are the problem for several calls.

Thanks.ARDA

[SansGrip]

Yes, I imagine that's the problem. I always forget only one instance is created. I made those static (at about 3am ;)) because of the compile problem you mentioned wrt ebx, but it didn't occur to me that it might screw something up.

I'll make a new release, probably tonight.

Thanks :).

[SansGrip]

New release, 1.1a:

Yet another "oops" release. Current pixel is once again considered in the averaging code -- I found the lack of it too aggressive, especially during fast motion. Also fixed stupid "3am bug" involving a couple of variables I'd declared static that shouldn't've been. Thanks to krieger2005 for spotting that one, and ARDA for diagnosing it.

Available here (http://www.indeus.com/sansgrip/avisynth/).

[Dali Lama]

Hi SansGrip,

Nice to see you here again. Yes, I have been using FluxSmooth, and interestingly enough, I only use its temporal portion. Thanks for improving the noise reduction and making a fast temporal version (as if the original wasn't fast enough ;) )

In a quick comparison, I cannot see much difference in noise removal from v1 to v1.1a in anime material.

Oh and make sure you check out kurosu's MVDenoise. Its a very good and relatively fast motion-compensated temporal noise remover.

Thanks,

Dali

[SansGrip]

Originally posted by Dali Lama
In a quick comparison, I cannot see much difference in noise removal from v1 to v1.1a in anime material.
That's good -- there shouldn't be any ;). The "improved noise reduction" was flawed and I removed it from the most recent release. The only benefit of using 1.1a is that temporal-only smoothing is much faster with FluxSmoothT().

Oh and make sure you check out kurosu's MVDenoise. Its a very good and relatively fast motion-compensated temporal noise remover.
I will -- thanks.

[Fizick]

Kurosu is a great man,
but I think, Manao is the author of MVDenoise
Or there are a lot MVDenoises ? :)

[Dali Lama]

You're right Fizick. Sorry Manao.

Thanks for that clarification SansGrip

-Dali

[Boulder]

SansGrip,

FluxSmoothST seems to get very slow with CCE if the filter is placed before resizing. I've got a 720x480 AVI clip and I tried to make a compressibility test with QCCE, the script is like this:


AVISource("c:\temp\test.avi",false)
FluxSmoothST(temporal_threshold=-1,spatial_threshold=7)
BicubicResize(656,304,0,0.6)
AddBorders(24,136,24,136)
ConverttoYUY2()
AssumeFPS(25.000)
SelectRangeEvery(500,15)


1) When I try to encode this with CCE, it takes a very long time. If I load this in VDub, it's not at all that slow. If I then move FluxSmoothST right after resize, CCE is as fast as ever, no slowdowns whatsoever.

2) If I add the line Crop(2,2,636,268,align=true) or with align=false, right after the AVISource line, there is no slowdown.

3) If I encode the same script without the sampling line (SelectRangeEvery), encoding time is almost the same with Flux before or after the resizing part.

I also remember having this problem with the earlier versions as well. I also tested it on several AVI clips and they all give similar results.

[sapient]

Where can I download the latest version... The link a few posts above is dead...

[Wilbert]

http://www.avisynth.org/warpenterprises/

[nomonhan]

The link
http://www.avisynth.org/warpenterprises/
is dead

[Reel.Deel]

The link
http://www.avisynth.org/warpenterprises/
is dead

http://avisynth.nl/index.php/FluxSmooth

Also the first link in Google search.