Peach Smoother

[jarthel]

It's an anime so there must be some still scenes in it. any other idea?

thanks

jayel

[High Speed Dubb]

Could you help with a few diagnostics?

There’s an easy way to tell if the clip has enough still material to estimate the noise. Just use the Dot option. If the dot doesn’t show up in still scenes, then the sequence has too much motion to figure out the noise.

Could you say what you mean by trembling? Do you mean that there appears to be noise behind moving objects? That edges of objects or macroblock boundaries don’t have as stable a color as their centers? That there is blurring within objects, or behind moving objects? Or that the whole scene appears to shake?

Also, was your clip compressed before you processed it? Recorded on a VCR? Is it fairly dark?

[soulfx]

One thing I really like about this filter is it's ability to pick up on motion and edges in motion and not temporal filter them. One big problem I had in the past was whenever a scene went into motion the temporal filters I was using caused the edges to disapear and/or have really strange temporal smoothing effects.

This extra ability to pick up on motion and edges better has helped out a ton, but it also causes it not to filter the edges, thus causing them to "shimmer". The one way I found that cuts down on the this is to keep NoiseLevel and Baseline values higher as auto Estimation seems to estimate too low of values for my taste.

High Speed Dubb,
Would it be possible to add a feature that shows what the filter is detecting as motion?

[High Speed Dubb]

Just to be clear — Are you seeing the shimmer on the edge of moving objects, or on the edge of stationary objects? Those are two different problems.

If it’s the former, then you’re seeing what I call the “noise shadow” — the area in which the filter avoids averaging because of recent or nearby motion. Good news is that this will be much subtler with the next release.

If it’s the latter, then it’s color crosstalk, and what you really need is a comb filter.

Yes, it would be possible to output the motion estimate. What kind of format would be useful for it? Internally it’s an array of Probability(motion), where probability is measured in 0x10000 fixed point.

[soulfx]

The shimmer appears mostly with movement. I don't really mind the shimmer as you mentioned it avoids averageing it because of recent or nearby motion, which in terms of noise reduction is good, because if it did it probably would wipe that edge out.

If the motion estimate could be figured as a percentage of motion in the current frame and output that way it could help determine which frames it's detecting as having high amounts of motion and which frames its seeing as low motion. Really anything, even the probablity number could be output, as something more then nothing would be a big help.

Maybe the motion detection could also be output as an overlay over the image. Have it display black wherever it detects motion. That might be a bit more involved then just outputting a number though.

[jarthel]

My problem is the same with solfx. The trembling/shimmering is on the edge like outline of the characters. And it only happens on movements. I'll try your suggestion of "dot=true".

Also my source are dvd vobs.

jayel

[High Speed Dubb]

Thanks to jarthel and soulfx for narrowing down the problem. I agree — that’s the most distracting aspect of the filter. But as soulfx noticed, it’s there for a reason.

Since version 1.0 is aimed specifically at this problem, I’ve put together a release. It may or may not help with MPEG2 material — Hopefully MPEG2 encoding doesn’t introduce any artificial motion.

http://students.washington.edu/ldubb/computer/PeachSmoother_v1_0a.zip

Version 1.0a reduces the appearance of noise on the edge of moving objects. It’s done through a couple tricks which let me cut down on the noise without blurring the edges. One trick is that the spatial smoothing is much more specific to moving areas than before, and can now safely be used at higher levels without messing up stationary parts of the picture.

The other is the result of some analysis of how the expected change at a pixel is altered by past averaging at that location. Areas which had been moving but now appear to be still are expected to have a larger variance than areas which have been still for a while.

Also, the formula for calculating the probability of motion got a small tweak to more closely resemble the calculated optimum.

The name of the estimates option has been changed to readout. That’s because I kept forgetting the old name.

Because of the algorithmic changes, you’ll find that you can drop your v0.9 Noise Reduction setting by about 10 points. Also, the noise readout got a small recalibration, which means that your NoiseLevel and Baseline estimates should be multiplied by 0.9192 before use with v1.0a

[jarthel]

will test later. How about SSE2? :)

thank you Lindsay. :)

jayel

[manono]

You don't need SSE2 optimizations. Your computer's too fast already.:)

[jarthel]

SSE2 optimizations if implemented by Lindsay is free. so why not? :)

jayel

[High Speed Dubb]

No, SSE2 isn’t on the slate. It would be complicated. And it would probably lead to artifacts unless each 8 pixel block were broken up into two 4 pixel blocks, which would toss away any speed gain.

My time is free for you, but has a cost. I’m a poor low level optimiser, but good with algorithms. You’ll get more useful stuff from me if I don’t waste my time with tweaking assembler instructions.

Besides, if you want speed, you’re much better off with the Grape.

[jarthel]

I didn't realize that SSE2 optimizations is lot of work. So I apologize for my comments. :)
-----------

Regarding the latest version, it seems to work very nicely. No more "trembling". Just a question regarding "dot=true". Are the dots random red/blue dots across the screen? Cause that's what I'm seeing.

I also tried the avs script with Xvid and it looks good too. I was using the latest stable build by Umaniacs (dated oct-2002).

Here's a filesize comparison between 3 encodes:

using Divx5 temporalsmoother(5) - around 250mb
using Divx5 PeachSmoother (NoiseReduction=50,Stability=25,Spatial=100,dot=true) - 290mb
using Xvid PeachSmoother(NoiseReduction=50,Stability=25,Spatial=100,dot=true) - 318mb

Once again thank you.

Jayel

ps. what is Grape?

[soulfx]

Grapes are yummy ^_^.

The green dot is a little 1-2 pixal dot (it took me a while to actually see it as I was looking way up in the upper left hand corner). It's about 3-4% away from the top left corners. It only shows up when there is little to no movement in the picture and is just their for debug purposes.

[High Speed Dubb]

jarthel,

Apology accepted — Yes, depending on the algorithm, SSE2 can be a fairly difficult change. And don’t even think about YV12 for this — It would probably be faster to just convert back and forth.

It’s also good to hear that the trembling is reduced — that was the main goal of the recent changes. (That and reduced spatial smoothing.) Now let’s see whether soulfx (or other people) find that edges are now blurred... I don’t think they will be, but it’s always worth checking.

Like soulfx said, Dot should just give a single tiny green dot toward the upper left of the picture. Red/blue dots sound like some kind of noise or artifact. Either that, or you turned on the debug=true option.

About the file sizes — Yeah, this wasn’t really designed for file size reduction. It’s just aimed for picture quality. Stuff like keeping low level noise is bound to increase the compressed size.

It looks like you’ve already found the Grape. ;)


soulfx,

About your earlier suggestion — I’m not sure what you mean by ‘percentage of motion.’ But a picture of Probability(motion) should still be reasonably informative (Y = Probability(moving)*256). Alternatively, I could enable a debugging setting which I’ve been using for algorithm evaluation. That flashes all pixels with Probability(motion) > threshold pink once per second.

I’ll fix up the docs on the Dot option so people won’t have to hunt for it so much.

[soulfx]

To tell you the truth I wasn't quite sure what I was trying to say about the percentage of motion either. The picture of the probability of motion sounds like a great idea and would help me and others see what the filter is actually seeing as motion.

I haven't seen any bad effects so far with edges under motion. I've got some tests to run tonight and I'll see if anything comes up.

It's sort of funny. You kept forgetting the old name of Readout (Estimates) and now I keep forgetting the new name for it.

[^^-+I4004+-^^]

didn't tested thoroughly but first tests look fine!
also some AWESOME(!) documentation you got there (Peach)

nice work!

[High Speed Dubb]

And now on to version 1.0b. This one is a minor update. It adds a new diagnostic called ShowMotion, which shows the internal motion estimate mixed in with the picture. Black is (inferred) stationary area, while white is in motion.

Also, Stability settings of less than 0 are now allowed. And I fixed a few documentation mistakes — I forgot to change the listed defaults in the help file to reflect the changes in 1.0a.

http://students.washington.edu/ldubb/computer/PeachSmoother_v1_0b.zip


It's sort of funny. You kept forgetting the old name of Readout (Estimates) and now I keep forgetting the new name for it.

Naturally. :( Well, I’ve enabled “estimates” as an undocumented synonym for readout in version 1.0b.

PS: And thanks for the compliment on the documentation. I tend to make the help file a pretty high priority, since these interfaceless filters can otherwise be pretty hard to understand.

[soulfx]

The new ShowMotion option works great. It really helps get an idea what settings to use for NoiseReduction. It also helps me figure out what frames to look at when judging spatial and temporal filtering settings.

I tried the Readout (Estimates) and it doesn't show anything anymore. Neither Readout or Estimates output any values onto the video.

[High Speed Dubb]

Very strange — I just downloaded it myself and the readout is showing up okay. I’ll take a close look at the parameter parsing there to see if I got something messed up.

I guess my preference for NoiseReduction is to use the normal output rather than the motion display. Internally, the filter uses various functions of the motion probability in its calculations — For example, it’s squared before determining the amount of motion when deciding where to apply spatial smoothing.

[soulfx]

Oops, It's my fault. I just realized why I wasn't getting any values to show up. Since Peach is faster then my older noise filters I've moved it before crop and resize. So it was cropping out the Estimates. Sorry about that :/

[High Speed Dubb]

No problem — It’s remarkably easy to get these sorts of issues confused. My first publicly available code was a port of a regular expression engine, for which about half of the seeming bugs — including those I found myself — were due to misunderstanding how regular expressions were supposed to work.

[High Speed Dubb]

Here’s a trivial update. This moves the readout a little further from the edge of the screen, so it won’t be so easily lost when cropping.

http://students.washington.edu/ldubb/computer/PeachSmoother_v1_0c.zip

[soulfx]

I was wondering if high and low NoiseLevel and BaseLine limits could be implimented. Something that would allow for auto noise determination, but would keep the filter from determining noise below or above limits set by the user.

The filters auto determination of noise works great, but there are sometimes it goes too high or too low. Like when there is smoke then the filter tends to think the smoke is part of the noise (thus raising it's determination of NoiseLevel and BaseLine above it's normal values). A High Estimate Limit would be useful in situations like this.

I've been using the filter sort of successfully for removing some crosstalk, but in order for me to do this I have to manually set the NoiseLevel and BaseLine a bit higher then what it normally auto Estimates. A Low Estimate Limit would allow it to not drop below a certain estimate value, but would allow for it to increase its Estimate if it needs to.

It's just an idea for more tweak-ability and fine tuning. Don't let it take away too much time from any work on a comb filter ;)

I can't thank you enough for all your excellent work you've done with this filter.

[jarthel]

isn't asking for more parameters make it harder for average joes to use the filter? :) There must be a line between usability and complexity. :)

jayel

[High Speed Dubb]

That’s been my opinion with the DScaler filters — All those options get confusing. That’s less of a problem with AVISynth, though, since the weird stuff can all just be ignored by people who don’t want to mess with it. I’ll probably split the settings into “important” and “esoteric” in the docs so people will know which part they need to understand.

Yeah, smoke and fog are definitely bad news for the Peach. It’s sort of fun to watch it try to cancel out fog, but I won’t claim that’s a feature. User set maximum/minimum values isn’t the most elegant way to take care of things, but it’s a possibility.

Part of the problem is that the filter was designed for DScaler, where the user may be switching back and forth between channels with different noise levels. So the filter needed to be tolerant of dramatic noise level changes. That isn’t as necessary here, so I can probably improve things some just by making it less prone to believe wild fluctuations.

[FuPP]

@Lindsey

Any chance to get Peach working with avisynth 2.5a ? That one is probably the best I've seen (I mean visual quality)

Regards

FuPP

[High Speed Dubb]

It’s just waiting for me to have the time to get the 2.5 API and port it over.

At a guess, it’ll still be faster in YUY2 than YV12, even if you need to convert the formats. Has anyone compared the YUY2 and YV12 versions of TemporalCleaner for speed? That’s probably the Peach’s closest algorithmic relative, though Peach Smoother should lose even more speed with YV12.

[SansGrip]

Originally posted by High Speed Dubb
Has anyone compared the YUY2 and YV12 versions of TemporalCleaner for speed? I haven't, but whether or not Peach will lose speed depends on how "interconnected" the different components are. If it's not possible to apply the filter to each separately then you probably won't gain any speed.

Out of interest, why do you think it would be slower?

[High Speed Dubb]

It uses Y and UV changes to jointly affect both Y and UV. So it’s pretty much a worst case for a planar format — The best thing to do (for picture quality) would probably be to sort the Y and UV values into a temporary structure, process them, and sort them back into their individual components again. That would be pretty similar to converting to and from YUY2 around the filter.

The alternative would be to do a ground up rewrite of the algorithm. Bleah. :( Peach Smoother uses a private field/frame sized structure to save cumulative evidence for motion. That could be used to allow the chroma blending to depend on the luma changes (but not vice-versa, unfortunately). The result wouln’t be quite as good as for YUY2, but it might be faster.

Then again, that might not even be faster, since it would need to access the motion evidence stucture twice. It’s a big structure, so that could mean twice as much access to memory outside the L2 cache.

[natami]

@High Speed Dubb
hello, i just got ahold of your amazing smoother a few days ago (i must have been living under a rock)...it is exactly what i was looking for, and fast!, just wanted to add my 2 cents...great job :)

[Guest]

Originally posted by High Speed Dubb
[B]The alternative would be to do a ground up rewrite of the algorithm. Bleah. :( Peach Smoother uses a private field/frame sized structure to save cumulative evidence for motion. That could be used to allow the chroma blending to depend on the luma changes (but not vice-versa, unforunately). The result wouln't be quite as good as for YUY2, but it might be faster.[b]
It's not an extensive modification to just run your inference engine on all three planes and OR (or otherwise combine) the results for your decision.

[High Speed Dubb]

That would require two passes over the data — Once to detect differences, and once to apply the result. The YUY2 version manages with one pass, so it will probably be faster.

It would be possible to simultaneously walk through the Y, U, and V planes with appropriate offsets, combine the information, and then apply that to each component. That’s the combine->process->separate method I was suggesting. My hunch is that -> YUY2 -> process in YUY2 -> YV12 would often be faster, since combining and separating would mean that three times as much data would be accessed at once. Also, it would take a lot more register shuffling.

Beside, I don’t want to maintain two very different versions of the code. That would be icky.

[SansGrip]

Originally posted by High Speed Dubb
It uses Y and UV changes to jointly affect both Y and UV. So it’s pretty much a worst case for a planar format Yep. You'll at least have to maintain a bunch of parallel pointers and that's a pain in the neck.

The best thing to do (for picture quality) would probably be to sort the Y and UV values into a temporary structure, process them, and sort them back into their individual components again. That would be pretty similar to converting to and from YUY2 around the filter. Well, except that the converstion filters are already MMX optimized :).

The alternative would be to do a ground up rewrite of the algorithm. Bleah. :( Ouch. Definitely one to try to avoid.

[FuPP]

hem, so what is your decision, doctor ? ;)

[terwin]

Will there be a compile for Avisynth 2.5 in the next time? It’s such a great filter and I can’t use it. :(
I'm also satisfied with the yuv2 version. :D

Thx terwin

[jarthel]

/me wants a avisynth2.5 version :)

[zyrill]

yes, that would be just so great! please make a 2.5 version... PLEASE!

[kilg0r3]

me2

[kaitsuburi]

*bump*

I am a Peach fan too.

-kaitsuburi

[trance]

NO AVS 2.5 version yet? I hear great things about it but can't use it since I have Avisynth 2.5...

[Aktan]

Try using LoadPluginEx plugin to use 2.0x plugins in 2.5

[kilg0r3]

could we speed processsing by feeding only luma to peach?

[zyrill]

loadpluginex doesn't work with peach... the output is completely crap...

[Wilbert]

I assume you converted to YUY2 before feeding to Peach?

[homersapien]

Originally posted by Aktan
Try using LoadPluginEx plugin to use 2.0x plugins in 2.5

LoadPluginEX causes sever problems on my systems, so here's another vote for a 2.5 version :(

[YY1020]

Is it for AviSynth 2.52??

[tempetai]

Originally posted by homersapien
LoadPluginEX causes sever problems on my systems, so here's another vote for a 2.5 version :(

Just make sure that you don't put LoadPluginEX inside avisynth's plugin directory.

[stevendm]

Peach looks very good. Does it work properly with avisynth 2.5?

Thanks

[stickboy]

See this thread:
Peach, Grape, and Guava for AVS 2.5.x (http://forum.doom9.org/showthread.php?s=&threadid=58674)