Peach Smoother

[High Speed Dubb]

Here is “Peach Smoother.” It is dedicated to an especially yummy peach I had a few weeks back.

http://students.washington.edu/ldubb/computer/PeachSmoother.zip

This is meant as a pure “does it crash” build. This is my first AVISynth plug-in, so I’d like to make sure that I haven’t done anything truly screwy.

Being a stability test, I haven’t enabled any options or included any documentation. To run it, the command is
PeachSmoother()

Basically, the Peach works by looking for good pixels and gathering orange smoke from them. When it has gathered enough orange smoke, it sprinkles that onto the bad pixels, making them better. When the Peach is seeing lots of good pixels, it gets so happy that a small green dot appears in the upper left corner of the image.

Um... Well, I’ll write real documentation once I’m sure it isn’t just crashing people’s computers.

PS: The link has been edited to point to the most recent version. (I noticed that version 0.9a was getting more new downloads than 0.9h)

PPS: Note that this requires SSE instructions (from a Pentium 3, Athlon, Celeron beginning with the Tualatins (~1.2 GHz), or anything more recent).

[sh0dan]

LOL - who needs more documentation than that? :rolleyes: :sly:

It works great, and seems to be a very good temporal smoother, much better than the built in.

The first frame is very often trashed, with strange artifacts, but that's the only problem I've seen. (Try opening in VDubAVS and refresh (F5) at at frame).

This combination does wonders for my noisy DV sourcePeachSmoother()
Convolution3d (preset="movieLQ")
lanczosresize(448,336)

[trbarry]

I don't see any green dots or orange smoke but it does not crash my machine. ;)

I can't really tell what it does.

- Tom

[ajp]

Didn't crash my computer either and the results look very promising on the short clip I tested. Will come in real handy for my DV caps from TV, keep up the good work!

When can we expect a version with parameters? :)

EDIT: I can see some "smearing" or ghosting, not sure how to explain it. Very nice for a testversion though... ;)

[vlad59]

I got no crash too!

But I got a strange effect on a dark blue background which has been transformed in some parts in pure black.

I can send png images if you like !

[High Speed Dubb]

Yay! No crash is a good sign, I think. Thank you for testing it.

My apologies to people with pre-SSE computers. I forgot to post that SSE instructions (from a P3, P4, Athlon, or recent Celerons) are necessary in order to run it. It does check for processor capabilities, so this shouldn’t have caused any crashes. But that would have saved some unnecessary downloads.

The Peach is mostly a temporal smoother, though it falls back on spatial smoothing in moving areas if the signal is very noisy. It’s a port of the “adaptive noise” filter from DScaler, though it has a few extra changes (mainly the spatial smoothing).

I’ll mostly leave the picture quality issues for later, once I’ve got parameter entry and the docs together. The main trick with this filter is patience. It blurs too much until the green dot shows up. You know, not enough orange smoke. ;)

Sh0dan,
I’m not sure about the first frame corruption. That sounds a lot like an uninitialized buffer — but double checking for that looks like everything is initialized properly. By design the first frame should be left through unaltered. Also, I haven’t been able to get a corrupted first frame on my computer. Could you rephrase your description of how to see it? There’s a typo in your explanation which makes it confusing.

[trbarry]

Lindsey -

I've recently taken to using DScaler's Gradual Noise filter on some of my Tivo channels. What is the difference between the Gradual Noise filter and (more Peachy?) Temporal Noise filter?

- Tom

[High Speed Dubb]

Temporal noise averages (75% previous field, 25% this field) so long as the difference is below a threshold. It also has some rounding glitches.

Gradual noise averages between the previous field and the current field using a weighting determined by the local sum of differences — The closer the pixels are, the more weight is placed on the old value at that pixel. It fixes the rounding glitches. (But there’s a small bug in the fdshow version which reintroduced some rounding problems.)

The Peach/adaptive noise works similarly to gradual noise, but uses a more effective scheme to detect motion, making use of temporal and spatial correlation in the motion. It also figures out the appropriate weighting scheme by estimating the noise (aka orange smoke) in the picture.

That’s why I was able to get away with posting a parameterless version — the same setting is designed to work regardless of the noise. But it does make sense to tweak depending on the amount of detail in the picture — The posted version has reasonable parameters for animation, since that seems to be popular in this group.

The FLT_GradualNoise.c and FLT_AdaptiveNoise.txt files have a fair amount of detail about the algorithms. They’re at
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/deinterlace/DScaler/Plugins/

Parameter input seems to be working. Now I just need to write up some documentation...

[Guest]

>Now I just need to write up some documentation...

Documenting is my favorite part of a project. It is so much easier than coding, testing, debugging.

Don't you agree? :p

[High Speed Dubb]

Well, there are definitely some programs for which the documentation is more complicated than the code.

This isn’t one of those. :) I’ve now got some docs.

The new version, with accessible parameters and a few tweaks, is available from
http://students.washington.edu/ldubb/computer/PeachSmoother_v0_9c.zip

Next up is changing it to properly handle interlaced material.

PS: Edited to put up the new version, which properly checks its parameters.
PPS: Look further down in the thread for a more recent version.

[soulfx]

@High Speed Dubb: Wow, this is really awesome work. You've been able to put a single filter together for what I was trying to accomplish with three seperate filters, nice work :)

I came across a little bug, but not that big of deal. PeachSmoother seems to take any ol' last frame to compare with the current frame. Like when I jump around from a position farther ahead it remembers that frame and tries to adjust it to the frame I land on farther back. It still works for strait through encodes though.

Thanks for an awesome filter. You have my 100% support for this filter. All I do all day every day is TV captures and the biggest problem I have is noise. I'm glad to add this filter to my arsenal for ultimate noise beatdown ^_^

Peace,
SoulFX

[High Speed Dubb]

Glad it’s working well. :)

Yeah, I haven’t yet put in a check for contiguous frames. I should probably figure out about the first frame problem before doing that.

[High Speed Dubb]

Here’s a minor update which allows higher spatial smoothing values. PS:... and now turns off when jumping between frames. (It would be better to turn off just the temporal part of the algorithm, but that would be harder to write.)

http://students.washington.edu/ldubb/computer/PeachSmoother_v0_9e.zip

[High Speed Dubb]

Here’s a more significant update. This version deals properly with interlaced video, averaging only between matching fields.

http://students.washington.edu/ldubb/computer/PeachSmoother_v0_9f.zip

Also, I rescaled the spatial smoothing parameter so that it is really measured in percent of the temporal smoothing. To convert your current value to the new scale, multiply it by 1.23 (yes, really ;) ).

That leaves just a couple small tweaks before I call it version 1.

To Tom, vlad59, and sh0dan (and anyone else, of course) — Now that the filter is a little further along, I’m interested in the picture quality issues. Have the updates fixed the first frame corruption or the black on blue problem?

[vlad59]

@High Speed Dubb

The black on blue problem was solved with the 0.9e release. Gonna recheck with the 0.9f to be sure.

Your suggested settings for cartoon seems to strong for my test I've lowered a little the NoiseReduction parameter to get rid of ghosting.

Very good work.

[JohnMK]

I'm very impressed with my UW comrade. :D

[iago]

@High Speed Dubb

First of all, before I report a few issues I observe in my test encodes (normal movie / average quality DVD / XviD fixed quants 2 and 3 / MPEG2Dec3.dll / Avisynth 2.06), I would like to congragulate and thank you for this very successful filter, which in terms of compressibility gain, quality and speed is really amazing. BTW, I used default values in the test encodes -> PeachSmoother()

Just a possible problem: it seems to cause some sort of instability and shaking with still parts of the picture, both when applying it after resize or before. This is very obvious when I compare the results with and without PeachSmoother. I dunno if others experienced such a problem too, but I just wanted to report it here. (Version 0.9f)

best regards,
iago

[High Speed Dubb]

vlad59,

Interesting — I would have guessed that 0.9b had the fix for it. (That’s where I corrected to allow for Y(black) = 16.) Well, I’m glad it’s taken care of.

iago,

Would those parts of the picture be at the edges of objects or of compression blocks? Do you see any (less obvious) shimmering in the unprocessed source at the same locations?


Yay, Dawgs? ;)

[High Speed Dubb]

Oops — I forgot to comment on some of that.

Vlad59,

Those values were from a flat colored, high contrast cartoon (Justice League). Shaded animation would probably have problems with those settings. I’ll put an explanation of that in the example scripts.


I tried compressing a few quick clips (with HuffYUV), and noticed something sort of surprising. A noise reduction value of 150 allowed only a little more compression than a value of 20. So for saving disk space, it may be that only very light noise reduction is necessary. (But I do think the output looks nicer when noise reduction is in the low 50s.)

[High Speed Dubb]

It’s me again. :)

Many thanks to everybody for the feedback! This makes it much easier to figure out bugs, quirks, tweaks, and defaults. And it’s great to hear that the filter is working well.

Here is version 0.9g . This one enables spatial (but not temporal) smoothing when jumping between frames, and adds a couple speed-ups. You can now get it to run faster by turning off spatial smoothing.

http://students.washington.edu/ldubb/computer/PeachSmoother_v0_9g.zip

This is likely to be the last 0.9 release. Now is a good time to suggest changes to the parameter names, since I won’t want to change them after a 1.0 version.

[iago]

Would those parts of the picture be at the edges of objects or of compression blocks? Do you see any (less obvious) shimmering in the unprocessed source at the same locations? @High Speed Dub

Those shimmering/trembling ;) parts of the picture are both edges and full compression blocks. I can't see any shaking/instability in the source and in the encode without PeachSmoother.

Also, are the defaults of the new Version 0.9g same as the Version 0.9f?

regards,
iago

[High Speed Dubb]

Nuts — That could mean Peach Smoother won’t work well on XviD compressed video.

My suspicion is that the filter is picking up on very low level shimmering in the source, and is interpreting it as motion. That’s the price of having a sensitive motion detector. There’s a similar problem with broadcast video and color crosstalk, but that at least is fixable.

Do you have any way of putting a short (~10 second) clip online for a little while? I can’t be sure of the problem (or possible solutions) without taking a close look at the video.

PS: Yes, the defaults should be the same. But I did rewrite the parameter reading for 0.9g — Did you notice any odd changes?

[iago]

@High Speed Dub

If you give me an e-mail address, I can send a short clip (~10 sec. / ~3mb) displaying the problem.

iago

edit: regarding v0.9g defaults -> no I haven't noticed any problematic changes, I just wondered if the defaults are the same ;).

[High Speed Dubb]

I’ve replied to iago with a private message. (Though people can also find my email address on my web page, or use the URL in the filter help.)

[iago]

High Speed Dubb

I just wanted to confirm if you want the XviD (without PeachSmoother) encoded clip or a part of the source (vob, etc.)?

iago

edit: I'm sending the XviD (without PeachSmoother) encoded clip (~5 mb)

[High Speed Dubb]

Go with what you had immediately before processing with Peach Smoother — That would be the XviD, I think.

[iago]

@High Speed Dubb

After 2 or 3 unsuccessful tries of e-mailing, I give up ;). Each time my server cuts the transfer either half way or delivers an error message. And since no one else except me seems to have or be able to reproduce this problem, then just forget it! ;)

best regards,
iago

[High Speed Dubb]

iago,

Thank you for making the attempt. The filter has only been around for a few days, so other people may well see similar problems. If someone does see an edge/block noise problem, I’d very much like to take a look at it. (But do send an email or private message in advance — I don’t want my inbox hit with multiple huge messages.)

[Rrrough]

Hi Lindsey,

here you can see an example for the problem attached as screenshots.
clip was compressed with XVID - mostly default options.

my script was :
MARC's MPEG2DEC - no pp
CROP()
LANCZOS RESIZE
PEACHSMOOTHER() - no options used

peachsmoother v0.9c was used.

hope this is of any help.

cheers

attachments
pic 1 - with PeachSmoother
pic 2 - without PeachSmoother

[High Speed Dubb]

Rrrough,

Could you send me a video clip of that? A still will show the problem, but not the cause.

[Rrrough]

sure, how many frames would you need ? it's a high motion clip with 1500 frames resulting in an email-unfriendly 20 megs...:rolleyes:

cheers

[High Speed Dubb]

Yeah, that’s the problem with a feedback-based filter like this — It takes a pretty long clip to figure out what’s happening.

That it’s a high motion clip already makes me slightly suspicious — 25 seconds of high motion stuff may not be enough for the filter to get a good read on the noise level. But that would tend to cause blurring, not shimmering.

20 MB will probably get rejected by the server here. Do you have a place where you could upload it? Barring that, could you mail me a choice 8 MB from a 10 second clip?

[High Speed Dubb]

Here is another 0.9 release. I thought of an easy-to-add feature, and didn’t want to wait until the v1.0 release time.

http://students.washington.edu/ldubb/computer/PeachSmoother_v0_9h.zip

This version allows three new options. “Estimates” shows the filter’s current estimate of the amount of noise (measured in expected absolute difference per pixel) along with the lowest amount of noise seen in the picture. The other new options (NoiseLevel and Baseline) allow you to specify the noise level instead of estimating it. So you can use the Estimates display to figure out the amount of noise, then use that amount to choose the NoiseLevel and Baseline for the next run.

[soulfx]

Could the initial noise estimates be set lower or to zero when first starting the video stream? Either that or disable temporal smoothing until a stable noise estimate is reached?

When first starting up videos with high amounts of motion I got a really bad temporal smear for the first couple of frames until the auto estimation kicks in and lowers the values and/or it figures out that it shouldn't be using temporal filtering.

As far as shimering and stuff goes maybe this could be prevented by implimenting a noise filter for the motion detection. I haven't noticed any type of shimering, but I don't use XviD for sources, just MPEG2 interlaced.

[magnanimous]

Looks like a great filter but has anyone got it to work within DVD2SVCD, when it gets to video encoding (cinemacraft) it says the resolution isn't supported, hope sombody can help.

[High Speed Dubb]

soulfx,

The difficulty isn’t that the filter starts with a high noise estimate. The problem is that it starts with no confidence in its initial estimate.

That’s on purpose, of course, since its initial estimate is just an arbitrary amount. The bad news is that the Peach will accept any signal — even a poor signal which is from motion rather than noise — until a better signal shows up. PS: Unfortunately, it isn’t possible to know what is a good or a bad signal in advance, since that is measured relative to the other signals which have been seen. (That’s a trick which allows the filter to make use of weak signals from video with bad interference.)

There are a few of ways out of the problem. You can use the new Estimates option to read a good NoiseLevel and Baseline value, and then run the filter supplying those values. If there is some uninteresting material before the start of the video, you can include a little of it, and cut it out later. The Peach will be able to use the extra video to get a noise estimate. Or you can start the processing in the middle of the video in a stationary part, and process the moving part afterward, once a good noise estimate has been found.

PS: It already uses a noise filter for motion detection. I would argue that temporal noise filtering and motion detection are essentially the same problem.

magnanimous,
I’m not familiar with DVD2SVCD, so I’ll have to leave that question to someone else. But this filter doesn’t change the output resolution, so I’m not sure where the problem might have come from.

It does require that the source video have a (pixel) width which is evenly divisible by 4 — It is written to work with odd widths, but I haven’t tested that yet, so it’s currently disabled.

[Rrrough]

@lindsey:

you have (hopefully:confused: ) mail

hmmm, I think something went wrong with the picture attachments, so I will try once more with the peachy one...

[indy69]

I have just tried the filter in dvd2svcd. When i used the filter with no options set (peachsmoother()) it worked ok. When I tried with some options set it came up with a resolution error. I think my syntax was wrong. Is there anymore information on what syntax to use.
Thanks

[High Speed Dubb]

The syntax is described in the filter docs, and there are a few examples in the accompanying script file. Could you copy your script into a post, and give the exact text of the error?

[High Speed Dubb]

Rrrough,

That clip has constant fast motion. That means the filter didn’t have a chance to estimate the noise, and is blurring too much. Take a look at my response to soulfx for some hints on how to avoid that problem.

[Rrrough]

well, blurring is ok, but where's this blocking/trembling coming from ?

cheers

[High Speed Dubb]

Hmm... I didn’t notice any blocking or trembling in the clip you sent me. Where should I look for it?

(The obvious artifacts in the picture you posted there are from blurring — they’re left over from the previous frame.)

[Rrrough]

next to lara's nose and mouth, looks like a block here. maybe it's my monitor. didn't notice that it's from previous frame, sorry then. gonna play a little with the settings.

cheers

[High Speed Dubb]

Yep, it’s left over from the beige curtains.

[indy69]

Using the filter with dvd2svcd i can get these 2 parameters to work.

[AVISYNTH_BicubicResize with PeachSmoother filter]
0=Loadplugin(!PeachSmoother.dll)
1=PeachSmoother(^NoiseReduction,^Stability,)
2=BicubicResize(^TargetWidth,^TargetHeight,^b_value,^c_value)
3=AddBorders(0,^BorderTop,0,^BorderBottom)
!PeachSmoother.dll=D:\Dvd2svcd\peachsmoother.dll
^NoiseReduction=50
^Stability=25
^b_value=0.33
^c_value=0.33

I was just wondering how to get the spatial values to work.
When I add ^spatial to the script with any value I get a resolution error which normally means there is something wrong with the syntax.

[High Speed Dubb]

I haven’t used dvd2svcd, so take this with a grain of salt. But I do have a hunch about the problem.

It looks like you’re specifying the parameters by position in the command, rather than by name. There are probably two ways to solve the problem.

You can specify the parameters by name rather than by position. I strongly recommend that — It’s better to use named parameters for all but the simplest commands. (That way your script doesn’t depend on the hard to remember parameter ordering.) So the command would be (I think)

1=PeachSmoother(NoiseReduction = 50, Stability = 25, Spatial = 80)

Alternatively, you can use the correct order for the options. The option order is:
NoiseReduction, Stability, DoSmoothing, Smoothing, Dot, Estimates (which I plan to rename to Readout), NoiseLevel, Baseline, Debug

Anyone who is familiar with dvd2svcd is welcome to correct me on my syntax.

[jarthel]

Very fast temporal/spatial smoother!!! more than twice as fast as convolution3d and 2.5x as fast as SSHiQ. Will know the results in an hour.

Is there any chance you can add SSE2? I can test it for you if need be. :)

thank you.

jayel

ps. I deal mainly with anime. :)

[High Speed Dubb]

Ha! In DScaler, the Peach’s twin (adaptive noise) is the slowpoke. And the AVISynth version is even slower. I’ll eventually move DScaler’s gradual noise over to AVISynth — That one is genuinely fast.

Peach smoother is only barely a spatial smoother. The next release (which is looking extremely nice) will be making the spatial smoothing even more specific to moving areas.

[jarthel]

Bad results is what I get after trying peachsmoother. moving scenes have trembling (dunno the right term). Also it's bigger than just using temporalsmoother(5).

But it is in its infancy stage so there is room for improvement. :)

jayel

[High Speed Dubb]

Remember that you can’t run it just on a moving scene — in order to work, it first needs to see a still scene in order to estimate the noise. After that, it can handle moving material just fine.