New blur constrained smoother

[neuron2]

Our esteemed colleague poptones wrote in another thread:Apply an edge detection matrix to your source clip, apply it as a mask to itself, then layer that over a blurred clip and you have damn near perfect MPEG artifact removal without losing any of the detail that makes images look sharp to us.Actually, this is far from perfect because the detail map should constrain the blurring process so that blurring does not cross edge details. This is not achieved by simple overlaying. For example, if you had two color regions separated by a small black line, you do not want the two color areas to be blended together in the vicinity of the line. Constraining the blurring requires an integrated approach that cannot be achieved with Layer().

It so happens that I have a constrained blur smoother available at http://shelob.mordor.net/dgraft/msmooth10b2.zip. :)

EDIT: This links now points to the fixed beta 2.

Msmooth stands for "masked smoother". Although designed specifically for anime, it may be useful elsewhere. The filter is very effective at removing mosquito noise and MPEG blocking artifacts as well as effectively smoothing flat areas in anime. Not only is the noise reduction valuable, but less bits are needed to encode the resulting clip.

The justification for the filter is simple. The biggest complaint about Smart Smoother (for example) is that setting the threshold high enough to smooth sufficiently destroys a significant amount of detail. What we need is to decouple the control of detail detection from the amount of smoothing applied. In Smart Smoother and other thresholded smoothers, one threshold controls both of these functions. Msmooth implements this new idea. You first set the 'threshold' parameter so that desired detail is preserved. Then you set the smoothing strength. You can set as high a smoothing strength as you like; the preserved detail will not be destroyed and the blurring will not cross detail boundaries (because the detail map is used to constrain the smoothing).

Please see the help file in the zip file for additional details. Feedback will be greatly appreciated. Note that the filter has not yet been optimized so it is quite slow.

[vlad59]

Strange, I had almost the same idea when I saw the limit of Convolution3D (a filter I made). Doing multiple simple 3*3 blur seemed to be a good solution when coupled with something like an edge detection.

I had no time to make a working version, but I tested your filter and I get a good result on an noisy anime.

[neuron2]

@vlad59

Thank you for the feedback. Of course I know about you and Convolution3D, since it is part of the world record longest thread. :)

The repeated 3x3 is not the only possibility, just the easiest to implement, so that is why the first hack of the concept uses it. You could use a bigger kernel but then the constraint is more difficult. You'd have to use a special kind of boundary fill starting from the center pixel to find the set of constrained pixels to include in the blur. I will be experimenting to see if it is faster, because doing repeated 3x3s to get a larger effective blur radius is slow.

Also, straight blurring is not the only option for the smoothing operation. One might consider constrained median filtering, for example.

I will focus in the short run on getting the speed to an acceptable level.

In the longer run I am thinking that a similar concept can be applied temporally. The basic concept is that when we consider a pair of pixels spatially and whether they should be blurred together, it is not enough to look only at information derived from those two pixels. You need more global information. In the case of Msmooth, that comes from the detail detection phase, where many other pixels are involved, thus adding more global information.

Now, for the temporal smoothers, in deciding whether to blend a pixel and its corresponding pixel in a succeeding frame, we need more global information. In the temporal case, the detail detector gets replaced with a motion detector. Of course, not blending moving pixels is not a new idea. But deciding whether the pixel is moving should not be based only on that pixel and its predecessor (as you might do by simply testing their difference against a threshold); it has to use more global information. One approach is deciding whether a small block of pixels around the target pixel differs from its predecessor by more than a threshold.

The idea of switching between temporal and spatial smoothing is well known. See, for example: http://www.cis.udel.edu/~dovrolis/PV99/pv99.html. But it seems to me that switching is not necessary if the two modes can independently decide. Why couldn't the best noise reduction for a given pixel at a given time come from using both spatial and temporal smoothing at the same time? A switching mode would preclude that.

[Marc FD]

Hi :)
I'm currently working on a avisynth-toolbox project.
maybe you could use it to design this kind of behaviour.

[neuron2]

@MarcFD

For projects such as the one you describe, you really should review the specifications with the intended users before beginning implementation work. Why don't you start a new thread where we can discuss it?

[Marc FD]

see the Mpack & focus2 thread ;)
thanks, you're right. i'm too young, didn't thinked it could be crap when i began implementation... it's why i expose it now (in fact i did only write several hundred lines of C/MMX)

[wotef]

i did a couple of quick tests on my vhs source and i like what i'm getting - msmooth certainly allows greater precision with none of smart/spatial smoother's or 2d cleaner's whoops-the-noise-is-cleaned- up-but-shucks-all-the-detail's-gone-with-that-last-little-tweak-to-threshold/strength

temporal consideration and requisite speed :devil: optimisations would be dandy

[zettai]

Hmmm. This filter is a wonderful idea but I'm not overly keen on the 3x3 blur matrix.

I'm testing the filter on a copy of the anime Ebichu Minds the House which, despite being a nice clean digital anime is still less than perfect as the DVD is quite highly compressed in places. My desire was to return this anime to the super-clean digital inking of the source using this filter.

The edge detection is very good and I'm finding that it is preserving the distinct edges of this anime very well. I'm finding that low thresholds are better for this particular anime, otherwise you start blurring gradient backgrounds and so on. However, given the thick lines and block colours used in this anime, low threshold masks are perfectly sufficient.

The blur itself does effect the right areas but at low strength isn't enough to sufficitenyly smooth out, say, blockiness in skin tones but at higher levels just makes everything look very very streaky as the macroblocks in the skin tones get transformed into lines instead of blocks.

If you would like me to post some screenshots of my tests I will.

[neuron2]

@zettai

There is a bug in my chroma smoothing. I am making a new version. Thank you for the feedback.

[soulfx]

@neuron2

Thumbs up to a promissing smoother. I've run it through some tests and like the results I'm getting.

@neuron2 or Marc
I have come across some strange behavior. The filter will drop the first frame of the video and to make up for it will repeat the second frame twice when used with the MAM filter. The total frame count will still be the same, but the video will be off by one frame for the rest of the video. Don't know which filter is causing trouble, because outside of MAM Msmooth works okay and without Msmooth MAM works okay.

Peace,
SoulFX

[neuron2]

Can't help with MAM, but the Msmooth beta I released is seriously deficient at even moderate strengths as the chroma smoothing is all messed up. I've withdrawn the beta 1 and have a new one coming shortly. You can't believe how hard it is to write a general configurable size blur kernel that does both luma and chroma correctly *in YUV space* and which is not super slow.

[trbarry]

You can't believe how hard it is to write a general configurable size blur kernel that does both luma and chroma correctly *in YUV space* and which is not super slow.

Amen.

I'm not currently working on noise filters now but for the thing I am working on I finally just decided to write a couple fast SplitYUY2Chroma and MergeYUY2Chroma filters that move all the luma bytes to the left half of the line and all the UV chroma words to the right. It's maybe not any faster but it sure simplifies things and lets me keep them separate yet still with a single same sized clip.

- Tom

[neuron2]

@trbarry

Interesting idea to retain coding sanity! I'm going to sleep on it right now. :)

Thank you.

[sh0dan]

Originally posted by neuron2
You can't believe how hard it is to write a general configurable size blur kernel that does both luma and chroma correctly *in YUV space* and which is not super slow.

Talk about - been there - my best solution was a quite massive unroll to avoid massive branch-penalties (smoothhiq avs source) (http://cultact-server.novi.dk/kpo/avisynth/smooth_hiq_as.html). And still fighting to do the same stuff in assember/mmx - it's a b*tch.

[vidiot]

@Neuron2,

i´m not to much a "brain" in vidiostuff (as my nickname may told already), but I tried your filter MSmooth with big success with an anime (Angel Sanctuary). Much better than 2DCleaner but somewhat slower.
I tried to compare what the filter is doing with an "split YUV" filter in VD writte by Simon walters (?)...
And the only noise I could see was in the middle (the U-plane?).

For still pictures, I would go with something like "gausschen blur" just for the middle plane and than recombine.
Maybe this is what is done already by "Layers ans Masks", but I´m to dump to realise the concept...

...otherwise I couldn´t say "thankyou" enough times for making telecide (decomb) that fast...

Harald

[neuron2]

@vidiot

You should change your handle; it is inaccurate. :)

The beta gives usable results at low smoothing strengths. Beyond low, serious artifacts in chroma are produced.

I'm happy you find Decomb useful as I put a lot of effort into it. :)

[bill_baroud]

the link is dead ? or i have missed something ?

if someone could up it somewhere, i'll be glad ... i want test a filter i was thinking for a long time ... and Donald do it for us.

big thanks !

[Marc FD]

*sigh* just wanted to try it to see if it's the idea i had for removing MPEG-2 quantization artifacts.

and i can't dl it ....

waiting for a fix.

PS : are you using a MMXed iterative bluring or simple a 1 pass blur kernel convolution ?

i really eager to take a look on the code...

[neuron2]

I've killed the link because I hate to have buggy stuff out there with my name on it. Please be patient, the new fixed beta is coming soon (possibly tonight). Please no-one upload the old one!

The new beta uses a one-pass kernel whose size can be selected. As you know from other threads, iteration in YUV subsampled space is problematic.

I never do low-level coding (MMX, etc.) until the functionality stabilises. You shouldn't either. :)

[Marc FD]

no pb, i'm not the kind of coder who like to loose time :)
but i would really enjoy to help you ( a _little_ bit ;) ) on this project because it sounds promising :)

[neuron2]

The new fixed beta can be downloaded from:

http://shelob.mordor.net/dgraft/msmooth10b2.zip

This is now production quality but still somewhat slow. The next release will add algorithmic and low-level optimizations.

I'm very surprised by the capabilities of this filter. I hope you will be too. The magic is in the detail-constrained smoothing.

After completing the optimizations, I plan to repeat this trick in the time domain. :)

As always, feedback will be gratefully received.

[zettai]

ok, testing time!

Source: Ebichu Minds the House recompressed to PICvideo mjpeg quality 10 to give it a nice amount of artifacts :)

avisynth script:

loadplugin("C:\filters\msmooth.dll")
AVIsource("tester.avi")
crop(14,8,692,468)
converttoRGB()

Attached images show - original avi file loaded into vdub on top of an avisynth and warpsharp altered version below.

Set 1 (anti-mosquito test) Msmooth(mask=false,threshold=18,strength=8)
Set 2 (attempt to combat macroblocking on face) Msmooth(mask=false,threshold=18,strength=18)

Both lower versions have been warpsharped afterwards to increase the clarity of the lines as even high thresholds like this can leave lines a little hairy and uneven. Unfortunately I don't recall the warpsharp settings, although the 2nd image is probably warpsharp setting "too much" :)

Filter strengths - I couldn't believe how good this was at removing mosquito artifacts. Really really nice :)

Filter Weaknesses - The filter is, well, boxy, which inevitably means that if you want to combat a macroblocking problem then you can turn small macroblocks into one big one, which isn't always what you want.

Also, cropping is very very important - if you have a thin black line at the edge of your image, this may be missed by the mask and can lead to the edge being blurred into the image. Not really a weakness, more of an observation.

Of course, a natural caveat with this filter is that the threshold to detect the mosquito artifacts will inevitably also detect noisy non-uniform backgrounds. That can't be helped though.


Possible recommendations: Perhaps allowing the user to select different blurring methods or maybe include the possibility of a larger blurring radius. I dunno, just something to combat the 'big block' effect perhaps.

This is a very nice filter and I'm looking forward to see if any further advancements can be made. I'm not particularly worried about speed (as all my work is for AMVs so they're not very long anyway) but I'm sure speed enchancements will be welcome.

[Edit: until the image is attached by a moderator, you can see the results here: http://www.absolutedestiny.org/msmoothtest3.jpg ]

[sh0dan]

Very impressive!!! It works very well on evenly colored surfaces!

Very impressed!
It does seem very agressive on finer details, but it looks very good for sharp lines. Perhaps it would be possible to distinguish sharp edges from smaller detail? Or is it just a matter of better parameters?

[zettai]

I think the problem is, essentially, that mosquito artifacts are "finer details" so in order to get rid of them you have to get rid of, for example, some of the background definition in the images above.

It's pretty clear from the debug images exactly what you will be smoothing - in this situation I incresed the threshold until I couldnt see any masking on the mosquito and blocking artifacts. In many sources the artifacts arent as heavy as this so dont need as high a threshold and hence will save many of the smaller details.

[Defiler]

Wow.

[vidiot]

@neuron2:

The first Version was "slow" with my configuration (~9fps)
This Version (beta2) is much slower in the same configuration (~5fps).
No wonder: I´ve to convert to RGB
if not I receive a message:
"Mssmooth:RGB Data only Use....."

Do i miss something, - why not stay in YUV (I guess it´s harder to code and you´ll stay with RGB until you´ve reached a special step?)?

Harald

[neuron2]

@vidiot

Going to RGB32 makes the code tractable for a human being but more importantly it enables some great speedups that are in the works. I am projecting 5-10 time speedup!

You see, I've discovered that the non-linear blur kernel is still separable. I'm sure you understand the implications of that! After changing to 1D convolutions, I can then do incremental update to the window summations. Then MMX etc.

[vidiot]

...pretty fast answer!
Thank you.

I couldn´t wait to see those speed improvements.

If your filter can reach those speeds, I can filter my tv captured material near realtime (it´s only 480 * 576/ PAL of course)
most times with telecide() and cnr2() in the filter chain...

That would be great!

Harald

[kyousuke]

really impressive on uniformly colored areas ! it's incredible !

i'm going to test it right now! :D

[Bulletproof]

Are you going to be also porting this to .VDF? I would really like that if you could, along with a preview button.

[Zarxrax]

Wow! This is very impressive! I've sworn by the 2d cleaner filter for probably over a year now, and almost pissed my pants when the optimised version was released. But this is so much better! You've really outdone yourself here, and I can't wait to see what further improvements can come from this.
I would also like to see it released as a vdub filter, as I rarely ever use avisynth. (I must be one of the only people left that still does IVTC manually in TMPGenc, instead of using decomb :rolleyes: )

[neuron2]

@Zarxrax

Can you please post some screenshots that compare it to 2d cleaner. I need to be objectively convinced that it really is better before investing a lot of time in it. Often the newest toy has the novelty effect if you know what I mean. :)

[Bulletproof]

Currently I think this filter is in a different class than 2D Cleaner (although it is very good). I think Smart Smoother and 2D Cleaner just cant properly clean edge artifacts unless you enter in a high threshold which ends up killing details.

I tried testing your filter today with some animation material, it worked quite well except for a few points. Some animation has pretty detailed backgrounds, this one happened to have a "ruins" background, rock-type details, the walls had natural noise and fine details to them, the filter ended up making the background look like "pastel" or a "painted" effect. But the foreground which had the actual characters came out quite well, this is with a strength of 3 btw. Right now it's really hard for me to tweak the threshold/mask. The older smoother in .VDF form lets you watch changes in realtime to properly tweak things. I'm not quite sure if this would work, but maybe you could introduce something that checks the old frame vs the filtered frame, and if the difference between the two is too high then maybe it could just skip that frame, although I'm thinking that the difference will already be high just from the artifact removal, if there was just some way...

[vlad59]

Hi neuron2,

I just tested your latest beta of Msmooth with my ultimate test for denoising filters : a very noise anime (transfered without any good matering from tape to DVD : so with flying black spot, some scratch, etc).

I had no time to tweak all the parameters (only the strength in fact), so excuse me if I post dumb informations.

On still scene, MSmooth does a very good job but the color look unnatural (too pastel) and the noise is mixed with the backgroud making it darker. I'll make a full test to see if with ffdshow and added noise it looks better.

On a long fade in (~ 7 frames) from full white to a detailed frame : and here the frame are way too much blurred not viewable at all (I can provide a hulfyuv clip if you want). This scene was also problematic for Convolution3d, only Tom's StMedianFilter made a very good work.

But on characters the result are awesome, the skin is softer but edges remain black and thin.

Waiting for the next release ......

Thanks for your work.

[zettai]

The threshold will make a very big difference to your results. With the testing I did, the threshold can be very different depending on the source. One of my sources only needed a threshold of 3, whereas others needed a much higher threshold to get all of the noise.

The debug is excellent in seeing what will be smoothed though so I recommend using it.

I just increased the threshold until none of the noise was shown in the mask - this may or may not remove background details depending on how noisy the source was. In any case, setting the right threshold is essential for this filter and there isn't a general value you can use, in my opinion. Luckily the debug display is excellent so this is very easy to do.

[edit: terrible spelling :)]

[Zarxrax]

@neuron: sorry, I can't post any comparisons right now, as my pc is dead. I should have a new system up and running this weekend though. My comments were solely based on the pics that Zettai had posted up (yea I know its no substitute for my own testing :p ) I'll end up doing a ton of test encodes when I get up and running again though. So ya can either wait a few days on me, or perhaps someone else would like to show you comparisons.

[vidiot]

@neuron2:

I made a more or less quick comparisation between your mssmooth and 2dcleaner(the VD one)

Test-setup:
Source: Anime captured from tv res. 448x544 (yes, I´ll crop the overscan while capturing...).
Codec picvideo Q 19

Avisynth 2.04+
----------
Telecide()
Cnr2(1,60,210,60,256,60,256)# somewhat strong, but no ghosting...
ConvertToRGB ## New for new Msmooth
Msmooth(mask=false,threshold=10,strength=7)
----------
For 2 d cleaner, i took the same script w/o Mssmooth

Telecide()
Cnr2(1,60,210,60,256,60,256)# somewhat strong, but no ghosting...

The RGB Conversion is done by VD then...
2D Cleaner with settings 20 / 2/2

Maybe I could have found better params for both Versions,
but of course for me speed is a concern.

I tried to compare the video not only frame by frame, so I encoded 2 min. of both versions. I can tell you that there is really no difference I can see (maybe because of my eyes or my old televison)
It´s hard to judge.

With frame by frame compare 2dcleaner is a bit ahead (in this case/ look at the attached image)
and 3-4 times faster.
But I´ll like the idea seeing mssmooth evolving in speed and maybe more params to tweak...

So please don´t give up to fast...

Harald

Edit: I think you´ll have to wait till the image is attached here...

[Bulletproof]

That's really weird that you have a strength setting of 7 there and it looks like it barely touched the video, yet when I used a strength of 3 on some of my video I was finding "pastel" effects. Maybe its the threshold but I think 10 is the default right?

[vidiot]

@Bulletproof:
Could it be that the effect of threshold depends on what Mssmooth gets?
Look at my filter chain and you saw a CNR2() with kind of hard settings. Maybe that is already to much to compare both filters?
(But it´s the only way to get rid of weavy lines from tv).
I think I should have make source pictures from after Telecide() and cnr2()...

But if you take a closer look at the sign "SOUND HOUSE" at the entrance on the right, you´ll see that it´s hard to read with mssmooth settings i used. So for me it seems a bit harder with teh 19/7 setting than 2dCleander with 20/2/2.

Harald

[WarpEnterprises]

What about this version using ONLY WaveletNoiseReduction?
Settings used:
VirtualDub.video.filters.Add("wavelet noise reduction (0.1)");
VirtualDub.video.filters.instance[0].Config(50, 12840, 10280, 0, 12850, 15420, 5120, 10280, 10280);

Another (additional) good way to get rid of those alternating lines (red-green-red) is to merge the color only of each two lines of each FIELD before any further processing (telecide).

[vidiot]

WarpEnterprises,

this is OT regarding to Neuron´s question.
But otherwise...
I tried those WNR filter for VD before and tried the original one witch was introduced in AVIUTIL- the aviutil one has got more settings and is really hard to setup because of that.

This filter was dicussed in an VD Thread I don´t remember (I looked for it, but haven´t found it...)

For removing PAL alternating lines I usually use rmpal.vdf (ReMovePal)
which does a great job, but kind of slow. I use it before any other processing.
That is a problem with using avisynth in VD, because how often should I change the colorspace??

But thankyou for the suggestion,
i´ll try those settings as I´m looking for speed in any way , if the quality is good enough.
For home made video it´s no question: quality before time!
(But I don´t want to chase rainbows)

Kind regards
Harald

[Bulletproof]

Ack, I didn't realize that your Msmooth video was to the right side of your 2D cleaner picture. I was accidentally looking at the source_mat one. The Msmooth one is off screen on my 1280x960 desktop so I had to scroll to the right to see the right picture.

[poptones]

Howdy. Well, I'm having to catch up as I just recently got mah shiny new machine to compile the project properly. Anyway, I saw this thread about a week ago but I couldn't find my samples (my machine has experienced great turmoil) so I didn't bother to reply.

I've not had a chance to try the neww filter, but I wanted to make the point my comments about getting rid of MPEG artifacts was more directed toward getting rid of some serious artifacts - like this...

http://personal.ayrix.net/~poptones/video/ick.jpg

That's why I made that filter that triggered basically off "how many pixels have changed substantially between the last two frames?"

I found my sample video clips. You can see the results
here (http://personal.ayrix.net/~poptones/video/filter.avi) and here. (http://personal.ayrix.net/~poptones/video/source_filter.avi)

Obviously it wasn't perfect (it still blurred too much the "good" video) but that's why I decided to try a different tack using more general purpose filters. Which led to making layer filters, which took me off the original quest for, like, a year. Man, it's easy to get sidetracked ;)

[MoonWalker]

Hi,

I have tried msmooth10b2.zip to remove the MPEG artifacts...But it doesn't quite work..Firstly the encoding goes with 2fps and secondtly is doesn't remove very well the MPEG artifacts..I think my avs is wrong..BTW is it able to remove the mosquito noise??

SetMemoryMax(40)
LoadPlugin("D:\DivX\GORDIA~1\mpeg2dec.dll")
LoadPlugin("J:\Matrix\1cd-xvid\640\New Folder\Msmooth.dll")
mpeg2source("J:\Matrix\matrix.d2v")
ConvertToRGB()
Msmooth(threshold=5,strength=7)
Crop(0,78,720,420)
BicubicResize(640,272,0,0.5)

Any suggestions??

Thanks,
MoonWalker

[neuron2]

@MoonWalker

This filter is currently in the shop for repairs.

[neuron2]

@Marc

It's also rude to hijack other people's threads like this. I'm beginning to like you less and less, to be quite frank about it.

[Marc FD]

it was just a joke :(

sorry i didn't intend to do something bad.

(are you in a bad mood today :confused: )

[Marc FD]

Originally posted by Bach
it is even more rude to delete a post which already have a reply...

just 2 cents from an ex-moderator.

???????????????????

I don't understand why.

I recognise i've made an error. i try to correct it.
why keep this post if it hurts donald ?

I want everybody to be :) :D

[neuron2]

@all

I feel the need to apologise to Marc. My comment about liking was really over the top and uncalled for. I'm usually pretty polite and I can't even blame Koepi for not keeping an eye on me. :)

So, sorry, Marc. I forgot that I was once your age, and I won't forget it again.