MipSmooth v0.9 released

[sh0dan]

I implemented some changes based on the feedback and my own tests.

Update: Look below for v0.9 notes!


v0.2:
* Added optional resizers.
* Added "strong" method - now default. Previous default was "soft".
* Added scalefactor to allow different mip scaling.
* Mod 16 restriction lifted with new downscalers.
* Lower thresholds, since blending is stronger now. Default settings are still fairly strong.


http://cultact-server.novi.dk/kpo/avisynth/MipSmooth.html

So basicly it is now possible to filter harder without getting artifacts. I get results far better than C3D - especially on animated sources with large uniformly, or shaded aras.

Film material also provide good results.

As always, I'm very interested in feedback. If you have no further comments, it's about to become 1.0! :cool:

MipSmoother is an attempt at reinventing SmoothHiQ and Convolution3D. For now this is a proof of concept, and the first versions are for evalutation of quality and speed. MipSmooth was made to enable smoothing of larger pixel areas than 3x3(x3), to remove blocks and smoothing out low-frequency noise.
It is my hope that when this filter has been tweaked it will be better than C3D at smoothing out flat surfaces and gradients. This should be good for cartoons/anime, but it actually also helps film footage quite nicely.

[neily]

shOdan,

Looks very promising on first inspection, even at the default values. It seems about 60% quicker than SmoothHiQ at default settings, but I can see that this will be highly dependent on the setting used.

Sorry to be picky, but in the documenation it refers to arguments spatialchroma and temporalchroma, whereas of course the real arguments are spatial_chroma and temporal_chroma.

[Bulletproof]

Cool, except I'm getting an access violation when I put TemporalSoften before this filter, if I put TemporalSoften after it then it works :) Still cool though.

[Boulder]

This could be interesting. I've been looking for a general spatial and temporal smoother for my TV caps for quite a while. I'll run some tests and see what kind of white rabbits you've got for us;)

[Ewi]

Originally posted by sh0dan
I get results far better than C3D - especially on animated sources with large uniformly, or shaded aras.


In terms of quality or in terms of compression gain (or both)?

[kilg0r3]

For normal film one should use downsizer and upsizer = bicubic. otherwise you might get speckled faces, especially when there is a lot of grain or other luma noise. Tested settings were:

MipSmooth(3,2,2,2,5,method="soft",scalefactor=0.75)

[sh0dan]

@kilg0r3: Seems like quite good preset for movieHQ

@Bulletproof: Could you post the AviSynth version, and script?

@Ewi: Primary in terms of quality - compression seem to be about the same.

[Bulletproof]

I think my build was outdated, I just downloaded the May 15th build and I can't reproduce the access violation, so everything is ok now.

[DDogg]

Sh0dan, how would you recommend applying settings and script for home DVCam footage?

[sh0dan]

Ddogg, it should not be interlaced when you use the filter, and the best would be for each field to be processed separately to avoid crosstalk, so:

avisource("vid.avi")
separatefields()
even=selecteven()
odd=selectodd()
even=Mipsmooth()
odd=Mipsmooth()
interleave(even,odd)
weave()

...would probably be the best method if you want the output to be interlaced. Otherwise deinterlace before mipsmooth.

As for the actual settings, something like kilgor3's for pretty clean material. Otherwise you might try the "strong" method.

[Belgabor]

Um, shouldn't that be


avisource("vid.avi")
separatefields()
even=selecteven()
odd=selectodd()
even=Mipsmooth(even)
odd=Mipsmooth(odd)
interleave(even,odd)
weave()

[Defiler]

Originally posted by kilg0r3
For normal film one should use downsizer and upsizer = bicubic. otherwise you might get speckled faces, especially when there is a lot of grain or other luma noise. Tested settings were:

MipSmooth(3,2,2,2,5,method="soft",scalefactor=0.75) Wouldn't using bicubic for both the downsizer and upsizer yield MipMapped frames that fell within the threshold more often (because they retain more detail), and thus increase the amount (or rather, frequency) of smoothing? Setting the scale factor higher would increase this effect even more, right?

[sh0dan]

Partially - it will however also lessen the effect, as the image is closer to the original. It'll provide different smoothing - not necesarily more or less. As for the impact of the resizer algorithm - I'm still not sure if it is better to use either one - or in which case which is better. I'm hoping for your feedback here.

[wotef]

some feedback, i can't see any differences in terms of visual quality (or even filesize!) when altering any of the downsizers or upsizers, save to say that lanczos makes it slower; so i'd stick with what's fastest

shodan is there a way to give mipsmooth parameter defaults with a registry entry (so you won't have to keep typing stuff in)

this is what i'm using currently:
spatial=5,temporal=5,spatial_chroma=5,temporal_chroma=5,scenechange=4,method="soft")

[Dark-Cracker]

someone have some presets for nmipsmooth filter ? i am try to made some tests but i am not sure i change the right settings.

preset for : light, medium,normal,strong,extreme settings

thank u :)
++

[Bulletproof]

I've had tremendous results when increasing the thresholds, however increasing the threshold litterally makes my system crawl. It brought my encoding speed litterally down to 1fps on my Athlon 1.7ghz (actual speed). I'm not surprised that it made it slow down, because increasing the threshold that high in SSHiQ will also bring the encoding speed that low. But the end result was that it helped the compressibility greatly, therefore overall allowing more bits to be used in important areas.

[N_F]

Is there any reason to change the scenechange value? Should it be low for semi-clean sources and high for noisy sources or something like that?

[sh0dan]

@wotef: Nice idea about storing custom presets - I'll look into doing this.

I agree that different resizers have little visual impact - it might have some effect on edges

@N_F: It should match the amount of temporal smoothing you want. Experiment with this script, to adjust thresholds:
vi = last
scriptclip("subtitle(string(YDifferenceFromPrevious(vi)), align =8)")
scriptclip("subtitle(string(YDifferenceToNext(vi)), align =2)")

The previous frame is at the top - the next is at the bottom. I'll look into embedding this functionality into MipSmooth to help you adjust the scenechange threshold.

@Bulletproof: Only the scenechange threshold should have performance impact - otherwise there is a serious bug somewhere. The spatial+temporal thresholds should not impact speed in any way.

@Dark-Cracker: Guess we all need them. There are some starting points in the defaults, and this thread, but you are very welcome to send which presets give you good results.

[wotef]

re: customising default parameters, i got the idea from decomb's help file

"Next, create defaults files as required in the plugins directory. For example, to set the default post=false for Telecide(), make a file called Telecide.def and put this line in it:

post=false

You can list as many parameter assignments as you like, one per line. Those not specified assume the values given below. Of course, you can always override the defaults in your scripts when you invoke the functions. NOTE: The lines in the defaults file must not contain any spaces or tabs.

but something like convolution3d's built-in variable preset="movieHQ"
would be cool, too

[Defiler]

Originally posted by sh0dan
@wotef: Nice idea about storing custom presets - I'll look into doing this.I think this would be convenient, but it's a little outside the typical Avisynth filter metaphor. It would also make it harder for people to paste in their script when they run into trouble.. Just something to think about.

[sh0dan]

It could be:

Mipsmooth(preset="custom", store=true/false)

When store is true, the current settings will be saved to the registry, when it is false, settings will be loaded from the registry. Then preset could also contain other presets. The store parameter will be ignored, if preset is not set to custom.

[Cyberia]

Shodan,

For simplicity, why not just use keywords like?:
loadpreset
savepreset

So you could do this:
Mipsmooth(loadpreset="custom1") //*load these values
MipSmooth(3,2,2,2,5,method="soft",scalefactor=0.75) //*modify them
Mipsmooth(savepreset="custom2") //*save as a new preset value


Just a suggestion.

[sh0dan]

It will only support one custom preset anyway - otherwise I have to go through hell to get the registry entries working. Besides your suggestion will never work, since each filter instance have their own settings. I will however make it possible to override specific preset settings (I hope).

25% of it done.

[Bulletproof]

I did some quick tests (Using XviD Null speed test w/ YV12 and no extra codec features (default settings)):

Gets me 1fps:

MipSmooth(spatial=10, temporal=6, spatial_chroma=12, temporal_chroma=5, scenechange=5, method="strong", downsizer="bilinear", upsizer="bicubic", scalefactor=2.0)

Gets me 27fps:

Mipsmooth()

Also gets me 1fps:

MipSmooth(downsizer="lanczos", upsizer="bilinear", scalefactor=2.0, method="strong")

Gets me 2fps:

MipSmooth(spatial=3, temporal=3, spatial_chroma=3, temporal_chroma=3, scenechange=5, method="strong", downsizer="point", upsizer="point", scalefactor=2.0)

Gets me 12fps:

MipSmooth(spatial=3, temporal=3, spatial_chroma=3, temporal_chroma=3, scenechange=5, method="strong", downsizer="point", upsizer="point", scalefactor=0.6)

Gets me 16fps:

MipSmooth(scenechange=15)


So you're right, the actual thresholds do not make much impact (if any) to the encoding speed, however it seems the scalefactor plays a very large role in the speed, using a scalefactor of 2.0 brings the fps down to 1.0 in almost every case. The default invoke seems to give a whopping 27fps, and changing the scale factor to 0.6 brings it up to 12fps. The scenechange also changes the speed but not as much as the scalefactor it seems. But I think using a scalefactor below 2.0 doesn't seem to impact the video as much.

(sh0: Inserted spaces to break lines)

[Defiler]

I'm finally going to get a chance to do some extended tests with MipSmooth tonight. I know you're all dying to hear my thoughts about this. :D

[sh0dan]

@Bulletproof: This is to be expected - these frames are created:

Current frame: Size*2, Size*4 & Size*8!
Previous and next frame: Size*2 & size*4

So it might not be that strange it is slow. ;)

(Btw, for other people - using point for both down and upscaling doesn't make sense in terms of quality - point in general isn't recommended for upscaling).

[sh0dan]

Preset system implemented. My suggestion for MovieHQ:

spatial = 3
temporal = 2
spatial_chroma = 5
temporal_chroma = 4
scenechange = 4
method = "strong"
downsizer = "bilinear"
upsizer = "bilinear"
scalefactor = 0.75

[Dark-Cracker]

hi,

i have made some little tests for findings presets and here what my tests have resulted :

light :
mipsmooth(3,2,2,2,5,method="soft",downsizer="bilinear",upsizer="bilinear",scalefactor=0.65)

medium :
mipsmooth(5,3,5,4,5,method="soft",downsizer="bilinear",upsizer="bilinear",scalefactor=0.65)

strong :
mipsmooth(6,4,10,8,6,method="soft",downsizer="bilinear",upsizer="bilinear",scalefactor=0.70)

extreme :
mipsmooth(8,4,6,6,6,method="strong",downsizer="bilinear",upsizer="bilinear",scalefactor=0.70)

Change the generical presets

screenshot (difference beetween source/medium/strong/extreme preset) the light preset was not in this screenshot :
http://www.eclipsedvd.firstream.net/autodub/Filter_screenshot/chihiro_mipsmooth.jpg

the green part is the pixels modified by the filter.
the result is correct for time used vs compressibility test.

Perhaps i have made some mistake in the parameters, post a msg to correct me :)

damn my english sux.
Bye.

[sh0dan]

One thing - unless you use downsizer="bilinear", scalefactor does not have any effect. You might want to try that.

I think it would be better just to have the straigt images - I really can't tell much from the green images. You might want to try the strong setting, with slightly lower thresholds - just to experiment. Your material seems like good material to test an "animeHQ" preset, since it's already very clean.

[Dark-Cracker]

ok,i think i will always use downsizer="bilinear", i think is a good param beetween speed and quality.
i genraly use the green images to be sure the settings was not to strong, i will try to post the filtered pictures as soon as possible.i will use a less strong thresholds but of course the input source was aproximatively a clean material.

Bye.

[drebel]

After a lot of testing with heavy noise (Life of Brian R2) ,and playing around with the settings, i must admit i really liked the filter, because of its flexibility and general optical impression.I'll just point out two things worthy of mentioning IMHO :

- Scene detection less than 10 couldn't handle camera zoom outs ,so in those scenes there was no temporal denoising :(
- Upscaling (scale factor >1) wan not an option for my poor Duron (yet,i hope ;) , dramatic fps drop),although image quality was great
- Downscaling (0.65<scale fact<0.75) had decent speed even with downsizer "lanczos" and upsizer "bilinear" (scalefactor =0.5 then ?)as long as there were no big spatial thresolds used with small s.factors (not speedwise, just a tendancy in blurring things)

mipsmooth(8,25,8,25,10,method="soft",downsizer="bilinear",upsizer="bilinear",scalefactor=0.7)

It seemed to me there 's a big tolerance in temporal thresholds without strange side-effects (value of 25 was pretty big for convolution3d)

One last thing @develloper : usually when encoding we resize by downscaling .Is there a way to implement a filter's specific resolution output as an option (the input could be assumed given after cropping ) and convert scale factor accordingly in order to avoid the BASIC RESIZE (usually Lanczos for me) after your filter ? I believe that could save us some encoding time (and quality ?) and place your filter as the absolut medium between cropping-resize procedure...
PS : forgive me if i misunderstood something ,but your posts here helped a lot more than the documentation

[Dali Lama]

I'd like to share some experiences.

Great filter Sh0dan!

I have tried to recreate a nice filter set I am using on anime and came to this:

MipSmooth(24,10,44,16,5,method="soft",downsizer="Lanczos",upsizer="Lanczos",scalefactor=0.5)

I am using a really high spatial filter and and average temporal threshold for anime. The chroma can be singificantly higher without hurting the details too much. At these thresholds I can recreate the 2DCleaner look...finally! Also more details are retained, but very slightly less noise removal is achieved in comparison to a 2DCleaner. The results are really very good.

One problem I encountered is that if I try to raise the scalefactor to 2 it causes an access violation when I scrub through the video too much. That is probably because it is pushing my system beyond its limits.

Good luck,

Dali

[Dali Lama]

@Drebel,

25 for temporal threshold caused blending in my anime clip. Just wanted to let you know. You can scrub frame by frame in a high motion scene and see it for yourself. But maybe it doesn't bother you under regular viewing.

Bye,

Dali

[HomiE FR]

@sh0dan : Hi and thanks for this new filter ! :)

I tested MipSmooth with the conditional functions, inside a ScriptClip. And I've got the same problem that I had before with some other filters : my memory usage starts growing up without stopping. You told me before that you thought it was a memory leak inside the filter (I believe it's true) so I guess there is a (little, less than in FluxSmooth for instance) memory leak inside MipSmooth 0.2.

I hope I'm right, and I hope you'll be able to find it without too much problems.

[sh0dan]

@HomiE FR: I'll investigate - I haven't found any leaks yet, but I'll keep trying.

@Dali Lama: Seems like a fairly agressive setting. What happends if you use method="strong" and lower the thresholds? Also - does lanczos actually give better results than bilinear?

@drebel:
- Scenechange detection is there to take out those cases, as zoom/pan/blends usually tend to give ghosting/trails when these cases are used for temporal cleaning.

Regarding temporal thresholds - yes, they can be pretty high, since they are applied after spatial softening. So you should be able to use fairly high (10-20) thresholds, as long as you have a reasonably low scenechange threshold (max 7-10).

I'll be adding some debug output option, and release a new version shortly.

Still looking for some anime material to test various presets on though.

[MrTibs]

Is there a 2.0x (YUY2) version avaible?

[sh0dan]

@HomiE FR: I think I found the leak!

@MrTibs: There will be limited YUY2 support for 2.5 in the next version.

[HomiE FR]

@sh0dan : You're quick ! ;) That would be great, since I'd like to test this filter within a sort of adaptative motion filtering script that I wrote using your brand new conditional filters.

Thanks again for you hard work.

[sh0dan]

Version 0.9 released.

Same page as before.
http://cultact-server.novi.dk/kpo/avisynth/MipSmooth.html

* Added presets.
* Added storecustom parameter to store a custom preset.
* Capable of running YUY2 material. For now chroma thresholds are ignored in YUY2.
* Capable of running in RGB mode (RGB24 and RGB32).
* Added "show=true" now displays information onscreen.
* Updated documentation.
* The Reduce downsizer is now interchangeable with bilinear, and only used when the settings allow it.
* Fixed: Luma thresholds were also being used as chroma thresholds.

[cipher]

Cool! These new features in v0.9 are really hot!
Off to do some tests.

P.S. I smell a final version coming soon when it jumps from v0.2 straight up to v0.9. :D

Edit: I noticed mod 16 restrictions for width and height have been deleted in Requirements. So I assume there are no such restrictions now? Which is great, coz I don't have to give up MipSmooth as I did when this requirment wasn't met. :)

[sh0dan]

@cipher: You are right - this limitation is no longer present!

The filter will still operate faster when there is mod16 width/height - but as always - qulity above speed! :)

v0.9 is to be last before 1.0 - if there should be any bugs. I already found a minor one - the _displayed_ scenechange in RGB mode is three times higher than the actual (and tested) scenechange.

[Defiler]

This is a cool filter. Heh.

[Dali Lama]

@Sh0dan

Sorry, for the late reply. I chose Lanczos resize, because if you zoom 4x (!!) in virtualdub, the edges of find detail, like hair is better defined. But its probably not practically worth the slowdown?

I'm glad you brought up the issue of soft vs. strong. In my tests, (which I half-forget ;) ) I achieved better gradation of detailed backgrounds and even clothing (depends on anime) with the soft setting with high thresholds. It still retained a lot of detail for anime, IMO more than the strong with low thresholds.

Also, a major plus to using the soft with high thresholds is that it removed "rainbowing" which is rather important for many sources.

__________________________________________________________________

In respose to to the new version. I am testing now, but would like to report that using the "animeLQ" preset, really "ugly" artifacts result in motion scenes. It may be due to the scenechange detection of 8 or the strong setting, or the thresholds, :confused:

I am sorry to say "ugly" like that, but a more descriptive term might be ghosting. It is like a plastic look.

Edit: The source I am using is DVD anime (hq), so that could explain a lot.

I have been trying a much more conservative script:

MipSmooth(10,10,5,10,5,method="soft")

With good results on that source.

Great filter once again Sh0dan. It has the capability to be a mainstay in our scripts.

Dali

[sh0dan]

Can anyone else confirm that MipSmooth(10,10,5,10,5,method="soft") is better for animeHQ - if so, I'll make it the default!

[Defiler]

Personally, I've been using higher scenechange detection values (assuming that they provide more sensitivity.. heh..) along with high-quality anime sources. I'm picky about temporal artifacts, so I usually use scenechange=10.
Edit: However, it's hard to determine exactly which setting has the most influence, since I tend to adjust all the temporal values simultaneously. Only so many two-pass encodes fit into a day. Heh.

[HomiE FR]

sh0dan : Thanks for this filter ! :) I will try to find out if the leak is gone. Moreover I think you wrote in a post that you were looking for some anime vob source for testing. Maybe I can help you (some anime high quality z1).

[sh0dan]

Sounds great - just drop a PM!

[Dali Lama]

@ Sh0dan,

Since, you are thinking of putting that setting in a preset, I re-evaluated to make sure it would work as an AnimeHQ in almost all cases.

For completeness, I have found *one* scene where there is slight ghosting if you go "frame by frame" in virtualdub. If seen in full motion you will not notice it.

If you lower the temporal thresholds to 5 no ghosting is exhibited. However, I will leave it up to you Sh0dan if you feel it is better to have absolutely no ghosting or undetectable ghosting (IMO) for your preset. Maybe you can do some tests too once HomieFR gets you a sample.

I hope others can test to make a good AnimeHQ setting too,

Dali

[PowerMacG4]

I found ghosting, too with animeLQ. Cowboy Bebop episode 20. I'll make some screens if you want them.

[digitize]

I definately like this filter very much, looks better than my old combo (deen a2d + temporalcleaner). It's able to retain detail, but also compress well by flatening surfaces well. About the ghosting, you can't expect just to use defaults and get optimal results. Now the AnimeLQ may ghost on some sources, but not others, tweak the values in order to prevent this (i.e. the temporal values). Well done sh0dan.