LimitedSharpenFaster MVDegrain--Haloing

[rkalwaitis]

I have a simple question.

I was playing around with one on my sources and noticed a slight haloing around the texts in the intro after the denoising and sharpening process. It did not appear on the original source. I was utilizing mvdegrain (fastdegrain.avs by Sagekilla) and LimitedSharpenFaster.avsi (Didee). Of course the obvious attempt would be to be rid of this haloing. What I did was reduce the limt in MVDegrain to 235 and the oshoot in LimitedsharpenFaster to 0 instead or 1.

So the simple question whould be is this a proper way to reduce the haloing? Should I have used a dehalo script? It appears to have reduced it significantly. Also another question is what do I actually change by changing the oshoot from 1 to 0 and reducing the limit of mvdegrain?

I know there are other sharpeners and denoisers, but these are the two which I am making myself use inorder to learn more.:thanks:

[foxyshadis]

Overshoot in LS literally means "how much haloing is acceptable"; if you clamp it to 0 it will never be brighter than the surrounding pixels (the halo effect), but otherwise it can. 1 shouldn't be that visible, but I've seen the same effect and often clamp it to 0 myself. Good work, since it's much better to learn your tools and prevent it in the first place than to attempt to fix it with more filters; that always leads to a bad case of filteritis and detail destruction. Fix your source's problems, not yours.

You might also want to look at LSFmod (http://avisynth.org/mediawiki/LSFmod), with a couple of fixes and a lot of changes.

[*.mp4 guy]

LimitedSharpen(Faster) will always produce haloing when ss_x and ss_y are not set to 1 (disabled). It is not possible to interpolate by non integer resolutions without producing frequency domain artifacts (most commonly observed as haloing), furthermore the supersampling routines in LS(F) are never capable of avoiding such artifacts. In smode=4 overshoot has very little to do with how much haloing is produced. IIRC Overshoot only really behaves as you would expect in smode one and two.

There are a few ways to get supersampling without halos (and supersampling is non-optional if you want good results) but they are all fairly slow, and most people don't appear to care about the fairly minor halos LS(F) generally causes.

[rkalwaitis]

Thank you for your responses. So the limit of 255 actually had nothing to do with the haloing. It was the overshoot. The haloing although light was noticeable around text more than during the actual film. It drives me crazy. I'am also starting to learn that more harm can be done with filters and that less in most cases may be best.



Ill play with lsfmod and mvdegrain. I think mvdegrain seems to be a good denoiser for most occassions and plan on trying to learn more about it and its use with scripts like lsf and lsfmod. I really do not mind slow and am interested in the other ways to supersample without halos! :thanks: again

[Nephilis]

To avoid Halos while using LSF you should;
- Firstly use Lmode=3 [In this mode; LSF sharpen the edges with zero OS & US (to avoid haloing), On not-edges, limited with specified OS + LS]
- As *.mp4 guy said disable supersampling. (ss_x=ss_y=1.0) but though you can see some aliasing on the lines and edges.
- because of sharpen filters are bitrate eater, you must give high bitrate than standart encoding and not high sharpen strength.



_____________________________________

[LaTo]

Some tips with LSFmod:

1. Try with "preblur" parameter:
http://forum.doom9.org/showthread.php?p=1307216#post1307216

2. Try with "source" parameter and turn OFF ss_x/ss_y:
original = last;
denoised = source.FastDegrain();
denoised.LSFmod(source=original,ss_x=1.0,ss_y=1.0,[...]);

3. Try 1+2 together

[Keiyakusha]

Personally I like the result with ss_x=ss_y=1.0 (and without "source" parameter) more than with any amount of supersampling. however I'm only using edgemode=1 and strength never higher than 150-200 (usually 100-150)
I also like speed gain.

[rkalwaitis]

Wow you guys are a wealth of knowledge. Ill try to play with these later today and see what happens.

Lato, I asked a question about prefiltering in another thread recently and Im not sure if I understand why blurring is important on the prefilter. Apparently blurring helps eliminate noise. I looked at your link and read through how to deblur via your script. Im gonna see if I can figure out how to use the blur that is packaged with mvtools with the lsfmod preblur option.

Thanks for your help everyone.

[foxyshadis]

Blurring is a way to figure out what's an edge and what isn't: Pixels that significantly change are edges. (In SeeSaw it has a second use, where some of the image is sharpened, some is blurred, and some is mostly left alone.) I've always had pretty good success with the included simple blurs.

While LSF can be useful as part of a surgical script to recover badly blurry or blocky movies, I rarely use it to encode, but all the time on playback. As long as real detail isn't removed during encoding, the lack of fake detail keeps the file sizes much more reasonable. An HTPC with simple avisynth sharpening can blow away a bad bluray. (And Bluray+Seesaw is a beautiful combo when studios use dvd-style soft focus.)

[rkalwaitis]

So are edgemasks made via a blur? And if so, is it necessary to do a prefilter blur or can I use an edgemask, or/and enhanced edgemask. Ill checkout seesaw again and see if I can figure out how Didee is doing his magical blurring. Speaking of simple blurs, that is me simple. I used the MFlowBlur with MVTools2. It seems to do a reasonable job. Of course Im not trying to squeeze a 2 hour source into 700mb either. I use 20-22 Constant quality. Here is how I did it.

selectrangeevery(400,16)
super = MSuper()
backward_vectorsblur = MAnalyse(super, isb = true)
forward_vectorsblur = MAnalyse(super, isb = false)
prefiltered = MFlowBlur(super, backward_vectorsblur, forward_vectorsblur, blur=15)# Blur default is 50, however the author's example had a 15% so that is what I used. Blur can not be more than 200% so anything over 200 will through an error. So Knowing that 15 isnt really to much of a blur, but I dont know how much blur I need to successfully find the edges well.

superfilt = MSuper(prefiltered)
backward_vectors = MAnalyse(superfilt, isb = true,delta = 2, overlap=2)
forward_vectors = MAnalyse(superfilt, isb = false,delta = 2, overlap=2)
forward_compensation = MCompensate(super, forward_vectors)
backward_compensation = MCompensate(super, backward_vectors)
interleave(forward_compensation, last, backward_compensation)
DeGrainMedian().sharpen(.3).sharpen(.1).sharpen(0.05) ## I used avisynth's internal sharpen in small bites until I got the sharpness I liked.
selectevery(3,1)

Im gonna have to figureout how to make a simple function out of my preblur with mflowblur, so I can experiment with LSFMod's preblur.

I think this is how to use it with MVDegrain2. Since this is my choice of poison until I figure it out well and good.

selectrangeevery(400,16)
super = MSuper()
backward_vectorsblur = MAnalyse(super, isb = true)
forward_vectorsblur = MAnalyse(super, isb = false)
prefiltered = MFlowBlur(super, backward_vectorsblur, forward_vectorsblur, blur=100)#blur=15)
backward_vec2 = MAnalyse(msuper(prefiltered), isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(msuper(prefiltered), isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(msuper(prefiltered), isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(msuper(prefiltered), isb = false, delta = 2, overlap=4)
MDegrain2(msuper(prefiltered), backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

#now I can post sharpen with avisynth's sharpener to taste.

[MatLz]

Also, aren't Lanczos resizers known to produce halos ?
A little of course...but inside a sharpening function that maybe grows the problem a little more.

[Didée]

It's a simplification to say "edgmasks are based on blurring", but yeah, that is more or less the general idea. There are many different methods of edge masking, but usually, they are based on pixel differences. If the difference between neighboring pixels is big, then the "edge value" is big. If the pixel differences are small, then the "edge value" is small.
(Note that when "strong noise" is present, then also the noise is causing big pixel differences....)


Regarding pre-blurring:

Understand that this is not a self-satisfying task. It's not to be done "because it should be done". It is suited for cases where filter X has a problem with some specific characteristic of the source. In that case, it is often better to filter that specific characteristic with another filter. (!another! filter. not the same filter!)

The usual rule-of-thumb is that the pre-processing filter and the actual processing filter should work "perpendicular", meaning they should work in a different way. When the actual filter is a temporal filter, then a possible preprocessing filter should be spatial, not temporal. (See e.g. "ttempsmooth" filter - for its "pclip" option, you usually want to aplly a spatial filter, not a temporal. In case of doubt, just try.) - Vice versa, if the actual processing filter is a spatial filter, then a possible preprocessor should be temporal, not spatial.

Reasoning is quite simple: if any given filter has a certain "problem" with the source, then it is very likely that another filter which works in a similar way, will have the same "problem". But, if the preprocessor has the same "problem" as the actual processing filter, then the preprocessing makes not much sense.

In particular, it is not the best idea to use MVTools-filtering as a preprocessor to MVTools-Filtering. Say that the source is very noisy, and MAnalyse accidentially "compensating the noise". (Result is unstable, moving textures, like in walls, sky, etc.)
Now, when using a MVTools-based preprocessor, then the following can happen:
- normal MVTools application results in swimming walls
- Use MVTools as preprocessor:
-- the preprocessor creates swimming walls
-- the actual MVTools filter now becomes swimming walls as input, and therefore necessarily creates ... swimming walls

==> The process has become (much) slower, but it has not become any better.

***

Then, pre-denoising for a sharpening operation (like in LS/F, SeeSaw, or whatever) has its own particularies. The goal usually is to exclude noise from the sharpening process, so that the noise won't get enhanced.
However, a true *blurring* operation is dangerous, in particular for "limiting" sharpeners that try to avoid halo creation. If an edge is blurred, then the blurring process creates additional headroom for the limited-sharpening process, which subsequently will lead to halos in the end result (because the original edge, to which the result is applied, did have less headroom than the preprocessed edge.)


All of that may sound very complicated, but it's much easier than it sounds. As usual, best way is practice. A description of a pitfall may or may not give a good picture. But once you actually have been fallen and been trapped in a pitfall, you'll surely reckognize it anytime in the future. ;)

[rkalwaitis]

Didee,

The use of spatial and temporal filters working together one as a prefilter and one as the denoiser makes sense to me. I can see how using mflowblur and mvdegrain, especially as I had used them may bring about problems and only compound them. Ill try to come up with something a bit better and let you critique if you so desire.