CalmGrain - FluxSmooth, now mostly on grain!

[Bloax]

Oh snap, it's me again! (Yipe!)

This time, I've been looking at temporal denoisers.
FluxSmooth was one of those I best liked, due to the fact that it was fast. (And about as good as everything non-complex.)

The problem is, however - that FluxSmooth, like many other Temporal DN'ers, tend to grab pretty large details at high thresholds.
Whilst I wanted to stabilize the grain, and not everything else. Coincidentally, grain only begins to get stable at high thresholds.

You can see what this leads to, hmm?

And so I came up with a so-so solution today, by using a spatial denoiser to get a difference mask of the noise, and using that to filter out everything but (mostly) noise that was temporally denoised.

The result? More or less stable grain! (Hooray!)
The gains turned out to be pretty minimal, but at least the noise artifacts post-encoding should be more stable now. ;P

Do report issues here, though I doubt I can do much about them. http://i988.photobucket.com/albums/af10/Vilde123/laff1010.gif

(The package of plugins required should be here.) (http://www.mediafire.com/?1ucjyg6tya0krsm)

[Old (first) version] (http://pastebin.com/KBUxWVkK)
[Previous version] (http://pastebin.com/YuN0ZHZA)
function CalmGrain(clip clp,int "tstr",int "mskstr",int "mskbias")
{
# Requirements: RemoveGrain(SSE3).dll, FluxSmooth.dll, RedAverage.dll (Just look up "RedAverage" on Google.)
# By yours truly, Bloax.

# ~~~~~ Wuzzis now?
# This 'should' calm grain down, I can now say that it "should" improve compressibility by a little bit.
# It certainly reduces the eye-tearing effect of heavy noise compression artifacts post-encoding though. :-)
# ~~~~~

# ~~~~~ Wuzzis do now?
# "tstr" Is the strength of FluxSmoothT, higher values cause more pixels to hang around for the ride.
# Use values around 10 (for very slight reduction) to about 40-60 (or 255 if you so wish), too high values can look horrible though.
# Watch out for artifacts at higher values (eg. Default :P), though!

# "mskstr" Is the intensity of the mask, though instead of the usual multiplication, here I set it to addition.
# Higher values make the mask more intense, values like 45 would result in something that looks like a binary mask.
# While low values, like -15, would look a bit less extreme.
# But honestly, just leave it be. Unless you feel like experimenting.
# Don't go below -17 though!

# "mskbias" Is the bias value for the mask, don't ask what exactly it does though!
# Just leave it there, or mess around with it if you feel like it.

tstr = Default(tstr, 58)
mskstr = Default(mskstr, 0)
mskbias = Default(mskbias, -32)

a = clp.IsYV12() ? clp : clp.ConvertToYV12()

noise = a.RemoveGrain(17,-1).RemoveGrain(4,-1) # You can replace this line with whatever denoiser you want.
# I recommend you to make said denoiser only denoise luma, as that's faster, and provides usable masks anyway.
msk = RAverageW(noise,19+mskstr,a,-18-mskstr,mode=8,u=0,v=0,bias=mskbias) # Set u & v to 3 to include chroma differences.
# It's faster to just do it with luma, and the default denoising is set to luma only. (* luma,-1 chroma)
RMerge(a,FluxSmoothT(a,tstr),msk,mode=255)
}

Latest version (RAverage/Merge one) ought to be about 20% faster than the previous.
Mostly due to a simplification in the denoising chain. It should in addition be less "blended"-ish now. (Oopsie.)

[SSH4]

mt_makediff(last,mt_lut(y=128,chroma="128").addgrainC(5).temporalsoften(2,255,255,255,2),u=3,v=3)

[Bloax]

I don't see what exactly that is, but it looks like a substitute for this little script.
The problem is though, that I'm going for really strong stabilization, and not slight like that seems to provide.

Not to mention that it adds some grain of it's own. :confused:

All in all, you can see what I mean by comparing the two on this video. (www.mediafire.com/?143936wk03g3jhs)

Thanks for the effort anyway. (?)

[Gavino]

mt_makediff(last,mt_lut(y=128,chroma="128").addgrainC(5).temporalsoften(2,255,255,255,2),u=3,v=3)

Do you mean y=-128?

[SSH4]

Gavino with y=128 == y= -(-128) => (256-128=128)
y=64 (y=-(-64)) => 256-64 => y=-192

Bloax yes and now.
use 128,128,128 YV12 gray and generate your own kind of grain: temp,size,anisotropic... and add your grain to original video with mt_makediff.
Check how made GrainFactory3 script for example.

[Didée]

@ Bloax:

In my opinion, the masking procedure is not correct.

> c = Mt_MakeDiff(a,b).Mt_Expand().Mt_Inflate()

And that's the mask, with the "measure" created by basically removegrain(4).removegrain(19).
Now, the mask is 128 where the RemoveGrain's did "nothing", more "blackish" where the RemoveGrains made pixels darker, and more "whiteish" where they made pixels brighter.

So you get 50% denoising where RemoveGrain did nothing, a tendency to "no denoising" where RG made pixels darker, and a tendency to "full strength" where RG made pixels brighter.

Of course, you can always excuse with "That's exactly how I want it to be!". ;)

Still, the "theoretically-correct" way without any mysterious bias would be this:

> c = Mt_lutxy(a,b,"x y - abs 2 *").Mt_Expand().Mt_Inflate()


Oh, and 1.66*0.66 is not "1", but ~1.096. Another mysterious bias. ;)

[Bloax]

This isn't about adding grain, this is about "calming" existing grain.

Just look at the video, and note how the floor texture flickers in the distance.
That's what this script fixes. It's a temporal denoiser, except that it only stabilizes what a spatial denoiser finds. (That is, noise!)

---

ASDF new post. D:

The weird nboost bias is the result of me getting tired of multiplication correlation. (Eg. (x*1.25)*0.8 = x)
Which is mostly why I leave a comment on just leaving it be.

Also, for some mysterious reason your version gives me worse results. Less noise ends up getting stabilized. :P
Or rather, stuff ends up being less stable. (It is a mystery!)
I also have no idea what 90% of the stuff Mt_LUTxy does is. (No documentation, you see. ;()

[Didée]

Oh ... yes, that's correct. With my "correction you get massively less denoising.

The reason is that you have a whacky measure of where the noise is, and that whackyness has been compensated by another whackyness (the grey-centered mask, that gives you 50% denoising even when "nothing" has been detected.)

Hence, the "correct" suggestion does not fit in your, erh, 'mysterious' framework. ;)

It would rather need sth like "x y - abs 100 *" to get full denoising strength ... but somehow, then the whole thing wouldn't make much sense anymore.

(Not sure if it does in the first place, but I won't start a debate on principles...)

[Bloax]

Just watch the video and look at the floor texture in the distance.
That's about all I designed this script to fix, without everything much bigger getting caught in the high thresholds. :P

(And so everything starts to make sense!!)

Let's just say that I'm happy as long as it works. And so it does, frankly.. Uh.. δωδ
Yeah, that'll do.

[Didée]

Which video - "LeTest.avi", which is 1990's game footage (or at least looks like that) ?

Gosh! In that video, there is not one single pixel that FluxSmooth could ever improve! (Because there is no noise or grain to start with). On that video, FluxSmooth can do nothing good. Nothing. If it does anything, it will be harm. So yes, everything that dimishes FluxSmooth's actual performance will be an "improvement" on that very "test video". But if that has anything to do with realworld performance, that is another question. (I reckon "no it hasn't".)

I mean, c'mon. The task is to remove grain, isn't it? What's the point in taking a perfectly grainless video, and just judge by the not-damage that is done?

For that case, the *perfect* filter is much easier and faster:

NOP()

This does absolutely no damage on "LeTest.avi". Ah. Purrfect.


For realworld tasks,

FluxSmooth(12,4).FluxSmoothT(4).Merge(last,0.49)

is an easy & useful filter combo, BTW.

[Bloax]

Well, that really depends on what one classifies as grain!
:stupid:

Edit: Oopsie, once again. This isn't about removing grain, it's about making it stable. (And thus more pleasing to look at in extreme cases, eg. Below.) :P

I'll just go ahead and post two images here, one with stuff I'd like to remove (http://i.imgur.com/A6F9E.gif), and one with it nicely reduced (http://i.imgur.com/YYxEq.gif). (See: Floor.)

Though I really don't know, you guys can go ahead and classify my filters as ones made for game recordings.

Because honestly, I have no "real" material to work with. :p

Also, looks like I'm seeing a reduction because of the spatial grain removal for masking purposes.
Though it looks like there's still something temporal-ish about it. (Crysis 2 without antialiasing still looks more stable.)

Ehh, looks like this stuff deserves another one.
:stupid:

[SSH4]

remove grain
add your own

[Mug Funky]

there was a filter floating around a couple of years ago called "grainoptimizer" which will achieve what i think you're looking for - to take what is essentially noise (be it alias noise that would have been signal if only the samplerate was high enough, or wouldn't be present if only the sampling was filtered before reconstruction) and make it sit still while maintaining the same "texture".

it was very good for compression, though IMHO the later versions of it (that required another denoiser) didn't result in what i thought was great about the original filter.

of course, after that x264 got really, really good (rather than just really good) and so filters of this kind became less necessary.

[DeathAngelBR]

ttempsmooth().fft3dfilter().gradfun3()

^^^^ does the trick for me

[chainring]

there was a filter floating around a couple of years ago called "grainoptimizer" which will achieve what i think you're looking for - to take what is essentially noise (be it alias noise that would have been signal if only the samplerate was high enough, or wouldn't be present if only the sampling was filtered before reconstruction) and make it sit still while maintaining the same "texture".

it was very good for compression, though IMHO the later versions of it (that required another denoiser) didn't result in what i thought was great about the original filter.

of course, after that x264 got really, really good (rather than just really good) and so filters of this kind became less necessary.IIRC, grainoptimizer was written by Dark Shikari, if that helps anyone.

[Keiyakusha]

IIRC, grainoptimizer was written by Dark Shikari, if that helps anyone.

Yeah but he has no interest in it. At all...

BTW I was looking for some solution that should do similar thing. Like if we have animation with static background, no noise only dither that jumps here and there pretty heavily. So I wanted a solution that will make static parts in 2 (or more) nearby frames looks the same (as much as possible of course). No other filtering/smoothing should be done. Only replacing similar parts of the frames with exactly the same stuff, maybe average of both, don't know.
But solution from this thread meant to be used for something different or it just does nothing.

[Bloax]

Well, there's no harm in trying.
Though I really don't know what this could be used for, because once again - I have no 'real' material to test/tweak for/work with.

Edit: Whoah, this here is one beastly script:

msk=Mt_MakeDiff(RemoveGrain(4)).Mt_Inflate()
CalmGrain().Merge(Mt_Merge(GrainOptimizer(RemoveGrain(4).RemoveGrain(17),strengt*h=3,tdist=16,minrep=4),msk,u=3,v=3),0.75)
AntiAliasRG(sharpen=false).Merge(last,0.4).AntiAliasRG(mode="prewitt",sharpen=fa*lse).Merge(last,0.33)
I did know of GrainOptimizer, I just didn't see that 2.01 was still available. (Whereas 2.02 is not.) It's pretty cool, though to achieve the results I want it ends up causing a lot of artifacts. :P

[Bloax]

I call forth the power of the damned to rise my minions from the grave!! [/necrobump]

On topic, script got a little update. It works better, it works faster. And it works a bit messier. (Whoops!)
Just posting it because well, it's a script of mine - after all.

Funnily enough, on the video I posted, it works much better than GrainOptimizer. (73% vs. 98%)
But yeah, a little update in case someone gets an use for this. (Oh, how I doubt that!)

[Bloax]

Ahh!!

I updated it again! (As kind-of promised in the RedAverage thread, heh.)

Now it ought to be more-or-less usable, and there ought to be none of the blending caused by my stupidity.
I'll go ahead and say that you could change the weights from (*,x,*,-(x+1)) to (*,x,*,-x) to gain a nice absolute noise filter, but for some reason I decided to mask some more. :P

Oh silly me, hmm? ;)

[Didée]

The masking still is flimflam. But hey, it's 20% faster now. ;)

[Bloax]

Oh but believe me, it's fine if you do what I mentioned there. :P
It's just that I'm silly and I prefer to have a wide area masked.

Alternately, I could of course use Mt_Expand/Inflate, but that would require yet another (common, haha) plugin.

[Didée]

The masking is strongly related to the local luma value. It does something, but it's not very meaningful. A closer explanation was already given earlier in this thread.

As an analogy, one could steer the denoising strength by the loudness of the audio track: loud parts get strong denoising, silent parts get little denoising. This would be similarly meaningful.