MakeDiff mask problem - static/dynamic noise

[ChaosKing]

I have an anime Bluray with static noise. But some parts of the video (within frames) are dynamic noise. My goal now is to remove the dynamic noise and replace the removed parts with static noise.

Sample to illustrate the static/dynamic noise problem:
https://www.dropbox.com/s/s5j2jmb8q13gjil/noise.mkv?dl=0


My attemt so far:
blur_clip = haf.SMDegrain(clip, tr=4, thSAD = 400, prefilter=2)

grain = blur_clip.grain.Add(var=100.0, uvar=0.0, hcorr=0.0, vcorr=0.0, seed=-1, constant=True)
grain = adjust.Tweak(grain, bright=200) # Just to see which parts are merged

diff_clip = core.std.MakeDiff(clip, blur_clip)
clip = core.std.MaskedMerge(clipa=clip, clipb=grain, mask=diff_clip)
It seems that the output of MakeDiff can not be used as a mask like I did. What could be used instead?

[WolframRhodium]

Replace the last two lines with

diff_clip = core.std.MakeDiff(grain, clip)
clip = core.std.MergeDiff(diff_clip, blur_clip)


or simply

clip = core.std.Expr([grain, clip, blur_clip], "x y - z +")

[ChaosKing]

Hmm nope. It merges everything, like there is no mask at all.

I tried also the example from here with same outcome VS-R43
http://www.vapoursynth.com/doc/functions/mergediff.html

[kgrabs]

Your intent was to remove the random grain and add the same amount of static grain in the same area, right? I think what you want for the mask clip is this:
diff_clip = core.std.Expr([clip,blur_clip], 'x y - abs')

btw integer mvtools only has Degrain# up to 3, you would have to use feisty2's mvtools-sf to use Degrain4 and up (SMDegrain can only use the integer version, though, so...)

[ChaosKing]

I found a mistake in my script "grain = clip.grain.Add..." <-- should be blur_clip and not clip!

I made a comparison between MakeDiff and Expr.
MakeDiff(clip, blur_clip) + MaskedMerge(clipa=clip, clipb=grain, mask=diff_clip1) looks best to me so far.
BUT if you add the Levels line you will see that the whole clip is altered and not only the denoised part like in the last row with Expr.

Is it possible to convert the MakeDiff output (gray) to a (green) mask like in row4 with Expr? :D

@kgrabs Does that mean that Degrain4+ will "convert" it to static grain or that it's just better in terms of stabilising the picture (or grain)?

Ready to use script:

#you need https://github.com/HomeOfVapourSynthEvolution/VapourSynth-AddGrain
import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()

clip = core.std.BlankClip(format=vs.YUV420P8, width=200, height=150, length=100, color=[206,235,135])
g_static = clip.grain.Add(var=100.0, constant=True)
g_dynamic = clip.grain.Add(var=100.0, constant=False)#haf.GrainFactory3(clip, g3str=150, temp_avg=60)#clip.grain.Add(var=100.0, constant=False)


clip = core.std.StackHorizontal([
core.std.StackVertical([g_static, g_static]),
core.std.StackVertical([g_static, g_dynamic])
])
goal = core.std.StackHorizontal([
core.std.StackVertical([g_static, g_static]),
core.std.StackVertical([g_static, g_static])
])
orig=clip



pre = clip.std.BoxBlur(hradius = 4, hpasses = 5, vradius = 4, vpasses = 5)
blur_clip = haf.SMDegrain(clip, tr=10, thSAD = 1000, prefilter=pre)


grain = blur_clip.grain.Add(var=120.0, constant=True)
#grain = grain.std.Levels(min_out=9, max_out=10,gamma=10) # Just to see which parts are merged

diff_clip1 = core.std.MakeDiff(clip, blur_clip)
#diff_clip2 =
diff_clip3 = core.std.Expr([clip,blur_clip], 'x y - abs')

final1 = core.std.MaskedMerge(clipa=clip, clipb=grain, mask=diff_clip1)
final2 = core.std.Expr([grain, clip, blur_clip], "x y - z +")
final3 = core.std.MaskedMerge(clipa=clip, clipb=grain, mask=diff_clip3)


row1 = core.std.StackHorizontal([
clip.text.Text("Source"),
blur_clip.text.Text("smdegrain"),
pre.text.Text("pre blur"),
])
row2 = core.std.StackHorizontal([
final1.text.Text("Final with MakeDiff mask"),
diff_clip1.text.Text("diff_clip1 MakeDiff(clip, blur_clip)"),
grain.text.Text("grain"),
])
row3 = core.std.StackHorizontal([
final2.text.Text('Fin Expr([grain, clp, blur_clp], "x y - z +")'),
diff_clip1.text.Text("IGNORE, diff1 copy"),
grain.text.Text("grain again"),
])
row4 = core.std.StackHorizontal([
final3.text.Text("Final with Expr mask"),
diff_clip3.text.Text("diff_clip3 Expr([clip,blur_clip], 'x y - abs')"),
goal.text.Text("My goal is this"),
])
clip = core.std.StackVertical([row1, row2, row3, row4])
clip.set_output()

[ChaosKing]

It seems that the mask was always to weak so see an effect, even a large addgrain values over 9000 were not visible.

Binarize seems to work nicely here and now I get a good mask for the moving mouth grain.
pre=clip.flux.SmoothT(temporal_threshold=6).std.BoxBlur(hradius = 4, hpasses = 2, vradius = 4, vpasses = 2)
blur_clip = haf.SMDegrain(clip, tr=6, thSAD = 600, prefilter=pre)
grain = blur_clip.grain.Add(var=400.0, constant=True)

diff_clip = core.std.Expr([clip, blur_clip], 'x y - abs')
diff_clip = diff_clip.std.Inflate(threshold=200)

diff_clip = diff_clip.std.Binarize(threshold=3.3, v0=0, v1=80)
clip = core.std.MaskedMerge(clipa=clip, clipb=grain, mask=diff_clip)

https://i.imgur.com/t9YdhuA.png

What is left now is to avoid this filtering on scenes with "full dynamic grain". I think with a value which represents the diff-value between filtered and unfiltered could work.

Somethink like this:
if changes_from_diff_clip < 30: #30%
apply_masked_filtering()

But how would I convert diff_clip to a "changes_from_diff_clip" value?

[kgrabs]

I'd probably binarize the diff clip the rest of the way and use FrameEval
And I meant haf.SMDegrain just silently falls back to tr=3 no matter how high you set it.

Altogether it'd be something like this I think:

import mvmulti as mv
import fvsfunc as fvf

pre = clip.flux.SmoothT(temporal_threshold=6).std.BoxBlur(hradius=4, hpasses=2, vradius=4, vpasses=2)
pre = haf.DitherLumaRebuild(pre, s0=1)
pre = fvf.Depth(pre, 32, range_in=1)

clip = fvf.Depth(clip, 32)

super = core.mvsf.Super(pre,16,16,1,0)
vectors = mv.Analyze(super, blksize=16, overlap=8, search=4, tr=6)
super = core.mvsf.Super(clip,16,16,1,1)
blur_clip = mv.DegrainN(clip, super, vectors, thsad=600, tr=6)

grain = blur_clip.grain.Add(var=400.0, constant=True)

diff_clip = core.std.Expr([clip, blur_clip], 'x y - abs').std.Inflate(threshold=200/255)

mask_clip = diff_clip.std.Binarize(threshold=[3.3/219, 3.3/224], v0=0, v1=80/255)

prop_clip = mask_clip.std.Binarize(threshold=0.00001, planes=0).std.PlaneStats()

# hipass is basically the percentage of the frame you'd be adding grain back to
# so by default if 75% or more of the frame would have static grain added, this skips the frame
# you could probably use some magic to do some sorta temporal check but uh no thx
def grain_inspector(n, f, hipass=0.75):
if f.props.PlaneStatsAverage > hipass:
return clip
else:
return core.std.MaskedMerge(clipa=clip, clipb=grain, mask=mask_clip)

clip = clip.std.FrameEval(grain_inspector, prop_src=prop_clip)

[ChaosKing]

Thank you very much!:thanks:

I had to change FrameEval to FrameEval(functools.partial(grain_inspector, c=clip), prop_src=prop_clip) otherwise I could not get any output.

Now I need to tweak the hipass value. It seems that I need also a lower limit, bcs sometimes the value is 0 or very small in "frozen frames".