Erroneous 'pre-cast' shadow with TemporalDegrain

[asarian]

I added a ".TemporalDegrain(degrain=3, ov=4, blksize=16)" command to an existing avs script, so as to denoise my 2010 BluRay. All seemed well, but at ca 20 secs into this 1 min CLIP (http://rapidshare.com/files/395372334/2010-td.m2ts.html) you'll see a brief, erroneous. what I'd call 'foreshadow' of the scene to come: a shadow-image of part of the lamp above the desk and some other stuff. Obviously, this error does not exist in the source.

Too bad, because TemporalDegrain degrained very well. But errors like these I can't live with, of course. So, is this something which can be done to avoid this on a new pass? Or maybe there are better temporal degrainers out there? I'd really like to degrain this source properly, error-free.

[poisondeathray]

I haven't looked at the clip yet, but what source filter did you use? I've seen directshowsource do weird things with temporal filtering . A frame accurate one is preferred

When you remove TemporalDegrain , the artifacts are gone?

[asarian]

I haven't looked at the clip yet, but what source filter did you use? I've seen directshowsource do weird things with temporal filtering . A frame accurate one is preferred

When you remove TemporalDegrain , the artifacts are gone?
Yeah, Í couldn't keep things frame accurate with DirectShowSource, so I switched to using FFVideoSource recently, which does the job perfectly. Basically I use a line like:

FFVideoSource("D:\Temp\video.mkv").ConvertToYV12().TemporalDegrain(degrain=3, ov=4, blksize=16)

And yes, without TemporalDegrain the fragment encodes just fine, without the distortion.

[Didée]

TemporalDegrain is a pretty strong filter - I don't know the source, does it really need that strong filtering? With what settings did you use TemporalDegrain? Custom settings, or just default?

I didn't load your sample (dislike Rapidshare), but the problem is "bleeding across" a scenechange, yes? Well ... internally in TemporalDegrain, FFT3DFilter is used with overly-strong settings (cleaning for motionvector search only). After all, the default settings were made for the horrendously-strong grain of the "300" movie ...
FFT3DFilter *generally* is not safe at scenechanges, and at very strong settings, it can happen to smear over scenechanges. This doesn't influence the output, yet. But if FFT3DFilter smears a scenechange, then this may cause the upfollowing MVTools to also miss the scenechange - and if MVTools misses the SC, then you have it in the output, too.

Reduce strength, use another prefilter for TemporalDegrain, use another denoiser, or use no filtering at all. Lots of opportunities.

[asarian]

TemporalDegrain is a pretty strong filter - I don't know the source, does it really need that strong filtering?
Yeah, it's pretty bad. It's the Blu-Ray of 2010.


With what settings did you use TemporalDegrain? Custom settings, or just default?
Pretty much the settings I used above: "degrain=3, ov=4, blksize=16"


I didn't load your sample (dislike Rapidshare), but the problem is "bleeding across" a scenechange, yes? Well ... internally in TemporalDegrain, FFT3DFilter is used with overly-strong settings (cleaning for motionvector search only). After all, the default settings were made for the horrendously-strong grain of the "300" movie ...
You guessed correctly. :) The "bleeding across" is precisely the problem.


FFT3DFilter *generally* is not safe at scenechanges, and at very strong settings, it can happen to smear over scenechanges. This doesn't influence the output, yet. But if FFT3DFilter smears a scenechange, then this may cause the upfollowing MVTools to also miss the scenechange - and if MVTools misses the SC, then you have it in the output, too.

Reduce strength, use another prefilter for TemporalDegrain, use another denoiser,
Pardon my daftness in these matters, but I presume FFT3DFilter is the prefilter you're talking about, right? I wonder why TemporalDegrain uses it then, internally, if it's unsafe to use. I mean, who wants to encode for a whole day, just to find your scenes got smeared?

More to the point, though, what other prefilter could I use instead of FFT3DFilter then? That's sort of new teritory for me. I just wanted a good degrainer. :) Seems x264 has built-in degraining now too, but it's not temporal, near as I can tell; and looks like I really need a temporal solution.

[Didée]

Pardon my daftness in these matters, but I presume FFT3DFilter is the prefilter you're talking about, right? I wonder why TemporalDegrain uses it then, internally, if it's unsafe to use. I mean, who wants to encode for a whole day, just to find your scenes got smeared?
Yes, FFT3DFilter is doing the pre-filtering.

Well, please note that I did not make the function "TemporalDegrain". Back when the question came up how to deal with the insane grain of "300", I showed a linear script that is technically able to deal with such strong grain. (Normal appliance of MDegrain would be confused by the grain itself, resulting in "swimming" textures.) But I never claimed that the provided script would be "safe for general use". If I had thought so, I would have made a function out of it by myself. But I didn't, and probably for a reason.

Probably it wouldn't be very difficult to make it scenechange-aware. Obvious possibilities:

a) combining FFT3DFilter with SCSelect

b) do a MCompensate-preparation for FFT3DFilter (yes it seems insane to use MVTools just for the preparation of a PREfilter ... but it works, and for this purpose very quick-n-dirty settings can be used)

c) just use another prefilter. MedianBlur+FluxSmooth is a quite fast alternative - the overall end result might be slightly less quality, but it's quite fast, and very robust regarding scenechanges.



BTW, care to post a sample of the untouched source? I don't know how it looks like, but I bet it's not half as serious as you think it was. ;)

[asarian]

Well, please note that I did not make the function "TemporalDegrain". Back when the question came up how to deal with the insane grain of "300", I showed a linear script that is technically able to deal with such strong grain. (Normal appliance of MDegrain would be confused by the grain itself, resulting in "swimming" textures.) But I never claimed that the provided script would be "safe for general use". If I had thought so, I would have made a function out of it by myself. But I didn't, and probably for a reason.
Well, please note that I didn't mean to imply you were responsible somehow. Just chalk that up as ignorance on my end. :) I simply knew little to nothing about the background of TemporalDegrain. And I do appreciate all the help.

I'll go try and make some of your suggestions work (if I can).


BTW, care to post a sample of the untouched source? I don't know how it looks like, but I bet it's not half as serious as you think it was. ;)
Sure. HERE (http://rapidshare.com/files/396070248/2010raw.m2ts.html) it is. I would call it 'pretty bad.' 2010 is probably one of the only few Blu-Rays I ever regret having spent money on, because of the heavy grain.

[Mug Funky]

haha. people want high-def, then complain about the extra information :)

all you get extra from a blu-ray with respect to the original film is sharper edges and more grain (and less digital noise). so we all filter our blu-rays to look like the DVD did...

these days i've been doing my personal encodes with no filtering at all - just CRF 20 x264. it handles that grain perfectly, and i'd rather watch something analog looking than be tortured with the consequences of my own filtering everytime i see a movie :)

...that said, i'm a film guy and know well what the stuff looks like uncompressed. there's much more grain than you realise. blu-ray has just shown the world how grainy camera negative can be, as previously standard def within 9mbps or so has just not been able to show it in it's full dirty glory.

you should watch a film shot on standard 16 scanned in high def. there's a grainfest for you. still beautiful if done right, but grain comes with it.

[asarian]

...that said, i'm a film guy and know well what the stuff looks like uncompressed. there's much more grain than you realise. blu-ray has just shown the world how grainy camera negative can be, as previously standard def within 9mbps or so has just not been able to show it in it's full dirty glory.

you should watch a film shot on standard 16 scanned in high def. there's a grainfest for you. still beautiful if done right, but grain comes with it.
All true. But not every Blu-Ray is grainy. For example, the 2001 Blu-Ray, which I also have, is much, much cleaner. Far as I'm concerned, they just did a bang-up job on 2010; and I want the grain gone.

[Didée]

Thanks for the sample. Won't dive into the "remove grain, or rather keep it?" question now (purists surely say "keep", but in last instance it's a matter of personal preference) ...

... but in any case: in this sample I only see "normal" film grain. Not particularly bad or serious. And probably not particularly difficult to remove. Needs some toying-around to find well-suited settings, but I don't see that a sledgehammer of a filter should be needed. Seems pretty normal to me, not at all bad or ugly. ("300" is several-orders-of-magnitude more severe!)

[Lyle_JP]

All true. But not every Blu-Ray is grainy. For example, the 2001 Blu-Ray, which I also have, is much, much cleaner. Far as I'm concerned, they just did a bang-up job on 2010; and I want the grain gone.

2001 was shot on 65mm and Stanley Kubrick lit the shit out of everything. Peter Hyams shot 2010 with scope lenses in low light with "fast" exposure 35mm film. You cannot compare the two! This isn't a case of good blu-ray/bad blu-ray. Both blu-rays accurately represent the look of the films they are presenting.

[tormento]

Back when the question came up how to deal with the insane grain of "300", I showed a linear script that is technically able to deal with such strong grain.

You meant this (http://forum.doom9.org/showpost.php?p=1073349&postcount=33)?

[Didée]

No. That's a low-frequency stabilisator.

What I meant was this (http://forum.doom9.org/showpost.php?p=1076276&postcount=61).


Also, your partial quote is misleading. When looking at the FULL paragraph,Well, please note that I did not make the function "TemporalDegrain". Back when the question came up how to deal with the insane grain of "300", I showed a linear script that is technically able to deal with such strong grain. (Normal appliance of MDegrain would be confused by the grain itself, resulting in "swimming" textures.) But I never claimed that the provided script would be "safe for general use". If I had thought so, I would have made a function out of it by myself. But I didn't, and probably for a reason.
it should've been clear anyway, and the last three posts superfluid.

In case it's still not clear - I was talking about a basic method that I had shown up, and about the functions that other people derived from it.

[tormento]

What I meant was this (http://forum.doom9.org/showpost.php?p=1076276&postcount=61).
I am sorry if my knowledge of english is not good enough, however, with some trial and error I have tried to port your script to MVTools2:

DGMultiSource("e:\in\2_48 Salvate il soldato Ryan\salvate_x64.dgi",fieldop=0)

o = last
fft = o.fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8)
fftD = mt_makediff(o,fft)

super = fft.MSuper(pel=2, sharp=2)
bv3 = super.MAnalyse(isb = true, delta = 3, overlap=4, blksize=16)
bv2 = super.MAnalyse(isb = true, delta = 2, overlap=4, blksize=16)
bv1 = super.MAnalyse(isb = true, delta = 1, overlap=4, blksize=16)
fv1 = super.MAnalyse(isb = false, delta = 1, overlap=4, blksize=16)
fv2 = super.MAnalyse(isb = false, delta = 2, overlap=4, blksize=16)
fv3 = super.MAnalyse(isb = false, delta = 3, overlap=4, blksize=16)

NR1 = o.MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)
NR1D = mt_makediff(o,NR1)

DD = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x = o.mt_makediff(DD,U=2,V=2)

NR2 = NR1x.MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)

s = NR2.minblur(1,1)
allD = mt_makediff(o,NR2)
ssD = mt_makediff(s,s.removegrain(11,-1))
ssDD = ssD.repair(allD,1) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssDD,U=2,V=2)

return(last)

#------------------------------------------
# Taken from MCBob.avs:
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

Is it correct?

[Didée]

No, it's not fully correct. The "super" clip needs to be re-created for each denoising instance. (With the way you have it now, you will get a super-blurry result...)

I'll post the correction later, also some suggestions for improvement. For 1080p, the script could need some adjustments.

[tormento]

No, it's not fully correct. The "super" clip needs to be re-created for each denoising instance. (With the way you have it now, you will get a super-blurry result...)
Thank you, Didée. I was getting a very blurry frame indeed.
I'll post the correction later, also some suggestions for improvement. For 1080p, the script could need some adjustments.
Near some objects, a "grain halo" remains and there are some not denoised frames. Do you think it's a result of my faulty implementation or it's a limit of motion compensation and needs more fine tuning?

Here a video sequence as example.

http://img19.imageshack.us/img19/5830/save0000h.th.png (http://img19.imageshack.us/i/save0000h.png/) http://img805.imageshack.us/img805/8446/save0001.th.png (http://img805.imageshack.us/i/save0001.png/) http://img821.imageshack.us/img821/9220/save0002.th.png (http://img821.imageshack.us/i/save0002.png/) http://img94.imageshack.us/img94/5443/save0003.th.png (http://img94.imageshack.us/i/save0003.png/)

[Nephilis]

How about this ? Can this script Salvate il soldato Ryan ? :)


DGMultiSource("e:\in\2_48 Salvate il soldato Ryan\salvate_x64.dgi",fieldop=0)

src = last
preNR = src.fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8)
fftD = mt_makediff(src,PreNR)

PreNR_super = PreNR. MSuper(pel=2, sharp=2)
src_super = src.MSuper(pel=2, sharp=2, levels=1)

bv3 = MAnalyse(PreNR_super, isb = true, delta = 3, search=5, blksize=16, overlap=8)
bv2 = MAnalyse(PreNR_super, isb = true, delta = 2, search=5, blksize=16, overlap=8)
bv1 = MAnalyse(PreNR_super, isb = true, delta = 1, search=5, blksize=16, overlap=8)
fv1 = MAnalyse(PreNR_super, isb = false, delta = 1, search=5, blksize=16, overlap=8)
fv2 = MAnalyse(PreNR_super, isb = false, delta = 2, search=5, blksize=16, overlap=8)
fv3 = MAnalyse(PreNR_super, isb = false, delta = 3, search=5, blksize=16, overlap=8)

NR1 = src.MDegrain3(src_super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)
NR1D = mt_makediff(src,NR1)

DD = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x = src.mt_makediff(DD,U=2,V=2)

NR_super = NR1x.MSuper(pel=2, sharp=2, levels=1)

NR2 = src.MDegrain3(NR_super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)

s = NR2.minblur(1,1)
allD = mt_makediff(src,NR2)
ssD = mt_makediff(s,s.removegrain(11,-1))
ssDD = ssD.repair(allD,1) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssDD,U=2,V=2)

return(last)

#---------------------------------------

function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

[Didée]

@ Nephilis: You are suggesting the script that tormento has posted three posts above. What's the point?



@ tormento:
When grain is "not touched" in certain spots, that's because MAnalyse couldn't compensate those parts. When a complete frame is not touched, that's because MVTools' scenechange threshold was triggered.


Below are some scripts to try.

Disclaimer#1: none of these scripts was actually tested, I just wrote them down. Hope I didn't make errors.

Disclaimer#2: These are just out of technical interest. For SPR, I'd rather try to keep the source, and let x264 compress stronger. With grain-tuned (psy-tuned) settings, it should do its job quite well.


1) The basic script, with errors corrected.
# Basic script, corrected

DGMultiSource("e:\in\2_48 Salvate il soldato Ryan\salvate_x64.dgi",fieldop=0)

o = last
fft = o.fft3dfilter(sigma=16,sigma2=10,sigma3=6,sigma4=4,bt=5,bw=16,bh=16,ow=8,oh=8)
fftD = mt_makediff(o,fft)

fft_sup = fft.MSuper(pel=2, sharp=1)
bv3 = fft_sup.MAnalyse(isb = true, delta = 3, overlap=8, blksize=16)
bv2 = fft_sup.MAnalyse(isb = true, delta = 2, overlap=8, blksize=16)
bv1 = fft_sup.MAnalyse(isb = true, delta = 1, overlap=8, blksize=16)
fv1 = fft_sup.MAnalyse(isb = false, delta = 1, overlap=8, blksize=16)
fv2 = fft_sup.MAnalyse(isb = false, delta = 2, overlap=8, blksize=16)
fv3 = fft_sup.MAnalyse(isb = false, delta = 3, overlap=8, blksize=16)

o_sup = o.MSuper(pel=2, sharp=2, levels=1)
NR1 = o.MDegrain3(o_sup,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)
NR1D = mt_makediff(o,NR1)

DD = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x = o.mt_makediff(DD,U=2,V=2)

NR1_sup = NR1.MSuper(pel=2, sharp=2, levels=1)
NR2 = NR1x.MDegrain3(NR1_sup,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=300)

s = NR2.minblur(1,1)
allD = mt_makediff(o,NR2)
ssD = mt_makediff(s,s.removegrain(11,-1))
ssDD = ssD.repair(allD,1) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssDD,U=2,V=2)

return(last)

#-----------------------------------------


2) The same script, with some adjustments more suited for 1080p# Enhanced script #1: Better suited for 1080p (probably)

DGMultiSource("e:\in\2_48 Salvate il soldato Ryan\salvate_x64.dgi",fieldop=0)
o = last

fft = o.fft3dfilter(sigma=20,sigma2=12,sigma3=7.5,sigma4=5,bt=5,bw=24,bh=24,ow=12,oh=12)
fftD = mt_makediff(o,fft)

fft_sup = fft.MSuper(pel=2, sharp=1)
bv3 = fft_sup.MAnalyse(isb = true, delta = 3, overlap=8, blksize=16)
bv2 = fft_sup.MAnalyse(isb = true, delta = 2, overlap=8, blksize=16)
bv1 = fft_sup.MAnalyse(isb = true, delta = 1, overlap=8, blksize=16)
fv1 = fft_sup.MAnalyse(isb = false, delta = 1, overlap=8, blksize=16)
fv2 = fft_sup.MAnalyse(isb = false, delta = 2, overlap=8, blksize=16)
fv3 = fft_sup.MAnalyse(isb = false, delta = 3, overlap=8, blksize=16)

o_sup = o.MSuper(pel=2, sharp=2, levels=1)
NR1 = o.MDegrain2(o_sup,bv1,fv1,bv2,fv2, thSAD=400)
NR1D = mt_makediff(o,NR1)

DD = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x = o.mt_makediff(DD,U=2,V=2)

NR1_sup = NR1.MSuper(pel=2, sharp=2, levels=1)
NR2 = NR1x.MDegrain3(NR1_sup,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400)

s = NR2.minblur(2,1)
allD = mt_makediff(o,NR2)
ssD = mt_makediff(s,s.removegrain(20,-1).removegrain(11,-1)).mt_lut("x 128 - 1.6 * 128 +")
ssDD = ssD.repair(ssD.repair(allD,1),12) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssDD,U=2,V=2)

return(last)

#------------------------------------------


3) Be more exhaustive in everything. Additional filter: Fluxsmooth

# Enhanced script #2: Put more effort into everything

DGMultiSource("e:\in\2_48 Salvate il soldato Ryan\salvate_x64.dgi",fieldop=0)
o = last

o_sup1 = o.MSuper(pel=1)
bv02 = o_sup1.MAnalyse(isb = true, delta = 2, blksize=32, overlap=4, search=4)
bv01 = o_sup1.MAnalyse(isb = true, delta = 1, blksize=32, overlap=4, search=4)
fv01 = o_sup1.MAnalyse(isb = false, delta = 1, blksize=32, overlap=4, search=4)
fv02 = o_sup1.MAnalyse(isb = false, delta = 2, blksize=32, overlap=4, search=4)

interleave(o.MCompensate(o_sup1,fv02),o.MCompensate(o_sup1,fv01),o,o.MCompensate(o_sup1,bv01),o.MCompensate(o_sup1,bv02))
fft = last.fft3dfilter(sigma=20,sigma2=12,sigma3=7.5,sigma4=5,bt=5,bw=16,bh=16,ow=8,oh=8).SelectEvery(5,2)
fftD = mt_makediff(o,fft)

fft_sup = fft.FluxSmoothT().MSuper(pel=2, sharp=1)
bv3 = fft_sup.MAnalyse(isb = true, delta = 3, overlap=8, blksize=16, search=5, searchparam=8)
bv2 = fft_sup.MAnalyse(isb = true, delta = 2, overlap=8, blksize=16, search=5, searchparam=6)
bv1 = fft_sup.MAnalyse(isb = true, delta = 1, overlap=8, blksize=16, search=5, searchparam=4)
fv1 = fft_sup.MAnalyse(isb = false, delta = 1, overlap=8, blksize=16, search=5, searchparam=4)
fv2 = fft_sup.MAnalyse(isb = false, delta = 2, overlap=8, blksize=16, search=5, searchparam=6)
fv3 = fft_sup.MAnalyse(isb = false, delta = 3, overlap=8, blksize=16, search=5, searchparam=8)

o_sup2 = o.MSuper(pel=2, sharp=2, levels=1)
NR1 = o.MDegrain2(o_sup2,bv1,fv1,bv2,fv2, thSAD=460)
NR1D = mt_makediff(o,NR1)

DD = mt_lutxy(fftD,NR1D,"x 128 - abs y 128 - abs < x y ?")
NR1x = o.mt_makediff(DD,U=2,V=2)

NR1x_sup = NR1x.MSuper(pel=2, sharp=2, levels=1)
# NR2 = NR1x.MDegrain3(NR1x_sup,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=460)
NR2 = NR1x.MinBlur(2).removegrain(11,0).MDegrain3(NR1x_sup,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=460)

bcomp1 = o.MCompensate(o_sup2,bv1)
fcomp1 = o.MCompensate(o_sup2,fv1)
pmax = mt_logic(bcomp1,fcomp1,"max").mt_logic(o,"max")
pmin = mt_logic(bcomp1,fcomp1,"min").mt_logic(o,"min")

s = NR2.minblur(2,1)
allD = mt_makediff(o,NR2)
ssD = mt_makediff(s,s.removegrain(20,-1).removegrain(11,-1)).mt_lut("x 128 - 2.0 * 128 +")
# ssDD = ssD.repair(ssD.repair(allD,1),12) .mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")

NR2.mt_adddiff(ssD,U=2,V=2)
last.mt_clamp(pmax,pmin,0,0,U=2,V=2)

return(last)

#------------------------------------------

[Nephilis]

@ Nephilis: You are suggesting the script that tormento has posted three posts above. What's the point?
@ tormento:
When grain is "not touched" in certain spots, that's because MAnalyse couldn't compensate those parts. When a complete frame is not touched, that's because MVTools' scenechange threshold was triggered.

- Its not the same script. The corrected version..

- Isn't motion estimation process making by Manalyse and motion compensation by MDeGrain ?

[Didée]

- Its not the same script. The corrected version..
Yes, indeed. Sorry. However, it pretty much looks like a script that I had posted somewhen in the past.

- Isn't motion estimation process making by Manalyse and motion compensation by MDeGrain ?
If you want to mince matters, yes. But don't worry, I'll escape. :p

MAnalyse does motion estimation. Motion compensation is done by MCompensate (explicitely), and by MDegrain (privately).
But, MAnalyse is also doing internal compensation, or at least sort of that. It's in the process of compairing one location with another location.
In the given context, it doesn't really matter if one says "Manalyse failed to compensate" or "Manalyse failed to estimate". In this context, both are the same, more or less.

[Nephilis]

Yes, indeed. Sorry. However, it pretty much looks like a script that I had posted somewhen in the past.


If you want to mince matters, yes. But don't worry, I'll escape. :p

MAnalyse does motion estimation. Motion compensation is done by MCompensate (explicitely), and by MDegrain (privately).
But, MAnalyse is also doing internal compensation, or at least sort of that. It's in the process of compairing one location with another location.
In the given context, it doesn't really matter if one says "Manalyse failed to compensate" or "Manalyse failed to estimate". In this context, both are the same, more or less.

You have forgotten MFlow :p

By the way yes the script was written by you not me. i only adapted it to MVtools2..

[Didée]

You have forgotten MFlow :p
No I did not forget about MFlow. MFlow is doing motion interpolation, which is yet another story. :)

[tormento]

Below are some scripts to try.
Well, thank you again...
3) Be more exhaustive in everything. Additional filter: Fluxsmooth
Groan.. if only it would exist in x64... ;)

MinBlur requires MediaBlur too, am I wrong? Is it ok to use MinBlur provided in MCTempDenoise or is it a different function?

[Didée]

Sure. MedianBlur is the main ingredient of MinBlur.

And sorry, but ... when suggesting basic techniques, I don't consider availability of x64 plugins. Supposely, 99% of all existing Avisynth plugins have not been ported yet to x64.

[Nephilis]

No I did not forget about MFlow. MFlow is doing motion interpolation, which is yet another story. :)

According to author, MFlow does a motion compensation of the frame not by blocks like MCompensate, but by pixels.

MFlowInter and MFlowFps are for motion interpolation to obtain a more fluid motion by 2 different ways.( By using bw and fw motion vectors to create picture at some intermediate time moment between current and next frame or by doubling the framerate)

[Didée]

According to author, MFlow does a motion compensation of the frame not by blocks like MCompensate, but by pixels.
Well, that's a "sufficient for the average user" description.

Of course, interpolation is ~similar~ to compensation. But under the surface, there's much more to it. The push/fetch mechanism(s) are different. Interpolation is much more tricky and complicated.

In particular:

- If estimation fails, then compensation can use the original reference as fallback solution (do-nothing), which results in legitimate data.

- If estimation fails, then there is nothing that interpolation could use. There is no fallback solution. The lame-fallback is blending, but taken strictly, the resulting data is not legitimate.


Finished from my side. I'm not interested in word mincing any more.

[tormento]

Sure. MedianBlur is the main ingredient of MinBlur.
Is the MinBlur of MCTempDenoise ok?
And sorry, but ... when suggesting basic techniques, I don't consider availability of x64 plugins. Supposely, 99% of all existing Avisynth plugins have not been ported yet to x64.
Don't be sorry for your kindness..

Well, AFAIK have been ported a lot indeed.

[tormento]

Didée: just tried the second script. It works and it's wonderful but gives 0.02 fps...

[tormento]

How about this ? Can this script Salvate il soldato Ryan ? :)
Have you tried it?

Nice crash ;)

[Nephilis]

Have you tried it?

Nice crash ;)

What was the error message? If you share the error message here, obviously you can find the solution pal ;)

[tormento]

Sure. MedianBlur is the main ingredient of MinBlur.
Would you try to make a script using only x64 available plugins (MVTools...)?

The script you posted is simply too slow in x86 to be used.

[Didée]

It's not so much a matter of x86 vs. x64. If 0.2 fps in x86, or 0.25 fps in x64 ... who cares?

The matter rather is that 1080p is not meant to be processed by such exhaustive filtering. (Or, if you prefer, it's a matter of currently available PC power still being insufficient for such processing.)

Did you already TRY to encode as 720p with some adequate x264 settings? That's much more sane-minded. ;)
(you'll also have a better karma when keeping the grain of SPR.) :D

[tormento]

Karma is something that scares me about ;)

I spent lot of money on 1080p compliant hardware, please don't let me surrender!

There should be something a genius like you =PpPp should create from notepad ;)

[tormento]

2) The same script, with some adjustments more suited for 1080p#
Didée the script you posted is awesome but a bit slow =P Is it possible to fasten it a bit for a pure bw source?

The #3 script plain crashes ;)

[Didée]

The #3 script plain crashes ;)
Yes indeed. Script#3 contains misnamed variables:
o_sup1 = o.MSuper(pel=1)
bv02 = o_super1.MAnalyse(isb = true, ...
bv01 = o_super1.MAnalyse(isb = true, ...
fv01 = o_super1.MAnalyse(isb = false, ...
fv02 = o_super1.MAnalyse(isb = false, ...

[...]

o_sup2 = o.MSuper(pel=2, sharp=2, levels=1)
NR1 = o.MDegrain2(o_super2,bv1,fv1,bv2,fv2, thSAD=460)
NR1D = mt_makediff(o,NR1)



Didée the script you posted is awesome but a bit slow =P Is it possible to fasten it a bit for a pure bw source?
Okay. I tried to melt down the script as much as possible. Here it is:
mpeg2source("source.d2v")

return( last )
The denoising is a little weak now, but the speed should be good.

[tormento]

The denoising is a little weak now, but the speed should be good.
Thanks, nice laugh after a weak day ;)

[onesloth]

Didée the script you posted is awesome but a bit slow =P Is it possible to fasten it a bit for a pure bw source?

Yes. Read the documentation for fft3dfilter, mvtools, masktools, and repair. :p I can't recall for certain if repair() has luma only modes.

[tormento]

Below are some scripts to try.
Didee, any idea on how to replace MedianBlur with a x64 existing plugin?

[Didée]

Currently, the only fallback solution is mt_luts from masktools. 5x5 Median :

mt_luts(o,o,mode="median",pixels=mt_square(2),expr="y",Y=3,U=2,V=2)

But be warned, it is extremely slow. :( - Example:

Medianblur(2,0,0) = 60.7 fps

mt_luts: 2.95 fps

Better bug someone to port MedianBlur or RemoveGrainHD to x64. mt_luts(median) is a nice-to-have for proof-of-concepts, but simply too slow for general usage.

[cretindesalpes]

More likely:
mt_luts(o,o,mode="median",pixels=mt_square(2),expr="y",Y=3,U=2,V=2)

[Didée]

Oops, indeed, you're right. expr="y" it must be.

(Haven't used it in a while, and looked up in the documentation. From what the documentation states, "x" would work.)

[Gavino]

From what the documentation states, "x" would work.
"x is the pixel from clip1, and y a pixel from the neighbourhood in clip2."

So "x" just gives you the original pixel, ie no-op.

[Didée]

"It computes the mode operation on the result of the function defined by expr, where x is the pixel from clip1, and y a pixel from the neighbourhood in clip2, defined by pixels."

Offhand, I read that "mode" is computed on the result of "expr".


Though ... after reading a few dozen times, the description is correct. It's just not immediately obvious that "expr" is computed as many times as there are pixels in y-neighborhood, and then "mode" is computed on the array of results.

One single letter to make it (a bit) less confusing: (?)

It computes the mode operation on the results of the function defined by expr, where x is the pixel from clip1, and y a pixel from the neighbourhood in clip2, defined by pixels.

[tormento]

mt_luts: 2.95 fps
Well, it's a real pity.

Your script gives me awesome results and sometimes I figure out launching a x264 encoding with it as AVS input. At 10% usually I abort as it's simply too slow to be realistically used to encode 1080p.
Better bug someone to port MedianBlur or RemoveGrainHD to x64
You have more "connections" (influence?) than me. =Pp