GTDegrain (function for faster temporal degraining)

[g-force]

Comments/suggestions welcome

-G

# GTDegrain ver.1.07 28May08
# Function by G-force, heavily borrows from Didée's TemporalDegrain which removes a bit more noise, but may be slower
# Requires mvtools.dll, RemoveGrain.dll, mt_masktools-26.dll
# Changelog:
# 1.07 Added Changelog
# 1.06 TemporalSoften(prefilt,255,255,24,2) was TemporalSoften(prefilt,255,255,25,2)
# Interleave(fw3,fw2,fw1,bw3,bw2,bw1,input,fw1,fw2,fw3,bw1,bw2,bw3) was Interleave(bw3,bw2,bw1,input,fw1,fw2,fw3)
# Interleave(fw2,fw1,bw2,bw1,input,fw1,fw2,bw1,bw2) was Interleave(bw2,bw1,input,fw1,fw2)
# Interleave(fw1,bw1,input,fw1,bw1) was Interleave(bw1,input,fw1)
# TemporalSoften(degrain,255,255,24,2) was TemporalSoften(degrain,255,255,255,2), 2 places
# SelectEvery(1+degrain*4,degrain*2) was SelectEvery(1+degrain*2,degrain)
# 1.05 Replaced TTempsmoothF with TemporalSoften, 3 places
# Added .MT_Lutxy(last,"x 128 - abs y 128 - abs < x y ?")
# 1.04 Switched from MVDegrain to MVCompensate/TemporalSoften
# 1.03 Release

function GTDegrain (clip input, clip "denoise", int "degrain", int "prefilt", int "pel", int "sharp",
\ int "blksize", int "overlap", int "dct", int "search", int "idx", int "thSAD")

{ # Denoise is a pre-filtered external clip for use instead of internal pre-filter
degrain = default(degrain,2) # Degrain temporal radius (1 to 3)
prefilt = default(prefilt,4) # prefilter strength (1 to 7)
pel = default(pel,2) # Mvector subpixel interpolation precision (1, 2, or 4)
sharp = default(sharp,1) # Mvector subpixel interpolation method (0, 1, or 2(best))
blksize = default(blksize,8) # Mvector blocksize (use 16 for more speed or HD resolution)
overlap = default(overlap,blksize/2) # Mvector block overlap amount (Max is blksize/2)
dct = default(dct,0) # Mvector using of DCT for blocks difference calculation (0 to 4)
search = default(search,2) # Mvector type of search at every level (0 to 3)
idx = default(idx,1) # MV initial idx value
thSAD = default(thSAD,400) # MV thSAD value (use 300 for additional artifact protection)

########## prefilter (for obtaining motion vectors)
defined(denoise) ? denoise : input.TemporalSoften(prefilt,255,255,24,2).Repair(input,9)

########## get motion vectors
bw1 = last.MVAnalyse(isb=true, delta=1,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
fw1 = last.MVAnalyse(isb=false,delta=1,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
bw2 = (degrain>=2) ?
\ last.MVAnalyse(isb=true, delta=2,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
\ : NOP()
fw2 = (degrain>=2) ?
\ last.MVAnalyse(isb=false,delta=2,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
\ : NOP()
bw3 = (degrain>=3) ?
\ last.MVAnalyse(isb=true, delta=3,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
\ : NOP()
fw3 = (degrain>=3) ?
\ last.MVAnalyse(isb=false,delta=3,pel=pel,sharp=sharp,blksize=blksize,overlap=overlap,dct=dct,search=search,idx=idx)
\ : NOP()

########## apply motion vectors
bw1 = input.MVCompensate(bw1,thSAD=thSAD,idx=idx+1)
fw1 = input.MVCompensate(fw1,thSAD=thSAD,idx=idx+1)
bw2 = (degrain>=2) ? input.MVCompensate(bw2,thSAD=thSAD,idx=idx+1) : NOP()
fw2 = (degrain>=2) ? input.MVCompensate(fw2,thSAD=thSAD,idx=idx+1) : NOP()
bw3 = (degrain>=3) ? input.MVCompensate(bw3,thSAD=thSAD,idx=idx+1) : NOP()
fw3 = (degrain>=3) ? input.MVCompensate(fw3,thSAD=thSAD,idx=idx+1) : NOP()

########## 1st degrain stage
inter = (degrain>=3) ? Interleave(fw3,fw2,fw1,bw3,bw2,bw1,input,fw1,fw2,fw3,bw1,bw2,bw3)
\ : (degrain==2) ? Interleave(fw2,fw1,bw2,bw1,input,fw1,fw2,bw1,bw2)
\ : Interleave(fw1,bw1,input,fw1,bw1)
inter.TemporalSoften(degrain,255,255,24,2)

########## limit 1st degrain stage
last.Repair(inter,9)

########## 2nd degrain stage
TemporalSoften(degrain,255,255,24,2)
SelectEvery(1+degrain*4,degrain*2)

########## contra-sharpening (sharpen, but don't add more than what denoising removed)
nr = last
nr.RemoveGrain(4,-1) # remove some spots before estimating sharpening
MT_MakeDiff(last,last.RemoveGrain(12,-1)) # difference of a simple kernel blur (overestimate sharpening)
last.Repair(MT_MakeDiff(input,nr),1,-1).MT_Lutxy(last,"x 128 - abs y 128 - abs < x y ?")
# limit the difference to only the sharpness that the denoising removed
# and make sure that the abs limited difference is less than the abs difference
nr.MT_AddDiff(last,U=2,V=2) # apply the limited difference (add the sharpness back in)

output = last
return(output)
}

[thetoof]

Oh I remember where that came from ;)

Will do some tests soon.

[g-force]

Updated the code. Now even faster!

[Didée]

g-force, is there a certain reason why you did remove the mt_lutxy - line in the contrasharpen section?

I had a certain reason to put it in, and now I'm curious about your reason to remove it again ...

[thetoof]

Hmmm... first test with GTDegrain (on the sample where TD removed the dirt removedirt couldn't)
TD = 2,2 fps
GT 1.03 = 2 fps
GT 1.04 = 2,2 fps
GTemporalDegrainFaster = 4,5 fps

Where did all your speed improvement go?

[g-force]

g-force, is there a certain reason why you did remove the mt_lutxy - line in the contrasharpen section?

I had a certain reason to put it in, and now I'm curious about your reason to remove it again ...

I couldn't see that it did much visually. So your comparing (denoised-blurred denoise) with the same thing lightly repaired by (original-denoised), and taking the one closest to grey (I think). What does this do exactly? It looks like some sort of edge detector. Are there some sources that this helps?

If you feel pretty strongly that it's better with it in, I'll put it back!

-G

[g-force]

Hmmm... first test with GTDegrain (on the sample where TD removed the dirt removedirt couldn't)
TD = 2,2 fps
GT 1.03 = 2 fps
GT 1.04 = 2,2 fps
GTemporalDegrainFaster = 4,5 fps

Where did all your speed improvement go?

Oh no! Really? I just can't see how that could be...

-G

[g-force]

thetoof,

make sure you're using TTempSmooth v0.9.4.

-G

[Didée]

Regarding speed, with the way you're using ttempsmoothF, you can as well replace it with plain temporalsoften. (You're actually not using any features that make a difference, and for plain averaging, temporalsoften is faster.)

Regarding contra-sharpen, look at THIS (http://www.mediafire.com/?ld6ysnmbeyn).

(Before changing things, better make sure you've *understood* what they're doing.)

[g-force]

Regarding speed, with the way you're using ttempsmoothF, you can as well replace it with plain temporalsoften. (You're actually not using any features that make a difference, and for plain averaging, temporalsoften is faster.)

Regarding contra-sharpen, look at THIS (http://www.mediafire.com/?ld6ysnmbeyn).

(Before changing things, better make sure you've *understood* what they're doing.)

Ugh, I am blocked from Mediafire here at work, but it must be good to deserve such a scolding! It's going back in. Thanks for the suggestion on TemporalSoften!

-G

[Didée]

Try this link (http://home.arcor.de/dhanselmann/_samples/contrasharp_correct_vs_false.avi).

The point is that the mentioned mt_lutxy prevents that unrelated noise gets "restored" (like e.g. block flicker, which the synthetic example shows). Without that line, you might get back some of the crud that already was successfully removed ...

BTW, the scolding was a late punishment for the previous versions of GTDegrain where you had included the impaired contrasharp version as being mine, without noting that you had randomly removed some stuff.

[g-force]

BTW, the scolding was a late punishment for the previous versions of GTDegrain where you had included the impaired contrasharp version as being mine, without noting that you had randomly removed some stuff.

Sorry Didée, I like to give credit where it's due, but I realize that it's frustrating when people say that something is yours when they have messed with it too much. I hope something like "inspired by" or "borrowed from" is more tasteful for things that get changed. If there is an even more appropriate etiquette, please let me know so that I can be respectful.

-G

[Nikos]

Why the scenechange parameter value is 255?
In Scripted_MVDegrain3 function the value is 24.
long.temporalsoften(3,255,255,24,2) # instead of TTempSmooth - if and only IF alt-clip is used

[Terranigma]

Why the scenechange parameter value is 255?
In Scripted_MVDegrain3 function the value is 24.
long.temporalsoften(3,255,255,24,2) # instead of TTempSmooth - if and only IF alt-clip is used
What're you talking about Nikos? It wasn't 24 either, but 99.9

[ChrisW77]

I tried this on 640x480 VHS, and it's not even close to Didée's TemporalDegrain.

Using

source = video
denoised = source.MedianBlur(2,2,2).FluxSmoothT()

SetMTMode(2,6)
TemporalDegrain(source,denoised,degrain=2)

Looks great, runs great (around 12fps), a little soft, but overall great.

Using the same script, but replacing the above with GTDegrain(), results in half the speed, and half the quality. Blurry, not enough noise removal, and slow.
Whats the point, when you can just use the superior TemporalDegrain ?

[Terranigma]

Whats the point, when you can just use the superior TemporalDegrain ?

Or mc_spuds, which doesn't get nearly half the credit it deserves. I guess I could add my reply for mc_spuds whenever I see someone recommending temporaldegrain. :)

[ChrisW77]

Or mc_spuds, which doesn't get nearly half the credit it deserves. I guess I could add my reply for mc_spuds whenever I see someone recommending temporaldegrain. :)


Yep, I agree, it's rather good. Thing is, it's also very slow. Until I can buy a 10Ghz QuadCore, I will have to do with alternatives. :)

[Nikos]

Terranigma read again the Scripted_MVDegrain3 function post:

function Scripted_MVDegrain3(clip c, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", clip "mvbw3", clip "mvfw3",
\ int "thSAD", int "plane", int "limit", clip "pelclip", int "idx")
{
thSAD = default(thSAD, 400)
plane = default(plane, 4)
limit = default(limit, 255)
_idx = default(idx, -11)

thSAD = thSAD / 8

# alt = c.FFT3DFilter(sigma=10,sigma2=6,sigma3=4,sigma4=2,bw=16,bh=16,ow=8,oh=8,bt=1)

SAD_fw3 = c.MVMask(mvfw3, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_fw2 = c.MVMask(mvfw2, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_fw1 = c.MVMask(mvfw, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw1 = c.MVMask(mvbw, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw2 = c.MVMask(mvbw2, kind=1, ml=thSAD, gamma=0.999, Ysc=255)
SAD_bw3 = c.MVMask(mvbw3, kind=1, ml=thSAD, gamma=0.999, Ysc=255)

comp_fw3 = c.MVCompensate(mvfw3, idx=_idx) # .mt_merge(alt,SAD_fw3,U=3,V=3)
comp_fw2 = c.MVCompensate(mvfw2, idx=_idx) # .mt_merge(alt,SAD_fw2,U=3,V=3)
comp_fw1 = c.MVCompensate(mvfw, idx=_idx) # .mt_merge(alt,SAD_fw1,U=3,V=3)
comp_bw1 = c.MVCompensate(mvbw, idx=_idx) # .mt_merge(alt,SAD_bw1,U=3,V=3)
comp_bw2 = c.MVCompensate(mvbw2, idx=_idx) # .mt_merge(alt,SAD_bw2,U=3,V=3)
comp_bw3 = c.MVCompensate(mvbw3, idx=_idx) # .mt_merge(alt,SAD_bw3,U=3,V=3)

black = blankclip(c,color_yuv=$008080)
long = interleave( comp_fw3,comp_fw2,comp_fw1, c, comp_bw1,comp_bw2,comp_bw3 )
long_SAD = interleave( SAD_fw3, SAD_fw2, SAD_fw1, black, SAD_bw1, SAD_bw2, SAD_bw3 )

long.TTempSmooth(3,255,255,1,1,strength=4,pfclip=long_SAD,fp=false,scthresh=99.9)
# long.temporalsoften(3,255,255,24,2) # instead of TTempSmooth - if and only IF alt-clip is used
SelectEvery(7,3)
}

[Terranigma]

Terranigma read again

long.TTempSmooth(3,255,255,1,1,strength=4,pfclip=long_SAD,fp=false,scthresh=99.9)
# long.temporalsoften(3,255,255,24,2) # instead of TTempSmooth - if and only IF alt-clip is used
SelectEvery(7,3)


No no, we're both right, but we were talking about different things. I could've sworn that temporalsoften read ttempsmooth. :p

[Nikos]

The same question:
Why the scenechange parameter in value in temporalsoften is 255? :confused:

[g-force]

The same question:
Why the scenechange parameter in value in temporalsoften is 255? :confused:

Nikos,

It doesn't really matter much what the scenechange value is as long as it's high enough to not falsely trip on slight changes in the compensated frames. The MVCompensate+SAD/Interleave makes sure that all the frames inside the temporal radius of TemporalSoften belong to the same compensated frame. So effectively, there ARE NO scenechanges inside that temporal radius.

-G

[g-force]

I tried this on 640x480 VHS, and it's not even close to Didée's TemporalDegrain.

Using

source = video
denoised = source.MedianBlur(2,2,2).FluxSmoothT()

SetMTMode(2,6)
TemporalDegrain(source,denoised,degrain=2)

Looks great, runs great (around 12fps), a little soft, but overall great.

Using the same script, but replacing the above with GTDegrain(), results in half the speed, and half the quality. Blurry, not enough noise removal, and slow.
Whats the point, when you can just use the superior TemporalDegrain ?

Okay, here are the numbers that I get on my 1.8GHz, 192MB RAM, AMD 3100+ with a 720x480 source.

GTDegrain(): 60 frames per minute (1 fps)
TemporalDegrain(): 3 frame per minute
TemporalDegrain(source,source.MedianBlur(2,2,2).FluxSmoothT(),degrain=2): 4 frames per minute

TemporalDegrain(source,source.MedianBlur(2,2,2).FluxSmoothT(),degrain=2) also results in MUCH more loss of moving detail.

-G

[ChrisW77]

4 frames per MINUTE ?
Utter rubbish.

Sorry, but there is NOT more loss in detail over your "Script".
Learn to script properly, BEFORE making posting such nonsense.

All I see here, is someone looking to make a name for himself, especially after you quite blatantly hijacked that other thread, and then had the NERVE to ask to have it put in the wiki :eek:

Sorry, but I won't bother with your script, it just doesn't cut it.

[Didée]

Hello, it's me, the nagging uncle. ;)

Without having tried or compared anything, the following can be said:

1) Your *extremely* sluggish speed of TemporalDegrain is because you just don't have enough memory. For sure it is slow, but not *that* slow. With less than 512MB RAM, TemporalDegrain should not be used, simple as that.

2) More detail loss with MedianBlur.FluxSmoothT probably is because that's a too strong prefilter for the specific source you tried. On other sources the picture might be different.

3) Scenechange value of TemporalSoften isn't too cruicial, since MVAnalyse+MVCompensate will take care of most scenechanges. However they tend to miss some scenechanges, and in those spots you'll get some unnecessary degradation. For saftey reasons, I'd probably put sth like 20 to 24 for scenechange threshold.

4) There's an inherent flaw in your chain of interleave().TS().repair().TS(), which will lead to unnecessary softening/blurring/ghosting/whatever. With a temporal window of +/-2, you create an interleaved chain like

A' A' A A' A' B' B' B B' B' C' C' C C' C'

The first TS averages over the compensated window, e.g. B is averaged with the surrounding B' frames. Correct. Then this is repaired by the not-averaged interleaved sequence. Correct. If at this point the middle frames would be SelectEvery()'ed, everything would be correct.
But you follow up with another instance of TemporalSoften. Now, this instance of TS requests frames from the so-far-already-processed interleaved chain. This means that the A' and B' and C' frames also undergo the filtering of the 1st TS/repair instance, which means that these A'/B'/C' frames have been averaged with frames that are outside of the compensated window. E.g. the leftmost B' frame is averaged over (A' A' B' B' B). This is not correct. The side effects get dimished because of the Repair() operation, sure. But there will be some leftover degradation, manifesting in slight blurring/softening or ghosting or something thelike.

To circumvent this issue without calculating even more compensations, you could interleave more compensated frames from "the other side". Example for degrain=2: Replace

Interleave( bw2,bw1,input,fw1,fw 2)

with

Interleave( fw2,fw1, bw2,bw1,input,fw1,fw2, bw1,bw2 )

then process as you're doing, and adopt the final SelectEvery() accordingly. This method doesn't cost much more computation (since no additional compensations have to be made), but widens the compensated window so that you don't average with frames from outside of the compensated window.

[g-force]

4 frames per MINUTE ?
Utter rubbish.

Sorry, but there is NOT more loss in detail over your "Script".
Learn to script properly, BEFORE making posting such nonsense.

All I see here, is someone looking to make a name for himself, especially after you quite blatantly hijacked that other thread, and then had the NERVE to ask to have it put in the wiki :eek:

Sorry, but I won't bother with your script, it just doesn't cut it.

Are you referring to post #82 on this page?

http://forum.doom9.org/showthread.php?t=136433&page=5

I don't think making suggestions for how to make TD go faster in a thread asking for suggestions on how to make TD go faster constitutes hijacking the thread.

I did suggest putting the prefilter in the wiki along with several other contenders. I just thought one of the alternatives to the FFT might be useful to someone. Obviously Didée's suggestion was helpful to you. Nikos had a good one too.

-G

[g-force]

Hello, it's me, the nagging uncle. ;)

Replace

Interleave( bw2,bw1,input,fw1,fw 2)

with

Interleave( fw2,fw1, bw2,bw1,input,fw1,fw2, bw1,bw2 )



I actually enjoy the nagging uncle posts. And I really like that suggestion!

-G

[thetoof]

Sorry, but there is NOT more loss in detail over your "Script". Learn to script properly, BEFORE making posting such nonsense.
Wow dude, calm down.... he got the results he got with the sample on which he used the filters, that's all. This is what happens when you tell that you get better results without uploading a sample of the source on which you tried your stuff. All the comparisons are therefore meaningless and criticizing them without seeing screenshots is even more.

All I see here, is someone looking to make a name for himself, especially after you quite blatantly hijacked that other thread, and then had the NERVE to ask to have it put in the wiki :eek:
As g-force pointed out in his previous post, he never asked to add his function in the wiki... only the option to use a faster prefilter inside TD instead of calling it externally. No need to have such an agressive tone, especially when you throw groundless arguments.

[Nikos]

With Didie's correction the function does a good job.
g-force make the correction in the first post please.

[ChrisW77]

edit mod: removed profanity.

[foxyshadis]

Seriously, chill. Profanity isn't welcome at this forum, bad day or not, and disagreements are handled in a civil manner or taken elsewhere. Striked for rule #4.

[g-force]

Function now updated on page one.

-G

[Adub]

changelog?

[g-force]

changelog?

Done.

-G

[Adub]

Thanks very much.