Local Deflicker

[prokhozhijj]

I've made local deflicker based on Didée idea (I took it from this post http://forum.doom9.org/showthread.php?t=149465&highlight=Deflicker)

This deflicker can deflick part of frame.

Usage very simple. You run it as follows

dest=dest.LDeflicker()

There is the code


#based on Didée idea from this post http://forum.doom9.org/showthread.php?t=149465&highlight=Deflicker

function LDeflicker(clip o, int "lumatg1", int "lumatg2", int "blury", int "blurc", int "lumatl1")
{
lumatg1 = default(lumatg1,100) #luma treshold for first pass of soften in GlobalDeflicker
lumatg2 = default(lumatg2,lumatg1/2) #luma treshold for second pass of soften in GlobalDeflicker

blury = default(blury,115/4) # blur strength for luma in LocalDeflicker
blurc = default(blurc,115/4) # blur strength for chroma in LocalDeflicker

lumatl1 = default(lumatl1,4) #luma treshold for first pass of soften

res=o.GlobalDeflicker(lumatg1, lumatg2)

res=res.LocalDeflicker(blury, blurc, lumatl1)

return res
}


# =============================================================================
# Local Deflicker
# It affects only part of frame
# =============================================================================
function LocalDeflicker(clip o,int "blury", int "blurc", int "lumat1")
{
res=o

# Motion search
super=res.MSuper(rfilter=4)

prefiltered = medianblurt( res , radiusy=1 , radiusu=0 , radiusv=0 , temporalradius=3 , mc=false , calcborder=true)
superfilt = prefiltered.MSuper(rfilter=4)

lambdav=1000
backward_vec1 = MAnalyse(superfilt, isb=true, delta=1, search=4, searchparam=2, lambda=lambdav)
forward_vec1 = MAnalyse(superfilt, isb=false, delta=1, search=4, searchparam=2, lambda=lambdav)

thSCDv=500
backward_comp1 = MFlow(res, super, backward_vec1, thSCD1=thSCDv)
forward_comp1 = MFlow(res, super, forward_vec1, thSCD1=thSCDv)

res=interleave(forward_comp1, res, backward_comp1)

# Source bluring
screen = res.gaussianblur(varY=blury, varC=blurc, border=3) #.greyscale

# Source deflickering and debanding
screenTS = screen.Soften2(lumat1).GradFun2DBmod(thr=1.5,mask=true,mode=3,temp=100,str=4,adapt=-1,radius=3,range=30)

# Find the difference in luma between source and deflickered clip
calmed = res.mt_makediff(mt_makediff(screen,screenTS),U=2,V=2)

# Simple brightness mask for the range that actually is concerned
masky = res.mt_lut("x 32 < x 16 - 16 * x 204 > 255 204 x - abs 255 30 / * - 255 ? ?").removegrain(11,-1)

# Merge-in the deflickered result by that mask
final_dumb = res.mt_merge(calmed,masky,U=2,V=2).SelectEvery(3,1)

# We will not process frames when scene schanges
o_curr = o
p_curr = final_dumb
final_dumb=SCSelect(o_curr , o_curr , o_curr , p_curr, dfactor=2.0)

return final_dumb

}

# =============================================================================
# Global defliker
# It affects whole frame
# =============================================================================
function GlobalDeflicker(clip o, int "lumat1", int "lumat2")
{
lumat1 = default(lumat1,100) #luma treshold for first pass of soften
lumat2 = default(lumat2,lumat1/2) #luma treshold for second pass of soften

res=o

# Average brightness calculating for the whole frame
screen = mt_lutf(res,res,mode="average",yexpr="x",U=1,V=1) #.greyscale

# Deflickering for intermaedite clip
screenTS = screen.Soften1(lumat1,lumat2)

# Find the difference in luma between source and deflickered clip
calmed = res.mt_makediff(mt_makediff(screen,screenTS),U=2,V=2)

# Simple brightness mask for the range that actually is concerned
masky = res.mt_lut("x 32 < x 16 - 16 * x 204 > 255 204 x - abs 255 30 / * - 255 ? ?").removegrain(11,-1)

# Merge-in the deflickered result by that mask
final_dumb = res.mt_merge(calmed,masky,U=2,V=2)

return final_dumb

}

function Soften1(clip clp, int "luma1", int "luma2")
{
res = clp
luma1 = default(luma1, 8)
luma2 = default(luma1, luma1/2)


# Luma averaging between 1 and 3 frames
# TemporalSoften(clip, int radius, int luma_threshold, int chroma_threshold, int "scenechange", int "mode")
res = clp.temporalsoften( 1, luma1, 0, 18, 2)
res = res.merge(clp,0.249)
res = res.temporalsoften( 3, luma2, 0, 9, 2)

return res
}

function Soften2(clip clp, int "luma1")
{
res = clp
res = medianblurt( res, radiusy=luma1 , radiusu=0 , radiusv=0 , temporalradius=4 , mc=false , calcborder=true , markscenechange=false)

return res
}



Required plug-ins:
===========

MV Tools 2
Mask Tools 2
VariableBlur 0.4 with neuron2 patch
Gradfun2db 1.0
AddGrainC 1.5
GradFun2DBmod 1.5
MedianBlur 0.8.4.1
RemoveGrain pre1.0
RemoveDirt 0.8


Possible artefacts that script can produce:
- tracking in scenes with fast motion. The reason is in probably incorrect prefiltering for motion search and insufficient motion compensation. I've tried to get rid tracking on fast scenes with MMask from MVtools2 but with no success.
- banding on smooth sources. This is due to preliminary blurring in LocalDeflicker function (screen variable).

There is source clip in lossless format (about 79Mb) http://www.mediafire.com/?7dhb66xphha6whh

PS. I am newbie.

Edit1: 20110227
- Added scene change detection (thanks to Didée);
- Changed some blurring parameters (they are located in very beginning of script. You can play with them);
- Motion estimation is calculated now on prefiltered clip for more accuracy
- border=3 in LocalDeflicker.

Edit2: 20110228
- Changed functions naming

[Lyris]

Can you tell me from where I can get the function "GaussianBlur"? Google's not turning anything up...

[Gavino]

It's part of the VariableBlur plugin.

[prokhozhijj]

So. No any comments or suggestions? :(

[TheProfileth]

So. No any comments or suggestions? :(

I want to give this a shot just give me a couple minutes

[Lyris]

Just tried it - impressive! Except for the fact that it produces a strange sort of glowing effect on the last frame of each scene, and the first frame of the next. Can you fix that?

[prokhozhijj]

Just tried it - impressive! Except for the fact that it produces a strange sort of glowing effect on the last frame of each scene, and the first frame of the next. Can you fix that?

I think that issue exists because of incorrect scene detection in some cases.

Try to play with temporalradius=4 in DedeeSoften2 function. May be it helps.

[Lyris]

Setting that to "0" does solve the problem, but it also seems that it kills the flicker correction - I'll need to do more thorough tests when I have time. It's often quite hard to tell if flicker has been fixed unless it's seriously bad.

[prokhozhijj]

Setting that to "0" does solve the problem, but it also seems that it kills the flicker correction - I'll need to do more thorough tests when I have time. It's often quite hard to tell if flicker has been fixed unless it's seriously bad.

All local deflickering is located in DedeeSoften2 procedure. medianblurt plugin can works with MVTools directly. But it seems with version 1 only. May be medianblurt can be replaced with some other luma smoother. I am newbie in Avisynth so I don't know all good plugins in that area. Sorry.

You can test if deflicker is still in effect if you compare screen and screenTS clips in LocalDeflicker function. screen is the clip before deflickering, and screetTS - after.

EDIT1. It looks like I understood where the problem is. If we can split somehow original clip on separate scenes and feed every scene to LocalDeflicker function then glowing artefact on scenes border will disappear. But for now I don't know how to do it better.

[Lyris]

Actually, I've just ended up using LocalDeflicker, since that is adequate for this source. Many thanks!

[Lyris]

OK, scratch that. I'd really love to be able to use the entire DeDeeDeflicker() command, but can't because of the fact that it causes artefacting on the first and last frames of a scene.

Is there any way of writing a script which detects the scene changes, so we can turn the filter off on these frames? Or better yet, incorporate it into the function so we can fix the issue?

[Didée]

http://forum.doom9.org/showthread.php?p=1140719#post1140719

;)

[Lyris]

Thanks DiDee!

Suggestion:

I found that using the function causes the edges of the frame to be shadowed, even if all borders are cropped off prior to applying the deflicker.

Changing border=0 to border=3 in the GaussianBlur function fixes this (it's part of the LocalDeflicker function).

I changed

# Source bluring
screen = res.gaussianblur(varY=blury,varC=blurc,border=0) #.greyscale

to


# Source bluring
screen = res.gaussianblur(varY=blury,varC=blurc,border=3) #.greyscale

and fixed this.

The remaining problem is that banding is now visible. If you Alter the LocalDeflicker function to return "screen" instead of "final_dumb" and pull the brightness up a little, we can see that the blurred image has horrible contouring which gets merged into the final deflickered output. I don't have a solution for this and sadly can't share the example clip... but imagine a scene from a black and white clip with a black background and a girl lit (appearing white) in front of it. On the deflickered clip, the flicker is gone (yay!) but there is now a sort of pulsating contoured glow around her head and body.

[Lyris]

Example images (from credits):

Before deflicker
http://img546.imageshack.us/img546/5695/beforek.jpg

Uploaded with ImageShack.us (http://imageshack.us)

After deflicker
http://img191.imageshack.us/img191/2227/deflickered.jpg

Result of "return screen" showing contours exist at this stage
http://img339.imageshack.us/img339/3619/screenpm.jpg

edit: I tried patching Gradfun2dbmod into the script - messy, I'd rather avoid banding in the first place than try to clean it up later... but it didn't do much good. It almost eliminated contouring in the "return screen", but not in the final result. Dark areas in the video still have a "tree rings" look to them.

[prokhozhijj]

Script has been changed. Please see first post for details.

[Lyris]

Thanks for the latest update. Do we have any better way to remove the banding other than throwing grain into the affected areas? This is not good for compression.

It still also creates artefacting which almost makes the film look like it is water damaged... it's frustrating because other than these two issues, it makes a huge improvement.

before

http://img16.imageshack.us/img16/8185/83866249.png (http://img16.imageshack.us/i/83866249.png/)

after

http://img801.imageshack.us/img801/6024/outbu.png (http://img801.imageshack.us/i/outbu.png/)


Can you tell me, is there any way to use an effective single-frame (Global) deflicker? I've tried running GlobalDeflicker on its own but it doesn't do a lot of good. For some reason, #ing out the LocalDeflicker part of the "DeDeeDeflicker" function makes the flicker worse.

[prokhozhijj]

Thanks for the latest update. Do we have any better way to remove the banding other than throwing grain into the affected areas? This is not good for compression.

It still also creates artefacting which almost makes the film look like it is water damaged... it's frustrating because other than these two issues, it makes a huge improvement.


I've implemented the first found solution for debanding. I'll try to find something else. But it'll take some time. Maybe someone from Avisynth gurus can help. I am just newbie. :)

EDIT1: http://forum.doom9.org/showthread.php?p=1386418#post1386418 in this post some expert says you can not get a clear video without banding. I am just blur a frame (screen variable) before deflickering it. So that is the reason of banding. Please try to move GradFun2DBmod(thr=1.5,mask=true,mode=3,temp=100,str=4,adapt=-1,radius=3,range=150) from screenTS to screen in LocalDeflicker. And also try to play with "range" and "mode" parameters. Maybe it helps. I've not changed my mind to find a better solution. But I am not sure if I can.


Can you tell me, is there any way to use an effective single-frame (Global) deflicker? I've tried running GlobalDeflicker on its own but it doesn't do a lot of good. For some reason, #ing out the LocalDeflicker part of the "DeDeeDeflicker" function makes the flicker worse.

Deflicker consists from two parts. Global - for the whole frame, it prepares frame for the following part of deflicker, i.e. local. Local deflicker tries to allign luma in different parts of frame. But for the good results it is needed that global luma has been aligned properly.

Of course you can use something else instead of GlobalDeflicker function. I've tried all deflickers I could find. But Didée's deflicker was one of the best in this area. So I've used it in my Deflicker.

[Lyris]

Still no success, the banding stays... I'm only beginning to get my feet wet with actual function making/altering too, so if anyone else can weigh in, I'd really appreciate it...

The other deflicker filters I've tried don't really have much effect. Is this consistent with other people's findings?

[Didée]

There are several open traps in this function, and most of the issues are coming up when the filtering stumbles into one of the traps.

- the prefiltering for Soften2 function is way too rough. TemporalMedian with radius=3 will create enough artifacts on motion to fool the motion search

- compensation for Soften2 is insufficient. With +/-1 compensations, you're only "safeguarding" a temporal radius of 1. Maybe 2, with some good will. But actually Soften2 is using a temporal radius of 4. In this combination, all the MVTools stuff is mostly a waste of time.
-- specifically, MedianBlurT with radius=4 is very slow. In combination with Motion Compensation, it is *ridiculously* slow. (For yet unknown reasons....)

- In the context in which they're used, the spatial parts of MedianBlur might do both harm and good. Possibly more harm. Related to the next point:

- It doesn't exactly help that ontop of the flicker, the source is noisy/grainy. This might interfere with some of the blurring operations. But that's a task for closer examination.


It's difficult to give specific "change this, change that" advice. I'd recommend to start from scratch, and think more closely about the implications of the used operations.

* * * *

BTW, who is this "Dedee" guy? I'd like to talk to him. :)

[prokhozhijj]

There are several open traps in this function, and most of the issues are coming up when the filtering stumbles into one of the traps.

- the prefiltering for Soften2 function is way too rough. TemporalMedian with radius=3 will create enough artifacts on motion to fool the motion search


In generally you are right probably. But in my two cases it works fine. The matter is that flicker is very strong sometimes. And I don't know how to make it weaker before motion search.


- compensation for Soften2 is insufficient. With +/-1 compensations, you're only "safeguarding" a temporal radius of 1. Maybe 2, with some good will. But actually Soften2 is using a temporal radius of 4. In this combination, all the MVTools stuff is mostly a waste of time.


I've tried with radius 2 but it produces more artefacts than radius 1 (in my cases of course). Maybe the reason is in incorrect prefiltering. I don't know yet.


-- specifically, MedianBlurT with radius=4 is very slow. In combination with Motion Compensation, it is *ridiculously* slow. (For yet unknown reasons....)


Yes. Again you are right. But with strong flicker it gets better results.


It's difficult to give specific "change this, change that" advice. I'd recommend to start from scratch, and think more closely about the implications of the used operations.


Thanks. I will try. I saw results of deflickering in following site http://www.algosoft-tech.com/ (section Demo). I was very impressed and after I've decided to reach simillar results. :) But I don't know for sure if it is possible with avisynth's plugins only.

* * * *


BTW, who is this "Dedee" guy? I'd like to talk to him. :)

Well. :) I thought that if I am using your algorithm then I need to keep this fact in new deflicker. But this mention can be removed from source at any time if you have objections.

[Didée]

I thought that if I am using your algorithm then I need to keep this fact in new deflicker. But this mention can be removed from source at any time if you have objections.
Fair enough. But, the point is that this sort of naming is too specific. First, those temporal-averaging methods are not particularly ingenious - give or take some small tidbits, they're rather basic. "Elementary methods of filtering". Second, it sounds like those a particular/important filters of a particular person. Kind of like, erh ... like if Leonardo did nothing in his life, except for painting Mona. Which is too much of a reduction.

Giving credit is good practice, but it would be vastly sufficient if there's a "using methods inspired by ..." somewhere in the script, or sth alike. And if not, it's no tragedy either.

However, IF you feel like mentioning me, THEN feel free to spell my Nickname correct. "Dedee" is a malapropism. "Thriller" was not by Mitchell Jason.

[prokhozhijj]

First, those temporal-averaging methods are not particularly ingenious - give or take some small tidbits, they're rather basic. "Elementary methods of filtering".

Well. Maybe for such professionals like you this method is not ingenious. But I am like a child in the world of digital video processing. :)


Giving credit is good practice, but it would be vastly sufficient if there's a "using methods inspired by ..." somewhere in the script, or sth alike. And if not, it's no tragedy either.

Ok. I've removed your name from functions naming.


However, IF you feel like mentioning me, THEN feel free to spell my Nickname correct. "Dedee" is a malapropism. "Thriller" was not by Mitchell Jason.

Sorry for this. I have no intention to insult you by incorrect spelling your name. I just worried that Avisynth can not process letters with diacritics. I mean "é" symbol in your name.

[lansing]

Just tried it, the LocalDeflicker work well on the my vhs source, where there's flickering on moving object.
However it was just too slow, on my AMD athlon 4 core, it took over 6 minutes to process a 17 seconds 720x480 clip to largarith lossless.

And i'm getting out of bounds memory access on medianblur upon closing virtualdub

[some dude]

I'm trying to use it in wine but I'm getting this error

Evaluate: Unrecognized exception!

in windows it just ends up locking up virtualdub

[kurish]

Effective deflickering is surprisingly difficult, it seems. I've spent the last few days tracking down and tweaking every flicker filter I can get my hands on, but none seem to do the trick for the 1960s B&W film I'm working on. (I used the opportunity to update the avisynth wiki, at least.)

prokhozhijj's Local Deflicker is the only one I've found that's had any real promise. But alas, while it does beautiful work on stationary objects, any motion seems to produce unacceptable artifacting. Two examples follow; source frame is first and ldeflicker() output is second.

Hand moving through light area:

http://i.imgur.com/6Zuvk.jpg http://i.imgur.com/v7v6r.jpg

Woman turning her face to the left:

http://i.imgur.com/4nTEa.jpg http://i.imgur.com/L6QsR.jpg

The only way I've found to eliminate these artifacts is to set Soften2 temporalradius=0. But then that also eliminates the deflickering and defeats the purpose of this plugin, of course.

Anyone have suggestions for tweaking this function to handle motion more effectively? Thanks!

[lisztfr9]

@kurish

Imho flicker can be addressed with overlaying the same clip for substrat and fine tuning it with the mask (mt_merge or overlay), and afterward increasing the luma range. Since the thread is about local deflicker.

[kurish]

Thanks for the helpful suggestion, lisztfr9. It led me to some experimentation with masktools. I tried masking out moving objects and merging with a ldeflicker() clip, but the flickering/non-flickering juxtaposition was too awkward.

I guess I have to live with a little flicker, it seems. For the record, I've found that a bit of cretindesalpes' GradFun3/Dither followed by Didée's GrainFactory3 does a lot to keep one's eyes from being irresistibly drawn to the flicker:

lvl_overshoot = 3.5
lvl_details = 2

o = last
w = Width ()
h = Height ()

edges = mt_edge (mode="prewitt", thY1=0, thY2=255)
details = mt_edge (mode="min/max", thY1=0, thY2=255)

mask_lines = edges.mt_lut ("x 6 - 50 *")
mask_lines = mask_lines.mt_expand ().mt_expand ().mt_expand ().mt_expand ()
mask_overshoot = edges.mt_lut ("x "+String(lvl_overshoot)+" - 2 ^ 40 *")
mask_overshoot = mask_overshoot.mt_inpand ().RemoveGrain (12, -1)
mask_overshoot = mt_logic (mask_overshoot, mask_lines, mode="max")
mask_details = details.mt_binarize (lvl_details)
mask_details = mask_details.RemoveGrain (21 ,-1).mt_expand ()
mask_final = mt_logic (mask_overshoot, mask_details, mode="min")
mask_final = mask_final.RemoveGrain (20, -1).RemoveGrain (20, -1)

o
BicubicResize (w/16 * 4, h/16 * 4, 1, 0).BicubicResize (w, h, 1, 0)
GradFun3 (radius=4, thr=0.6, mask=0, mode=6) # Huge blur
mt_merge (last, o, mask_overshoot, luma=true) # W/o overshoot
GradFun3 (thr=0.6, mask=0) # W/o banding
mt_merge (last, o, mask_final, luma=true) # Details protected

GrainFactory3(g1str=3, g2str=6, g3str=6, temp_avg=20)

(Everything except the GrainFactory3 call is directly from the Dither documentation. My source has a lot of jumpy color banding, so this helps.)

[lisztfr9]

Didée pointed me to Temporalsoften, with this script :


video=AVISource("C:\Documents and Settings\main\My Documents\TC_sample.avi")


last = video


o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2) # most important !
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

# o2 = mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

stackhorizontal(o2,o)



I don't know exactly what's essential here, so imvho, i think that temporal soften does miracles :

http://avisynth.org/mediawiki/TemporalSoften

"These filters can work miracles, and I highly encourage you to try them. But they can also wipe out fine detail if set too high, so don't go overboard."

[kurish]

:goodpost:

Wow, miracles indeed. That produces very nice results! (I changed the bicubicresize parameters to 0.25 that of my own video, of course.)

I see some minor artifacting (shimmering on hard lines) during movement but it's quite acceptable given the otherwise signficant improvement.

I saw that you posted that in the other deflicker thread (http://forum.doom9.org/showthread.php?p=1599767#post1599767) and thought I'd tried it out, but apparently I had not.

Thank you very much! And thank you to Didée, of course. Time for re-encodes...

[`Orum]

Didée pointed me to Temporalsoften, with this script
I'm doing some restoration work and stumbled across this, which finally takes care of some nasty flicker that only affected some areas of some of the frames.

I've formalized this as a function to make it easier for others to use; obviously there's still work to be done here if you want more parameters to tune. I've also lowered the default chroma thresholds (to be the same as the luma ones) so it doesn't completely butcher chroma temporally. That said, I find these to still be very aggressive defaults for most clips (I tend to use something around LumaTh1 = 6, LumaTh2 = 3), so be careful:

function TSdeflick(clip c, float "divisor", int "LumaTh1", int "ChromaTh1", int "SCTh1", int "LumaTh2", int "ChromaTh2", int "SCTh2") {
d = default(divisor, 2)
lt1 = default(LumaTh1, 32)
ct1 = default(ChromaTh1, lt1)
st1 = default(SCTh1, 24)
lt2 = default(LumaTh2, 12)
ct2 = default(ChromaTh2, lt2)
st2 = default(SCTh2, 20)

sm = c.bicubicresize(Width(c) / d, Height(c) / d)
smm = sm.temporalsoften(1, lt1, ct1, st1, 2).merge(sm, 0.25)
smm = smm.temporalsoften(2, lt2, ct2, st2, 2)
return c.mt_makediff(mt_makediff(sm, smm, U=3, V=3).bicubicresize(width(c), height(c), 0, 0), U=3, V=3)
}


It's not clear to me though as the original values were hard-coded, is the downscaled clip default supposed to be 25% of the total area, or 25% in each dimension? As you can see in my script, I set it to be the former, but let me know if you think it's better the other way around.