Deflicker

[lisztfr9]

Hi,

This sort of flicker is local, or rather randomly local, it's a sort of vertical waving, stroboscopic.

http://www.mediafire.com/?paymcw5gfwmf7ug

Deflicker does something globally, but can't reach this local pattern.

I replaced the vertical artifacts with my own, witch are much nicer. I decimate frames manually, and Unshake parts of the video. Anyway on bad VHS there is much to learn ! I wonder what else i can do ? Trying Ylevels or...

Thanks, L

[cobo]

I experimented with your clip. Mdegrain3 with the Manalyse set to dct=1 gets rid of a lot of the flicker, but dct=1 makes it very slow. Combining with the virtualdub deflicker filter (http://neuron2.net/deflick/flick.html) called first in the chain reduces flicker even more.

Set thSAD 600 or higher. A blocksize of 16 and overlap of 8 works better than any other blocksize or smaller overlap. I used MVTools 2.6.0.5 from cretindesalpes' Dither package (http://forum.doom9.org/showthread.php?p=1386559#post1386559) because it makes for a shorter script.

LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\All\ffms2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\All\mvtools2.6.0.5.dll")
LoadVirtualDubPlugin("C:\Program Files\AviSynth 2.58\plugins\All\deflick.vdf","DeFlick")
import("C:\Program Files\AviSynth 2.56a\plugins\All\FFMS2.avsi")

FFmpegSource2("T et C flicker_3.avi", atrack=-1, fpsnum=25, fpsden=1)

ConvertToRGB()
DeFlick(7, 7, 0, 24, 0)#window size,softening,interlaced,scene change threshold,show scene change
ConvertToYV12()

Super=MSuper()
mvmulti=MAnalyse(Super, delta=3, blksize=16, overlap=8, dct=1, multi=true)
MDeGrainN( Super, mvmulti, 3, thSAD=800)

result: Mdeflick.avi (http://speedy.sh/QGy75/Mdeflick.avi) 2.66MB

[lisztfr9]

Hi,

Thanks for pointing me to Mdegrain. I will post a better video for the deflicker issue, it's rather a partial flicker like here, against the windows :

http://www.youtube.com/watch?v=I9k6uCLcFDY&list=HL1352194642&feature=mh_lolz

Just to illustrate the problem of this awful source, it was TV, stored since 1992 on VHS.

At 25 sec against the window and after on the wall, this is some partial flicker, some kind of luma beat.

L

[yup]

lisztfr9!
Please upload lossless AVI Ut or huffyuv compressed. XviD not lossless and can introduce additional flicker.
I am try motion compensated 3 before and 3 after current frame ReduceFlicker (kassandro plugin) without success.
cobo!
Try dct=5 faster than dct=1 and use close approach.
yup.

[cobo]

I know dct-5 is the generally preferred setting, but it doesn't reduce the flicker as much as dct=1 will. What do you mean by "use close approach"?

[yup]

cobo!
dct=1 at frequency space and dct=5 also but Hadamard transform. From doc dct=1 work slowly for blksize=8 and very slowly for other blksizes.
Tomorrow morning I try dct=1 with Reduceflicker and report.
yup.

[cobo]

Oh, I see what you mean. Using blocksize 16 is slower than using blocksize 8, but I found it reduces flicker more than either blocksize 8 or 32.

[lisztfr9]

Yup,

Wait, i'm uploading a 155 MB file which holds all the things to cure, or to enhance.

huffyuv, RGB to YUY2 which is the only compression i get.

Edit : below is the link :

http://www.mediafire.com/?5wjhpxcoidm4dzi

Every 3 frames there is one with more noise, but also imho a little clearer.

[yup]

lisztfr9!
I download two times and can not open file in VirtualDub. VLC play with diagonal distortion. Better resize to 352x480.
yup.

[lisztfr9]

lisztfr9!
I download two times and can not open file in VirtualDub. VLC play with diagonal distortion. Better resize to 352x480.
yup.


Yes i saw that it was displayed in diagonal in VLC, but it's due to the YUY2 colors pace format imho. Anyway, if i leave it in RGB the file is much larger...

And i dunno, but i can open it in VD, i wouldn't have sent a file without testing it. Simply set a statement for RGB in your script... or better, YV12

L

Edit : Sorry but you told me to use compressed huff in VD so there is one single way to do this, by mean of the color space option, so i did that. I thought you knew that the color space would modified.

[StainlessS]

@lisztfr9,
Assuming your colorspace into VDUB is YUY2, set HuffYUV "Convert to YUY2, Predict Median(best)"
and in VDUB select Menu/Video/Fast Recompress.
This (Fast Recompress) causes VDub to enter negotiation with HuffYUV and avoids the YUY2->RGB->YUY2
conversions.

EDIT: The Fast Recompress option is also a good idea if using eg UT video assuming that input and
codec colorspaces are same. If eg input YV12 and output YUY2 then converts YV12->YUY2, skipping
RGB altogether.
By the way, UT Video v12 is out, 22 Oct 2012.

[yup]

lisztfr9!
No succes diagonal distortion remaining script
AVISource("T&C sample.avi",pixel_type="YUY2")
StainlessS good advice.
I am not need 700 frames, 100 wil be enough.
For space saving use Ut codec or better lagarith codec.
yup.

[lisztfr9]

C'mon, this is only YUY2... it's nothing exotic.

The file was YV12, and converted by VD WITHOUT fast recompress because i don't know what it means.

The file is OK in VD when you press the RIGHT ARROW, not the play button.


http://cjoint.com/12nv/BKhov4ghN1q.htm

there is nothing much to select here

I will install Ut and lagar

Edit : You are right, Avsp won't display it correctly ! gosh

[yup]

You use old version. See other link
http://neuron2.net/www.math.berkeley.edu/benrg/huffyuv.html
For Ut
http://forum.doom9.org/showthread.php?p=1226110#post1226110
For Lagarith
http://lags.leetcode.net/codec.html
yup.

[StainlessS]

The file was YV12, and converted by VD WITHOUT fast recompress because i don't know what it means

Assuming "Full Processing mode", then would convert YV12->RGB->YUY2 ie two unnecessary colorspace conversions.
(YV12->YUY2 when negotiation is entered into via Fast Recompress, is not much of a change, VD and codec compromise
on what conversions to do based on input and output colorspaces)
.
Full Processing mode always converts to RGB and then to the target colorspace.

EDITED:

[lisztfr9]

@StainlessS

Ah ok Stainless. I think also my huff codec is outdated, version 1.3.1. The last is about 2.

I'm uploading a 65MB lags file...

http://www.mediafire.com/?2o2ctz54k5d55gu

With Lagarith codec, and less frames :)

Edit : Crop is .Crop(2, 46, -4, -50)

The codec wasn't happy with this......

[cobo]

I experimented some more with the first clip and found that the following script reduces flicker as much as the first script I posted, but it's faster. The difference on my 9year old PC is ~0.05fps with the first script compared to ~0.98fps with this one when encoding with xvid in virtualdub.

LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\All\ffms2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\All\ReduceFlickerSSE2.dll")
LoadVirtualDubPlugin("C:\Program Files\AviSynth 2.58\plugins\All\deflick.vdf","DeFlick")
LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\RemoveGrain1.0b\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.58\plugins\All\mvtools2.6.0.5.dll")
import("C:\Program Files\AviSynth 2.58\plugins\All\FFMS2.avsi")

FFmpegSource2("T et C flicker_3.avi", atrack=-1, fpsnum=25, fpsden=1)

ReduceFlicker(strength=3,aggressive=true)

ConvertToRGB()
DeFlick(7, 7, 0, 24, 0)
ConvertToYV12()

o1=last o1x=o1.width() o1y=o1.height()
o1sup = o1.MSuper(pel=2,sharp=2)
bv1 = o1sup.MAnalyse(isb=true, delta=1,blksize=16,overlap=8,search=4,dct=5)
fv1 = o1sup.MAnalyse(isb=false,delta=1,blksize=16,overlap=8,search=4,dct=5)
bc1 = o1.MCompensate(o1sup,bv1)
fc1 = o1.MCompensate(o1sup,fv1)
Interleave(fc1,o1,bc1)
Clense(reduceflicker=false)
SelectEvery(3,1)

Super=MSuper()
mvmulti=MAnalyse(Super, delta=3, blksize=16, overlap=8, dct=5, multi=true)
MDeGrainN( Super, mvmulti, 3, thSAD=1000)

[lisztfr9]

Thanks Cobo,

I will exercise i bit on artificial clips. I think about drawing in Gimp some B&W gradient, with a picture inside, and let the gradient flip around. Then the filter should fight against the gradient and preserve the picture. So i need to flip the image every frame, overlay the picture, and that will be the test clip.

[Didée]

Regarding "flicker reduction", what about the old trick of subsampled temporalsoften?

avisource("D:\__DL\TC_Sample__Lagarith.avi")

o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

stackhorizontal(o,o2)

That's not bad as-it-is, and it is very fast.

If a more timeconsuming filterchain is used anyway, in particular with MVTools-Denoising etc., then the result "o2" makes a good source for motion vector search. No need for sluggish DCT=1.

[Jenyok]

Is it so ???
.

LoadVirtualDubPlugin("C:\PROGRAM FILES\VIRTUALDUB 1.9.11\PLUGINS\deflick.vdf", "DeFlick")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\REDUCEFLICKER_0_5\reduceflicker.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\MVTOOLS-V2_5_11_3\mvtools2.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\REMOVEGRAIN-1_0\removegrain.dll")



function DeFlicker(clip clp)
{
last = clp
ReduceFlicker(strength=3, aggressive=true)

ConvertToRGB()
DeFlick(7, 7, 0, 24, 0)
ConvertToYV12()

o1 = last
o1x = o1.width()
o1y = o1.height()

o1sup = o1.MSuper(pel=2, sharp=2)
bv1 = o1sup.MAnalyse(isb=true, delta=1, blksize=16, overlap=8, search=4, dct=5)
fv1 = o1sup.MAnalyse(isb=false, delta=1, blksize=16, overlap=8, search=4, dct=5)
bc1 = o1.MCompensate(o1sup, bv1)
fc1 = o1.MCompensate(o1sup, fv1)

Interleave(fc1, o1, bc1)
Clense(reduceflicker=false)
SelectEvery(3, 1)

Super = MSuper(sharp=2)
bvec3 = MAnalyse(Super, isb=true, delta=3, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
bvec2 = MAnalyse(Super, isb=true, delta=2, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
bvec1 = MAnalyse(Super, isb=true, delta=1, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
fvec1 = MAnalyse(Super, isb=false, delta=1, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
fvec2 = MAnalyse(Super, isb=false, delta=2, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
fvec3 = MAnalyse(Super, isb=false, delta=3, blksize=16, overlap=8, search=3, truemotion=true, dct=5)
MDegrain3(clp, Super, bvec1, fvec1, bvec2, fvec2, bvec3, fvec3, plane=4, thsad=1000)

return last
}

[lisztfr9]

@Didée,


o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)



Imho there is only 1 clip in the second mt_makediff bracket pair ? i can see only one, so how is it supposed to operate ? that's a big mystery :)

Also the double makediff seems a bit recursive ?

[Didée]

Don't ask. Just trust. :)


o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)
See? A small clip is created. The small clip is temporal-softened. The difference achieved by temporalsoften is taken from the small clip, then re-scaled, and applied to the original clip.
Effectively, this is (very similar to) a spatial lowpass on the difference of a normal temporalsoften.
(This little script does also chroma processing just for the heck of it. Since the clip effectively is B/W, it's not really needed. You could just call greyscale() .)

Maybe the initial downsize even could be made a little smaller. Try (68,112,0,0) instead of (88,144,0,0)

[lisztfr9]

Ok, but the first bracket :



o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)




I can see only 1 clip there ... so i assume the second clip is the leading o ?

[Didée]

Yes, there is only one clip IN the parantheses. This one clip is the 2nd clip argument of mt_makediff(). The 1st clip argument is the leading "o." oh-dot.

o2 = o.mt_makediff( mt_makediff(sm,smm,U=3,V=3) .bicubicresize(width(o),height(o),0,0) , U=3,V=3)

===== edit =====
Generally, it doesn't matter if you write "Filter(clip1,clip2)" or "clip1.Filter(clip2)". It's mostly a matter of habit.

In the special case of mt_makediff (or mt_adddiff), it is my personal preference to mix both ways of syntax, depending on the "intention" of the action:

- when getting a difference from two clips, I write "mt_makediff(clip1,clip2)"
- (Having two "normal" clips, and getting a 128-centered difference-clip as result)

- when applying a difference-clip to a normal clip, I write "clip1.mt_makediff(clip2)"
- (clip2 being the difference-clip)

There is no "must be so" behind this. It's just my personal way using the freedom of syntax.

===== /edit =====


o2 = o.mt_makediff( something )
<==> (the same as)
o2 = mt_makediff( o, something )

# with #

something = mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0)


Or, dissecting the script in another way:

o = last
small = o.bicubicresize(...)
small_ts = small.temporalsoften(...)

diff_small = mt_makediff(small,small_ts)
diff_small_resized = diff_small.bicubicresize(width(o),height(o))

o2 = o.mt_makediff(diff_small_resized)

[StainlessS]

Yes, first clip is o, second is result of "mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0)"


EDIT: Beaten to it, as Didée said. :)

[Gavino]

@lisztfr9: see "OOP notation" here: http://avisynth.org/mediawiki/Grammar

[lisztfr9]

This is working great ! But i don't know why, i tried to play a bit with the code :


o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2) # .merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

# o2 = mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

stackhorizontal(o2,o)



imho the first merge 0.25 is not the key point, also the last mt_makediff...

All is in the title, the subsampled temporal softening, i guess the antiflicker effect is there, mt_makediff(sm,smm

I just see that the subsampling does the trick, but why ? i never would have thought it, because the reduced image isn't that small....

Edit : now i believe the temporal soften does it all.
Edit : this seems most effective :

smm = smm.temporalsoften(2,12,255,20,2)

[yup]

Hi all!
I try two motion compensated scripts
with ReduceFlicker
SetMTMode(3,8)
source=AVISource("TC_Sample.avi")
SetMTMode(2,8)
super=source.MSuper(pel=2, sharp=1)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
bc1 = source.MCompensate(super, backward_vec1, thSAD=1000)
fc1 = source.MCompensate(super, forward_vec1, thSAD=1000)
bc2 = source.MCompensate(super, backward_vec2, thSAD=1000)
fc2 = source.MCompensate(super, forward_vec2, thSAD=1000)
bc3 = source.MCompensate(super, backward_vec3, thSAD=1000)
fc3 = source.MCompensate(super, forward_vec3, thSAD=1000)
Interleave(bc3,bc2,bc1,source,fc1,fc2,fc3)
ReduceFlicker(strength=3,aggressive=true)
SelectEvery(7,3)

with Deflicker from Fizick
SetMTMode(3,8)
AVISource("TC_Sample.avi")
SetMTMode(2,8)
super=source.MSuper(pel=2, sharp=1)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec4 = MAnalyse(super, isb = true, delta = 4, blksize=16, overlap=8, chroma=false, dct=1)
bc1 = source.MCompensate(super, backward_vec1, thSAD=1000)
bc2 = source.MCompensate(super, backward_vec2, thSAD=1000)
bc3 = source.MCompensate(super, backward_vec3, thSAD=1000)
bc4 = source.MCompensate(super, backward_vec4, thSAD=1000)
Interleave(bc4,bc3,bc2,bc1,source)
DeFlicker(scene=30,lag=4)
SelectEvery(5,4)
Script with ReduceFlick get reasonable output. Problem with source not only flicker but also ghosting.
yup.

[Jenyok]

yup
Is it so ?
.

#
# http://forum.doom9.org/showthread.php?t=166355&page=2
# motion compensated scripts with ReduceFlicker
#
function ReduceFlicker2(clip clp)
{
source = clp
super = source.MSuper(pel=2, sharp=1)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
bc1 = source.MCompensate(super, backward_vec1, thSAD=1000)
fc1 = source.MCompensate(super, forward_vec1, thSAD=1000)
bc2 = source.MCompensate(super, backward_vec2, thSAD=1000)
fc2 = source.MCompensate(super, forward_vec2, thSAD=1000)
bc3 = source.MCompensate(super, backward_vec3, thSAD=1000)
fc3 = source.MCompensate(super, forward_vec3, thSAD=1000)
Interleave(bc3, bc2, bc1, source, fc1, fc2, fc3)
ReduceFlicker(strength=3, aggressive=true)
SelectEvery(7, 3)

return (last)
}


#
# http://forum.doom9.org/showthread.php?t=166355&page=2
# motion compensated scripts with Deflicker from Fizick
#
function DeFlicker2(clip clp)
{
source = clp
super = source.MSuper(pel=2, sharp=1)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=16, overlap=8, chroma=false, dct=1)
backward_vec4 = MAnalyse(super, isb = true, delta = 4, blksize=16, overlap=8, chroma=false, dct=1)
bc1 = source.MCompensate(super, backward_vec1, thSAD=1000)
bc2 = source.MCompensate(super, backward_vec2, thSAD=1000)
bc3 = source.MCompensate(super, backward_vec3, thSAD=1000)
bc4 = source.MCompensate(super, backward_vec4, thSAD=1000)
Interleave(bc4, bc3, bc2, bc1, source)
DeFlicker(scene=30, lag=4)
SelectEvery(5, 4)

return (last)
}

[yup]

Jenyok!
I do not read Your post. First script written 3 days before http://forum.doom9.org/showthread.php?p=1599435#post1599435 when I am waiting sample.
Two different man can think like one.
For me also interesting using Deflicker but one need a lot of MC frames only from one side.
yup.

[kurish]

o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

Thanks for this. With a couple small tweaks, it did some beautful work on a clip that was resistant to every other deflicker function I tried.

The script as written above presented two issues for me. First, it produced shimmering in areas with high-contrast and movement (eg, a dark shirtsleeve moving against a light background). Second, the temporal softening was a bit too strong for my source, and produced "ghost edges" during movement (remnants/foreshadowing of adjacent frames). The shimmering was eliminated with an edge (contrast) mask, and the ghosting required some tuning of the temporal softening.

I ended up with this:

o = last
sm = o.bicubicresize(174,88) #25% of source
smm = sm.temporalsoften(1,12,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,7,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)

overlay(o2,o,mask=o.mt_edge(mode="min/max",u=2,v=2).blur(1.5).blur(1.5))

It's weaker than your original script, but it was quite sufficient for my source and produced very minimal artifacting.

[GMJCZP]

I've been testing this script to mitigate flickering and I've adapted it to be less aggressive (now exists presets):

# Small Deflicker 2.4
# Based in idea by Didée, kurish (http://forum.doom9.org/showthread.php?p=1601335#post1601335)
# Adapted by GMJCZP
# Requirements: Masktools2, RGTools, vsCnr2, vsMSmooth.

# USAGE:
# Small_Deflicker(clip, width_clip=width(clip)/4, height_clip=height(clip)/4, preset=2, cnr=false, rep=false) (default values)

# PRESETS:
# preset = 1 Soft deflickering (GMJCZP values)
# preset = 2 Medium deflickering (kurish values, preset by default)
# preset = 3 Hard deflickering (Didée values)

# REPAIR MODE:
# - Chroma noise reduction it is an experimental attempt to mitigate the side effects of the script
# By default it is disabled (only for presets 2 and 3)
# - Repair is an option for certain sources or anime/cartoon content, where ghosting may be evident
# By default it is disabled (maybe for preset = 1 it is not necessary to activate it)


function Small_Deflicker(clip o, int "width_clip", int "height_clip", int "preset", bool "cnr", bool "rep")
{

width_clip = default(width_clip,AdjustMOD(width(o)/4,16)) # Width of small clip
height_clip = default(height_clip,AdjustMOD(height(o)/4,16)) # Height of small clip
preset = default(preset,2) # Medium Deflickering
cnr = default(cnr,false) # Chroma noise reduction (experimental)
rep = default(rep,false) # Repair mode

Assert(((width_clip % 16) == 0), "width_clip must be mod 16")
Assert(((height_clip % 16) == 0), "height_clip must be mod 16")

sm=o.bicubicresize(width_clip,height_clip,0,0) # can be altered, but ~25% of original resolution seems reasonable
smm = (preset==1) ? sm.Preset1() : (preset==2) ? sm.Preset2(cnr) : (preset==3) ? sm.Preset3(cnr) : Assert(false, "Preset not valid (1, 2 or 3)")
o2=o.mt_makediff(mt_makediff(sm,smm,u=3,v=3).bicubicresize(width(o),height(o),0,0),u=3,v=3)
o2 = rep ? Repair(o2, o, mode=10, modeU=10, modeV=10) : o2


return o2
}

# Function that adjusts parameters to mod 16

Function AdjustMOD(int value, int "factor")
{
factor = default(factor,16)
adjust = value - (value % factor)
return adjust
}

# Presets

Function Preset1(clip sm)
{
smm=sm.temporalsoften(1,6,9,10,2).vsMSmooth(threshold=0.8,strength=25.0,chroma=true,luma=false).merge (sm,0.25).merge (sm,0.25)
smm=smm.temporalsoften(2,3,5,6,2).vsMSmooth(threshold=2.0,strength=1.0,chroma=true,luma=false)
Return smm
}

Function Preset2(clip sm, bool cnr)
{
smm=sm.temporalsoften(1,12,255,24,2).vsMSmooth(threshold=0.8,strength=25.0,chroma=true,luma=false).merge (sm,0.25).merge (sm,0.25)
smm=smm.temporalsoften(2,7,255,20,2).vsMSmooth(threshold=2.0,strength=1.0,chroma=true,luma=false)
smm = cnr ? smm.vsCnr2(mode="ooo",ln=5,un=40,vn=40, scdthr=2.0) : smm
Return smm
}

Function Preset3(clip sm, bool cnr)
{
smm=sm.temporalsoften(1,32,255,24,2).vsMSmooth(threshold=0.8,strength=25.0,chroma=true,luma=false).merge (sm,0.25).merge (sm,0.25)
smm=smm.temporalsoften(2,12,255,20,2).vsMSmooth(threshold=2.0,strength=1.0,chroma=true,luma=false)
smm = cnr ? smm.vsCnr2(mode="ooo",ln=10,lm=255,un=35,vn=35, scdthr=2.0) : smm
Return smm
}

[johnmeyer]

I'll have to try that out the next time I'm doing a film transfer. Currently I use the Deflicker function and, if the flicker is really bad, I the DCT=1 setting in the MVTools2 desnoising. That setting, which unfortunately is turtle slow, does make a huge reduction in flickering. That benefit is a side effect, i.e., that was not its original intended purpose.

I'll have to see if this function works as well, or better, and whether the performance (fps) is faster.

Thanks for doing this!

[GMJCZP]

Thanks to you, john.
I hope this enriches the script that has worked so hard VideoFred, StainlesS and you.

[manono]

Hi,

I was just trying this out and got the message:

AviSynth open failure:
BlindPP: Need mod 16 height

But the DVD sourced video is 704x480. The total script used is:

AVISource("Movie3.avi")
Small_Deflicker()
return last

Do you need more information?

[StainlessS]

I've been testing this script to mitigate flickering and I've adapted it to be less aggressive:

# Small Deflicker based in idea by Didée, kurish (http://forum.doom9.org/showthread.php?p=1601335#post1601335)
# Adapted by GMJCZP
# Requirements: Masktools2, DGDecode (small clip must be mod 16)

function Small_Deflicker(clip o, int "width_clip", int "height_clip")
{

width_clip= default(width_clip,width(o)/4)
height_clip= default(height_clip,height(o)/4)

sm=o.bicubicresize(width_clip,height_clip,0,0) # can be altered, but ~25% of original resolution seems reasonable
smm=sm.temporalsoften(1,6,9,10,2).BlindPP(quant=1,moderate_h=1,moderate_v=1,cpu2="oxxxxx").merge (sm,0.25)
smm=smm.temporalsoften(2,3,5,6,2).BlindPP(quant=2,moderate_h=20,moderate_v=20,cpu2="oxxxoo")
o2=o.mt_makediff(mt_makediff(sm,smm,u=3,v=3).bicubicresize(width(o),height(o),0,0),u=3,v=3)
o2=overlay(o2,o,mask=o.mt_edge(mode="min/max",u=2,v=2).Blur(1.5).Blur(1.5))
return o2
}

I note that for the downsize Didée did NOT supply the b, and c args, ie used default 1.0/3.0 for both (marked in RED above).
For the upsize he used 0.0. for both, what is that called (the b=0.0,c=0.0) and what is the effect ?

The parameters b and c can be used to adjust the properties of the cubic; they are sometimes referred to as "blurring" and "ringing" respectively.
http://avisynth.nl/index.php/Resize#BicubicResize

I suspect that using b=0.0 and c=0.0 for downsize is wrong (does not reflect Didée intent). :helpful:

EDIT:
GMJCZP,
You get better CODE block layout if converting TAB's to SPACE's prior to copy onto forum, I use PSPad
"Menu/Edit/Special Conversion/Convert Tabs To Spaces/".
Also, Tab step setting best set to 4 in text editor settings.

EDIT: Also, for the downsize / 4, see ResizePadded(), probably be able to persuade it to cooperate (although lots of code).
https://forum.doom9.org/showthread.php?t=174496&highlight=ResizePadded
Perhaps of use, perhaps not, would it cause probs if not exact /4.0 reduction ?
(it of cause would cause probs if src / 4, not colorspace compatible).
ResizePadded(), would solve both my above points. [EDIT: as would eg width(o)/(4*2)*2 # for YV12/YUY2, for colorspace width point only]

[GMJCZP]

manono, TinMan, let me digest all these things.

TinMan, you have given me so much information that I will have to look for a digestive. :o

For SD video I have used for example Small_Deflicker (height_clip = 128) without problems, but I understand that the function must be robust. Thanks for your feedback.

[GMJCZP]

At the moment manono can test the function without problems, I hope.

- Small_Deflicker (https://forum.doom9.org/showthread.php?p=1812060#post1812060) function, now takes into account if any value is not mod 16.

[manono]

It works now, thanks. Just not very well, but I appreciate the effort.

[GMJCZP]

It works now, thanks. Just not very well, but I appreciate the effort.
I don't understand you.

[manono]

I don't understand you.
Maybe we don't use the same definition for flicker. I'm looking for something that improves on the luma fluctuations common in old films and Small_Deflicker doesn't seem to do much, if anything. I can provide a sample if requested.

In addition, this one seems to add some nasty color to the lower part of the video from time to time. The picture has been resized from the 704x480 of the video to 640x480. Except for that, the script remains the same as earlier.

https://s4.postimg.org/8384e2knx/Artifact.jpg (https://postimg.org/image/v4opjtkbd/)

[StainlessS]

I have just discovered that GamMac also can be used as a deflicker filter, with LockChan -1 . The results are truly amazing!

Fred. :)

https://forum.doom9.org/showthread.php?p=1795200#post1795200

Perhaps worth a bash. RGB only (for the moment, I was working on YUV version of GamMatch, S power curve for chroma).

EDIT:
(Uploading to here doesn't seem to work. No picture.)
http://postimages.org/
Easy Peasy, no registration required (I think).

[GMJCZP]

Maybe we don't use the same definition for flicker. I'm looking for something that improves on the luma fluctuations common in old films and Small_Deflicker doesn't seem to do much, if anything. I can provide a sample if requested.

In addition, this one seems to add some nasty color to the lower part of the video from time to time. The picture has been resized from the 704x480 of the video to 640x480. Except for that, the script remains the same as earlier.

(Uploading to here doesn't seem to work. No picture.)

So I clarified at the beginning, the function is designed not to be as aggressive as the proposal of Didée and Kurish. It works fine for sporadic flickering situations in which it avoids using the option of dct = 1 with MDegrain.

[manono]

Perhaps worth a bash. RGB only (for the moment...
I tried it earlier - soon after VideoFred posted about his accidental discovery - and wasn't impressed.

Thanks for the link to that picture place. I've added the picture now.

[johnmeyer]

I can't speak for others, but the kind of flicker I deal with all the time is shown on the left side of this "before/after" film transfer:

Film Restoration Before/After (https://www.youtube.com/watch?v=tBAHzO7rJS0&t=1m55s)

I queued this video to start at the point a woman docks and old Chris-Craft boat, and if you look at the sky and the background at the top of the "before" section, you will see tremendous flicker.

Some of the other clips in that video also show flicker, but not all. The old deflicker plugin works pretty well, as you can see.

[StainlessS]

Manono, can you post small sample that exhibits GMJCZP KillerSpots green stuff, thanx.

What version MaskTools2 you using ?

GMJCZP,

I take back what I said about Didee intents, he seems kinda erratic in this post, not using 0,0 in code block, but then
using it when referencing the code. (guess it may remain a Mystery what the intent was).

Don't ask. Just trust. :)


o = last
sm = o.bicubicresize(88,144) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)
See? A small clip is created. The small clip is temporal-softened. The difference achieved by temporalsoften is taken from the small clip, then re-scaled, and applied to the original clip.
Effectively, this is (very similar to) a spatial lowpass on the difference of a normal temporalsoften.
(This little script does also chroma processing just for the heck of it. Since the clip effectively is B/W, it's not really needed. You could just call greyscale() .)

Maybe the initial downsize even could be made a little smaller. Try (68,112,0,0) instead of (88,144,0,0)

Not using 0,0 is equivalent to 1.0/3.0,1.0/3.0, Bicubic b and c defaults.
I kinda trust the missing 0,0 a bit more, presuming that he actually tried the code, whereas he could just have made
a booboo when talking about the code.

[GMJCZP]

To be honest, Tinman, I have long postponed reading the Masktools masking theory (including RPN, which I learned very little from an HP calculator). I would have to read a lot to adequately decipher the idea of Didée, and there is no doubt that he is a specialist in this field, but since this is ultimately a science, although inaccurate but science at last, everything is subject to improvement.

I venture to say that with those values of BicubicResize perhaps Didée was not concerned with a particular performance with this resizing parameters, or not?

Edit: as a consequence of this I say I agree with you that this is a mystery.

[manono]

Manono, can you post small sample that exhibits GMJCZP KillerSpots green stuff...
I have no problems with KillerSpots. It works fine. The pic results from using Small_Deflicker.

[StainlessS]

OK, so Didee intent along with Masking Theory and RPN, shall remain a mystery to us both.

Using Mt_Polish in XP compatible MaskTools is much easier, and the awkwardness of keep having to switch probably means I'll mostly stick with Fizick version.

I guess we need someone with good eyes to figure out which of the Didee numbers produces best results.

[GMJCZP]

I have no problems with KillerSpots. It works fine. The pic results from using Small_Deflicker.

Are you using the latest version of Masktools2?

Edit: that green band brings back memories of when in other cases it touched the values of "u" and "v", which in the script are equal to 3.

EDIT2: manono, is your video black and white? mmm...

[GMJCZP]

OK, so Didee intent along with Masking Theory and RPN, shall remain a mystery to us both.

Using Mt_Polish in XP compatible MaskTools is much easier, and the awkwardness of keep having to switch probably means I'll mostly stick with Fizick version.

I guess we need someone with good eyes to figure out which of the Didee numbers produces best results.

That's why I do not like black boxes, sooner or later I'll take a look at this monster called Masktools. That is why I developed Khameleon.

[StainlessS]

I have no problems with KillerSpots. It works fine. The pic results from using Small_Deflicker.

Oops, guess I retrieved the script function name from corrupt memory cells (guess my flip flops went all floppy), I meant Small_Deflicker.
I was going to have a play with it and though that I would try figure out where the greenery originated,
looks like plugin chroma problems, perhaps due to erroneous settings as suggested by GMJCZP in post #50.

EDIT:
GMJCZP,

Your AdjustMOD(value,factor) function is exactly same as value/factor*factor.
Round value down to next lower multiple of factor, when value is not an exact multiple of factor.

In your usage AdjustMOD(width(o) /4,16) is exact same as width(o)/4/16*16 OR width(o)/(4*16)*16

Round value down to next lower multiple of factor ALWAYS
(value-1)/factor*factor

To round up, to the next higher multiple of factor, when value is not an exact multiple of factor.
(value+factor-1)/factor*factor.

To round up, to the next higher multiple of factor, ALWAYS.
(value+factor)/factor*factor

To round to nearest multiple of factor.
(value+factor/2)/factor*factor)

Maybe you like to make some black boxes :)

EDIT:
I seem to have missed out round down, [think it was already discussed, and so I missed it out]
value/factor*factor.

[GMJCZP]

Maybe you like to make some black boxes :)

No. I know that the function rounds to the lowest value because I am the father of that creature. I did so because after all it does not matter if you take the closest value, the lowest or the highest value, the result should not vary. I just wanted that small clip to be mod 16. ;-)

Thanks anyway for clarifying this, I will take it for the future.

[StainlessS]

Why particularly mod 16 ? [your lucky number, perhaps]

[GMJCZP]

The culprit is BlindPP. That is written in the script in the part of the requirements.

[StainlessS]

Thanks, did not know anything bout that.

EDIT:
written in the script in the part of the requirements

Yep, now I've spotted it.

[GMJCZP]

If the video of manono is black and white maybe explain the green band, we will wait for your response.

Edit: now that I remember, I tried small_deflicker on an anime video to black and white with no problems. Anyway the information is not bad.

[StainlessS]

It dont look quite black and white to me.

I'm currently in process of using VDUB MSU Old Cinema to add flickering and shimmering to a DVD clip (Cabaret sequence),
gonna run it through several times chopping off a prime number of frames from beginning (idea that it adds flicker in different
positions each time), and I think I'll do 3 or four iteration on clip, and then try your function on it.
Not though changing it to Black and white.

[GMJCZP]

The reason why the video is black and white is because I remembered this episode where you participated:

Here (https://forum.doom9.org/showthread.php?p=1699768#post1699768)

What do you think?

[StainlessS]

I just did quick test on Small_Deflicker on 1st generation MSU Old Cinema flickering, did not do very good, pretty much same as input.

Ran same through original Didee (packaged a bit) and got rid of pretty much all flickering.


Function Fn(clip c) {
o = c
sm = o.bicubicresize(o.width/16*4,o.height/16*4) # can be altered, but ~25% of original resolution seems reasonable
smm = sm.temporalsoften(1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
o2 = o.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(width(o),height(o),0,0),U=3,V=3)
o2
}


EDIT: I did however see no greenery via Small_Deflicker, perhaps masktools version problem.

[manono]

Are you using the latest version of Masktools2?
No, something from 2005.
manono, is your video black and white? mmm...
No, it's a color film. I'll enclose another pic. Sometimes that color band is purpley looking, but most often green.

https://s3.postimg.org/wpjkv44yb/Artifact2.jpg (https://postimg.org/image/qolvy1ibz/)

Edit: I saw the StainlessS post after I posted. I tried that Fn he included. No more green and, as he also thought, the flickering is pretty much gone. For this particular source the flickering is mild. I've seen much worse. But, on quick comparison it's much much better now.

[GMJCZP]

New settings for Small_Deflicker (https://forum.doom9.org/showthread.php?p=1812060#post1812060):

- Adjusted the values of temporalsoften to those of kurish, reason why the force has increased.

- The ghosting effect is now mitigated (repair mode).

- Additional requirement: RGTools.

[GMJCZP]

manono, last version of Masktools2. (https://github.com/pinterf/masktools/releases/)

[GMJCZP]

Small_Deflicker 2.0 (https://forum.doom9.org/showthread.php?p=1812060#post1812060):

- Added presets (soft, medium, hard)
- Now the repair mode is optional
- Eliminated the Overlay line (maybe a bug in green screen with u = 2, v = 2 values)

[manono]

manono, last version of Masktools2. (https://github.com/pinterf/masktools/releases/)
Thank you. But it won't load and asks if maybe I'm missing a library.

"Module not found. Install missing library?"

Is there anything required for it to work that isn't required for older versions?

The new Small_Deflicker does seem to work much better at default settings. Still has the colored bottom part from time to time with the older version of MaskTools2 I've been using.

[GMJCZP]

1) Download the version of Masktools V2 as I indicated.
2) In the requirements section of the script it says "Requirements: Masktools2, RGTools, DGDecode (small clip must be mod 16)"
DGDecode: Groucho2004 version.
3) If you follow the error message enter here (http://repacks.net/forum/viewtopic.php?t=125) and install "Visual C ++ Runtimes AIO"

[StainlessS]

manono,
Module not found. Install missing library?

Always handy to have a copy of Dependency Walker around (or Groucho2004 AvsInfo)

http://www.dependencywalker.com/

Point it at dll or Exe and gives info on it including eg MS CPP runtime requirement.
M$ use and distribute it as part of various toolkits.

EDIT:
You may already have an older version, it was supplied as part of M$ Windows Support Tools on W2K and WXP
setup disks, as depends.exe

Depends.exe: Dependency Walker
Overview | Notes | Related Tools Run Depends Run Depends Help


--------------------------------------------------------------------------------

This tool scans any 32-bit or 64-bit Windows module (including .exe, .dll, .ocx, and .sys, among others) and builds a hierarchical tree diagram of all dependent modules. For each module found, it lists all the functions that are exported by that module, and identifies which of those functions are actually being called by other modules. Another view displays the minimum set of required files, along with detailed information about each file including a full path to the file, base address, version numbers, computer type, debug information, and more.

Depends is very useful for troubleshooting system errors related to module load problems. During the module scan, this tool detects dozens of common application problems such as missing modules, invalid modules, import/export mismatches, circular dependency errors, and mismatched computer types of modules.

Depends is useful for developers, testers, system administrators, creators of setup scripts, and anyone else who needs to examine what dependencies are required to make a specific module load successfully.

Depends runs on Windows 95, Windows 98, Windows NT 3.51, Windows NT 4.0, Windows 2000, and Windows XP. It can process any 32-bit or 64-bit Windows modules, including ones designed for Windows CE. It can run as a graphical application or as a console application. New features in version 2.0 include C++ function name undecoration, detection of delay-load and dynamic dependencies, application profiling, user-definable search paths, and the ability to save results to several file formats.

Details on running Depends can be found in Dependency Walker 2.0 Help (Depends.hlp), a separate documentation file.

System Requirements
The following are the system requirements for Depends:

Windows 95, Windows 98, Windows NT 3.51, Windows NT 4.0, Windows 2000, or Windows XP Professional
Files Required
Depends.cnt
Depends.exe
Depends.hlp
Dwinject.dll


EDIT: SUPPORT/TOOLS/ directory on Setup CD.

When you first set it up, suggest you point at dll, and "open With", and find the exe and "Always use" (or whatever its called)
tick box, so that you can just double click a dll and always auto open Dependency Walker.

[manono]

1) Download the version of Masktools V2 as I indicated.

I did already.
2) In the requirements section of the script it says "Requirements: Masktools2, RGTools, DGDecode (small clip must be mod 16)"
I have them already. Anyway, the message is about MaskTools2.
DGDecode: Groucho2004 version.
I'm not changing out my perfectly good version of DGDecode for a different one. It has no bearing on the MaskTools2 problem anyway.
3) If you follow the error message enter here and install "Visual C ++ Runtimes AIO"
This is what I thought would fix it, but nothing. The problem persists.

All these deflicker filters suck anyway because they rely on temporal cleaners and create ghosting during movement. I don't know why an AviSynth deflicker filter hasn't been created that averages the luma over a user-chosen number of frames, with some sort of scene change detection. Easy for me to say, right?

Anyway, thanks for trying.

[GMJCZP]

Small_Deflicker 2.1 (https://forum.doom9.org/showthread.php?p=1812060#post1812060):

- Added experimental option "cnr" for mitigate the side effects of the script (additional requirement: cnr2).

[Groucho2004]

Thank you. But it won't load and asks if maybe I'm missing a library.

"Module not found. Install missing library?"

Is there anything required for it to work that isn't required for older versions?Run AVSMeter with the switch "-avsinfo", it will tell you if any dependencies are missing for the plugins in your auto-load directory.

[kedautinh12]

Asd-g releases vs-cnr2 fork from vapoursynth-cnr2 can support HBD, can you replace cnr2 to vs-cnr2 with different parameters??
https://github.com/Asd-g/AviSynth-vsCnr2

[Reel.Deel]

Asd-g releases vs-cnr2 fork from vapoursynth-cnr2 can support HBD, can you replace cnr2 to vs-cnr2 with different parameters??
https://github.com/Asd-g/AviSynth-vsCnr2

Aside from the function name difference, all of the parameters are the same except for vsCnr2 does not have the "log" parameter. And the script in this thread that uses Cnr2 does not use that parameter so just switching the name to vsCnr should work. But there are other things in the script that are 8-bit only.

[GMJCZP]

There is a problem, BlindPP only works in YUY2 and YV12, there are two alternatives, either DG (videoh) modifies DGDecode to operate in HBD or I will have to rethink this.

[real.finder]

There is a problem, BlindPP only works in YUY2 and YV12, there are two alternatives, either DG (videoh) modifies DGDecode to operate in HBD or I will have to rethink this.

chikuzen try to port it but didn't finish it back then (https://github.com/chikuzen/MPEG2DecPlus/blob/mpeg2decplus/src/dgdecode/BlindPP.cpp), Asd-g 1st said " I would prefer to move BlindPP to a standalone plugin." then after he see the code "I'll not bother with BlindPP." then I asked DG http://rationalqm.us/board/viewtopic.php?f=7&t=1000

so unless pinterf did it, since it's old hard inline asm

[GMJCZP]

Version 2.2: (https://forum.doom9.org/showthread.php?p=1812060#post1812060)

Now Small_Deflicker operates in HBD (not ideal for BlindPP reconversions, vsCnr2 has been adopted). Feel free to testing.

[kedautinh12]

Thanks

[GMJCZP]

Small_Deflicker version 2.3: (https://forum.doom9.org/showthread.php?p=1812060#post1812060)

- Replaced BlindPP by vsMsmooth.

[kedautinh12]

Thank man

[GMJCZP]

Small_Deflicker 2.4: (https://forum.doom9.org/showthread.php?p=1812060#post1812060)

- Adjust vsCnr2 value for preset=3.

[kedautinh12]

Thanks