Drop-out compensation

[Floatingshed]

I am trying to clean up a telerecording (kinescope) made from a 1964 video recording.
I can remove most of the film-based artifacts, but I'm a bit stuck with some of the tape drop-out.
I assume that either the videotape method (2" Quad) and/or the tele-recording process has caused tape drop-out to be spread over two or even three film frames.
This appears as randomly placed bright white flecks on a generally dark image, naturally they are immune to standard de-spot techniques.
Any ideas anyone? Thanks.

[johnmeyer]

Kinescopes are difficult because the continuous video image was captured by a non-continuous recording medium (film takes the entire image at the same, brief moment in time). You end up getting some artifacts appearing on adjacent frames, which makes them difficult to remove.

RemoveDirt and the motion compensated RemoveDirtMC sometimes work on dropouts. The AVISynth function DeSpot also works, although it is an extremely difficult function to tune. Despot has the advantage that it can be tuned to only look for white spots (I assume your dropouts are white) and ignore black spots.

[johnmeyer]

Here is a script I got from Didée, a long time ago. It was supposed to remove dropouts.

separatefields()
a=last
mot=removegrain(8,0).DepanEstimate(range=2)
take2=a.depaninterleave(mot,prev=2,next=2,subpixel=2)
clean1=take2.DeGrainMedian(limitY=255,limitUV=255,mode=5,norow=true,interlaced=false).selectevery(5,2)

sup1 = clean1.minblur(1).removegrain(11,0)
\ .mt_lutxy(clean1,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
\ .msuper(pel=2,sharp=0)
sup2 = a.msuper(pel=2,levels=1,sharp=2)
###### Good mvtools values, Do not change, checked##########
bv22=sup1.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
bv21=sup1.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv21=sup1.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv22=sup1.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)

interleave(a.mcompensate(sup2,fv22),a.mcompensate(sup2,fv21),a,a.mcompensate(sup2,bv21),a.mcompensate(sup2,bv22))
mergeluma(TMedian2(),0.9).mergechroma(TMedian2(),1).selectevery(5,2)

sup3 = last.msuper(pel=2,sharp=2)
bv33=sup3.manalyse(isb=true, truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv32=sup3.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv31=sup3.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv31=sup3.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv32=sup3.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv33=sup3.manalyse(isb=false,truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
last
weave()


#################################################################
function EdgeCleaner(clip c, float "strength", bool "rep", int "rmode", int "smode", bool "hot", bool "fix") {

strength = default(strength, 8.0)
rep = default(rep, true)
rmode = default(rmode, 17)
smode = default(smode, 0)
hot = default(hot, false)
fix = default(fix, true)

c = (c.isYV12()) ? c : c.ConvertToYV12()
strength = (smode==0) ? strength : strength+4

main = c.aWarpSharp(strength,1)
main = (rep) ? Repair(main,c,rmode) : main

mask = c.mt_edge("prewitt",4,32,4,32).mt_invert().mt_convolution()

final = (!hot) ? mt_merge(c,main,mask) : Repair(mt_merge(c,main,mask),c,2)
final = (fix) ? Overlay(final,c.ConvertToRGB24().Crop(0,1,-c.width+1,-c.height+2),x=0,y=1) : final
final = (smode != 0) ? mt_merge(final,c,c.StarMask(smode)) : final

return final

}

function StarMask(clip c, int "mode") {

mode = default(mode, 1)

clean = (mode==1) ? c.RemoveGrain(17) : Repair(c.Deen("a3d",4,12,0),c,15).RemoveGrain(21)
diff = (mode==1) ? mt_makediff(c,clean) : NOP

final = (mode==1) ? diff.Greyscale().Levels(40,0.350,168,0,255).removegrain(7,-1).mt_edge("prewitt",4,16,4,16) : \
Subtract(mt_merge(clean,c,c.mt_edge("roberts",0,2,0,2).mt_expand(mode=mt_circle(1)).mt_invert()),c).mt_edge("roberts",0,0,0,0).mt_deflate()

return final

}



function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
\ : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r<=1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}



# median of 5 clips from Helpers.avs by G-force
Function Median2(clip "input_1", clip "input_2", clip "input_3", clip "input_4", clip "input_5", string "chroma")
{
chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"
#MEDIAN(i1,i3,i5)
Interleave(input_1,input_3,input_5)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
m1 = selectevery(3,1)
#MAX(MIN(i1,i3,i5),i2)
m2 = input_1.MT_Logic(input_3,"min",chroma=chroma).MT_Logic(input_5,"min",chroma=chroma).MT_Logic(input_2,"max",chroma=chroma)
#MIN(MAX(i1,i3,i5),i4)
m3 = input_1.MT_Logic(input_3,"max",chroma=chroma).MT_Logic(input_5,"max",chroma=chroma).MT_Logic(input_4,"min",chroma=chroma)
Interleave(m1,m2,m3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
selectevery(3,1)
chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last
ConverttoRGB32(matrix="rec601",interlaced=false)
LoadVirtualDubPlugin("C:\Program Files (x86)\VirtualDub\plugins\Camcorder_Color_Denoise_sse2.vdf", "CCD", 1)
CCD(7,1) # de 0 à 100 # Défaut =30 /
converttoyv12(matrix="rec601",interlaced=false)
Return(last)
}

function TMedian2(clip c) {
Median2( c.selectevery(1,-2), c.selectevery(1,-1), c, c.selectevery(1,1), c.selectevery(1,2) ) }


function RemoveDirt(clip input, int limit, bool _grey)
{
clensed=input.Clense(grey=_grey, cache=4)
alt=input.RemoveGrain(2)
return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,dmode=2,debug=false,noise=limit,noisy=4, grey=_grey)

# Alternative settings
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=true)
# return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=6,cthreshold=8, gmthreshold=40,dist=3,tolerance= 12,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey,show=false)
}

[Floatingshed]

This script gives the error:
DePanInterleave: Input must be planar YUV or YUY2
If I put ConverttoYUY2() before the filter I get:
RemoveGrain works only with planar colorspaces

Help!

[wonkey_monkey]

YV16 (ConverttoYV16) is the planar equivalent of YUY2.

[StainlessS]

Floating Shed,
You might need some plugins updated, eg Pinterf Mvtools with depan stuff.
Also Pinterf Masktools.

It is also possible that, that particular script suffers from problem pointed out by real.finder,
where some scripts and plugins only work with YUY2, and some only with YV16,
due to a little 'short sighted-ness' on the part of developers thinking that YUY2 aint needed anymore.

[Floatingshed]

I'm not getting anywhere with this, no colorspace conversion seems acceptible to the script!

[pinterf]

due to a little 'short sighted-ness' on the part of developers thinking that YUY2 aint needed anymore.
Really, why would anybody need YUY2 when there is a parallel YV16 option? Why should tp7 or me or any other developer put back YUY2 support after a full plugin rewrite? YUY2 was not even supported in its native form in many old plugins only with a 'planar hack' which was essentially a conversion to YV16, because it was known in old times as well that working with YUY2 is a pain in general.

[StainlessS]

Real.finder pointed out somewhere recently that scripts that used to support eg YUY2 and YV12, now are restricted to YV12 only because some filters only work in YUY2
and others YV16, and so script now limited to YV12 only. [so at the least, they require multiple conversions added inside the script.
Particularly problematic where plugin source is not available, eg LaTu].

[pinterf]

This script gives the error:
DePanInterleave: Input must be planar YUV or YUY2
If I put ConverttoYUY2() before the filter I get:
RemoveGrain works only with planar colorspaces

Help!
Thanks for the report.

DepanInterleave
In an old source I can see this one
if (!vi.IsYV12() && !vi.IsYUY2())
env->ThrowError("DePanInterleave: input must be YV12 or YUY2 !"); //v1.6

but presently is seems it allows _only_ YUY2 and no planar.
I'll do a fix for it.

[pinterf]

Depan/DepanInterleave was (hopefully) fixed.

Please download the pack and test.
https://github.com/pinterf/mvtools/releases/tag/2.7.41.b

There were mutiple problems,
non-YUY2 was not allowed,
passing YUY2 signalled me that some inputlog file was not found (though none was provided)
then when these issues were fixed, came a YUY2->planar conversion error

Now it is giving me similar results for YV16 and YUY2 (latter is a bit slower since the work is done in planar YV16 inside and there is an internal conversion before-after the process)

[real.finder]

YUY2 is bad indeed, the first who drop support YUY2 was masktools2 2.6 by Manao

but the problem is there are many many plugins still life in 2.5 age and as StainlessS mention it will be only YV12 for the scripts that made with YUY2 in mind unless with some changes to did some YV16<>YUY2

in this period seems there are no enough interest to update plugins even when there are VS ports as I said before, we should be very grateful for pinterf since he did update many plugins side by side with avs+ core which is a lot of work!

indeed there are few other peoples like jpsdr and MeteorRain did some updates and we also grateful for them but they are far from pinterf impact