Masking/correcting line dropouts in captured video (bounty)

[juhok]

When capturing analog material some bad sources will cause line dropouts in the beginning of the field. It seems to be V -channel only.

Samples:
http://www.siluriformes.net/doom9/Sample-1_DVSD.avi PAL DV 4:2:0 AVI BFF (frames 5 & 7)
http://www.siluriformes.net/doom9/Sample-1_ULY2.avi PAL UtVideo 4:2:2 AVI TFF (frames 5 & 7)
http://www.siluriformes.net/doom9/Sample-2_DVSD.avi PAL DV 4:2:0 AVI BFF (frame 4)
http://www.siluriformes.net/doom9/Sample-2_ULY0.avi PAL UtVideo 4:2:0 AVI TFF (frame 5)

I'm looking for a way to fix these. Detect dropped line, repeat last or next line or copy U -> V or mocomp - anything goes as long as it's less noticeable than now and not very expensive CPU cycles wise.

$15 bounty for working script. PayPal, SEPA or Bitcoin will do.

[Mounir]

Try this:
directshowsource("D:\Sample-1_DVSD.avi",fps=25.000) # use avisource preferably
assumetff()
pointresize(last.width, last.height*2).Converttoyv12(matrix="rec601",interlaced=true).pointresize(last.width, last.height)# forget this line if decoded in yv12
separatefields()
pointresize(last.width,last.height*2)
turnright()
NLMeansCL(A=4, S=2, B=1, aa=1.0, h=2,HC=5.5, plane=3,cpu=false,buffer=false,sse=true)
turnleft()
pointresize(last.width,last.height/2)
weave()

result (before/after) click here (http://www.hostingpics.net/viewer.php?id=852535result.jpg)
frame 5

[Mounir]

Actually the code below is enough:
directshowsource("D:\Sample-1_DVSD.avi",fps=25.000)
assumetff()
pointresize(last.width, last.height*2).Converttoyv12(matrix="rec601",interlaced=true).pointresize(last.width, last.height)
separatefields()
NLMeansCL(A=4, S=2, B=1, aa=1.0, h=2,HC=5.5, plane=3,cpu=false,buffer=false,sse=true)
weave()

edit:
i think the source is bff because it's dv but with assumebff it wouldn't work so i'm confused...

[juhok]

Thanks for the effort, but problem with this approach is that I lack OpenCL device and cannot even run NLMeansCL :/ Also the repairing should be limited only to dropped lines. Changes/denoising to whole frame/all frames is not desirable.

Sample-1_DVSD is BFF. Maybe DSS is messing it up somehow.

[Mounir]

nlmeans tackle the artefacts on UV only (chroma) here
for white or black drop out horizontal lines there are solutions, try to search a little

[juhok]

Processing all UV frames isn't an option either, it's overkill and I use other methods for denoising. I've probably seen every topic there is regarding to white and black dropouts and streaks and have 100 analog video related topics in my favorites. The proposed dropout scripts don't work very well most of the time even for their intended purposes. In my case we have (edit: ) digital V dropouts which are the same color 100% of the time and limited to the upper part of the field. This should be pretty easy to select and then process only the few affected lines. I just don't have the avs knowledge to do it properly myself.

[raffriff42]

A TBC w/ dropout compensator integrated with the videotape machine's raw (RF) output is essential, as this is the only way to sense dropouts without false positives. http://en.wikipedia.org/wiki/Time_base_correction
Some TBCs featured a Drop Out Compensation (DOC) circuit that enabled videotape flaws caused by oxide drop-out to be temporarily corrected. The DOC circuit required dedicated cabling between the videotape player and the TBC in which irregularities were detected in portions of the video image. Previously captured and stored lines of video would then be superimposed over the flawed video lines.
Any way to remove video drop outs? (videohelp.com) (http://forum.videohelp.com/threads/290729-Anyway-to-remove-video-drop-outs?p=1765034&viewfull=1#post1765034)
If you are the copyright owner of a rare, one-of-a-kind video, you need to take care to do it right. There are companies that re-master old and damaged tapes. One is specsbros.com (http://www.specsbros.com/). Otherwise, you should incorporate a time base corrector (TBC) that has dropout compensation, but...it is a hardware solution that may cost you some bucks. Even then, perfection may not be a realistic expectation. Good luck.

[juhok]

That stuff don't really exist for VHS (the decks that have DOC out are otherwise old crap) and is a bit offtopic.

These "digital" dropouts are caused by Ensemble Designs BE-75 ADC which IS a modern Framesync/TBC. BE-75 doesn't handle certain deck with certain tapes well. So we're not talking about analog tape dropouts but deterministic digital dropouts which are easy to single out.

This problem is fixed by either changing deck or using more tolerant TBC inbetween, like Datavideo TBC-1000. Using another deck or TBC-1000 would introduce loss of quality tho which in this case I'm not comfortable with. So I'd rather fix the few line drops. :)

[poisondeathray]

I've used chubbyrain2 for these types of defects

Does it only affect top section ? If so, you can crop to that section, apply filter, and overlay . You can adjust the crop/overlay values as you see fit

eg



AVISource()
ConvertToYV12(interlaced=true)
Original=last

Original
Crop(0,0,0,-544)
Chubbyrain2()
Filtered=Last

Overlay(original, filtered)






function ChubbyRain2(clip c, int "th", int "radius", bool "show", int "sft")
{
#based on Mug Funky's ChubbyRain

th = default(th,10)
radius = default(radius,10)
show = default(show,false)
sft = default (sft, 10)

u = c.utoy()
v = c.vtoy()

uc = u.mt_convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0)
vc = v.mt_convolution(horizontal="1",vertical="1 -2 1",Y=3,U=0,V=0)

cc = c.mt_convolution(horizontal="1",vertical="1 2 1",Y=2,U=3,V=3).bifrost(interlaced=false).cnr2().temporalsoften(radius,0,sft,2,2)

rainbow=mt_lutxy(uc,vc,Yexpr=string("x y + "+string(th)+" > 256 0 ?")).pointresize(c.width,c.height).mt_expand(y=3,u=-128,v=-128)#.blur(1.5)

overlay(c,cc,mask=rainbow)

show==true? rainbow : last

}

[juhok]

Thanks. It's a bit messy (plugins wise) and leaves some artifacts in Sample1 but looks ok on other stuff I tried it on. Bounty claimed! :)

[poisondeathray]

Yes, it's a quick & dirty way of treating those buggers.

I await for a better , more elegant solution too ! Waiting for one of the gurus to chime in..... It's one of these things I've seen pretty common on various VHS transfers , not sure of the underlying cause.

If you feel like providing a "bounty" Donate it to doom9, or your favorite charity

[juhok]

Donation sent to doom9.

[raffriff42]

Glad you found a solution. Here's another one, maybe. It seems to work OK on your two samples.
EDIT 1 disable false compensation on scene change.
EDIT 2 process both U and V; still not quite as good as poisondeathray's script below. LoadModule("avslib", "filters", "channels")

top_pix = 32 ## number of pixels from top to process
thresh_u = 1.4 ## U channel AverageLuma difference threshold
thresh_v = 1.4 ## V channel AverageLuma difference threshold
scene_chg = 25.0 ## scene change detection threshold
demo_mode = false ## if true, show "U" and/or "V" overlay on corrected frames

C = AviSource("Sample-1_DVSD.avi") + AviSource("Sample-2_DVSD.avi")
Assert(C.IsYV12, "video must be YV12 format")

U = C.ShowU
UT = U.Crop(0, 0, U.Width, top_pix)
UP = UT.Trim(1, UT.FrameCount-1) + UT.Trim(UT.FrameCount-2, -1) ## previous frame
UP = demo_mode ? UP.Subtitle("U") : UP
UC = StackVertical(
\ C.ConditionalFilter(UP, UT, "UT.AverageLuma-UP.AverageLuma", ">", String(thresh_u), show=false),
\ U.Crop(0, top_pix, U.Width, U.Height-top_pix))
UC = UC.ConditionalFilter(U, UC, "C.YDifferenceToNext", ">", String(scene_chg), show=false)

V = C.ShowV
VT = V.Crop(0, 0, V.Width, top_pix)
VP = VT.Trim(1, VT.FrameCount-1) + VT.Trim(VT.FrameCount-2, -1) ## previous frame
VP = demo_mode ? VP.Subtitle(" V") : VP
VC = StackVertical(
\ C.ConditionalFilter(VP, VT, "VT.AverageLuma-VP.AverageLuma", ">", String(thresh_v), show=false),
\ V.Crop(0, top_pix, V.Width, V.Height-top_pix))
VC = VC.ConditionalFilter(V, VC, "C.YDifferenceToNext", ">", String(scene_chg), show=false)

return MergeYUV(C.ShowY, UC, VC)Assumes:
- YV12 source
- artifacts limited to U & V channels
- artifacts always occur in top 32 lines
- artifacts only occur on isolated frames, not adjacent ones
- artifact area U/V AverageLuma greater than non-artifact counterpart
Requires AVSLib (http://avslib.sourceforge.net/)

[poisondeathray]

I tested this on a some other clips with similar defects , not just limited to the top area, seems to work better than the chubbyrain2 approach

In your samples, both U and V were affected, Y completely unaffected. I found this to be the case in other examples with similar defects

-Uses RemoveDirt & RemoveDirtMC variants (requires MVtools2.dll, RemoveGrain.dll) , applied to U,V odd/even fields

-YV12, keeps interlace intact

-Important to process linearly (don't jump around the timeline too much, or at least go back 10-20 frames before seeking to the frame you want), as with all dirt removal plugins

-Overlay crop is applied at the very end (instead of cropping each U, V odd or even field by a certain smaller pixel dimension variable), so you could have the choice of processing all U,V plane, or just the cropped version that is overlayed . But this means more pixels are processed (slower), instead of just processing a small strip - you can modify this if you want

-Yes, the "FilterStack" seems excessive!, but you can change it, or the strengths. I found that at least some strong combinations were required with this approach

(Don't shoot me because I am terrible at scripting clean code, so it's not organized efficently)





Orig=AVISource()

#Important to process linearly (don't jump around the timeline)
TopPixels=32 #Number of Pixels from top to keep

function FilterStack(clip input)
{
RemoveDirtMC_NEW(input,100,false)
RemoveDirtMC(last,100,false)
RemoveDirt(last)
}

###Odd Fields
Orig
AssumeBFF()
SeparateFields()
SelectOdd()
Od=last

Od.UToY()
FilterStack()
OdU_filtered=last

Od.VtoY()
FilterStack()
OdV_filtered=last

YtoUV(OdU_filtered, OdV_filtered, Od)
Od_filtered=last

###Even Fields
Orig
AssumeBFF()
SeparateFields()
SelectEven()
Ev=last

Ev.UToY()
FilterStack()
EvU_filtered=last

Ev.VtoY
FilterStack()
EvV_filtered=last

YtoUV(EvU_filtered, EvV_filtered, Ev)
Ev_filtered=last

Interleave(Od_filtered, Ev_filtered)
Weave()
Filtered=last

Filtered
#levels(0,0.5,255,0,255) #make overlay obvious
Crop(0,0,0,-(Orig.height-TopPixels))
FilteredCrop=last

#Overlay(Orig, Filtered) #If you want to apply without cropped overlay
Overlay(Orig, FilteredCrop)








function RemoveDirt(clip input, int limit, bool _grey)
{
clensed=input.Clense(grey=_grey, cache=4)
alt=input.RemoveGrain(2)
return RestoreMotionBlocks(clensed,input,alternative=alt,pthreshold=4,cthreshold=6, gmthreshold=40,dist=1,dmode=2,debug=false,noise=limit,noisy=12,grey=_grey)
}

function RemoveDirtMC(clip,int limit, bool "_grey")
{
_grey=default(_grey, false)
limit = default(limit,6)
bvec = clip.MVAnalyse(isb=false, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
fvec = clip.MVAnalyse(isb=true, blksize=8, delta=1, pel=2, truemotion=true, idx=1)
backw = clip.MVFlow(bvec)
forw = clip.MVFlow(fvec)
clp=interleave(backw,clip,forw)
clp=clp.RemoveDirt(limit,_grey)
clp=clp.SelectEvery(3,1)
return clp
}


function RemoveDirtMC_NEW(clip,int limit, bool "_grey")
{
_grey=default(_grey, false)
limit = default(limit,6)

prefiltered = RemoveGrain(clip,2)
superfilt = MSuper(prefiltered, hpad=32, vpad=32,pel=2)
super=MSuper(clip, hpad=32, vpad=32,pel=2)

bvec = MAnalyse(superfilt,isb=true, blksize=16, overlap=2,delta=1, truemotion=true)
fvec = MAnalyse(superfilt,isb=false, blksize=16, overlap=2,delta=1, truemotion=true)

bvec_re = Mrecalculate(super,bvec,blksize=8, overlap=0,thSAD=100)
fvec_re = Mrecalculate(super,fvec,blksize=8, overlap=0,thSAD=100)

backw = MFlow(clip,super,bvec_re)
forw = MFlow(clip,super,fvec_re)

clp=interleave(backw,clip,forw)
clp=clp.RemoveDirt(limit,_grey)
clp=clp.SelectEvery(3,1)
return clp
}

[juhok]

raffriff42, this kind of script approach is what I was looking for (also this introduced me to avslib for the first time). When running the script against longer videos there are false positives and also it doesn't seem to clean up the real problem frames completely all the time. Amazingly the ChubbyRain2 script works pretty much 99% of the time in the longer videos. I'll take your script as a baseline and try to stab it and make detection more accurate. Maybe use only the detection part and run ChubbyRain2 against positive frames.

poisondeathray, I think I got the general idea what that's doing. I have a bad relationship with RemoveDirt(MC) and it would bog my existing script a bit. I'll give it a try if I get reliable detection working.

I'll post my findings if I can make it better than ChubbyRain2 thingy.

Btw if anyone who is reading this is using ChubbyRain2 for a similar interlaced source, change the hardcoded Bifrost(interlaced=false) to Bifrost(interlaced=true) in ChubbyRain2 code. :)