Script for removal horizontal stripes (alpha release)

[yup]

Hi folk!
Alpha release :scared:!
For script need the same plugin like for QTGMC
http://forum.doom9.org/showthread.php?t=156028
and AVSRecursion, RemoveGrainT from kassandro
http://home.arcor.de/kassandro/AvsRecursion/AvsRecursion.htm
http://home.arcor.de/kassandro/prerelease/RemoveGrainT-1.0.rar
Using
AVISource("selnew.avi")
removeline(thicknessline=2, distser=1, halflength=3,comparing=false)
Script tested with Set last build Avisynth 2.6.
short parameters description
work with interlaced source (for film need some small adaptation)
thicknessline thickness horizontal spike in pixels at field scale (not frame) 2 good start for many VHS capture (if choose thicker than at source scrit do not work)
distser distance between short lines, if source have single spike (not series) no sense, but for speed use distser=1
halflength detect line longer 2*halflength+1 pixels, if source have spike 7 pixels (not longer) could be equal 3, if set higher than at source script do not work
comparing if true ouput will be stacked fields source, filtered and diff for tuning parameters
For some captures need sequintial using script, always need starting with biger thicknessline (first pass with thicknessline=2 second thicknessline=1, for very bad may be need three pass and starting from thicknessline=3).
Script can remove two sequintial spikes at the same place frame, if we have three sequintial spikes need use script 2 times with same thicknessline value.
Work very slowly 1.5-2 fps on i7 Core.
Any suggestion for optimization script welcome.

yup.

[Guest]

Post a screenshot showing the problem. Sounds like motor noise interference or something like that.

(With 165 posts I suppose you know about interlacing.)

[yup]

top field from source
http://img443.imageshack.us/img443/4222/stripes9vt.png
yup.

[Guest]

Yup, looks like analog shot noise is affecting your capture. Check all your cabling and grounds. Make sure you have no big motors, generators, refrigerators, etc., running nearby or on the same circuit. No car engines running nearby. That sort of thing. This is really hard to remove with filters.

[yup]

neuron2!
Thank you for reply. For power supply I use online UPS with isolated transformer (for PC and VCR). Early I made one capture without this stripes. I think this coupled with tape. I try capture 2 times and get stripes at same places.
yup.

[DeathWolf]

yes, it sounds like tape reading heads arent able to read these parts, i had the exact same problem when digitizing tapes. Unfortunatelly i couldnt find any decent way of removing them without hurting the rest of the video. And apparently the only solution is to get a better tape reader which can come very expensive for some formats(even though they are very old...)

[yup]

DeathWolf!
I use Sony 4-head VCR. VHS cassete made during edit from one VCR to other.
yup.

[yup]

Hi folk!
After many try and error i find script which remove stripes
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fco = o.MVCompensate(fvo, idx=1, thSCD1=500)
bco = o.MVFlow(bvo, idx=1, thSCD1=500) MVCompensate interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fce = e.MVFlow(fve, idx=1, thSCD1=500)
bce = e.MVFlow(bve, idx=1, thSCD1=500)
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)
Degrainmedian need use in pure median mode. Now I try find other median filter for time plane and postfilter.
yup.

[Didée]

Hi folk!
After many try and error i find script which remove stripes
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fco = o.MVCompensate(fvo, idx=1, thSCD1=500)
bco = o.MVFlow(bvo, idx=1, thSCD1=500) MVCompensate interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fce = e.MVFlow(fve, idx=1, thSCD1=500)
bce = e.MVFlow(bve, idx=1, thSCD1=500)
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)

From the idea, that should work. But the script as you posted is buggy.

Blue seems fishy: one time flow, one time compensate?

Green will cause some syntax error.

Red is a big no-no: you must not use the same idx value for even and odd fields. These have to be different. (At least with pel=2, which is default.)

Apart from that, the script is fine. ;)

[Didée]

A small demo of wrong idx usage:

Correct idx usage:
http://img255.imageshack.us/img255/9890/twoclipsvu9.th.png (http://img255.imageshack.us/my.php?image=twoclipsvu9.png)

Wrong idx usage:
http://img259.imageshack.us/img259/6923/twoclipswrongidxsk5.th.png (http://img259.imageshack.us/my.php?image=twoclipswrongidxsk5.png)


v = BlankClip(width=304,height=224, pixel_type="yuy2")
v1 = v.Subtitle("this one (v1)",y=32).AddBorders(8,8,8,8,color=$FF0000)
v2 = v.Subtitle("that one (v2)",y=64).AddBorders(8,8,8,8,color=$00FF00)

bw1 = v1.MVAnalyse(isb=true, idx=1)
fw1 = v1.MVAnalyse(isb=false, idx=1)
comp1bw = v1.MVCompensate(bw1,idx=1)
comp1fw = v1.MVCompensate(fw1,idx=1)
x1 = Interleave(comp1fw,v1,comp1bw).TemporalSoften(1,255,255,255,2).SelectEvery(3,1)

bw2 = v2.MVAnalyse(isb=true, idx=1) # idx = 1 is WRONG! (just for demonstration!)
fw2 = v2.MVAnalyse(isb=false, idx=1) # --""--
comp2bw = v2.MVCompensate(bw2,idx=1) # --""--
comp2fw = v2.MVCompensate(fw2,idx=1) # --""--
x2 = Interleave(comp2fw,v2,comp2bw).TemporalSoften(1,255,255,255,2).SelectEvery(3,1)

#StackHorizontal(v1,v2)
StackHorizontal(x1,x2)

[yup]

Didée!
Thanks for reply.
I make cut and paste from script with Russian comments.
Also please advise postfilter more suitable for pure median prefiltering.
yup.

[Didée]

What kind of postfilter do you mean now?

Your script is basically fine, you just have to use idx=2 in the 2nd part of your script.

[yup]

Didée!
Now i do not use postfilter I only search him:)
yup.

[yup]

Hi folk!
I remove error from script.
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1)
fco = o.MVFlow(fvo, idx=1, thSCD1=500)
bco = o.MVFlow(bvo, idx=1, thSCD1=500)
interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
#DeGrainMedian(limitY=255,limitUV=255,mode=1)
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 2, overlap=4, dct=1)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 2, overlap=4, dct=1)
fce = e.MVFlow(fve, idx=2, thSCD1=500)
bce = e.MVFlow(bve, idx=2, thSCD1=500)
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true)
#DeGrainMedian(limitY=255,limitUV=255,mode=1)
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)

I totaly remove all black stripe only after 3 pass through this script my clip and clip little blured. I try use compensation from -2 frame and +2 frame positoion (DegrainMedian work on 5 frames), but result get worst. If somebode have any idea please advise.
With kind regards yup.

[yup]

Hi folk!
After more try and error. :thanks: tsp, Fizick, Manao, Mug Funky and other i find better way for remove black stripes.
First I use MVTools for prefiltering:
source=AVISource("clip.avi")
fields=source.AssumeBFF().SeparateFields()
backward_vec = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
forward_vec = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
fields.MVDegrain1(backward_vec,forward_vec,thSAD=400,idx=1)
Weave()
I use dct=1 for better accuracy motion estimation as recomended Fizick, and save this file in VirtualDub as degr1.avi. After I use ml3dex function from MedianBlur plugin, for estimation motion vector I use prefiltered source and for filtering use source.
AviSource("clip.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
SeparateFields()
source=last
AviSource("degr1.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
SeparateFields()
filtered=last
bv1 = filtered.MVAnalyse(blksize=8, isb = true, truemotion=true, delta=2, overlap=4,idx = 1,dct=1)
fv1 = filtered.MVAnalyse(blksize=8, isb = false, truemotion=true, delta=2, overlap=4,idx = 1,dct=1)
bc1 =source.MVFlow(bv1, idx=2, thSCD1=1000)
fc1 = source.MVFlow(fv1, idx=2, thSCD1=1000)
interleave(bc1, source, fc1)
ml3dex(mc=false,Y=3, U=3,V=3)
SelectEvery(3,1)
Weave()
Source clip DV YUY2, but ml3dex work only with YV12. For many parts my source enough one pass, but for worst parts I use 2 pass (for second pass I prefiltered pass1 use MVTools). Before encoding, after ml3dex need use MVDegrain1 for remove grain which bring ml3dex. Also I find that use 2 pass 1 step motion estimation (prefiltering MVDegrain1 and ml3dex) work better than use MVDEgrain2 for prefiltering and 2 call ml3dex in script.
See one field for comparing (top filtered, bottom source):
http://www.sendspace.com/file/ig8pgo
If any have idea for improvement please advice.
With kind regards yup.

[Fizick]

you get blurring on dress.
Do you try Despot plugin in seg=1 mode or median mode?
with width=big and height=1?

[yup]

Fizick! Thank You for reply.
Please precise parameter set.
I try for spot removal mode i use seg=1, pheight=1, pwidth=30 (maximum length black stripe)
for median mode
median=true, mheight=1, mwidth=30 instead ml3dex in my script. Picture less blured than ml3dex but many stripe do not removed. Please advise way for tuning other parameters.

With kind regards yup.

[Fizick]

sign = 2

[yup]

Hi folk!
Fizick I try find magic parameter set for Despot but without success.
I little modified supposed by gzarkadas script for creation spike detection index using motion compensated frames, see below
LoadPlugin("mt_masktools.dll") # v2.0a30
SetMemoryMax(256)
AVISource("seldv2.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
backward_vec = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
forward_vec = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
fc = fields.MVFlow(forward_vec, idx=1, thSCD1=10000)
bc = fields.MVFlow(backward_vec, idx=1, thSCD1=10000)
masksdi=SDIPixelMask(bc, fields, fc, 16, 0.8) # define your thresholds here
StackVertical(fields,masksdi)
#Weave()


# this is for Abs(p - f) or Abs(p - b)
Function absdiff(clip c1, clip c2) { return mt_lutxy(c1, c2, "x y - abs") }

# this build the expression for the SDI function
Function SDI_RPN(string expr_d1, string expr_d2, string expr_t1, string expr_t2) {
_d1 = expr_d1 + " "
_d2 = expr_d2 + " "
_t1 = expr_t1 + " "
_t2 = expr_t2 + " "
return _d1 + _t1 + "> " + _d2 + _t1 + "> | 1.0 " \
+ _d1 + _d2 + "- " + _d1 + _d2 + "+ / abs - 0.0 ? " \
+ _t2 + "> 1.0 " + _d1 + _t1 + "> " + _d2 + _t1 + "> | 1.0 " \
+ _d1 + _d2 + "- " + _d1 + _d2 + "+ / abs - 0.0 ? "
}

# this creates the mask
Function SDIPixelMask(clip prev, clip curr, clip next, float t1, float t2) {

c_d1 = absdiff(curr, next)
c_d2 = absdiff(curr, prev)

# since SDI returns 0..1 we multiply with 255

mask = mt_lutxy(c_d1, c_d2, SDI_RPN("x", "y", String(t1), String(t2)) + " 255 *")
return mask
}
see image
http://imgplace.com/directory/dir3096/1179845621_4088.png (http://www.imgplace.com/getcode.php?img=directory/dir3096/1179845621_4088.png)
This script successfully detecting black and white lines, but also find place where exist occlusion during motion estimation. My idea very simple find occlusion mask use MVMask with kind=2 (advice how use ml parameter), for both direction, forward and backward, binarize both mask to 0 and 255 level, use logical AND for find mask occlusion in both directions, because spike detection index do not sensitive for occlusion in one direction and last use logical XOR for remove from SDI occlusion mask. Unfortunately I not familiarly with mt_masktools. Please advice.
With kind regards yup.

[yup]

Hi!
After small break I back to this problem. I upgrade hardware for capture (Panasonic DMR-ES35V DVD recorder with VCR, time base corrector Cypress CTB-100, Ixos S-Video cables), rewrite script. Script based on ideas which i find on this forum.
LoadPlugin("mt_masktools.dll")
AVISource("leb1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
snf=bob(fields,0,0.5)
f=fields.mt_luts(fields,mode="median",pixels="0 -1 0 0 0 1",U=1,V=1)
source=bob(f,0,0.5)
bv1 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
bv2 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
fv1 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
fv2 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
#bc1 =snf.MVFlow(bv1, idx=2, thSCD1=1000)
#fc1 = snf.MVFlow(fv1, idx=2, thSCD1=1000)
bc1 =snf.MVCompensate(bv1, idx=2, mode=1)
bc2 =snf.MVCompensate(bv2, idx=2, mode=1)
fc1 = snf.MVCompensate(fv1, idx=2, mode=1)
fc2 = snf.MVCompensate(fv2, idx=2, mode=1)
interleave(bc2, bc1, source, fc1, fc2)
medianblurt(radiusy=0,radiusu=0,radiusv=0,temporalradius=2)
SelectEvery(5,2)
SeparateFields()
SelectEvery(4,0,3)
StackVertical(last,fields)

source and processing image
http://img405.imageshack.us/img405/1136/blacklineremovesp3.th.png (http://img405.imageshack.us/my.php?image=blacklineremovesp3.png)
Quantity black line decrease, horizontal stability increase, that help motion compensated filtering. Small problems I decrease color space from YUY2 to YV12 and after need increase sharpness. May be I need postprocessing after median filtering before sharpening? Please advice.
With kind regards yup.

[R3Z]

Hi!
After small break I back to this problem. I upgrade hardware for capture (Panasonic DMR-ES35V DVD recorder with VCR, time base corrector Cypress CTB-100, Ixos S-Video cables), rewrite script. Script based on ideas which i find on this forum.
LoadPlugin("mt_masktools.dll")
AVISource("leb1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
snf=bob(fields,0,0.5)
f=fields.mt_luts(fields,mode="median",pixels="0 -1 0 0 0 1",U=1,V=1)
source=bob(f,0,0.5)
bv1 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
bv2 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
fv1 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
fv2 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
#bc1 =snf.MVFlow(bv1, idx=2, thSCD1=1000)
#fc1 = snf.MVFlow(fv1, idx=2, thSCD1=1000)
bc1 =snf.MVCompensate(bv1, idx=2, mode=1)
bc2 =snf.MVCompensate(bv2, idx=2, mode=1)
fc1 = snf.MVCompensate(fv1, idx=2, mode=1)
fc2 = snf.MVCompensate(fv2, idx=2, mode=1)
interleave(bc2, bc1, source, fc1, fc2)
medianblurt(radiusy=0,radiusu=0,radiusv=0,temporalradius=2)
SelectEvery(5,2)
SeparateFields()
SelectEvery(4,0,3)
StackVertical(last,fields)

source and processing image
http://img405.imageshack.us/img405/1136/blacklineremovesp3.th.png (http://img405.imageshack.us/my.php?image=blacklineremovesp3.png)
Quantity black line decrease, horizontal stability increase, that help motion compensated filtering. Small problems I decrease color space from YUY2 to YV12 and after need increase sharpness. May be I need postprocessing after median filtering before sharpening? Please advice.
With kind regards yup.


Great script yup :) works really well. I dont completely understand why though :(

Could you (or someone else) explain to me what the mt_luts function does ? I dont understand why its there !

Cheers,

R3Z

[yup]

Hi R3Z!
This Didée idea http://forum.doom9.org/showthread.php?p=1017230#post1017230
Pure median only vertical filter. All remain as usual, use prefiltering for motion vectors estimation and use nonfiltering clip for motion compensation, and last use pure median temporal filter. If you have idea for improve let me know.
yup.

[yup]

Hi all!
Change code and added masks. Thanks Manao for hacking planar Colorspace for mt_masktools ans thanks to Kassandro for plugin VerticalCleaner http://videoprocessing.11.forumer.com/viewtopic.php?t=107
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("VerticalCleanerSSE2.dll")
LoadPlugin("SSE2Tools.dll")
LoadPlugin("RemoveGrainTSSE2.dll")
DirectShowSource("seldv.avi")
AssumeBFF()
fields=SeparateFields()
Interleaved2Planar(fields)
fieldsfpl=VerticalCleaner(mode=2,planar=true)
fieldsf=Planar2Interleaved()
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
mf1=Greyscale(fieldsf).MVMask(fv1,kind=1,Ysc=255)
mf1pl=Interleaved2Planar(mf1).mt_binarize(threshold=254,upper=true)
mb1=Greyscale(fieldsf).MVMask(bv1,kind=1,Ysc=255)
mb1pl=Interleaved2Planar(mb1).mt_binarize(threshold=254,upper=true)
mmaskpl=mt_logic(mf1pl,mb1pl,"max")
#clense(Interleaved2Planar(bc1), Interleaved2Planar(fields), Interleaved2Planar(fc1),increment=0)
mcf=clense(bc1, fields,fc1,increment=0)
mcfpl=Interleaved2Planar(mcf)
mcfmaskpl=mt_merge(mcfpl,fieldsfpl,mmaskpl,u=3,v=3)
#Planar2Interleaved()
#Weave()
mcfmasked=Planar2Interleaved(mcfmaskpl)
StackVertical(fields,mcfmasked)
See picture
http://img338.imageshack.us/img338/7571/girlbz7.th.png (http://img338.imageshack.us/img338/7571/girlbz7.th.png)
Any advice for impovement. Now script work in YUY2 and will be usefull for analog capture (huffyuv or DV) without colorspace conversion.
yup.

[2Bdecided]

yup,

I think your script is very clever, and tries to solve a near-impossible problem. Sadly, the script also seems to change the "good" image content quite a lot.

I have the same problem on some of my tapes, so I have been looking for a solution. Reading the videohelp.com forums, the most likely cause is worn heads on the playback machine, the second most likely cause is worn heads on the original record machine, and the third most likely cause is dirty heads on either machine.

So, first thing you can try is a head cleaner. The second thing is playing back the tape with the original record VCR (if available). The third thing is to try any other VCR - it doesn't have to be a good one - some times really bad (cheap or worn) VCRs are good at solving the problem (though make sure they don't break your precious tapes).


If the solution must be within AVIsynth, I have two suggestions (based on what I see on my tapes - yours may be different).

1. The black lines appear in slightly different positions each time I capture the same tape on the same VCR. Maybe you can capture the tape two or three times, and then keep the maximum value for each pixel across the three captures?

2. The black lines appear in slightly predictable places - often near a dark/light or light/dark luma border - maybe only apply any filtering in such areas? (This sounds very difficult to me).

Hope this is some help. If I find a reasonable solution from any of this, I'll post back.

I never dreamt capturing a few old tapes would cause me so many problems. I guess the good news is that, at least for now, they still play back surprisingly well. If we wait another ten years, the tapes may decay, and there will be very few decks left to play them back on!

Cheers,
David.

[yup]

@2Bdecided!
1. The black lines appear in slightly different positions each time I capture the same tape on the same VCR. Maybe you can capture the tape two or three times, and then keep the maximum value for each pixel across the three captures?
Yes, I try it.
http://videoprocessing.11.forumer.com/viewtopic.php?t=109
This only first stage.
Problem only with dubbed VHS, frequently the different capture not sync.
yup.

[2Bdecided]

By "in sync" do you mean frame loss, or movement within/of frame?

that should be an easier thing to correct than all the other problems you face, though I don't know how to do it in AVIsynth.

In general, assuming vertical (i.e. line numbers) are correct, I would simply convolve (maybe I mean cross correlate?) the signals horizontally, and the peak in the convolution result shows the offset between the two signals. This allows you to match them both.

Cheers,
David.

[yup]

Hi all!
Now mask for bad motion estimation could work
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("VerticalCleanerSSE2.dll")
LoadPlugin("SSE2Tools.dll")
LoadPlugin("RemoveGrainTSSE2.dll")
DirectShowSource("seldv.avi")
#AVISource("flower.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
fieldsf=fields.VerticalCleaner(mode=2)
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
mf1=fieldsf.MVMask(fv1,kind=2,ml=30,Ysc=255)
mb1=fieldsf.MVMask(bv1,kind=2,ml=30,Ysc=255)
mmask=mt_logic(mf1,mb1,"max",u=-128,v=-128)
mcf=clense(bc1, fields,fc1,increment=0)
mcfmask=mt_merge(mcf,fieldsf,mmask,luma=true)
#Weave()
#StackVertical(fields,mcfmask)
Weave(mcfmask)
ConvertToYUY2(interlaced=true)
LetterBox(16,16,16,16)

But now only in YV12 colorspace. This work better than for kind=1 MVMask. Next stage preserve detail in source video.
yup.

[Didée]

This work better than for kind=1 MVMask.

Really? If that's your experience, fine. My experience is that MVMask(kind=2) is WAY less reliable to detect bad motion.

Random examples:

http://img371.imageshack.us/img371/6803/mvmaskstuffath7.th.jpg (http://img371.imageshack.us/my.php?image=mvmaskstuffath7.jpg) - http://img370.imageshack.us/img370/9853/mvmaskstuffbny8.th.jpg (http://img370.imageshack.us/my.php?image=mvmaskstuffbny8.jpg)

Left-top: source
Left-bottom: compensation (delta=2)
Middle-top: your mask, exactly the code from above (max-merging forward+backward kind=2 mask)
middle-bottom: the usual MVMask(kind=1)
right-top: compensation corrected by your mask
right-bottom: compensation corrected by MVMask(kind=1)

That's the typical kind of "performance" I'm used to get from kind=2. For me it marks too much areas "bad" that infact would be usable, still it often does not detect areas that are really unusable.

[yup]

Didée!:thanks:
For my source with pulse noise, if using kind=1 I get only Verticalcleaner output i. e. motion compensated filter practicaly do not work. May be main reason for this problem output median prefilter, which segregate picture and move border. kind=2 give less restriction. May be use prefilter for source now I try Deblock:
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("VerticalCleanerS.dll")
LoadPlugin("RemoveGrainT.dll")
AVISource("selnew.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
fieldsf=fields.VerticalCleaner(mode=2)
fieldsf=fieldsf.Deblock(quant=25)
bv1f = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
fv1f = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
bc1 =fields.MVCompensate(bv1f, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1f, idx=2, mode=1, thSAD=16000)
mf1f=fieldsf.MVMask(fv1f,kind=2,ml=100,Ysc=255)
mb1f=fieldsf.MVMask(bv1f,kind=2,ml=100,Ysc=255)
mmaskf=mt_logic(mf1f,mb1f,"max")
mmmaskf=mmaskf.Deblock(quant=25)
mcf=clense(bc1, fields,fc1,increment=0)
mcfmask=mt_merge(mcf,fieldsf,mmaskf,luma=true)
#
#try preserve good motion compensated part of frame
fieldspre=fields.Deblock(quant=25)
bv1 = fieldspre.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 3, overlap=4, dct=0,chroma=false)
fv1 = fieldspre.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 3, overlap=4, dct=0,chroma=false)
mf1=fieldspre.MVMask(fv1,kind=1,ml=15,Ysc=255)
mb1=fieldspre.MVMask(bv1,kind=1,ml=15,Ysc=255)
mmask=mt_logic(mf1,mb1,"max")
mmask=mt_invert(mmask,U=-128,V=-128)
mmask=mmask.Deblock(quant=25)
mcfmasktot=mt_merge(mcfmask,fields,mmask,luma=true)
StackVertical(fields,mcfmasktot)

yup.

[yup]

Hi all!
Folowing script:
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("VerticalCleanerSSE2.dll")
LoadPlugin("SSE2Tools.dll")
LoadPlugin("RemoveGrainTSSE2.dll")
LoadPlugin("RemoveGrainHDS.dll")
AVISource("flower.avi")
#DirectShowSource("seldv.avi")
AssumeTFF()
fields=SeparateFields()
fieldspl=Interleaved2Planar(fields)
fieldsfpl=fieldspl.VerticalCleaner(mode=2,planar=true)
fieldsf2pl=Quantile(fieldspl,planar=true)
fieldsf=Planar2Interleaved(fieldsfpl)
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=3, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=3, delta = 4, idx = 1, overlap=4, dct=0,chroma=false)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=3, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=3, delta = 4, idx = 1, overlap=4, dct=0,chroma=false)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 =fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 =fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)
mlth=100
mf1=fieldsf.MVMask(fv1,kind=1,ml=mlth,Ysc=255)
mf1pl=Interleaved2Planar(mf1)
mf2=fieldsf.MVMask(fv2,kind=1,ml=mlth,Ysc=255)
mf2pl=Interleaved2Planar(mf2)
mb1=fieldsf.MVMask(bv1,kind=1,ml=mlth,Ysc=255)
mb1pl=Interleaved2Planar(mb1)
mb2=fieldsf.MVMask(bv2,kind=1,ml=mlth,Ysc=255)
mb2pl=Interleaved2Planar(mb2)
maskcenter=mt_invert(mt_logic(mf1pl,mb1pl,"max"))
maskbw=mt_invert(mt_logic(mb1pl,mb2pl,"max"))
maskfw=mt_invert(mt_logic(mf1pl,mf2pl,"max"))
mcfcenter=clense(bc1, fields,fc1,increment=0)
mcfcenterpl=Interleaved2Planar(mcfcenter)
mcfbw=clense(bc1, fields,bc2,increment=0)
mcfbwpl=Interleaved2Planar(mcfbw)
mcffw=clense(fc1, fields,fc2,increment=0)
mcffwpl=Interleaved2Planar(mcffw)
mcfmaskedpl=mt_merge(fieldsfpl,mcfbwpl,maskbw)
mcfmaskedpl=mt_merge(mcfmaskedpl,mcffwpl,maskfw)
mcfmaskedpl=mt_merge(mcfmaskedpl,mcfcenterpl,maskcenter)
#Weave()
mcfmasked=Planar2Interleaved(mcfmaskedpl)
StackVertical(Greyscale(fields),Greyscale(mcfmasked))

give better result than all my previous try. Former I try remove luma and chroma noise based on only luma value, not very good approach. Also I could be increase search parameter MVTools to 3 for better motion estimation. Change to search=3 more effective than decrease thSAD. Small step done.
:thanks: to all for advice and support.
yup.

[yup]

Hi all!
After some try and error i find script which practicaly remove 1 pixel thickness black and white lines
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("VerticalCleanerSSE3.dll")
LoadPlugin("SSE3Tools.dll")
LoadPlugin("RemoveGrainTSSE3.dll")
LoadPlugin("RemoveGrainSSE3.dll")
AVISource("melted3.avi")
source=AssumeTFF()
ConvertToYV12(source,interlaced=true)
fields=SeparateFields()
fieldsf2=fields.VerticalCleaner(mode=2)
fieldsf=fieldsf2.RemoveGrain(11)
bv1 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
bv2 = fieldsf.MVAnalyse(blksize=8, isb = true, truemotion=true, search=2, delta = 4, idx = 1, overlap=4, dct=0,chroma=false)
fv1 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 2, idx = 1, overlap=4, dct=0,chroma=false)
fv2 = fieldsf.MVAnalyse(blksize=8, isb = false, truemotion=true, search=2, delta = 4, idx = 1, overlap=4, dct=0,chroma=false)
bc1 =fields.MVCompensate(bv1, idx=2, mode=1, thSAD=16000)
bc2 =fields.MVCompensate(bv2, idx=2, mode=1, thSAD=16000)
fc1 =fields.MVCompensate(fv1, idx=2, mode=1, thSAD=16000)
fc2 =fields.MVCompensate(fv2, idx=2, mode=1, thSAD=16000)
mf1=fieldsf.MVMask(fv1,kind=1,ml=100,Ysc=255)
mf2=fieldsf.MVMask(fv2,kind=1,ml=100,Ysc=255)
mb1=fieldsf.MVMask(bv1,kind=1,ml=100,Ysc=255)
mb2=fieldsf.MVMask(bv2,kind=1,ml=100,Ysc=255)
maskcenter=mt_invert(mt_logic(mf1,mb1,"max"))
maskbw=mt_invert(mt_logic(mb1,mb2,"max"))
maskfw=mt_invert(mt_logic(mf1,mf2,"max"))
mcfcenter=clense(bc1, fields,fc1,increment=0)
mcfbw=clense(bc1, fields,bc2,increment=0)
mcffw=clense(fc1, fields,fc2,increment=0)
mcfmasked=mt_merge(fieldsf2,mcfbw,maskbw)
mcfmasked=mt_merge(mcfmasked,mcffw,maskfw)
mcfmasked=mt_merge(mcfmasked,mcfcenter,maskcenter)
Weave(mcfmasked)
convertToYUY2()
MergeChroma(Source)

Very interesting thing MVAnalyse better work with dct=0 than with dct=1. Also this script can use for analog capture with large grain noise and use instead VerticalCleaner RemoveGrain(1) only as first stage denoising. Second stage MVDegrainX.
yup.

[yup]

Hi all!
Time by time I try back to solving this problem. :thanks: to plugin writer which constantly expand feature especially tritical.
New version black line remover
SetMemoryMax(768)
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("RemoveGrainTSSE3.dll")
SetMTMode(5)
source=DirectShowSource("cap1.avi")
SetMTMode(2)
source=source.Trim(1019,0).AssumeTFF().ConvertToYV12(interlaced=true)
bobnn=source.eedi3(sclip=nnedi3(source,field=-2,nsize=3,qual=2,U=false, V=false),field=-2,vcheck=3,U=false, V=false)
bobnnmed=bobnn.mt_luts(bobnn,mode="median",pixels="0 -3 0 -2 0 -1 0 0 0 1 0 2 0 3",U=2,V=2)
#bobnnmed=bobnn.mt_luts(bobnn,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",U=2,V=2)
bobnnmedf=bobnnmed.dfttest(tbsize=1,ftype=1,sbsize=8,sosize=6,sigma=1000,U=false, V=false)
().SelectEvery(4,0,3)
super=MSuper(bobnn)
superf=MSuper(bobnnmedf)
bv1 = MAnalyse(superf, blksize=16, isb = true, delta = 2, overlap=8, dct=5,chroma=false)
bv2 = MAnalyse(superf, blksize=16, isb = true, delta = 4, overlap=8, dct=5,chroma=false)
fv1 = MAnalyse(superf, blksize=16, isb = false,delta = 2, overlap=8, dct=5,chroma=false)
fv2 = MAnalyse(superf, blksize=16, isb = false,delta = 4, overlap=8, dct=5,chroma=false)
bc1 = MCompensate(bobnn, super, bv1, thSAD=16000)
bc2 = MCompensate(bobnn, super, bv2, thSAD=16000)
fc1 = MCompensate(bobnn, super, fv1, thSAD=16000)
fc2 = MCompensate(bobnn, super, fv2, thSAD=16000)
mf1=bobnnmedf.MMask(fv1,kind=1,ml=100,Ysc=255)
mf2=bobnnmedf.MMask(fv2,kind=1,ml=100,Ysc=255)
mb1=bobnnmedf.MMask(bv1,kind=1,ml=100,Ysc=255)
mb2=bobnnmedf.MMask(bv2,kind=1,ml=100,Ysc=255)
maskcenter=mt_invert(mt_logic(mf1,mb1,"max",U=2,V=2))
maskbw=mt_invert(mt_logic(mb1,mb2,"max",U=2,V=2))
maskfw=mt_invert(mt_logic(mf1,mf2,"max",U=2,V=2))
mcfcenter=clense(bc1, bobnn,fc1,increment=0)
mcfbw=clense(bc1, bobnn,bc2,increment=0)
mcffw=clense(fc1, bobnn,fc2,increment=0)
mcfmasked=mt_merge(bobnnmed,mcfbw,maskbw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcffw,maskfw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcfcenter,maskcenter,U=2,V=2)

fieldf=mcfmasked.AssumeTFF().SeparateFields().SelectEvery(4,0,3)
#StackVertical(Separatefields(source),fieldf.MergeChroma(Separatefields(source)))
Weave(fieldf.MergeChroma(Separatefields(source)))

eedi3 work more better especially at artificial objects with straight borders (windows, wall and fence). Also post-filtering using dfttest give better motion search, because after using median vertical filter picture very segregated. I am almost satisfied result, but want change priority for use different temporal median filtered frames (mcfbf,
mcff and mcfcenter) and need help
http://forum.doom9.org/showthread.php?p=1445155#post1445155
yup.

[yup]

I try write code using priority based on minimum SAD
SetMemoryMax(768)
LoadPlugin("RemoveGrainTSSE2.dll")
source=AVISource("sel97.avi")
source=source.AssumeTFF().ConvertToYV12(interlaced=true)
bobnn=source.nnedi3(field=-2)
bobnnmed=bobnn.mt_luts(bobnn,mode="median",pixels="0 -3 0 -2 0 -1 0 0 0 1 0 2 0 3",U=2,V=2)
bobnnmedf=bobnnmed.dfttest(tbsize=1,ftype=1,sbsize=16,sosize=12,sigma=1000,U=false, V=false)
super=MSuper(bobnn)
superf=MSuper(bobnnmedf)
bv1 = MAnalyse(superf, blksize=16, isb = true, delta = 2, overlap=8, dct=5,chroma=false)
bv2 = MAnalyse(superf, blksize=16, isb = true, delta = 4, overlap=8, dct=5,chroma=false)
fv1 = MAnalyse(superf, blksize=16, isb = false,delta = 2, overlap=8, dct=5,chroma=false)
fv2 = MAnalyse(superf, blksize=16, isb = false,delta = 4, overlap=8, dct=5,chroma=false)
bc1 = MCompensate(bobnn, super, bv1, thSAD=16000, thSCD1=16000)
bc2 = MCompensate(bobnn, super, bv2, thSAD=16000, thSCD1=16000)
fc1 = MCompensate(bobnn, super, fv1, thSAD=16000, thSCD1=16000)
fc2 = MCompensate(bobnn, super, fv2, thSAD=16000, thSCD1=16000)
mf1=bobnnmedf.MMask(fv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf2=bobnnmedf.MMask(fv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
mb1=bobnnmedf.MMask(bv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mb2=bobnnmedf.MMask(bv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
maskcenter=mt_logic(mf1,mb1,"max",U=2,V=2)
maskbw=mt_logic(mb1,mb2,"max",U=2,V=2)
maskfw=mt_logic(mf1,mf2,"max",U=2,V=2)
mcfcenter=clense(bc1, bobnn,fc1,increment=0)
mcfbw=clense(bc1, bobnn,bc2,increment=0)
mcffw=clense(fc1, bobnn,fc2,increment=0)
#
output=bobnnmed
# center filtered
mt_lutxyz(maskcenter,maskbw,maskfw,"x y <= x z <= & x 255 < &")
output=mt_lutxyz(last,mcfcenter,output,"x y z ?")
# backward filtered
mt_lutxyz(maskbw,maskcenter,maskfw,"x y < x z < &")
output=mt_lutxyz(last,mcfbw,output,"x y z ?")
# forward filtered
mt_lutxyz(maskfw,maskcenter,maskbw,"x y < x z < &")
output=mt_lutxyz(last,mcffw,output,"x y z ?")

fieldf=output.AssumeTFF().SeparateFields().SelectEvery(4,0,3)
StackVertical(Separatefields(source),fieldf.MergeChroma(Separatefields(source)))

frame example
http://img69.imageshack.us/img69/9223/removeline.th.png (http://img69.imageshack.us/i/removeline.png/) Uploaded with ImageShack.us (http://imageshack.us)
Any suggestion welcome.
yup.

[yup]

Hi all!
I little change condition for SAD and change hard threshold to soft.

source=AVISource("selnew.avi")
source=source.AssumeTFF().ConvertToYV12(interlaced=true)
bobnn=source.nnedi3(field=-2,U=false, V=false)
bobnnmed=bobnn.mt_luts(bobnn,mode="median",pixels="0 -3 0 -2 0 -1 0 0 0 1 0 2 0 3",U=2,V=2)
bobnnmedf=bobnnmed.dfttest(tbsize=1,ftype=1,sbsize=16,sosize=12,sigma=1000,U=false, V=false)
super=MSuper(bobnn,chroma=false)
superf=MSuper(bobnnmedf,chroma=false)
bv1 = MAnalyse(superf, blksize=16, isb = true, delta = 2, overlap=8, dct=5,chroma=false)
bv2 = MAnalyse(superf, blksize=16, isb = true, delta = 4, overlap=8, dct=5,chroma=false)
fv1 = MAnalyse(superf, blksize=16, isb = false,delta = 2, overlap=8, dct=5,chroma=false)
fv2 = MAnalyse(superf, blksize=16, isb = false,delta = 4, overlap=8, dct=5,chroma=false)
bc1 = MCompensate(bobnn, super, bv1, thSAD=16000, thSCD1=16000)
bc2 = MCompensate(bobnn, super, bv2, thSAD=16000, thSCD1=16000)
fc1 = MCompensate(bobnn, super, fv1, thSAD=16000, thSCD1=16000)
fc2 = MCompensate(bobnn, super, fv2, thSAD=16000, thSCD1=16000)
mb1sad=bobnnmedf.MMask(bv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mb2sad=bobnnmedf.MMask(bv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf1sad=bobnnmedf.MMask(fv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf2sad=bobnnmedf.MMask(fv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
centersad=mt_logic(mf1sad,mb1sad,"max",U=2,V=2)
bwsad=mt_logic(mb1sad,mb2sad,"max",U=2,V=2)
fwsad=mt_logic(mf1sad,mf2sad,"max",U=2,V=2)
mcfcenter=clense(bc1, bobnn,fc1,increment=0, grey=true)
mcfbw=clense(bc1, bobnn,bc2,increment=0, grey=true)
mcffw=clense(fc1, bobnn,fc2,increment=0, grey=true)
# center filtered
mt_lutxyz(centersad,bwsad,fwsad,"x y <= x z <= & x 255 < &",U=2,V=2)
maskcenter=mt_lutxy(last,centersad,"x 255 y - 0 ?",U=2,V=2)
# backward filtered
mt_lutxyz(bwsad,centersad,fwsad,"x y < x z <= &",U=2,V=2)
maskbw=mt_lutxy(last,bwsad,"x 255 y - 0 ?",U=2,V=2)
# forward filtered
mt_lutxyz(fwsad,centersad,bwsad,"x y < x z < &",U=2,V=2)
maskfw=mt_lutxy(last,fwsad,"x 255 y - 0 ?",U=2,V=2)
#
mcfmasked=mt_merge(bobnnmed,mcfbw,maskbw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcffw,maskfw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcfcenter,maskcenter,U=2,V=2)


fieldf=mcfmasked.AssumeTFF().SeparateFields().SelectEvery(4,0,3)
#StackVertical(Separatefields(source),fieldf.MergeChroma(Separatefields(source)))
Weave(fieldf.MergeChroma(Separatefields(source)))
Any advice for improvement.
yup.

[Hagbard23]

Sorry for interfering here, but i would simply use the

DESCRATCH() (http://avisynth.org.ru/descratch/descratch.html)

Plugin and set it up for black lines which are horizontal (90 degrees). This seems like a perfect scenario for that plugin. I definitely would not use Despot nor mask operations (a little too much, for my taste)

Sorry - just my 2 cents...

[yup]

Hi Hagbard23!
I try descratch before without success.
See comparing
http://img814.imageshack.us/img814/9349/compare.th.png (http://img814.imageshack.us/i/compare.png/)

Uploaded with ImageShack.us (http://imageshack.us)
At top source, medium median motion compensated filtered, at bottom descratch output with suggested set for VHS.
yup.

[Hagbard23]

i still believe, a little tweaking should get you nicer results. Maybe flipping the image 90Degrees before feeding to descratch or something like that.

I KNOW that it works quite well, because i used it some couple of weeks ago with strongly scratched black/white material. The Results were very good.

sorry - i do not want to disturb here - do it like you want to do it... ;)

[StainlessS]

Have not really looked at your prob, but you might want to take a look here:-

http://forum.doom9.org/showthread.php?t=151426&highlight=depulse

About horizontal lines on analog captures & Depulse plugin to fix such.


EDIT:- Did not look at your prob as Depulse temporal in nature and only stills
supplied (I think). Also, In "New Plugins Sticky", "KillPulse" is mentioned and
seem to do the same thing, however could not locate a link that works for
this. There is also a plugin called "DepulseC" which I think I saw somewhere
does similar with coloured streaks. Link to that, below (which does work):

http://www5e.biglobe.ne.jp/~hoe/dv-and-movie/tipsxxx/index.html

[jmac698]

I have the same problem. Your script didn't work well for me, however clense with DePulse was great. Median of 5 captures was even better. By sync he means dropped frames in the captures. Virtualdub will drop frames by default without any indication. I developed Glitch Analyzer (see new plugins) to investigate this and find settings which work. There is also each line offset horizontally. I'm writing a plugin like 2Bedecided mentioned, except I think a direct minimum SAD is easier.
The problem isn't dirty heads. Tapes play fine in the same machine and also the tape itself only developed the problem after 15 years. The black lines are a different problem, and I can recreate that with test signals to see what causes them. Black to white transitions, for example. I've made a dramatic improvement already.

[yup]

jmac698
This script work fine for me. Need small tuning. Vertical median filter could remove horizontal lines. If line thicker need use more point for median filter (will introduce artifacts) if thiner less point. Now I think about building mask for bad part frame.
Also can use hierarchical approach with different vertical median filter.
I try like You median5 for my capture without success.
I have relatively new VCR/DVD writer 2006 year model.
It is very good that this problem interesting not only for me.
yup.

[jmac698]

yup,
Would you like to try your script on another example? Here is mine:
http://www.sendspace.com/file/l1500b

[yup]

jmac698
http://img513.imageshack.us/img513/5971/filt.th.png (http://img513.imageshack.us/i/filt.png/)
LoadPlugin("mt_masktools-25.dll")
LoadPlugin("RemoveGrainTSSE3.dll")
source=AVISource("stripes demo.avi")
source=source.AssumeTFF().ConvertToYV12(interlaced=true)
#med=source.mt_luts(source,mode="median",pixels="0 -4 0 -3 0 -2 0 -1 0 0 0 1 0 2 0 3 0 4",U=2,V=2)
med=source.mt_luts(source,mode="median",pixels="0 -3 0 -2 0 -1 0 0 0 1 0 2 0 3",U=2,V=2)
#med=source.mt_luts(source,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",U=2,V=2)
medf=med.dfttest(tbsize=1,ftype=1,sbsize=16,sosize=12,sigma=1000,U=false, V=false)
super=MSuper(source,chroma=false)
superf=MSuper(medf,chroma=false)
bv1 = MAnalyse(superf, blksize=16, isb = true, delta = 1, overlap=8, dct=5,chroma=false)
bv2 = MAnalyse(superf, blksize=16, isb = true, delta = 2, overlap=8, dct=5,chroma=false)
fv1 = MAnalyse(superf, blksize=16, isb = false,delta = 1, overlap=8, dct=5,chroma=false)
fv2 = MAnalyse(superf, blksize=16, isb = false,delta = 2, overlap=8, dct=5,chroma=false)
bc1 = MCompensate(source, super, bv1, thSAD=16000, thSCD1=16000)
bc2 = MCompensate(source, super, bv2, thSAD=16000, thSCD1=16000)
fc1 = MCompensate(source, super, fv1, thSAD=16000, thSCD1=16000)
fc2 = MCompensate(source, super, fv2, thSAD=16000, thSCD1=16000)
mb1sad=medf.MMask(bv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mb2sad=medf.MMask(bv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf1sad=medf.MMask(fv1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf2sad=medf.MMask(fv2,kind=1,ml=100,Ysc=255, thSCD1=16000)
centersad=mt_logic(mf1sad,mb1sad,"max",U=2,V=2)
bwsad=mt_logic(mb1sad,mb2sad,"max",U=2,V=2)
fwsad=mt_logic(mf1sad,mf2sad,"max",U=2,V=2)
mcfcenter=clense(bc1, source,fc1,increment=0, grey=true)
mcfbw=clense(bc1, source,bc2,increment=0, grey=true)
mcffw=clense(fc1, source,fc2,increment=0, grey=true)
# center filtered
#mt_lutxyz(centersad,bwsad,fwsad,"x y <= x z <= & x 255 < &",U=2,V=2) replace to lutxy
mt_logic(mt_logic(mt_lutxy(centersad,bwsad,"x y <= "), mt_lutxy(centersad,fwsad,"x y <="),"and"),mt_lut(centersad,"x 255 <="),"and")
maskcenter=mt_lutxy(last,centersad,"x 255 y - 0 ?",U=2,V=2)
# backward filtered
#mt_lutxyz(bwsad,centersad,fwsad,"x y < x z <= &",U=2,V=2) replace to lutxy
mt_logic(mt_lutxy(bwsad,centersad,"x y < "), mt_lutxy(bwsad,fwsad,"x y <="),"and")
maskbw=mt_lutxy(last,bwsad,"x 255 y - 0 ?",U=2,V=2)
# forward filtered
#mt_lutxyz(fwsad,centersad,bwsad,"x y < x z < &",U=2,V=2) replace to lutxy
mt_logic(mt_lutxy(fwsad,centersad,"x y < "), mt_lutxy(fwsad,bwsad,"x y < "),"and")
maskfw=mt_lutxy(last,fwsad,"x 255 y - 0 ?",U=2,V=2)
#
mcfmasked=mt_merge(med,mcfbw,maskbw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcffw,maskfw,U=2,V=2)
mcfmasked=mt_merge(mcfmasked,mcfcenter,maskcenter,U=2,V=2)

StackVertical(source,mcfmasked.MergeChroma(source))
I applied script for median5 filtered source.
Your source very bad quality (my also bad but better:))
yup.

[jmac698]

Ok, I try it now. It does remove a lot of noise. If you also enable my depulse depulse(80,79,0)
source=last it's even a little better. Even some black streaks are gone. However it is more blurry. I will have to try some more video. Does it help with rainbows that seem to appear at the start of a new recording?
thanks.

[yup]

@jmac698!
Last version script. Little comments.
SetMemoryMax(768)
SetMTMode(3,4)
source=DirectShowSource("lecture1.avi",pixel_type="YUY2")#.Trim(750,1250)
SetMTMode(2,4)
source=source.AssumeTFF().ConvertToYV12(interlaced=true)#.ColorYUV(autogain=true, autowhite=true)
bobnn=source.nnedi3(field=-2)
bobnnmed=bobnn.mt_luts(bobnn,mode="median",pixels=mt_rectangle(0,5),U=3,V=3)# rectangle 5 for 3 pixels thikness line, 3 for 2, 1 for 1
bobnnmedf=bobnnmed.dfttest(tbsize=1,ftype=1,sbsize=8,sosize=6,sigma=1000,U=false,V=false)
bobnnf=bobnn#.dfttest(tbsize=1,ftype=1,sbsize=16,sosize=12,sigma=1000,U=false,V=false)
super=MSuper(bobnn)
superf=MSuper(bobnnmedf,chroma=false)
superff=MSuper(bobnnf)
bw1 = MAnalyse(superf, blksize=8, isb = true, delta = 2, overlap=4, dct=5,chroma=false)
bw2 = MAnalyse(superf, blksize=8, isb = true, delta = 4, overlap=4, dct=5,chroma=false)
fw1 = MAnalyse(superf, blksize=8, isb = false,delta = 2, overlap=4, dct=5,chroma=false)
fw2 = MAnalyse(superf, blksize=8, isb = false,delta = 4, overlap=4, dct=5,chroma=false)
bc1 = MCompensate(bobnnf, super, bw1, thSAD=16000, thSCD1=16000)
bc2 = MCompensate(bobnnf, super, bw2, thSAD=16000, thSCD1=16000)
fc1 = MCompensate(bobnnf, super, fw1, thSAD=16000, thSCD1=16000)
fc2 = MCompensate(bobnnf, super, fw2, thSAD=16000, thSCD1=16000)
bwabs1=mt_lutxy(bobnnf,bc1,"x y - abs")#,U=-128,V=-128)
bwabs2=mt_lutxy(bobnnf,bc2,"x y - abs")#,U=-128,V=-128)
fwabs1=mt_lutxy(bobnnf,fc1,"x y - abs")#,U=-128,V=-128)
fwabs2=mt_lutxy(bobnnf,fc2,"x y - abs")#,U=-128,V=-128)
SDIc=SDIAdapt(bwabs1,fwabs1)
SDIb=SDIAdapt(bwabs1,bwabs2)
SDIf=SDIAdapt(fwabs1,fwabs2)
#
mb1sad=bobnnmedf.MMask(bw1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mb2sad=bobnnmedf.MMask(bw2,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf1sad=bobnnmedf.MMask(fw1,kind=1,ml=100,Ysc=255, thSCD1=16000)
mf2sad=bobnnmedf.MMask(fw2,kind=1,ml=100,Ysc=255, thSCD1=16000)
centersad=mt_logic(mf1sad,mb1sad,"max")#,U=2,V=2)
bwsad=mt_logic(mb1sad,mb2sad,"max")#,U=2,V=2)
fwsad=mt_logic(mf1sad,mf2sad,"max")#,U=2,V=2)
centermask=mt_invert(centersad)
bwmask=mt_invert(bwsad)
fwmask=mt_invert(fwsad)
mcfcenter=clense(bc1, bobnn,fc1,increment=0, grey=false)
mcfbw=clense(bc1, bobnn,bc2,increment=0, grey=false)
mcffw=clense(fc1, bobnn,fc2,increment=0, grey=false)
#
# sort SAD
# center filtered
mt_logic(mt_logic(mt_lutxy(centersad,bwsad,"x y <= "), mt_lutxy(centersad,fwsad,"x y <="),"and"),mt_lut(centersad,"x 255 <="),"and")
maskcentersad=mt_lutxy(last,centersad,"x 255 y - 0 ?",U=3,V=3)
# backward filtered
mt_logic(mt_lutxy(bwsad,centersad,"x y < "), mt_lutxy(bwsad,fwsad,"x y <="),"and")
maskbwsad=mt_lutxy(last,bwsad,"x 255 y - 0 ?",U=3,V=3)
# forward filtered
mt_logic(mt_lutxy(fwsad,centersad,"x y < "), mt_lutxy(fwsad,bwsad,"x y < "),"and")
maskfwsad=mt_lutxy(last,fwsad,"x 255 y - 0 ?",U=3,V=3)
#
mcfmaskedsad=mt_merge(bobnnmed,mcfbw,maskbwsad,luma=true)
mcfmaskedsad=mt_merge(mcfmaskedsad,mcffw,maskfwsad,luma=true)
mcfcentersad=mt_merge(mcfmaskedsad,mcfcenter,maskcentersad,luma=true)
mcfbwsad=mt_merge(bobnnmed,mcfbw,maskbwsad,luma=true)
mcffwsad=mt_merge(bobnnmed,mcffw,maskfwsad,luma=true)
#end sort SAD
#

#
# sort SDI
# center SDI
maskcentersdi=mt_logic(mt_logic(mt_lutxy(SDIc,SDIb,"x y >= "), mt_lutxy(SDIc,SDIf,"x y >="),"and"),SDIc,"and").mt_lut("x 255 0 ?")
# backward SDI
maskbwsdi=mt_logic(mt_lutxy(SDIb,SDIc,"x y > "), mt_lutxy(SDIb,SDIf,"x y >="),"and").mt_lut("x 255 0 ?")
# forward SDI
maskfwsdi=mt_logic(mt_lutxy(SDIf,SDIc,"x y > "), mt_lutxy(SDIf,SDIb,"x y >"),"and").mt_lut("x 255 0 ?")
#
maskedsdi=mt_merge(bobnn,mcfcentersad,maskcentersdi,luma=true)
maskedsdi=mt_merge(maskedsdi,mcffwsad,maskfwsdi,luma=true)
maskedsdi=mt_merge(maskedsdi,mcfbwsad,maskbwsdi,luma=true)
#end sort SDI
#


fieldmaskedsdi=maskedsdi.AssumeTFF().SeparateFields().SelectEvery(4,0,3)
#fieldmaskedsdi=SDIc.AssumeTFF().SeparateFields().SelectEvery(4,0,3)
StackVertical(Separatefields(source),fieldmaskedsdi)
#StackVertical(Separatefields(source),mt_makediff(fieldmaskedsdi,Separatefields(source)))
#Weave(fieldmaskedsdi)


# Spike Detection Function
#

function SDIAdapt(clip b1,clip b2)
{
threshsp=10
THP=string(threshsp)
threshavg=2
THAVG=string(threshavg)
SDI=mt_lutxy(b1,b2,"x "+THP+" > y "+THP+" > | 1 x y - x y + / abs - 255 * 0 ? ")
SDIavg=SDI.mt_luts(SDI,mode="avg",pixels=mt_square(5))
SDIAvgblk=SDIavg.PointResize(90,72)
SDIavgblkX=SDIAvgblk.mt_luts(SDIAvgblk,mode="median",pixels="1 -1 0 0 -1 1 1 1 -1 -1")
SDIavgblkCross=SDIAvgblk.mt_luts(SDIAvgblk,mode="median",pixels="1 0 0 0 -1 0 0 1 0 -1")
SDIAvgblk=clense(SDIAvgblk,SDIavgblkCross,SDIavgblkX,increment=0, grey=true).mt_lut("x")
SDIavg=SDIAvgblk.PointResize(720,576)
SDIad=mt_lutxy(SDI,SDIavg,"x "+THAVG+" y * > x 128 > & 255 0 ?")
SDImask=SDIad.mt_luts(SDIad,mode="median",pixels=mt_rectangle(0,4))# rectangle 4 for 3 pixels thikness line, 2 for 2, not need for 1
SDImask=SDIad.mt_luts(SDIad,mode="median",pixels=mt_rectangle(4,0))# remove line shorter 5 pixels
SDImaskleft=SDImask.mt_luts(SDIad,mode="max",pixels="0 0 -1 0 -2 0 -3 0 -4 0 -5 0 -6 0 -7 0 -8 0 -9 0 -10 0 -11 0")# try
SDImaskright=SDImask.mt_luts(SDIad,mode="max",pixels="0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0") # join
SDIMask=mt_logic(SDImaskleft,SDImaskright,"min") # sequental horizontal line
SDImask=SDImask.mt_luts(SDImask,mode="min",pixels=mt_rectangle(18,0)) # remove line shorter 37 pixels
SDImask=SDImask.HorlLinExp() # fill line with noise
SDIad=mt_logic(SDIad,SDImask,"min") # refine source SDI
SDIad=SDIad.mt_luts(SDIad,mode="median",pixels=mt_rectangle(0,4)) # rectangle 4 for 3 pixels thikness line, 2 for 2, not need for 1
return(SDIad)

}

# fill full horizontal line if exist one pixel per line

function HorlLinExp(clip mask1) {

mask2 = mask1.mt_lut(Y=-255, u=-128, v=-128)
black = mask1.mt_lut(Y=0, u=-128, v=-128)

v1=interleave(mask1,black).assumefieldbased.assumetff.weave
v2=interleave(mask2,black).assumefieldbased.assumetff.weave

mt_hysteresis(v1,v2, u=-128, v=-128)
separatefields.selecteven.assumeframebased
}

Now my main computer busy filtering multipass this script wit avs2avi. 2 pass wit 5 median vertical filter and 2 pass with 3., total time near 160 hour for 68 min clip. Screenshot will be at friday. Also I think about repairing using filtered and source clip.
Any advice welcome.
yup.

[yup]

Please!
Screenshot.
http://thumbnails36.imagebam.com/15398/af6df3153976503.jpg (http://www.imagebam.com/image/af6df3153976503) http://thumbnails34.imagebam.com/15398/1c91b0153976540.jpg (http://www.imagebam.com/image/1c91b0153976540)
Pictures from 4 and 5 sequential fields with horizontal lines.
yup.

[hansima99]

hi yup!

I tried tu use your script but I always get the error

"Script error: Invalid arguments to function "clense""

line 97 and line 23

I tried several filters but nothing worked. which versions of plugins are you useing

thats my project, try to get rid of the black stripes on the left

http://i52.tinypic.com/wkpser.jpg

[yup]

hansima99!
welcome to forum!
Need Kassandro plugin
http://www.removegrain.de.tf/
yup.

[Mounir]

Try THIS (http://avisynth.org/mediawiki/DeVCR) hansima99

[fonzzie31]

Hello,

I got some black horizontal stripes/lines on Laserdiscs digitizations I made.
The video format is : 720x576 PAL 25fps or 720x480 NTSC 29.97 fps - Enterlaced - YUV 10 Bits 4:2:2 Uncompressed.
The laserdisc players I used : PIONEER CLD-D925 PAL/NTSC and PIONEER Elite CLD-97 NTSC.

Here are some captures of the black stripes/lines I would like to remove :

http://revival03.free.fr/temp/blines1.jpg

http://revival03.free.fr/temp/blines2.jpg

http://revival03.free.fr/temp/blines3.jpg

These black strips/lines can be visible between half a frame to one complete frame.

I tried to use the Yup script, but I was unable to run it. I got an error message on very small red letters and It was impossible to read it. I'm not an Avisynth expert.

So I would like to know if the Yup script can help to remove the black stripes/lines, and how to use it without any problem.

I would like to obtain an enterlaced video, because the video source is enterlaced too.

Thanks beforehand for your help.

[yup]

fonzzie31!
Welcome to forum.
You need some plugin
masktools
http://forum.doom9.org/showthread.php?t=98985
MVTools2
http://www.avisynth.org.ru/mvtools/mvtools2.html
Kassandro plugin Removegrain RemovegrainT
only AVSRecursion (need install to Windows\system32 directory)
http://videoprocessing.11.forumer.com/
Tritical nnedi3
http://web.missouri.edu/~kes25c/
All plugin place to plugin directory Avisynth (for example C:\Program Files\Avisynth\plugin)
Try this. Do nor worry speak and we try answer.
Load avs script to VirtualDub
http://www.virtualdub.org/
and see error message.
yup.