Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th January 2007, 06:02   #1  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Script for removal horizontal stripes (alpha release)

Hi folk!
Alpha release !
For script need the same plugin like for QTGMC
http://forum.doom9.org/showthread.php?t=156028
and AVSRecursion, RemoveGrainT from kassandro
http://home.arcor.de/kassandro/AvsRe...sRecursion.htm
http://home.arcor.de/kassandro/prere...GrainT-1.0.rar
Using
Code:
AVISource("selnew.avi")
removeline(thicknessline=2, distser=1, halflength=3,comparing=false)
Script tested with Set last build Avisynth 2.6.
short parameters description
work with interlaced source (for film need some small adaptation)
thicknessline thickness horizontal spike in pixels at field scale (not frame) 2 good start for many VHS capture (if choose thicker than at source scrit do not work)
distser distance between short lines, if source have single spike (not series) no sense, but for speed use distser=1
halflength detect line longer 2*halflength+1 pixels, if source have spike 7 pixels (not longer) could be equal 3, if set higher than at source script do not work
comparing if true ouput will be stacked fields source, filtered and diff for tuning parameters
For some captures need sequintial using script, always need starting with biger thicknessline (first pass with thicknessline=2 second thicknessline=1, for very bad may be need three pass and starting from thicknessline=3).
Script can remove two sequintial spikes at the same place frame, if we have three sequintial spikes need use script 2 times with same thicknessline value.
Work very slowly 1.5-2 fps on i7 Core.
Any suggestion for optimization script welcome.

yup.
Attached Files
File Type: 7z removeline.7z (2.9 KB, 171 views)

Last edited by yup; 16th August 2012 at 08:41.
yup is offline   Reply With Quote
Old 24th January 2007, 06:08   #2  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Post a screenshot showing the problem. Sounds like motor noise interference or something like that.

(With 165 posts I suppose you know about interlacing.)
Guest is offline   Reply With Quote
Old 24th January 2007, 06:13   #3  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
top field from source
http://img443.imageshack.us/img443/4222/stripes9vt.png
yup.
yup is offline   Reply With Quote
Old 24th January 2007, 06:22   #4  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Yup, looks like analog shot noise is affecting your capture. Check all your cabling and grounds. Make sure you have no big motors, generators, refrigerators, etc., running nearby or on the same circuit. No car engines running nearby. That sort of thing. This is really hard to remove with filters.
Guest is offline   Reply With Quote
Old 24th January 2007, 07:11   #5  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
neuron2!
Thank you for reply. For power supply I use online UPS with isolated transformer (for PC and VCR). Early I made one capture without this stripes. I think this coupled with tape. I try capture 2 times and get stripes at same places.
yup.
yup is offline   Reply With Quote
Old 24th January 2007, 10:11   #6  |  Link
DeathWolf
Registered User
 
DeathWolf's Avatar
 
Join Date: Mar 2002
Location: France
Posts: 85
yes, it sounds like tape reading heads arent able to read these parts, i had the exact same problem when digitizing tapes. Unfortunatelly i couldnt find any decent way of removing them without hurting the rest of the video. And apparently the only solution is to get a better tape reader which can come very expensive for some formats(even though they are very old...)
DeathWolf is offline   Reply With Quote
Old 24th January 2007, 11:27   #7  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
DeathWolf!
I use Sony 4-head VCR. VHS cassete made during edit from one VCR to other.
yup.
yup is offline   Reply With Quote
Old 29th January 2007, 10:49   #8  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
MVTools and DeGrainMedian help me

Hi folk!
After many try and error i find script which remove stripes
Code:
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fco = o.MVCompensate(fvo, idx=1, thSCD1=500)
bco = o.MVFlow(bvo, idx=1, thSCD1=500) MVCompensate interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fce = e.MVFlow(fve, idx=1, thSCD1=500)
bce = e.MVFlow(bve, idx=1, thSCD1=500) 
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)
Degrainmedian need use in pure median mode. Now I try find other median filter for time plane and postfilter.
yup.
yup is offline   Reply With Quote
Old 29th January 2007, 11:17   #9  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Quote:
Originally Posted by yup View Post
Hi folk!
After many try and error i find script which remove stripes
Code:
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fco = o.MVCompensate(fvo, idx=1, thSCD1=500)
bco = o.MVFlow(bvo, idx=1, thSCD1=500) MVCompensate interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=0)
fce = e.MVFlow(fve, idx=1, thSCD1=500)
bce = e.MVFlow(bve, idx=1, thSCD1=500) 
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)
From the idea, that should work. But the script as you posted is buggy.

Blue seems fishy: one time flow, one time compensate?

Green will cause some syntax error.

Red is a big no-no: you must not use the same idx value for even and odd fields. These have to be different. (At least with pel=2, which is default.)

Apart from that, the script is fine.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 29th January 2007 at 11:55.
Didée is offline   Reply With Quote
Old 29th January 2007, 12:27   #10  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
A small demo of wrong idx usage:

Correct idx usage:


Wrong idx usage:



Code:
v  = BlankClip(width=304,height=224, pixel_type="yuy2")
v1 = v.Subtitle("this one (v1)",y=32).AddBorders(8,8,8,8,color=$FF0000)
v2 = v.Subtitle("that one (v2)",y=64).AddBorders(8,8,8,8,color=$00FF00)

bw1 = v1.MVAnalyse(isb=true,  idx=1)
fw1 = v1.MVAnalyse(isb=false, idx=1)
comp1bw = v1.MVCompensate(bw1,idx=1)
comp1fw = v1.MVCompensate(fw1,idx=1)
x1 = Interleave(comp1fw,v1,comp1bw).TemporalSoften(1,255,255,255,2).SelectEvery(3,1)

bw2 = v2.MVAnalyse(isb=true,  idx=1)  #  idx = 1 is WRONG! (just for demonstration!)
fw2 = v2.MVAnalyse(isb=false, idx=1)  #  --""--
comp2bw = v2.MVCompensate(bw2,idx=1)  #  --""--
comp2fw = v2.MVCompensate(fw2,idx=1)  #  --""--
x2 = Interleave(comp2fw,v2,comp2bw).TemporalSoften(1,255,255,255,2).SelectEvery(3,1)

#StackHorizontal(v1,v2)
StackHorizontal(x1,x2)
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 29th January 2007, 12:40   #11  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Didée!
Thanks for reply.
I make cut and paste from script with Russian comments.
Also please advise postfilter more suitable for pure median prefiltering.
yup.
yup is offline   Reply With Quote
Old 29th January 2007, 12:57   #12  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
What kind of postfilter do you mean now?

Your script is basically fine, you just have to use idx=2 in the 2nd part of your script.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 29th January 2007, 14:23   #13  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Didée!
Now i do not use postfilter I only search him
yup.
yup is offline   Reply With Quote
Old 5th February 2007, 08:27   #14  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi folk!
I remove error from script.
Code:
AviSource("sel.avi",pixel_type="YUY2")
AssumeTFF()
Crop(16,16,-16,-16)
SeparateFields()
e=SelectEven()
o=selectOdd()

bvo = o.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1)
fvo = o.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1)
fco = o.MVFlow(fvo, idx=1, thSCD1=500) 
bco = o.MVFlow(bvo, idx=1, thSCD1=500) 
interleave(fco, o, bco)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
#DeGrainMedian(limitY=255,limitUV=255,mode=1) 
od=selectevery(3,1)

bve = e.MVAnalyse(isb = true, truemotion=true, delta = 1, idx = 2, overlap=4, dct=1)
fve = e.MVAnalyse(isb = false, truemotion=true, delta = 1, idx = 2, overlap=4, dct=1)
fce = e.MVFlow(fve, idx=2, thSCD1=500)
bce = e.MVFlow(bve, idx=2, thSCD1=500) 
interleave(fce, e, bce)
DeGrainMedian(limitY=255,limitUV=255,mode=1, norow=true) 
#DeGrainMedian(limitY=255,limitUV=255,mode=1) 
ed=selectevery(3,1)

interleave(ed,od)
Weave()
AddBorders(16,16,16,16)
I totaly remove all black stripe only after 3 pass through this script my clip and clip little blured. I try use compensation from -2 frame and +2 frame positoion (DegrainMedian work on 5 frames), but result get worst. If somebode have any idea please advise.
With kind regards yup.
yup is offline   Reply With Quote
Old 2nd May 2007, 11:00   #15  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi folk!
After more try and error. tsp, Fizick, Manao, Mug Funky and other i find better way for remove black stripes.
First I use MVTools for prefiltering:
Code:
source=AVISource("clip.avi")
fields=source.AssumeBFF().SeparateFields() 
backward_vec = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
forward_vec = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
fields.MVDegrain1(backward_vec,forward_vec,thSAD=400,idx=1)
Weave()
I use dct=1 for better accuracy motion estimation as recomended Fizick, and save this file in VirtualDub as degr1.avi. After I use ml3dex function from MedianBlur plugin, for estimation motion vector I use prefiltered source and for filtering use source.
Code:
AviSource("clip.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
SeparateFields()
source=last
AviSource("degr1.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
SeparateFields()
filtered=last
bv1 = filtered.MVAnalyse(blksize=8, isb = true, truemotion=true, delta=2, overlap=4,idx = 1,dct=1)
fv1 = filtered.MVAnalyse(blksize=8, isb = false, truemotion=true, delta=2, overlap=4,idx = 1,dct=1)
bc1 =source.MVFlow(bv1, idx=2, thSCD1=1000) 
fc1 = source.MVFlow(fv1, idx=2, thSCD1=1000) 
interleave(bc1, source, fc1)
ml3dex(mc=false,Y=3, U=3,V=3)
SelectEvery(3,1)
Weave()
Source clip DV YUY2, but ml3dex work only with YV12. For many parts my source enough one pass, but for worst parts I use 2 pass (for second pass I prefiltered pass1 use MVTools). Before encoding, after ml3dex need use MVDegrain1 for remove grain which bring ml3dex. Also I find that use 2 pass 1 step motion estimation (prefiltering MVDegrain1 and ml3dex) work better than use MVDEgrain2 for prefiltering and 2 call ml3dex in script.
See one field for comparing (top filtered, bottom source):
http://www.sendspace.com/file/ig8pgo
If any have idea for improvement please advice.
With kind regards yup.
yup is offline   Reply With Quote
Old 3rd May 2007, 06:08   #16  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
you get blurring on dress.
Do you try Despot plugin in seg=1 mode or median mode?
with width=big and height=1?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 3rd May 2007, 09:18   #17  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Fizick! Thank You for reply.
Please precise parameter set.
I try for spot removal mode i use seg=1, pheight=1, pwidth=30 (maximum length black stripe)
for median mode
median=true, mheight=1, mwidth=30 instead ml3dex in my script. Picture less blured than ml3dex but many stripe do not removed. Please advise way for tuning other parameters.

With kind regards yup.
yup is offline   Reply With Quote
Old 3rd May 2007, 19:40   #18  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
sign = 2
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 22nd May 2007, 16:56   #19  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi folk!
Fizick I try find magic parameter set for Despot but without success.
I little modified supposed by gzarkadas script for creation spike detection index using motion compensated frames, see below
Code:
LoadPlugin("mt_masktools.dll")  # v2.0a30
SetMemoryMax(256)
AVISource("seldv2.avi")
AssumeBFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
backward_vec = fields.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
forward_vec = fields.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=2, idx = 1,dct=1)
fc = fields.MVFlow(forward_vec, idx=1, thSCD1=10000)
bc = fields.MVFlow(backward_vec, idx=1, thSCD1=10000) 
masksdi=SDIPixelMask(bc, fields, fc, 16, 0.8)  # define your thresholds here
StackVertical(fields,masksdi)
#Weave()


# this is for Abs(p - f) or Abs(p - b)
Function absdiff(clip c1, clip c2) { return mt_lutxy(c1, c2, "x y - abs") }

# this build the expression for the SDI function
Function SDI_RPN(string expr_d1, string expr_d2, string expr_t1, string expr_t2) {
    _d1 = expr_d1 + " "
    _d2 = expr_d2 + " "
    _t1 = expr_t1 + " "
    _t2 = expr_t2 + " "
    return _d1 + _t1 + "> " + _d2 + _t1 + "> | 1.0 " \
        + _d1 + _d2 + "- " + _d1 + _d2 + "+ / abs - 0.0 ? " \
        + _t2 + "> 1.0 " + _d1 + _t1 + "> " + _d2 + _t1 + "> | 1.0 " \
        + _d1 + _d2 + "- " + _d1 + _d2 + "+ / abs - 0.0 ? "
}

# this creates the mask
Function SDIPixelMask(clip prev, clip curr, clip next, float t1, float t2) {

    c_d1 = absdiff(curr, next)
    c_d2 = absdiff(curr, prev)
    
    # since SDI returns 0..1 we multiply with 255
    
    mask = mt_lutxy(c_d1, c_d2, SDI_RPN("x", "y", String(t1), String(t2)) + " 255 *")
    return mask
}
see image

This script successfully detecting black and white lines, but also find place where exist occlusion during motion estimation. My idea very simple find occlusion mask use MVMask with kind=2 (advice how use ml parameter), for both direction, forward and backward, binarize both mask to 0 and 255 level, use logical AND for find mask occlusion in both directions, because spike detection index do not sensitive for occlusion in one direction and last use logical XOR for remove from SDI occlusion mask. Unfortunately I not familiarly with mt_masktools. Please advice.
With kind regards yup.
yup is offline   Reply With Quote
Old 2nd October 2007, 20:57   #20  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi!
After small break I back to this problem. I upgrade hardware for capture (Panasonic DMR-ES35V DVD recorder with VCR, time base corrector Cypress CTB-100, Ixos S-Video cables), rewrite script. Script based on ideas which i find on this forum.
Code:
LoadPlugin("mt_masktools.dll")
AVISource("leb1.avi")
AssumeTFF()
ConvertToYV12(interlaced=true)
fields=SeparateFields()
snf=bob(fields,0,0.5)
f=fields.mt_luts(fields,mode="median",pixels="0 -1 0 0 0 1",U=1,V=1)
source=bob(f,0,0.5)
bv1 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
bv2 = source.MVAnalyse(blksize=8, isb = true, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
fv1 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 1, idx = 1, overlap=4, dct=1,chroma=false)
fv2 = source.MVAnalyse(blksize=8, isb = false, truemotion=true, delta = 2, idx = 1, overlap=4, dct=1,chroma=false)
#bc1 =snf.MVFlow(bv1, idx=2, thSCD1=1000) 
#fc1 = snf.MVFlow(fv1, idx=2, thSCD1=1000) 
bc1 =snf.MVCompensate(bv1, idx=2, mode=1) 
bc2 =snf.MVCompensate(bv2, idx=2, mode=1) 
fc1 = snf.MVCompensate(fv1, idx=2, mode=1) 
fc2 = snf.MVCompensate(fv2, idx=2, mode=1) 
interleave(bc2, bc1, source, fc1, fc2)
medianblurt(radiusy=0,radiusu=0,radiusv=0,temporalradius=2)
SelectEvery(5,2)
SeparateFields()
SelectEvery(4,0,3)
StackVertical(last,fields)
source and processing image

Quantity black line decrease, horizontal stability increase, that help motion compensated filtering. Small problems I decrease color space from YUY2 to YV12 and after need increase sharpness. May be I need postprocessing after median filtering before sharpening? Please advice.
With kind regards yup.
yup is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.