MatchClips 0.2 a filter to align clips in time

[tsp]

From the readme. Hope it makes sense else just try it.

Get version 0.2 here (http://www.avisynth.org/tsp/matchclips02.zip)
Get version 0.1 here (http://www.avisynth.org/tsp/matchclips01.zip)


MatchClips version 0.2

By Tonny Petersen (tsp@person.dk)

Matchclip is a filter that will try to align two similar clips in time. This can be useful if you want to restore
some old vhs tapes where the noise will appear different places from each capture because the tracking will be
different. The different captures of the same part of the tape will have dropped frames so this filter will try
to align two captures when the dropped frames are at different positions.

Usage:

MCAnalyse(clip src1,clip src2,int framestoseekafter,int framestoseekbefore,int framestolock,bool searchfromlock)

MCAnalyse takes two clips and find the frame offset between them. It returns a FrameInfo Clip that is to be used with the following filters

src1,src2: This is the two clip to align. They should be the same colorspace and src1's height and width should be less than or equal to src2's.
Note that if the colorspace is not YV12 both the luma and chroma plane will be processed and this will be slower. If YV12 then only the luma
plane will be processed. Remember to align the two clips at the first frame.

framestoseekafter, framestoseekbefore: This two parameters defines the searchrange. The filter will search from current frame - framestoseekbefore
to current frame + framestoseekafter in src2 and compare each frame to current frame in src1. The offset is then the distance between the current
frame and the frame that is most similar to src1.
You will have to set these two parameters bigger than the largest block of dropped frames.
Default: 20

framestolock: This is the number of frames with the same offset that is needed to lock the offset.
Default: 5

searchfromlock: If set to true the locked offset is added to current frame when searching. This enable the offset between src1 and src2 to be greater
than framestoseekbefore and framestoseekafter.
Default: true


MCShowFrame(clip src1,clip src2,clip FrameInfo,bool uselock,int threshold)

This filter will return the frame from src2 that matches the frame from src1


src1,src2: Same as above. The sound will come from src1 and the frame from src2.

FrameInfo: This is the clip from MCAnalyse that contains the offset information.

uselock: If set to true the frame returned will use the locked offset instead of the offset that gives the best match. This is good if there
is no good match i.e. if the current frame in src1 is very noise and the matching frame in src2 is not.
Default: true

threshold: if the average difference between the best match between src 2 and src1 are greater than threshold, the frame from src1 are returned else src2 are returned.
if a negativ number are provided a frame from src1 are returned if the average difference is smaller than the absolute value of threshold.
if zero the frame are always returned from src2 (the default behavior). Valid values are between [-255;255]


MCDebug(clip src,clip FrameInfo)

This filter will return information about the Locked Offset, Offset and if the offset is locked and the average difference between the best match and the source.

src: clip to use as background.

FrameInfo: This is the clip from MCAnalyse that contains the offset information.



MCIsLocked(clip FrameInfo)
MCLockedOffset(clip FrameInfo)
MCOffset(clip FrameInfo)
MCSAD(clip FrameInfo)

These 4 filters must be used inside conditional filter, scriptclip or Frame evaluate.
MCIsLocked returns true if the current frame offset is set to lock false if not.
MCLockedOffset returns the locked offset.
MCOffset returns the current offset.
MCSAD return the average difference between the best match and the source.


Example on usage.

source1=AVISource("c:\testclip1.avi").converttoyv12
source2=AVISource("c:\testclip2.avi").converttoyv12
c=mcanalyse(source1,source2,20,20)
mcshowframe(source1,source2,c)


or this complicated script that returns the frame from src2 if src1 is corrupted by noise. Usefull for vhs captures (this was the reason I created this filter.):

source1=AVISource("c:\testclip1.avi").converttoyv12
source2=AVISource("c:\testclip2.avi").converttoyv12
c=mcanalyse(source1,source2,20,20)
global d=mcshowframe(source1,source2,c)

global Fmask = DEdgeMask(source1,thY1=79,thY2=79,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
global Pmask = source.crop(source1.width/2-12,0,24,source1.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
source1.scriptclip("(AverageLuma(Pmask)>0.001&& AverageLuma(Fmask)>0.05 && (AverageLuma(Fmask.trim(current_frame-1,-1))>0.05||UDifferenceFromPrevious()>2))?last:d")


TODO: make this doc more readable :-)

[tsp]

found a better script to select the frame with the least severe noise:


source1=AVISource("c:\testclip1.avi").converttoyv12
source2=AVISource("c:\testclip2.avi").converttoyv12
c=mcanalyse(source1,source2,20,20)
global d=mcshowframe(source1,source2,c)

global Pmask1 = source1.crop(source1.width/2-12,0,24,source1.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
global Pmask2 = source2.crop(source2.width/2-12,0,24,source2.height).DEdgeMask(thY1=60,thY2=60,thC1=140,thC2=140,matrix="-2 1 -2 0 6 0 -2 1 -2" )
source1.scriptclip("AverageLuma(Pmask1)>0.1&&(AverageLuma(Pmask1)>AverageLuma(Pmask2))?d:last")

[DeathWolf]

i'm having trouble using the filter, it's obviously not working well on scene changes... i often get offsets of 1 frame...(ie scene change in clip1 happends 1 frame earlier/later than in the "matched" clip2)
Note that i might have frames that need to be deleted... would it be possible to get a relative lock?(like for each frame it searchs in [lastgoodframenumber-framestoseekbefore, lastgoodframenumber+framestoseekafter])
That way it'll always get the closest frame even after dupes.

Edit:
here's the way i used it:
test1 = Avisource("source1.avi",audio=false).subtitle("source1")
test2 = Avisource("source2.avi",audio=false).subtitle("source2")
ca1 = mcanalyse(test1,test2,6,6)
da1 = mcshowframe(test1,test2,ca1)
return stackhorizontal(da1,test1)

[Mug Funky]

hey, cool idea! thanks.

[tsp]

DeathWolf: Currently the default behavevior is to use a relative offset(it search from [lastlockframenumber-framestoseekbefore, lastlockframenumber+framestoseekafter] (the mcanalyse paramter searchfromlock controls this. The default value is true so a relative offset is used). To get a lock mcanalyse needs 5 frames in a row with the same offset(you can change this by using the parameter framestolock.) you can disable the lock by setting framestolock=1 in mcanalysis or uselock=false in mcshowframe. I added the lock option to replace noisy captured frames(that you get if you capture from a severely damaged vhs tape) in src1 with good frames in src2. If you only have dropped/added frames you don't need the lock so set framestolock=1 so your script look like this:

test1 = Avisource("source1.avi",audio=false).subtitle("source1")
test2 = Avisource("source2.avi",audio=false).subtitle("source2")
ca1 = mcanalyse(test1,test2,6,6,1)
da1 = mcshowframe(test1,test2,ca1)
return stackhorizontal(da1,test1)

[DeathWolf]

thanks, uselock=false indeed improved a lot things... now a thing that would be pretty useful would be a switch like copyiftoobad=threshold which would copy a frame from the source to match in case the match isnt good enough. Would that be possible?

[tsp]

Sure shouldn't be to great a problem.

[tsp]

DeathWolf: Threshold added to mcshowframe. If the average pixel difference between the best match and source frame are larger(or smaller than the absolute value of threshold if a negative number are used) than threshold the source frame are returned.