How to synchronize two videos by comparing video frames

[SilSic]

Hi all,

I need a script which makes it possible to synchronize(video/audio) two videos by comparing video frames which includes cutting the videos to be absolutely identical.
Especially for animes which differs e.g. by eyecatcher commercial break, black transits.

I've been looking for that since so many years but always ended up with no luck which really surprises me, because there are so many people who really need that! Right now search and cut scenes is all I can do but it is a pain in the neck(time-consuming!!!) which I'll never get used to and we are talking about 200 episodes and many others. There is absolutely no software which does that whole process what I'm looking for! But there a few which does have the function like autopano video(Automatic motion-based synchronization of the video frames) but without cut function which means it is absolutely useless. But avisynth seem to be promising.

No concessions, it must be be comparing pictures, to compare two videos by audio doesn't always work out for me.
Even though it is better than nothing.

Help is greatly appreciated!
SilSic

[johnmeyer]

You can use StainlessS' scene cutting software to cut into scenes, and then align along scene boundaries.

You say you can't use audio alignment, but you don't really say why that is so. I own and use a program called PluralEyes which lets me align video from multiple cameras which are used to simultaneously film an event, like a ballet performance or sporting event. It aligns using the audio track, and it works amazingly well, and lets me get all the cameras aligned on the timeline in my NLE, with one push of a button. It doesn't get much better than that.

I know dozens of people who use this software and they all love it.

[SilSic]

Hey thx for your quick reply.
I don't know how StainlessS works but it seems to be also a headache due of making a lot by hand. The whole process should work automatically of course or there is no sense at all in this!

How should audio alignment work, if the first video is only in english and the second only in japanese??

I tried really anything and I also tried PluralEyes but it doesn't cut the videos to ensure synchronizing for the rest of the video parts and it's only for audio alignment.

I thought about this: MSU Scene Change Detector analyses the video and the differences between the two videos can be determined, another plugin could cut the spots marked as different and another plugin synchronises by knowing these different spots. One video stays always untouched and is used as reference.
I also thought about commercial detector but it can't differ main video from eyecatcher in animes because they're mostly the same as the main video plus the eyecatchers changes every 26 Episodes on and on.

[StainlessS]

Can you specify Exactly what is required (anything and everything known about both clips).
Does only one clip have adverts, or both ?, if so, are they in exact same position (although maybe not same length ads in both clips).
Are dimensions, FrameRate etc exactly the same ?
What is an EyeCatcher ? (some kind of advert for another similar film ?)

some links
http://forum.doom9.org/showthread.php?p=1490778&highlight=cut+scene#post1490778
http://forum.doom9.org/showthread.php?p=1186331&highlight=cut+scene#post1186331
http://forum.doom9.org/showthread.php?p=1669306&highlight=findcuts#post1669306
http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566

Using last link for inspiration:-
You could write two files of ranges, one for each clip, for use by SelectFrames.avs (which uses Prune)
to cut and splice both clips separately (including audio).
(I would prefer using GScript for that rather than ScriptClip)

Also Matchframes/LocateFrames might be of use.

[creaothceann]

http://tvtropes.org/pmwiki/pmwiki.php/Main/EyeCatch
https://www.youtube.com/watch?v=RA6lH7xVGM0&t=0m7s

[StainlessS]

Thanx for that creaothceann, think I've seen the "We'll be right back after these messages" type thing on US shows transmitted in the UK.
(They might do similar here on Quiz or Shopping type Channels, but I would not know about that).

[SilSic]

in that case both clips doesn't have adverts, but dimensions are different, framerate is the same.

The script should work even when one of the both videos differs in framerate, dimension and so on. No matter what it is supposed to synchronize any kind of two videos.

thx creaothceann for the explanation of eyecatch.
And big thx to you StainlessS I never seemed to be so close to a solution :)

I tried findcuts but the giving output of timecodes are wrong. One exception is the starting frame(black frame) of the eyecatch
which is given in the timecode but at this rate how should synchronisation be done by cutting still missing the other timecodes? I switched video input for Full and Cut, the given timecode is absolutely different from the first and useless too.

I figured out using findcuts with BilinearResize instead of Spline64Resize doesn't even find the spot where the eyecatch begins. But why?
I have no clue how to process further now...

[StainlessS]

How bout this to start you off [Scavenged from LocateFrames] :-


Import("SelectRanges.avs") # Included with FrameSel() and Prune() [need both + RT_Stats, GScript and DebugView].

Function LocateCuts(clip Cut,clip UnCut,float "Thresh",int "ScanAhead",Int "Start",Int "Stop",
\ Float "ChromaWeight",String "Cmd",String "DB",bool "Debug",Bool "EAbort") {
myName = "LocateCuts: "
Cut = Cut.Killaudio() UnCut = UnCut.Killaudio()
Thresh = Float(Default(Thresh, 2.0))
ScanAhead = Default(ScanAhead,0)
Start = Default(Start,0)
Stop = Default(Stop,UnCut.FrameCount)
CW = Float(Default(ChromaWeight,1.0/3.0))
cmd = Default(cmd,"")
DB = Default(DB,"")
Debug = Default(Debug,True)
EAbort = Default(EAbort,True)
ScanAhead = (ScanAhead==0) ? UnCut.Framecount-1 : ScanAhead
ScanAhead = Min(Max(ScanAhead,1),UnCut.Framecount-1)
Start = Min(Max(Start,0),UnCut.Framecount-1)
Stop = Min(Max(Stop,Start),UnCut.Framecount)
Assert(UnCut.Width==Cut.Width,myName+"- Clip Width mismatch")
Assert(UnCut.Height==Cut.Height,myName+"- Clip Height mismatch")
Assert(RT_VarIsSame(Cut,UnCut),myName+"- ColorSpace Mismatch") # Same colorspace ?
Assert(Thresh>=0.0 && Thresh<=255.0,myName+"- Invalid Thresh")
Assert(CW>=0.0 && CW<=1.0,myName+"- Invalid ChromaWeight")
(Cmd!="") ? RT_FileDelete(Cmd) : NOP
Cut_UnMatched = "Cut_UnMatched_Frames.txt" # Frames in Cut clip that failed to be matched to UnCut clip
Cut_UnMatched_Ranges = "Cut_UnMatched_Ranges.txt" # as ranges
RT_FileDelete(Cut_UnMatched)
RT_FileDelete(Cut_UnMatched_Ranges)
Notfound = 0
Override = 0
MaxFoundDiff = 0.0 MaxFoundDiff_Frm=-1
FMT="CUT=%-6d UNCUT=%-6d DIFF=%f START=%d END=%-6d XPCNT=%-3d MATCH=%d : OVERRIDE=%d NOTFOUND=%d"
GScript("""
(DB != "") ? RT_DBaseAlloc(DB,Cut.Framecount,"iifiiii") : NOP
MatchLen = 0
IxEnd = Cut.Framecount-1
for (ix = 0, IxEnd) { # Iterate through find frames clip.
sS = Start # Init Start search position
BestSoFar = sS # Search start position
BestSoFarDiff = 256.0 # Impossible
mS= 0 # NOT FOUND
eterm=Min(sS+ScanAhead,Stop-1)
e=eterm # Init to 'scanned all the way'
Assert(sS<=e,myName+"- Internal Error, sS > e") # Oops.
xpCnt=0 # Prep XP skip Count
for(n=sS, eterm) { # Search c clip(s to eterm) or until satisfactory match found. (eterm evaluated on entry only)
CurrDiff=RT_FrameDifference(Cut,UnCut,n=ix,n2=n,ChromaWeight=CW)
if(CurrDiff < BestSoFarDiff) { # Better match than found so far ?
BestSoFar = n # Remember frame number
BestSoFarDiff = CurrDiff # Remember
if(BestSoFarDiff <= Thresh) { # Good enough to satisfy user requirement ?
e=BestSoFar # Where we've scanned to so far
mS=1 # Found satisfactory frame, (Flag Thresh)
if(BestSoFar < eterm && BestSoFarDiff > 0.0) {
tmp = BestSoFar # Remember
for(n=BestSoFar+1,eterm) {
CurrDiff=RT_FrameDifference(Cut,UnCut,n=ix,n2=n,ChromaWeight=CW)
e=n # We scanned more frames past original BestSoFar
if(CurrDiff < BestSoFarDiff) { # Better match ?
if(ix < ixEnd) {
# Check if XP advance makes problem for next ix find
ChkDiff = RT_FrameDifference(Cut,UnCut,n=ix+1,n2=n,ChromaWeight=CW)
if(ChkDiff>CurrDiff) { # OK, no problem for next ix iteration
BestSoFar = n # Remember frame number
BestSoFarDiff= CurrDiff # Remember
} Else { # Override XP advance
RT_DebugF("OVR: %d] [%d]CurrDiff=%f [%d]ChkDiff=%f",n,ix,CurrDiff,ix+1,ChkDiff,name=myName)
Override = Override + 1
n = eterm+1 # Force Exit with current_frame == eterm + 2 (normally exit @ eterm+1)
}
} Else {
BestSoFar = n # Remember frame number
BestSoFarDiff= CurrDiff # Remember
}
} else {
n = eterm+1 # Force Exit with current_frame == eterm + 2 (normally exit @ eterm+1)
}
}
if(BestSoFar != tmp) {
mS=2 # Found satisfactory frame, (Flag Thresh, and Xtra Paranoid found better frame)
XpCnt=BestSoFar-tmp # How many frames Xtra Paranoid skipped to find better match than Thresh.
}
}
Start=BestSoFar+1
n = eterm + 1 # Force Exit with current_frame == eterm + 2 (normally exit @ eterm + 1)
}
}
}
if(mS == 0) {
NotFound = NotFound + 1
# Always write Cut_UnMatched file if any not found, Cut[ix]
RT_WriteFile(Cut_UnMatched,"%d # Cut[%d] Best Match = UnCut[%d] Dif=%f",ix,ix,BestSoFar,BestSoFarDiff,Append=True)
if(EAbort) {
RT_DebugF("Cut[%d] Max Found Difference = %f : OVERRIDE Count=%d : NotFound Count = %d",
\ MaxFoundDiff_Frm, MaxFoundDiff,Override,NotFound,name=myName)
Assert(False,RT_String("%s *** EABORT *** Cut[%d] NOT FOUND : Best Match = UnCut[%d] Dif=%f",myName,ix,BestSoFar,BestSoFarDiff))
}
} Else if(BestSoFarDiff > MaxFoundDiff) {
MaxFoundDiff_Frm = ix
MaxFoundDiff = BestSoFarDiff
}
(DEBUG) ? RT_DebugF(FMT,ix,BestSoFar,BestSoFarDiff,sS,e,XpCnt,mS,Override,NotFound,name=myName) : NOP
(DB != "") ? RT_DBaseSet(DB,ix, ix,BestSoFar,BestSoFarDiff,sS,e,XpCnt,mS) : NOP
If(Cmd!="" && mS>0) {
RT_WriteFile(Cmd,"%d",BestSoFar,Append=True) # UnCut[BestSoFar]
}
}
RT_DebugF("Cut[%d] Max Found Difference = %f : OVERRIDE Count=%d : NotFound Count = %d",
\ MaxFoundDiff_Frm, MaxFoundDiff,Override,NotFound,name=myName)
If(Cmd!="") {
UnCut.FrameSel_CmdReWrite(Cmd,cmd=Cmd,Range=true) # Convert frames from UnCut to Ranges, easy reading
}
if(NotFound != 0) {
# Also write unmatched Cut Frames as Ranges file
Cut.FrameSel_CmdReWrite(Cut_UnMatched_Ranges,cmd=Cut_UnMatched,Range=true)
}
""") # End Of GScript
return NotFound
}

Function MakeCutTestClip(clip c,int "SZ") {
c
SZ = Default(SZ,100)
FC = FrameCount
c = 0
GSCript("""
For(n=0,FC-1,SZ*2) {
e = Min(n + SZ - 1,FC-1)
e = (e == n) ? -1 : e
c2 = Trim(n,e)
c = (c.IsClip) ? c ++ c2 : c2
# RT_DebugF("%d ] %d",n,e)
}
""")
return c
}

######## CONFIG ########
SZ = 100 # For Demo TEST ONLY clips
#
SCANAHEAD = Int(SZ * 1.5 + 0.5) # At least as big as any cut out (can set 0, but will search till end of UnCut clip where missing frames)
CMD = "UnCut_Cmd.Txt" # The Frame numbers in CMD are Frame numbers of UnCut clip that sequentially match ALL frames
# in the Cut clip, eg the first file frame number in UnCut clip will be replaced with frame 0 of Cut clip
# etc. On successful completion, converts UnCut_Cmd.Txt to a file of Ranges instead of frames for your
# perusal. On completed but failure, writes Cut_UnMatched_Frames.txt and Cut_UnMatched_Ranges.txt
# so you can see where frames / ranges where not found.
DB = "Test.DB"
THRESH = 0.5 # Adjust for source (MUST use DebugView, and need adjust if change BLURD/BLUR_ARG)
BLURD = True # Blur Detection Clips if True
BLUR_ARG = 1.58 # Blurring will likely require different Thresh (0.0 -> 1.58)
EABORT = True # If True, Abort scan on first NOTFOUND and throw error alert, else continue and write files.
######## MAKE Demo TEST CLIPS ########
UnCut=Avisource("D:\avs\Test.avi").RoboCrop(WMod=4,HMod=2,Laced=False)
Cut=UnCut.MakeCutTestClip(sz=SZ).Addborders(4,4,4,4)
UnCut=UnCut.Spline36Resize(UnCut.Width-32,UnCut.Height-32).Addborders(8,8,8,8)
# Lower Qual UnCut clip (spline36 will introduce differences [compared with detection clip Bilinear])
#
######## MAKE SIMILAR (resize to low dimension clip [Bilinear]) ########
UnCut=UnCut.RoboCrop(WMod=4,HMOD=2,Laced=False)
Cut=Cut.RoboCrop(WMod=4,HMod=2,Laced=False)
UBIG = (UnCut.Width > Cut.Width)
UnCutD = (UBIG) ? UnCut.BilinearResize(Cut.Width,Cut.Height) : UnCut # Detection clips to LOWER dim rez (USE BILINEAR!!!)
CutD = (!UBIG) ? Cut.BilinearResize(UnCut.Width,UnCut.Height) : Cut
UncutD = (BLURD) ? UnCutD.Blur(BLUR_ARG) : UnCutD
CutD = (BLURD) ? CutD.Blur(BLUR_ARG) : CutD
UnCut = (!UBIG) ? UnCut.Spline36Resize(Cut.Width,Cut.Height) : UnCut # Output clips to Higher rez (Resizer of your choice)
Cut = (UBIG) ? Cut.Spline36Resize(UnCut.Width,UnCut.Height) : Cut
#########################################################################
#########################################################################
#########################################################################
ERR=0 # Set no Error, [if comment out below LocateCuts line() ]

# ONCE below line executed SUCCESSFULLY to create CMD file, can comment out to view optional Return clips without re-scan.
ERR=LocateCuts(CutD,UnCutD,Thresh=THRESH,ScanAhead=SCANAHEAD,Cmd=CMD,db=DB,EAbort=EABORT)

Assert(ERR==0, RT_String("*** ERROR *** Some Frames Not Found(%d)",ERR))

Return FrameRep(UnCut,Cut,cmd=CMD,Show=FALSE) # FINAL OUTPUT:- Replace Uncut frames with Cut Frames

###
UnCutS = UnCut.Subtitle("UnCut",Align=5)
CutS = Cut.Subtitle("Cut",Align=5)
###
#Return StackHorizontal(CutS.ShowFrameNumber,SelectRanges(UnCutS.ShowFrameNumber,Cmd=CMD)) # Show matching frame numbers
#Return StackHorizontal(Cut,SelectRanges(UnCut,Cmd=CMD)) # No frame numbers
#Return RejectRanges(UnCutS.ShowFrameNumber,Cmd=CMD) # Show Frames Missing in CUT clip
#Return FrameRep(UnCutS,CutS,cmd=CMD,Show=True) # Replace Uncut frames with Cut Frames
###
#### Return any of above, OR Drop through to below
# Stripped out of SelectRanges, Return UnCut clip, cut to same as Cut Clip.
REJECT = FALSE # Select OR Reject
SHOW = True
PruneCmd = "~Prune_"+RT_LocalTimeString+".txt"
UnCut.FrameSel_CmdReWrite(PruneCmd,cmd=CMD,Reject=REJECT,Ordered=True,Prune=True,Range=true) # Re-write command file for Prune().
UncutS.Prune(Cmd=PruneCmd,FadeIn=True,FadeSplice=True,FadeOut=True,Fade=1.0,Show=SHOW) # Splice Selected/Rejected avoiding audio clicks
RT_FileDelete(PruneCmd) # Kill Prune Command file
Return Last


EDITED:
Need to check on Debug output to see that its doing OK, any NOTFOUNDs then maybe Thresh too low
(EDIT: or too hi and gone out of frame sync, see first NOTFOUND frame).
Creates a DBase which you can further scan to find eg frames not in Cut clip.

[StainlessS]

Added this line to previous post.


Return FrameRep(UnCutS,CutS,cmd=CMD,Show=True) # Replace Uncut frames with Cut Frames


Replaces matched sequences in UNCUT clip with same ones from CUT clip.

EDIT: Clips need to be same size (RoboCrop, Resize) / FrameRate (and same sequence order).

EDIT: Added RoboCrop/Resize (still needs same framerate).

[SilSic]

Hey StainlessS,
I'm pretty impressed of your work. Thanks a lot! :)
Testing the script gives me an error to avoid that could you send me SelectRanges.avs included with FrameSel() and Prune() please?

[StainlessS]

http://forum.doom9.org/showpost.php?p=1683559&postcount=22

You can find both FrameSel and Prune @Mediafire in my sig, below this post (recent versions have script in AVS folder).

EDIT: You need both plugins not just the script, also RT_Stats and GScript.

EDIT: You will also want DebugView (Google).

EDIT: For the detection clips only, I think I remember that we (Jmac698 and I) decided it was better to resize to lower rez clip size,
so I did it the wrong way around in post #8.

[StainlessS]

Updated post #8.
Downscale to lower res for Detection (lower Thresh required).

[SilSic]

I got this error: framesel_cmdrewrite: cannot open cmd file selectcmd
Where is the output of the final clip to be set?
"MAKE TEST CLIPS" is only for test purpose or is it necessary?

[StainlessS]

Oops sorry,


# ONCE below line executed to create CMD file, can comment out to view optional Return clips without re-scan.
#Assert(LocateCuts(CutD,UnCutD,Thresh=THRESH,ScanAhead=SCANAHEAD,Cmd=CMD,db=DB)==0, "LocateCuts: Some Frames Not Found")


Left the comment character in because of testing., just delete '#' (EDITED post).

"MAKE TEST CLIPS" is only for test purpose or is it necessary?

Yes, just for test, I (and anybody else that wants to try it) dont have your clips.

EDIT:
Where is the output of the final clip to be set?
Added this line to post #8

Return FrameRep(UnCut,Cut,cmd=CMD,Show=FALSE) # FINAL:- Replace Uncut frames with Cut Frames

[SilSic]

finally it did his work for over a hour. I got now this error "LocateCuts: Some Frames Not Found"
Is it possible to use it with SetMTmode to speed it up?
I still don't understand where the new clip will be saved to the hdd?

[StainlessS]

"LocateCuts: Some Frames Not Found"
Looks like you need to adjust Thresh, you would need to view the DebugView output for that (I guess that you did not do that).
The script will not work unless all sequences are found.
To save the script, you need to load it into eg VirtualDub/Mod and save Lossless to Hard Drive.

Although if you comment out the line (EDIT: after successful call to LocateCuts)

Assert(LocateCuts(CutD,UnCutD,Thresh=THRESH,ScanAhead=SCANAHEAD,Cmd=CMD,db=DB)==0, "LocateCuts: Some Frames Not Found")

then you can view any of the return results, and then save to HDD or eg feed straight into MeGUI, or whatever.

In DebugView, if you see any NOTFOUNDS then kill Whatever you are using Vdub/Mplayer investigate the first not found frame.
I'm guessing that the Thresh may be too low, view differences in DebugView (EDIT: Try with/without BLUR and watch DebugView).

EDIT: Try it out on any clean 5 minute SD clip as a test clip so you know what you are doing and how it works.

EDIT:
Is it possible to use it with SetMTmode to speed it up?
No idea, I dont use it.

[SilSic]

frames were not found so I changed mS=0 to mS=1 in hope to tolerate a few missing frames(does it??) and indeed it does seem to work but then the following error came up:
frameRep: *ERROR* Replace frame count != replace clip lenghth(538,540)

Now I see about the output, but it should create a new clip automatically without virtualdub which is again time-consuming to do so for eg. each 200eps,
also frame detection should work automatically without adjustment for any clip no matter what, like already autopano video does all by itself(Automatic motion-based synchronization of the video frames).
What about pattern detection, face detection, black outlines for anime but beside that it should also work for real shows.

Maybe an optional audio alignment could be also very helpful(as guideline or not by considering to tolerate missing frames when audio is same or vice versa) in cases when both clips has japanese audio but only one has english audio so that the english audio can be synchronized.
Or only based of background music which is both in english and japanese the same.
I know is a totally a new way to do so but if frame detection isn't accurate by itself and need adjustment it can be sometimes a better solution.

[StainlessS]

DONT change the function as you clearly have no idea what you are doing, you can change the arguments to it but not the code.
Changing mS to 1 means that you found all frames even before you looked for them.
FrameRep can only replace every frame frame from CUT into the position indicated in the cmd file, if there is not one, then it dont know where to put it, FAIL.
Avisynth is a FrameServer, ie it issues frames when requested by another application eg VirtualDub, it does not export any kind of video itself, however,
TWriteAvi plugin can export AVI video but not audio.
Suggest you stick with AutoPano or have a word with Harry Potter.

[StainlessS]

Thank you SilSic, got your PM and downloading sample right now.
Am still working on this, I shall put up what I have working right now, but am currently attacking from another completely different direction.
Update to post in within a few minutes.

EDIT: Post #8 updated.

[StainlessS]

Hi, Downloaded sample twice, both times comes up as successful download but both times short file (ie corrupt zip).
Can you upload somewhere else, eg MediaFire or SendSpace, thanx.

EDIT:
Thanx for the new links, will D/L in a little while.

Currently got new version (significantly different) which will not screw up just because it failed to find a single frame.
You will be able to search selected range of source clip whose sequences are to be inserted into same sequences in
selected range of dest clip. Sequences do not have to be in same order in both clips (but will be able to reduce timings
if they are). Switched to using RT_QwikScan() to search for frames and is currently searching SD material at about
1500 FPS on 2.4GHz Core Duo (Although that depends greatly upon settings).
Implemented MatchLen, which is number of frames at beginning of sequence that have to be good match or sequence
is rejected (avoid mistakes where similar scenes and high Thresh, currently Default 50 frames).
As implemented, will be able to insert sequences from Cut clip into UnCut clip, OR, just by swapping over clips
in call to function, then can do it the other way around instead (need rescan).

[StainlessS]

ReplaceCuts.avs

EDIT: Script Removed, see post #41 and 42

[StainlessS]

Client for dissimilar sour/dest clips

EDIT: Script removed, see posts #41 and 42.

[SilSic]

sorry didn't know that. But as far as I know convertfps does that or is it that something else :confused: Ether way I can convert the videos by myself if necessary shouldn't be a prob :)
Can you use the samples by converting them or reup of samples?

[StainlessS]

Cannot guarentee, Nottin but I is still trying.

[StainlessS]

Post #21 updated, still trying to knock it into shape but it is sorta working OK.

Output length slightly different as inputs are different framerate and also have to bob to obtain progressive.
As dealing directly with framerate conversion/bobbing/resizing/ etc, is considerably slower than would otherwise be the case,
at least 4x faster when scanning clips bobbed etc and recompressed as UT_Video.

Here some output using your sample clips.


00000048 168.30076599 ReplaceCuts: Out( 0,1259 ) Len=1260 from Dest( 0,1259 )
00000049 168.31784058 ReplaceCuts: Out( 1260,1309 ) Len=50 from Sour( 0,49 )
00000050 168.32705688 ReplaceCuts: Out( 1310,1341 ) Len=32 from Dest( 1312,1343 )
00000051 168.37007141 ReplaceCuts: Out( 1342,1501 ) Len=160 from Sour( 73,232 )
00000052 168.81121826 ReplaceCuts: Out( 1502,2584 ) Len=1083 from Sour( 244,1326 )
00000053 168.81266785 ReplaceCuts: Out( 2585,2585 ) Len=1 from Dest( 2588,2588 )
00000054 168.86462402 ReplaceCuts: Out( 2586,2723 ) Len=138 from Sour( 1328,1465 )
00000055 168.87957764 ReplaceCuts: Out( 2724,2777 ) Len=54 from Sour( 1470,1523 )
00000056 168.88047791 ReplaceCuts: Out( 2778,2778 ) Len=1 from Dest( 2784,2784 )
00000057 169.26531982 ReplaceCuts: Out( 2779,4144 ) Len=1366 from Sour( 1525,2890 )
00000058 169.26902771 ReplaceCuts: Out( 4145,4151 ) Len=7 from Dest( 4152,4158 )
00000059 169.30186462 ReplaceCuts: Out( 4152,4235 ) Len=84 from Sour( 2899,2982 )
00000060 169.78233337 ReplaceCuts: Out( 4236,5336 ) Len=1101 from Sour( 2987,4087 )
00000061 169.78379822 ReplaceCuts: Out( 5337,5337 ) Len=1 from Dest( 5349,5349 )
00000062 169.85607910 ReplaceCuts: Out( 5338,5479 ) Len=142 from Sour( 4089,4230 )
00000063 169.89158630 ReplaceCuts: Out( 5480,5580 ) Len=101 from Dest( 5493,5593 )
00000064 169.98034668 ReplaceCuts: Out( 5581,5882 ) Len=302 from Sour( 4262,4563 )
00000065 170.01301575 ReplaceCuts: Out( 5883,6003 ) Len=121 from Sour( 4568,4688 )
00000066 170.01391602 ReplaceCuts: Out( 6004,6004 ) Len=1 from Dest( 6022,6022 )
00000067 170.29168701 ReplaceCuts: Out( 6005,6988 ) Len=984 from Sour( 4691,5674 )
00000068 170.29258728 ReplaceCuts: Out( 6989,6989 ) Len=1 from Dest( 7009,7009 )
00000069 170.38987732 ReplaceCuts: Out( 6990,7273 ) Len=284 from Sour( 5676,5959 )
00000070 170.51103210 ReplaceCuts: Out( 7274,7592 ) Len=319 from Dest( 7295,7613 )
00000071 170.51380920 ReplaceCuts:
00000072 170.51385498 ReplaceCuts: Total (Including QwikScan DBase creation) Time taken = 463.11 secs (7.72 mins)
00000073 170.51400757 ReplaceCuts: Frames Searched = 8032763 FPS=17345.25 : O/P FPS=16.40
00000074 170.51417542 ReplaceCuts: Excluding QwikScan DBase creation, Time taken = 440.44 secs (7.34 mins)
00000075 170.51437378 ReplaceCuts: Frames Searched = 8032763 FPS=18238.00 : O/P FPS=17.24


EDIT: Where sour contains sequences that will be inserted into Dest [sour (containing cuts) about 2mins, dest about 2:32]

EDIT: The output would need to be re-interlaced or kept at 50FPS.

[SilSic]

I tried several hours, it detects too many wrong frames(although same clip properties eg framerate, etc.) and I tried another several hours adjusting thresh, even then it doesn't detect properly(eg "cut at frame 202" but detected 229 as cut, no matter how slightly the thresh has been adjusted).

The output above shows too many wrong detected cut scenes, in fact it only has 5 but the output has 23.

Also needed about 17 mins for 2 mins clip(another sample clip of mine), I assume that it always changes the framerate, resizing, etc,
even when not necessary in case when both clips have the same framerate, size, etc.. usually with the previous script it took only about 4 mins(mostly depends on thresh value).

I'm exhausted... but at least it cuts the audio now compared to the previous script and if both clips are absolutely similar, it detects all properly like a charm.

[StainlessS]

Yeh well that is all due to not being both same framerate and progressive/interlaced type, got to allow for either
source or dest (or both) having extra frames/dupes and more rubbish to ignore due to eg bobbing etc.
The script given was for your mismatched clips, you would not need that bob resize etc if both clips were compatible Progressive.
Also, calling resize only where both source and dest are same size should ignore the resize filter (ie not linked into filter chain by avisynth).
I'm still working on it.

I'm exhausted
me too.

Edit: Would probably work better too if done in YV24 so as to crop pixel accurate, what you using (Avisynth version) ?

Additional problem would be audio where mismatched clips, where replacment number of frames differ and so audio length
of dest and output differ, use original(dest) and would be out of sync, use replacing audio(sour) and would would eg be at differing volume level.

EDIT: Also, I would prefer AviSource with lossless codec, dont know of any other source filter that I trust as being
frame accurate except for DGIndex. (also faster if eg avi/ut_video)

In the given output log you will see several places where there are two 'Sour' ranges together, that is where number of
dupes inserted resulted in too much mis-aligned matches and so dropped (hopefully) dupes from sour.
There are a couple of bugs (at least) in given script, should be able to improve a little.

[SilSic]

I'm usin latest avisynth version 2.6.
Don't know if you got me right, I tested 2 clips which has both the same framerate and same video type and also same audio rate.
But still it doesn't detect properly and too many wrong frames. I wasn't able to cut out the parts in your script like
"Make Same FrameRate" or "Bob to Remove Interlacing"
as it always resulted to give me errors.

Only if I take the "same source" to make 2 clips, one uncut, one cut(I added cuts for testing) it works properly.

Both clips I sent you have the same video speed although one have 23,976 framerate and one 29.976.
So changing framerate may slow the video speed down and leads to an async. result.

Please rank the accuracy of detection first and if you still got some time and fun to do the stuff with Framerate conversion, interlaced, progressive and so on its up to you.
They aren't as important as the detection.

I sent you another sample clips which has same video properties which makes it easier to figure out the real cause for mismatch.

[StainlessS]

Matching clips is YOUR SIDE, I dont care how you do it, the only requirement is same size, same framerate, both Progressive and similar picture area (ie no borders).
I would not know if need IVTC, deinterlacing, speed up/down (AssumeFPS) etc, you know that and your job to match.

The only part on my side is the ReplaceCuts function and call to Prune to splice.
The bits about which one is source and which dest if only to test if it works both ways around.
MatchLen is number of frames that must match a start of any sequence.
LumaTol is a weird one and if it is missing matches then may need to be increased, see the weird description of it in RT_Stats RT_QwikScan.
If brightness contrast is far apart then needs hi LumaTol (8.0 -> 16.0), exact (and I mean bit identical) then 0.0.
Thresh is average pixel difference between corresponding pixels, ideally it would be 0.0.

I'm going to pub for an hour or so soon, I'll grab you samples then.

[StainlessS]

Can you please re-upload to a site that does not bombard you with bucket loads of ads and porn and other stuff.
Try send space or media fire.

[SilSic]

How can I exclude eg lumatol or "Make Same FrameRate", Bob, trying to exclude gives me errors?
Is there something else in your mind for detection frames accurately?
Is it possible to use any good analyser or some other plugins, scripts for better detection or MSU Quality Measurement Tool?
What about this Motion filter? http://forum.doom9.org/showthread.php?t=101859
I don't know how to implement it..

[StainlessS]

You do not want to exclude lumatol, if you noted the search readings, you would see that (in your 2 mns clip) there was 8 million (or so frames searched),
do that in real time if you wish, I cant really help with un-realistic expectation.

EDIT: on my (crappy) Core duo duel core, probably a bit faster on yours.

[StainlessS]

I shall (try) to complete this task with 1:1 frame match, is for someone else to further that, I have other things I have to do.

[SilSic]

Don't push yourself, gotta lot of time, I'd be very glad if it works like a charm without a lot of readjustments.

About my previous suggestions, I thought if there were more than matchlen, lumatol, brightness, contrast, chromaweight, which constitute detection,
the more new ones be added the more these mentioned ones can tolerate that partial not found particular image properties. Which increases the accuracy(like an increasing majority with decreasing denial) and the more it gets closer to his maybe unreachable 1:1 frame match destination.

[StainlessS]

Sorry for being a bit grumpy, I just lost the stylus for my Advent Nvidia Tegra Note 7 tablet and am a bit narked about it.
Found that there are a few on ebay for about £8.00, but I just dont like PayPal at all (heard toooo many bad stories about them).
Anyway, gonna try out a suggestion of using a solid graphite pencil, they are supposed to work fine, think the stylus is magnetic based
rather than standard capacitance stylus (8 magnets in Tegra Note 7).
Anyhow, got your samples and gonna have another bash at it.

EDIT: Suggest you take a peek at this post (and previous two) to explain the LumaTol thing.

http://forum.doom9.org/showthread.php?p=1693002#post1693002

EDIT:
For those wishing to read of the woes of using paypal see FYPaypal.com
where you have to use your imagination a little bit to fill in the FY part.
Have just tried to visit there and my ISP tells me that it does not exist, yet google it and you find a domain tools WHOIS entry for it.
Looks like I'm being barred from using it (thank you Three).
The guy there usually has the personal phone and mobile number of the CEO (despite however many times he changes it :) )
EDIT: Apparently FYP.com is closed down and for sale :(

[SilSic]

Yeah you can be really grumpy. I know a lot of that kinda, I'm used to it ;)
I really appreciate your help. And I wonder what you're up to now?

[StainlessS]

Yeah you can be really grumpy
You is a cheeky-chappy :)

Been up to this and mods to your script (for use in your script),
http://forum.doom9.org/showthread.php?t=172136

probably post something later today.

[StainlessS]

Posts #21 and #22 updated.

EDIT: Posts 21 and 22 removed, see post 41 and 42.

SpatialAlign output

SpatialAlign: Not already temporally aligned (Worst LumaDif of all tested frames=158.694)
SpatialAlign: Searching for temporal alignment
RT_QwikScanCreate: RT_QwikScanCreateDB() by StainlessS
RT_QwikScanCreate: DBaseAlloc DB
RT_QwikScanCreate: DBaseAlloc NextDB
RT_QwikScanCreate: Filling DB with FingerPrint data (Will take some time)

...

RT_QwikScanCreate: Total time = 17.27 seconds (0.29 mins)
SpatialAlign: Scanning 50 frames (1.7%) of Fix clip @ LumaTol==5 LdTh=12.00
SpatialAlign: Fix[231] Match Frame found at Src[164] MaxLumaDif=3.909 (over 50 frames)
SpatialAlign: Shifting clip HDir=2 VDir=-2 (orig correlation=0.975770 shifted=0.993354
SpatialAlign: Global Crop_L=2 Global Crop_T=0 Global Crop_R=0 Global Crop_B=2
AutoContrast: Levels( 17,1.0,234,16,235,Coring=False)


ReplaceCuts output (LC and LD conditions).

RT_QwikScanCreate: RT_QwikScanCreateDB() by StainlessS
RT_QwikScanCreate: DBaseAlloc DB
RT_QwikScanCreate: DBaseAlloc NextDB
RT_QwikScanCreate: Filling DB with FingerPrint data (Will take some time)

...

RT_QwikScanCreate: Total time = 20.69 seconds (0.34 mins)

ReplaceCuts: Dst[ 76->168 ] Src[ 10->102 ] Len= 93 LC= 0.287 LD= 3.105 FD= 2.372 PD= 12.469 (Fail@ LC=255.000 LD= 92.584 FD= 66.195 PD=251.146)
ReplaceCuts: Dst[ 170->313 ] Src[ 103->246 ] Len= 144 LC= 1.320 LD= 3.961 FD= 3.056 PD= 20.387 (Fail@ LC=188.887 LD= 50.069 FD= 42.122 PD=222.265)
ReplaceCuts: Dst[ 369->401 ] Src[ 302->334 ] Len= 33 LC= 3.100 LD= 5.520 FD= 4.210 PD= 33.005 (Fail@ LC= 3.097 LD= 5.511 FD= 4.201 PD= 32.667)
ReplaceCuts: Dst[ 407->528 ] Src[ 340->461 ] Len= 122 LC= 2.980 LD= 5.399 FD= 4.121 PD= 29.557 (Fail@ LC=184.719 LD= 53.807 FD= 41.904 PD=238.789)
ReplaceCuts: Dst[ 533->652 ] Src[ 466->585 ] Len= 120 LC= 2.738 LD= 4.789 FD= 3.548 PD= 37.176 (Fail@ LC=255.000 LD=100.180 FD= 70.088 PD=243.199)
ReplaceCuts: Dst[ 694->756 ] Src[ 627->689 ] Len= 63 LC= 0.541 LD= 3.687 FD= 2.870 PD= 38.207 (Fail@ LC=255.000 LD= 76.941 FD= 53.954 PD=226.844)
ReplaceCuts: Dst[ 806->1048 ] Src[ 739->981 ] Len= 243 LC= 3.555 LD= 5.002 FD= 3.745 PD= 52.956 (Fail@ LC=255.000 LD= 83.704 FD= 60.731 PD=245.342)
ReplaceCuts: Dst[ 1148->1286 ] Src[ 1082->1220 ] Len= 139 LC= 2.709 LD= 4.798 FD= 3.559 PD= 35.975 (Fail@ LC=251.790 LD= 68.521 FD= 51.870 PD=243.722)
ReplaceCuts: Dst[ 1288->1412 ] Src[ 1221->1345 ] Len= 125 LC= 3.638 LD= 3.611 FD= 3.022 PD= 27.508 (Fail@ LC=212.976 LD= 52.877 FD= 46.162 PD=226.671)
ReplaceCuts: Dst[ 1483->1684 ] Src[ 1417->1618 ] Len= 202 LC= 0.298 LD= 2.038 FD= 1.703 PD= 4.301 (Fail@ LC=237.094 LD= 49.827 FD= 38.537 PD=232.244)
ReplaceCuts: Dst[ 1686->1715 ] @ Src[ 1619->1648 ] Len= 30 LC= 4.127 LD= 4.523 FD= 3.514 PD= 23.937
ReplaceCuts: Dst[ 1686->1717 ] Src[ 1620->1651 ] Len= 32 LC= 4.136 LD= 4.510 FD= 3.502 PD= 23.790 (Fail@ LC= 26.797 LD= 9.371 FD= 7.323 PD= 55.956)
ReplaceCuts: Dst[ 1729->1778 ] Src[ 1662->1711 ] Len= 50 LC= 4.474 LD= 4.652 FD= 3.655 PD= 26.876 (Fail@ LC=255.000 LD= 76.840 FD= 56.902 PD=241.975)
ReplaceCuts: Dst[ 1781->1879 ] Src[ 1714->1812 ] Len= 99 LC= 1.150 LD= 3.344 FD= 2.506 PD= 12.076 (Fail@ LC=255.000 LD= 95.281 FD= 66.135 PD=246.780)
ReplaceCuts: Dst[ 1986->2015 ] Src[ 1919->1948 ] Len= 30 LC= 1.336 LD= 3.372 FD= 2.455 PD= 27.121 (Fail@ LC=255.000 LD= 46.801 FD= 41.505 PD=241.970)
ReplaceCuts: Dst[ 2073->2105 ] Src[ 2032->2064 ] Len= 33 LC= 0.964 LD= 0.104 FD= 0.239 PD= 0.398 (Fail@ LC=253.917 LD= 89.314 FD= 60.271 PD=254.517)
ReplaceCuts: Dst[ 2107->2181 ] Src[ 2065->2139 ] Len= 75 LC= 3.904 LD= 3.410 FD= 2.487 PD= 19.158 (Fail@ LC=255.000 LD= 30.621 FD= 22.221 PD=205.464)
ReplaceCuts: Dst[ 2232->2335 ] Src[ 2191->2294 ] Len= 104 LC= 3.882 LD= 3.212 FD= 2.497 PD= 24.339 (Fail@ LC=255.000 LD= 33.569 FD= 24.807 PD=222.377)
ReplaceCuts: Dst[ 2337->2385 ] Src[ 2295->2343 ] Len= 49 LC= 0.935 LD= 1.527 FD= 1.316 PD= 3.060 (Fail@ LC=255.000 LD= 38.099 FD= 27.235 PD=234.623)
ReplaceCuts: Dst[ 2476->2569 ] Src[ 2435->2528 ] Len= 94 LC= 4.720 LD= 4.579 FD= 3.506 PD= 48.446 (Fail@ LC=255.000 LD= 49.541 FD= 36.361 PD=239.908)
ReplaceCuts: Dst[ 2578->2681 ] Src[ 2537->2640 ] Len= 104 LC= 4.018 LD= 3.548 FD= 2.646 PD= 22.991
ReplaceCuts: Out( 0,75 ) Len=76 from Dest( 0,75 )
ReplaceCuts: Out( 76,168 ) Len=93 from Sour( 10,102 )
ReplaceCuts: Out( 169,169 ) Len=1 from Dest( 169,169 )
ReplaceCuts: Out( 170,313 ) Len=144 from Sour( 103,246 )
ReplaceCuts: Out( 314,368 ) Len=55 from Dest( 314,368 )
ReplaceCuts: Out( 369,401 ) Len=33 from Sour( 302,334 )
ReplaceCuts: Out( 402,406 ) Len=5 from Dest( 402,406 )
ReplaceCuts: Out( 407,528 ) Len=122 from Sour( 340,461 )
ReplaceCuts: Out( 529,532 ) Len=4 from Dest( 529,532 )
ReplaceCuts: Out( 533,652 ) Len=120 from Sour( 466,585 )
ReplaceCuts: Out( 653,693 ) Len=41 from Dest( 653,693 )
ReplaceCuts: Out( 694,756 ) Len=63 from Sour( 627,689 )
ReplaceCuts: Out( 757,805 ) Len=49 from Dest( 757,805 )
ReplaceCuts: Out( 806,1048 ) Len=243 from Sour( 739,981 )
ReplaceCuts: Out( 1049,1147 ) Len=99 from Dest( 1049,1147 )
ReplaceCuts: Out( 1148,1286 ) Len=139 from Sour( 1082,1220 )
ReplaceCuts: Out( 1287,1287 ) Len=1 from Dest( 1287,1287 )
ReplaceCuts: Out( 1288,1412 ) Len=125 from Sour( 1221,1345 )
ReplaceCuts: Out( 1413,1482 ) Len=70 from Dest( 1413,1482 )
ReplaceCuts: Out( 1483,1684 ) Len=202 from Sour( 1417,1618 )
ReplaceCuts: Out( 1685,1685 ) Len=1 from Dest( 1685,1685 )
ReplaceCuts: Out( 1686,1717 ) Len=32 from Sour( 1620,1651 )
ReplaceCuts: Out( 1718,1728 ) Len=11 from Dest( 1718,1728 )
ReplaceCuts: Out( 1729,1778 ) Len=50 from Sour( 1662,1711 )
ReplaceCuts: Out( 1779,1780 ) Len=2 from Dest( 1779,1780 )
ReplaceCuts: Out( 1781,1879 ) Len=99 from Sour( 1714,1812 )
ReplaceCuts: Out( 1880,1985 ) Len=106 from Dest( 1880,1985 )
ReplaceCuts: Out( 1986,2015 ) Len=30 from Sour( 1919,1948 )
ReplaceCuts: Out( 2016,2072 ) Len=57 from Dest( 2016,2072 )
ReplaceCuts: Out( 2073,2105 ) Len=33 from Sour( 2032,2064 )
ReplaceCuts: Out( 2106,2106 ) Len=1 from Dest( 2106,2106 )
ReplaceCuts: Out( 2107,2181 ) Len=75 from Sour( 2065,2139 )
ReplaceCuts: Out( 2182,2231 ) Len=50 from Dest( 2182,2231 )
ReplaceCuts: Out( 2232,2335 ) Len=104 from Sour( 2191,2294 )
ReplaceCuts: Out( 2336,2336 ) Len=1 from Dest( 2336,2336 )
ReplaceCuts: Out( 2337,2385 ) Len=49 from Sour( 2295,2343 )
ReplaceCuts: Out( 2386,2475 ) Len=90 from Dest( 2386,2475 )
ReplaceCuts: Out( 2476,2569 ) Len=94 from Sour( 2435,2528 )
ReplaceCuts: Out( 2570,2577 ) Len=8 from Dest( 2570,2577 )
ReplaceCuts: Out( 2578,2681 ) Len=104 from Sour( 2537,2640 )
ReplaceCuts: Out( 2682,2882 ) Len=201 from Dest( 2682,2882 )
ReplaceCuts:
ReplaceCuts: Ranges Replaced = 20 Frames Replaced = 1954
ReplaceCuts: Total (Including QwikScan DBase creation) Time taken = 266.96 secs (4.45 mins)
ReplaceCuts: Frames Searched = 986129 FPS=3693.93 : O/P FPS=10.80
ReplaceCuts: Excluding QwikScan DBase creation, Time taken = 246.27 secs (4.10 mins)
ReplaceCuts: Frames Searched = 986129 FPS=4004.30 : O/P FPS=11.71


Prune Cmd.txt file

0 0,75 # 0,75 Len=76
1 10,102 # 76,168 Len=93
0 169,-1 # 169,169 Len=1
1 103,246 # 170,313 Len=144
0 314,368 # 314,368 Len=55
1 302,334 # 369,401 Len=33
0 402,406 # 402,406 Len=5
1 340,461 # 407,528 Len=122
0 529,532 # 529,532 Len=4
1 466,585 # 533,652 Len=120
0 653,693 # 653,693 Len=41
1 627,689 # 694,756 Len=63
0 757,805 # 757,805 Len=49
1 739,981 # 806,1048 Len=243
0 1049,1147 # 1049,1147 Len=99
1 1082,1220 # 1148,1286 Len=139
0 1287,-1 # 1287,1287 Len=1
1 1221,1345 # 1288,1412 Len=125
0 1413,1482 # 1413,1482 Len=70
1 1417,1618 # 1483,1684 Len=202
0 1685,-1 # 1685,1685 Len=1
1 1620,1651 # 1686,1717 Len=32
0 1718,1728 # 1718,1728 Len=11
1 1662,1711 # 1729,1778 Len=50
0 1779,1780 # 1779,1780 Len=2
1 1714,1812 # 1781,1879 Len=99
0 1880,1985 # 1880,1985 Len=106
1 1919,1948 # 1986,2015 Len=30
0 2016,2072 # 2016,2072 Len=57
1 2032,2064 # 2073,2105 Len=33
0 2106,-1 # 2106,2106 Len=1
1 2065,2139 # 2107,2181 Len=75
0 2182,2231 # 2182,2231 Len=50
1 2191,2294 # 2232,2335 Len=104
0 2336,-1 # 2336,2336 Len=1
1 2295,2343 # 2337,2385 Len=49
0 2386,2475 # 2386,2475 Len=90
1 2435,2528 # 2476,2569 Len=94
0 2570,2577 # 2570,2577 Len=8
1 2537,2640 # 2578,2681 Len=104
0 2682,2882 # 2682,2882 Len=201


EDIT: Left the bob stuff in client script. Function requires same frameRate and will not work with eg dupes inserted.
You clip has varying shift for each sequence, missing frames in both clips (I think) and one of them enlarges slightly at frame before scene change.
The results shown for eg LC and LD are for the first frame in sequence, and first frame after sequence ends.

[SilSic]

Hey looks pretty amazing. But I can see in your cmd.txt(posted above) that it finds toooo much, there are just 3 spots to be found. Besides I tried your SpatialAlign script but it is incomplete. I cannot test it untill you put a complete version of it ;)
I'm sure this time it'll work. Can't await to test it :)

[StainlessS]

Besides I tried your SpatialAlign script but it is incomplete
http://forum.doom9.org/showpost.php?p=1722142&postcount=1

You can insert the demo utility stuff into client script of the SpatialAlign, it is not needed for this script.

just 3 spots to be found
No there aint, for the reasons below
will not work with eg dupes inserted.
Your clip has varying shift for each sequence, missing frames in both clips (I think) and one of them enlarges slightly at frame before scene change.
Look at the results before you condemn it.

You can eg reduce MatchLen to 1, but expect lots of problems with flashing colors as it rapidly changes from 1 clip to the other, the posted script is just about as good as you can get, you can tweak settings as you seem fit and you will need to do this and also get a little more acquainted with scripting. I know you want a completely automatic solution, avisynth is not such a tool and no-one is going to spoon feed you for every clip.

EDIT:
Posts #21 and #22 updated.
Will not fit in single post 16KB limit.

EDIT: Look at EFail results in debug stuff to see why a frame failed, you could eg raise the failing threshold but it will likely just fail again at
the next misaligned frame. I have used compomise settings, you can tweak the settings yourself as you see fit.

EDIT:
Dst[ 369->401 ] Src[ 302->334 ] Len= 33 LC= 3.100 LD= 5.520 FD= 4.210 PD= 33.005 (Fail@ LC= 3.097 LD= 5.511 FD= 4.201 PD= 32.667)

Cant explain that, I'll have to take a bit more of a look at it.

[StainlessS]

Here is part one of Replacecuts.avs (too big for single post)


Function ReplaceCuts(clip Dst,clip Src,Int "Flags",Float "LC",Float "LD",Float "FD",Float "PD",int "PDThresh",
\ Float "LumaTol",Int "MatchLen",Bool "InSeq",
\ Int "DstStart",Int "DstStop",Int "SrcStart",Int "SrcStop",
\ Float "ChromaWeight",String "Cmd",String "DB",bool "Debug",Bool "Debug2") {
/*
ReplaceCuts.avs, by StainlessS @ Doom9 : http://forum.doom9.org/showpost.php?p=1720446&postcount=41
Replaces sequences in Dst clip with similar sequences in Src clip.

Requires:- GScript, RT_stats, RoboCrop'ed before calling. DebugView for viewing debug info.
Frame Accurate Progressive sources of same Size, FrameRate, Colorspace.

Dst, Clip which will have sequences of at least MatchLen frames inserted from similar sequences in Src clip.
Src, clip which will replace similar sequences in Dst clip.
Flags, Default $03 (1+2==LC+LD). Bit flags
Condition flags(add together), All conditions set must succeed for each frame (ie AND)
1 = LC = Scaled Half range Pearson's Distance [ie Min((1.0-RT_LumaCorrelation * 255.0, 255.0)]
2 = LD = LumaDifference
4 = FD = FrameDifference
8 = PD = LumaPixelsDifferent (255.0 = 100%)
PD, ie RT_LumaPixelsDifferent() is not a good choice if frames are likely to be shifted WRT each other (esp anime).
Four conditions implemented as some may produce better results than others under certain circumstances.
LC, Default 6.0. Threshold for Scaled half range Pearson's Distance condition. [ie Min((1.0-RT_LumaCorrelation * 255.0, 255.0)]
LD, Default 6.0. Threshold for RT_LumaDifference. See RT_LumaDifference.
FD, Default 6.0. Threshold for RT_FrameDifference. See RT_FrameDifference.
PD, Default 6.0. Threshold for RT_LumaPixelsDifferent. See RT_LumaPixelsDifferent. (255.0 = 100.0%)
PdThresh Default 10, Arg to RT_LumaPixelsDifferent. See RT_LumaPixelsDifferent.
LumaTol, Default 7.0. See RT_QwikScan.
MatchLen, Default 25. Minimum length of similar sequences that may be replaced from Src into Dst.
InSeq, Default True. Set True if similar sequences in both Src and Dst are in same temporal order (takes much longer if false).
DstStart, Default 0. Start frame to search in Dst clip.
DstStop, Default Dst.FrameCount. Frame in Dst clip, including and after which, will not be searched.
SrcStart, Default 0. Start frame to search in Src clip.
SrcStop, Default Src.FrameCount. Frame in Src clip, including and after which, will not be searched.
ChromaWeight, Default 1.0/3.0. Arg to RT_FrameDifference, ie FD = (1.0-ChromaWeight) * YDiff + ChromaWeight *((UDiff+VDiff)/2.0)
Cmd, Default "Cmd.Txt". Output Prune() command file for cut and splice of input clips.
DB, Default "". If default "" then creates a temporary DBase file which is deleted after use. If user set to a filename,
then the DBase will be written to that filename and will not be deleted after use.
DBase has two fields, Field 0 = int Clip number (0=Dst clip, 1 = Src Clip). Field 1 = int frame number in Field 0 clip.
The DBase has the same number of records as both input Dst and Prune(cmd="Cmd.txt") output.
Debug, Default = True. Output Debug Info to DebugView (Google).
NOTE, When Debug True, outputs ALL condition results (not just the selected Flags ones), also outputs results at frame following
detected sequences, ie fail results.
Debug2, Default False. If set True then will also show indented results for earlier BestSoFar results when a better match is
found later. Default False as messes up debug formatting a little bit, and not generally needed.

*/
myName = "ReplaceCuts: "
Flags = RT_BitAnd(Default(Flags,$0F),$0F)
LC = Float(Default(LC, 6.0))
LD = Float(Default(FD, 6.0))
FD = Float(Default(FD, 6.0))
PD = Float(Default(PD, 10.0))
PDThresh = Default(PDThresh, 6)
LumaTol = Float(Default(LumaTol,7.0))
MatchLen = Default(MatchLen,25)
InSeq = Default(InSeq,True)
DstStart = Default(DstStart,0)
DstStop = Default(DstStop,Dst.FrameCount)
SrcStart = Default(SrcStart,0)
SrcStop = Default(SrcStop,Src.FrameCount)
CW = Float(Default(ChromaWeight,1.0/3.0))
Cmd = Default(Cmd,"")
DB = Default(DB,"")
Debug = Default(Debug,True)
Debug2 = (Default(Debug2,False) && Debug)
Assert(Src.Width==Dst.Width,RT_String("%sClip Width mismatch",myName))
Assert(Src.Height==Src.Height,RT_String("%sClip Height mismatch",myName))
Assert(RT_VarIsSame(Src,Dst,sig=false),RT_String("%sClip ColorSpace mismatch",myName))
Assert(Flags != 0,RT_String("%sInvalid Flags Cannot be 0($%X)",myName,Flags))
Assert(LC>=0.0 && LC<=255.0,RT_String("%sInvalid LC 0.0 -> 255.0(%f)",myName,LC))
Assert(LD>=0.0 && LD<=255.0,RT_String("%sInvalid LD 0.0 -> 255.0(%f)",myName,LD))
Assert(FD>=0.0 && FD<=255.0,RT_String("%sInvalid FD 0.0 -> 255.0(%f)",myName,FD))
Assert(PD>=0.0 && PD<=255.0,RT_String("%sInvalid PD 0.0 -> 255.0(%f)",myName,PD))
Assert(PDThresh>=0 && PDThresh<=255,RT_String("%sInvalid PDThresh 0->255(%d)",myName,PDThresh))
Assert(LumaTol>=0.0 && LumaTol<=16.0,RT_String("%sInvalid LumaTol 0.0 -> 16.0(%f)",myName,LumaTol))
Assert(MatchLen>0,RT_String("%sInvalid MatchLen, Must be greater than 0(%d)",myName,MatchLen))
Assert(DstStart>=0 && DstStart<Dst.Framecount,RT_String("%sInvalid DstStart 0 -> %d(%d)",myName,Dst.Framecount-1,DstStart))
Assert(DstStop>DstStart && DstStop<=Dst.Framecount,RT_String("%sInvalid DstStop %d -> %d(%d)",myName,DstStart+1,Dst.Framecount,DstStop))
Assert(SrcStart>=0 && SrcStart<Src.Framecount,RT_String("%sInvalid SrcStart 0 -> %d(%d)",myName,Src.Framecount-1,SrcStart))
Assert(SrcStop>SrcStart && SrcStop<=Src.Framecount,RT_String("%sInvalid SrcStop %d -> %d(%d)",myName,SrcStart+1,Src.Framecount,SrcStop))
Assert(CW>=0.0 && CW<=1.0,RT_String("%sInvalid ChromaWeight(%f)",myName,CW))
Cmd = (Cmd!="") ? Cmd : "Cmd.txt" RT_FileDelete(Cmd)
FSearched = 0 False = False # Make local
TIMSTART = RT_TimerHP


In zip form, includes SpacialAlign.avs (11KB):- http://www.mediafire.com/download/teug3d0te727d1r/ReplaceCuts.zip

[StainlessS]

Part 2

GScript("""
Tim = RT_LocalTimeString(File=True)
SDB = (DB!="") ? DB : RT_String("~DB_%s.txt",Tim)
QDB = RT_String("~QDB_%s.txt",Tim) NDB = RT_String("~NDB_%s.txt",Tim)
RT_QwikScanCreate(Src,QDB,prevdb="",nextdb=NDB,debug=Debug)
TIMSTART2 = RT_TimerHP
RT_DBaseAlloc(SDB,0,"ii")
if(DstStart>0) {
(DEBUG) ? RT_DebugF("Using Dest frames 0 -> %d",DstStart-1,name=myName) : NOP
for(i=0,DstStart-1) {RT_DBaseAppend(SDB,0,i)} # Set DB records prior to DstStart to clip 0(Dst)
}
SiEnd = SrcStop - MatchLen
sS = SrcStart
for(Di = DstStart, DstStop - 1) {
BstF = -1 # Best match frame Not found
# We do not track Black/White/Gray start frames, too easy to get it wrong and screw up eg audio
if(RT_YPlaneMinMaxDifference(Dst,n=Di,Threshold=1.0) >= 16) {
BstLen = -1 # Best len of match to Dst[Di]
For(Si = (InSeq) ? sS : SrcStart, SiEnd) { # Searching through Src[Si] for Dst[Di] frame.
MaxDistance = SiEnd - Si
Result=RT_QwikScan(Src,Si,Dst,Di,QDB,NDB,lumatol=LumaTol,Flags=Flags,lc=LC,ld=LD,fd=FD,pd=PD,pdthresh=PDThresh,
\ maxdistance=MaxDistance,xp=0)
if(Result>=0) { # Any Exit Conditions succeed ?
FSearched = FSearched + (Result - Si) + 1
if(QWKS_EXIT_FLAGS==Flags) { # ALL Exit Conditions succeed ?
jEnd=Min(SrcStop-1-Result,DstStop-1-Di)
Len = 1
for(j=1,jEnd) {
OKFlags = 0
if(RT_BitTst(Flags,0)) {
lcd = Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di+j,n2=Result+j)) * 255.0, 255.0 )
OKFlags = OKFlags + (lcd <= LC ?1:0)
}
if(RT_BitTst(Flags,1)) {
ldd = RT_LumaDifference(Dst,Src,n=Di+j,n2=Result+j)
OKFlags = OKFlags + (ldd <= LD ?2:0)
}
if(RT_BitTst(Flags,2)) {
fdd = RT_FrameDifference(Dst,Src,n=Di+j,n2=Result+j,ChromaWeight=CW)
OKFlags = OKFlags + (fdd <= FD ?4:0)
}
if(RT_BitTst(Flags,3)) {
lpd = RT_LumaPixelsDifferent(Dst,Src,n=Di+j,n2=Result+j,Thresh=PDThresh)
OKFlags = OKFlags + (lpd <= PD ?8:0)
}
FSearched = FSearched + RT_BitSetCount(Flags)
if(OKFlags!=Flags && j < jEnd) { # Out of whack frame !
# Tolerate a single frame that is out of whack, ie is next frame OK ?
OKFlags = 0
if(RT_BitTst(Flags,0)) {
lcd = Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di+j+1,n2=Result+j+1)) * 255.0, 255.0 )
OKFlags = OKFlags + (lcd <= LC ?1:0)
}
if(RT_BitTst(Flags,1)) {
ldd = RT_LumaDifference(Dst,Src,n=Di+j+1,n2=Result+j+1)
OKFlags = OKFlags + (ldd <= LD ?2:0)
}
if(RT_BitTst(Flags,2)) {
fdd = RT_FrameDifference(Dst,Src,n=Di+j+1,n2=Result+j+1,ChromaWeight=CW)
OKFlags = OKFlags + (fdd <= FD ?4:0)
}
if(RT_BitTst(Flags,3)) {
lpd = RT_LumaPixelsDifferent(Dst,Src,n=Di+j+1,n2=Result+j+1,Thresh=PDThresh)
OKFlags = OKFlags + (lpd <= PD ?8:0)
}
FSearched = FSearched + RT_BitSetCount(Flags)
}
if(OKFlags==Flags) {
Len = Len + 1
} Else {
j = jEnd # Break, not found
}
} # End for j
if(Len >= MatchLen && Len >= BstLen) {
SW = (Len > BstLen)
if(!SW) {
# If Same len as previous best then only switch if ALL conditions are better than previous best.
SW = (
\ !RT_BitTst(Flags,0) ||
\ Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di,n2=Result)) * 255.0, 255.0 ) <
\ Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di,n2=BstF)) * 255.0, 255.0 ))
SW = (
\ SW &&
\ (!RT_BitTst(Flags,1) ||
\ RT_LumaDifference(Dst,Src,n=Di,n2=Result) <
\ RT_LumaDifference(Dst,Src,n=Di,n2=BstF)))
SW = (
\ SW &&
\ (!RT_BitTst(Flags,2) ||
\ RT_FrameDifference(Dst,Src,n=Di,n2=Result,ChromaWeight=CW) <
\ RT_FrameDifference(Dst,Src,n=Di,n2=BstF,ChromaWeight=CW)))
SW = (
\ SW &&
\ (!RT_BitTst(Flags,3) ||
\ RT_LumaPixelsDifferent(Dst,Src,n=Di,n2=Result,Thresh=PDThresh) <
\ RT_LumaPixelsDifferent(Dst,Src,n=Di,n2=BstF,Thresh=PDThresh)))
FSearched = FSearched + RT_BitSetCount(Flags) * 2
}
if(SW) {
if(DEBUG2&&BstLen>0) { # Delayed write of previous best (dont count debug scanned frames)
lcD=Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di,n2=BstF)) * 255.0, 255.0 )
ldD=RT_LumaDifference(Dst,Src,n=Di,n2=BstF)
fdD=RT_FrameDifference(Dst,Src,n=Di,n2=BstF,ChromaWeight=CW)
pdD=RT_LumaPixelsDifferent(Dst,Src,n=Di,n2=BstF,Thresh=PDThresh)
RT_DebugF(" Dst[%4d,%-4d] Src[%4d,%-4d] Len=%4d LC=%7.3f LD=%7.3f FD=%7.3f PD=%7.3f",
\ Di,Di+BstLen-1,BstF,BstF+BstLen-1,Bstlen,lcD,ldD,fdD,pdD,name=myName)
}
BstLen = Len
BstF = Result
if(Di+BstLen >= DstStop || BstF + BstLen >= SrcStop) {
Result = SiEnd # no point in looking for longer match
}
}
}
}
Si = Result # Continue Qwikscan search
} Else { # Not found by QwikScan
FSearched = FSearched + MaxDistance + 1 # Searched all of the way (start inclusive)
Si = SiEnd # Break
}
} # End for Si
if(BstF != -1) { # Found
if(DEBUG) { # dont count debug scanned frames
lcD=Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di,n2=BstF)) * 255.0, 255.0 )
ldD=RT_LumaDifference(Dst,Src,n=Di,n2=BstF)
fdD=RT_FrameDifference(Dst,Src,n=Di,n2=BstF,ChromaWeight=CW)
pdD=RT_LumaPixelsDifferent(Dst,Src,n=Di,n2=BstF,Thresh=PDThresh)
if(Di + BstLen < DstStop && BstF + BstLen < SrcStop) {
ElcD=Min( (1.0 - RT_LumaCorrelation(Dst,Src,n=Di+BstLen,n2=BstF+BstLen)) * 255.0, 255.0 )
EldD=RT_LumaDifference(Dst,Src,n=Di+BstLen,n2=BstF+BstLen)
EfdD=RT_FrameDifference(Dst,Src,n=Di+BstLen,n2=BstF+BstLen,ChromaWeight=CW)
EpdD=RT_LumaPixelsDifferent(Dst,Src,n=Di+BstLen,n2=BstF+BstLen,Thresh=PDThresh)
RT_DebugF("Dst[%4d,%-4d] Src[%4d,%-4d] Len=%4d LC=%7.3f LD=%7.3f FD=%7.3f PD=%7.3f : Fail@ LC=%7.3f LD=%7.3f FD=%7.3f PD=%7.3f",
\ Di,Di+BstLen-1,BstF,BstF+BstLen-1,Bstlen,lcD,ldD,fdD,pdD,ElcD,ELdD,EfdD,EpdD,name=myName)
} Else {
RT_DebugF("Dst[%4d,%-4d] Src[%4d,%-4d] Len=%4d LC=%7.3f LD=%7.3f FD=%7.3f PD=%7.3f",
\ Di,Di+BstLen-1,BstF,BstF+BstLen-1,Bstlen,lcD,ldD,fdD,pdD,name=myName)
}
}
for(j=0,BstLen-1) {
RT_DBaseAppend(SDB,1,BstF+j)
}
sS = BstF + BstLen # sS @ frame after replaced range
Di = Di + BstLen - 1 # Di + Bstlen @ next Di iteration
}
}
if(BstF == -1) {
RT_DBaseAppend(SDB,0,Di)
}
}
if(DstStop<Dst.FrameCount) {
(DEBUG) ? RT_DebugF("Using Dest frames %d -> %d",DstStop,Dst.Framecount-1,name=myName) : NOP
for(i=DstStop,Dst.Framecount-1)
{RT_DBaseAppend(SDB,0,i)} # Set DB records post DstStop(incl) to clip 0(Dst)
}
ReplacedRanges=0 ReplacedFrames=0
Records = RT_DBaseRecords(SDB) # o/p FrameCount
PrvClp=RT_DBaseGetField(SDB,0,0) PrvSFrm=RT_DBaseGetField(SDB,0,1) PrvFrm=PrvSFrm - 1 OutSFrm=0
for(rec=0,Records-1) {
Clp=RT_DBaseGetField(SDB,rec,0) Frm=RT_DBaseGetField(SDB,rec,1)
if(Clp != PrvClp || Frm != PrvFrm+1) {
# Switching clip and/or range
RT_WriteFile(Cmd,"%d %6d,%-6d # %6d,%-6d Len=%-6d",
\ PrvClp,PrvSFrm,(PrvSFrm==PrvFrm)?-1:PrvFrm,OutSFrm,rec-1,rec-OutSFrm,Append=True)
(DEBUG) ? RT_DebugF("Out[%6d,%-6d] Len=%-6d from %s[%6d,%-6d]",
\ OutSFrm,rec-1,rec-OutSFrm,(PrvClp==0)?"Dest":"Sour",PrvSFrm,PrvFrm,name=myName) : NOP
if(Prvclp==1) {
ReplacedRanges = ReplacedRanges + 1
ReplacedFrames = ReplacedFrames + rec-OutSFrm
}
PrvSFrm = Frm # new range start frame
OutSFrm = rec # new ouput start frame
}
PrvClp=Clp PrvFrm=Frm
}
# close final range
RT_WriteFile(Cmd,"%d %6d,%-6d # %6d,%-6d Len=%-6d",
\ PrvClp,PrvSFrm,(PrvSFrm==PrvFrm)?-1:PrvFrm,OutSFrm,Records-1,Records-OutSFrm,Append=True)
(DEBUG) ? RT_DebugF("Out[%6d,%-6d] Len=%-6d from %s[%6d,%-6d]",
\ OutSFrm,Records-1,Records-OutSFrm,(PrvClp==0)?"Dest":"Sour",PrvSFrm,PrvFrm,name=myName) : NOP
(DB=="") ? RT_FileDelete(SDB) : NOP # Dont delete output DBase if user supplied DB name (he/she wants it).
RT_FileDelete(QDB) RT_FileDelete(NDB)
if(DEBUG) {
TSTOP = RT_TimerHP
T = TSTOP - TIMSTART
OutFrms = Records - (Dst.FrameCount - (DstStop-DstStart)) # exclude non selected (copied) ranges from output count
RT_DebugF("\nRanges Replaced = %d Frames Replaced = %d",ReplacedRanges,ReplacedFrames,name=myName)
RT_DebugF("Total (Including QwikScan DBase creation) Time taken = %.2f secs (%.2f mins)",T,T/60.0,name=myName)
RT_DebugF("Frames Searched = %d FPS=%.2f : O/P FPS=%.2f",FSearched,FSearched/T,OutFrms/T,name=myName)
T = TSTOP - TIMSTART2
RT_DebugF("Excluding QwikScan DBase creation, Time taken = %.2f secs (%.2f mins)",T,T/60.0,name=myName)
RT_DebugF("Frames Searched = %d FPS=%.2f : O/P FPS=%.2f",FSearched,FSearched/T,OutFrms/T,name=myName)
}
""") # End Of GScript
}

[StainlessS]

Client script

Import("ReplaceCuts.avs")
Import("SpatialAlign.avs") # http://forum.doom9.org/showthread.php?p=1722142#post1722142

Function AutoContrast(clip c,Float "Strength",Float "Ignore") {
c myName="AutoContrast: "
Strength=Max(Min(Float(Default(Strength,0.5)),1.0),0.0)
Ignore = Default(Ignore,0.4)
CSMin = (IsRGB) ? 0 : 16 CSMax = (IsRGB) ? 255 : 235
Eval(RT_QueryLumaMinMax(X=8,Y=8,W=-8,H=-8,ignore=Ignore)) QLRng=QLMMMax-QLMMMin+1 CSRng=CSMax-CSMin+1
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH))
Doit = (ALMin!=CSMin || ALMax != CSMax)
(Doit) ? RT_DebugF("Levels(%3d,1.0,%3d,%d,%d,Coring=False)",ALMin,ALMax,CSMin,CSMax,name=myName) : NOP
(!Doit) ? RT_DebugF("Skipping Levels",name=myName) : NOP
(Doit) ? Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) : NOP
Return Last
}

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}

Function Stack4Delta(clip c1,clip c2,bool "gray") {
Gray =Default(gray,false)
c1=(gray) ? c1.GrayScale:c1
c2=(gray) ? c2.GrayScale:c2
L=StackVertical(c1,c2)
D1=ClipDelta(c1,c2)
D2=ClipDelta(c1,c2,AMP=True)
R=StackVertical(D1,D2)
StackHorizontal(L,R)
Return Last
}


############ FILE STUFF ##########
FINAL = False # True (AFTER ReplaceCuts) output final clip.
FN1 = "UnCut-hq.avi" # Higher rez 1024x584 @ 23.976 FPS
FN2 = "Cut-hq.avi" # 720x480 @ 23.970 FPS (ie FrameRate mismatch)

DEST_FN1 = False # True, FN1 is Dest file, else FN2
USE_BOB = False # True, Bob if at least one of the clips is interlaced, Else dont Bob, both Progressive
# May need to AssumeTFF or AssumeBFF for both clips prior to bobbing.
BLURD = True # Blur Detection Clips if True
BLUR_ARG = 1.58 # Blur arg if BLURD, Blurring will likely require different Thresh (0.0 -> 1.58)
ALIGN = True # Auto align clips
USE_ASSUMEFPS = False # Choose method to change FrameRate, True = AssumeFPS, False = ChangeFPS
CONTRAST = True # Auto Set Levels
AUD_SAMPLERATE= 48000 # Audio SampleRate
RAD = 4 # Arg to SpatialAlign()
######## ReplaceCuts CONFIG ########
CMD = "Cmd.Txt"
FLAGS = 1+2 # Condition flags(add together), All conditions set must succeed for each frame (ie AND)
# 1 = LC = Scaled Half range Pearsons Distance [ie Min((1.0-RT_LumaCorrelation * 255.0, 255.0)]
# 2 = LD = LumaDifference
# 4 = FD = FrameDifference
# 8 = PD = LumaPixelsDifferent (255.0 = 100%)

# Condition args, adjust for source (MUST use DebugView, and need adjust if change BLURD/BLUR_ARG)
LC = 6.0 # Pearsons Distance
LD = 6.0 # LumaDifference
FD = 6.0 # FrameDifference
PD = 10.0 # LumaPixelsDifferent
PdTHRESH = 6 # 2nd Arg to LumaPixelsDifferent

LUMATOL = 7.0 # See explanation for RT_QwikScan.
MATCHLEN = 25 # 1 or More, minimum number of frames that must match FLAGS selected conditions.
INSEQ = True # True, both clips share same sequences, false Search entire source clip for each MatchLen sequence.
CW = 1.0/3.0 # ChromaWeight for FrameDifference, ie FD = (1.0-ChromaWeight) * YDiff + ChromaWeight *((UDiff+VDiff)/2.0)

#####
D_FN = (DEST_FN1) ? FN1 : FN2
S_FN = (DEST_FN1) ? FN2 : FN1
######## Load Source Clips ####################
ORG_Sour = Avisource(S_FN).Trim(0,0).ConvertToYV24(interlaced=USE_BOB)
ORG_Dest = Avisource(D_FN).Trim(0,0).ConvertToYV24(interlaced=USE_BOB)
Sour = ORG_Sour
Dest = ORG_Dest
######## Bob to Remove Interlacing ############
Sour = (USE_BOB) ? Sour.Bob : Sour
Dest = (USE_BOB) ? Dest.Bob : Dest
######## Crop after Bob ############
# Use RoboCrop Thresh -64.0 as thin border line may remain on frame edge (-64 probably always OK for anime)
Sour = Sour.RoboCrop(WMod=1,HMOD=1,Laced=False,Thresh=-64.0)
Dest = Dest.RoboCrop(WMod=1,HMOD=1,Laced=False,Thresh=-64.0)
######## Make Same FrameRate ##################
DHiFPS = (Dest.FrameRate>Sour.FrameRate)
Sour = (!DHiFPS) ? Sour :(USE_ASSUMEFPS) ? Sour.AssumeFPS(Dest.FrameRate,sync_audio=True) : Sour.ChangeFPS(Dest.FrameRate,Linear=True)
Dest = (DHiFPS) ? Dest :(USE_ASSUMEFPS) ? Dest.AssumeFPS(Sour.FrameRate,sync_audio=True) : Dest.ChangeFPS(Sour.FrameRate,Linear=True)
######## MAKE SIMILAR Detection clip (resize to low dimension clip [Bilinear]) ########
DBIG = (Dest.Width > Sour.Width)
DestD = (DBIG) ? Dest.BilinearResize(Sour.Width,Sour.Height) : Dest # Detection clips to LOWER dim rez
SourD = (!DBIG) ? Sour.BilinearResize(Dest.Width,Dest.Height) : Sour
######## Auto Contrast #####
DestD = (CONTRAST) ? DestD.AutoContrast(1.0,1.0) : DestD
SourD = (CONTRAST) ? SourD.AutoContrast(1.0,1.0) : SourD
######## Align shifted clip #####
DestD = (ALIGN) ? SpatialAlign(DestD,SourD,Rad=RAD,Prefix="Crop_") : DestD
DestD = (ALIGN) ? DestD.Crop(Crop_L,Crop_T,-Crop_R,-Crop_B) : DestD
SourD = (ALIGN) ? SourD.Crop(Crop_L,Crop_T,-Crop_R,-Crop_B) : SourD
######## Get rid of any thin edge borders left over after RoboCrop and make mod 4 #####
DestD = DestD.Crop(4,4,(DestD.Width/4-2)*4,(DestD.Height/4-2)*4)
SourD = SourD.Crop(4,4,(SourD.Width/4-2)*4,(SourD.Height/4-2)*4)
DestD = (BLURD) ? DestD.Blur(BLUR_ARG) : DestD
SourD = (BLURD) ? SourD.Blur(BLUR_ARG) : SourD
#####
#####
#####
Dest = (!DBIG) ? Dest.Spline36Resize(Sour.Width,Sour.Height) : Dest # Output clips to the Higher rez (Resizer of your choice)
Sour = (DBIG) ? Sour.Spline36Resize(Dest.Width,Dest.Height) : Sour
Dest = Dest.ConvertAudioTo16bit.ReSampleAudio(AUD_SAMPLERATE)
Audio = Dest.KillVideo
Sour = Sour.ConvertAudioTo16bit.ReSampleAudio(AUD_SAMPLERATE)
WW=(Dest.Width + (4-1)) / 4 * 4
HH=(Dest.Height + (4-1)) / 4 * 4
Dest = Dest.AddBorders(0,0,WW-Dest.Width,HH-Dest.Height).ConvertToYV12
Sour = Sour.AddBorders(0,0,WW-Dest.Width,HH-Dest.Height).ConvertToYV12
#Return SourD.ConvertToYV12.Info
#Return DestD.ConvertToYV12.Info

(!FINAL) ? ReplaceCuts(DestD,SourD,Flags=FLAGS,lc=LC,ld=LD,fd=FD,pd=PD,PdThresh=PdTHRESH,LumaTol=LUMATOL,
\ MatchLen=MATCHLEN,InSeq=INSEQ,ChromaWeight=CW,Cmd=CMD) : NOP

#Return MessageClip("DONE")

SHOW=(!FINAL)

Nick=" Dest = 0 ; Sour = 1"

Prune(Dest,Sour,Cmd=CMD,NickName=Nick,FadeSplice=True,Show=SHOW) # FadeSplice, avoid clicks & cracks in audio
AudioDub(Last,Audio) # Restore original audio (avoid volume mismatch)
(!FINAL) ? Stack4Delta(Last,Dest) : NOP
Return Last

[StainlessS]

Debug

AutoContrast: Skipping Levels
AutoContrast: Levels( 17,1.0,234,16,235,Coring=False)
SpatialAlign: Not already temporally aligned (Worst LumaDif of all tested frames=83.174)
SpatialAlign: Searching for temporal alignment
RT_QwikScanCreate: RT_QwikScanCreateDB() by StainlessS
RT_QwikScanCreate: Total time = 43.06 seconds (0.72 mins)
SpatialAlign: Scanning 50 frames (1.9%) of Fix clip @ LumaTol==5 LdTh=12.00
SpatialAlign: Fix[106] Match Frame found at Src[173] MaxLumaDif=4.640 (over 50 frames)
SpatialAlign: Shifting clip HDir=-1 VDir=1 (orig correlation=0.990456 shifted=0.995786
SpatialAlign: Global Crop_L=0 Global Crop_T=1 Global Crop_R=1 Global Crop_B=0
RT_QwikScanCreate: RT_QwikScanCreateDB() by StainlessS
RT_QwikScanCreate: Total time = 50.68 seconds (0.84 mins)
ReplaceCuts: Dst[ 3,699 ] Src[ 70,766 ] Len= 697 LC= 0.724 LD= 3.486 FD= 2.632 PD= 17.316 : Fail@ LC= 65.650 LD= 21.323 FD= 15.617 PD=159.951
ReplaceCuts: Dst[ 701,729 ] Src[ 767,795 ] Len= 29 LC= 0.838 LD= 2.534 FD= 2.202 PD= 15.216 : Fail@ LC= 6.355 LD= 1.503 FD= 1.341 PD= 0.715
ReplaceCuts: Dst[ 739,1416] Src[ 806,1483] Len= 678 LC= 1.610 LD= 3.836 FD= 2.963 PD= 43.234 : Fail@ LC=255.000 LD= 44.528 FD= 32.835 PD=205.277
ReplaceCuts: Dst[1501,1713] Src[1567,1779] Len= 213 LC= 1.400 LD= 3.566 FD= 2.788 PD= 28.315 : Fail@ LC=120.031 LD= 25.471 FD= 17.900 PD=103.506
ReplaceCuts: Dst[1714,1996] Src[1781,2063] Len= 283 LC= 1.533 LD= 3.832 FD= 2.835 PD= 22.360 : Fail@ LC=255.000 LD= 14.161 FD= 10.247 PD=244.266
ReplaceCuts: Dst[2067,2132] Src[2108,2173] Len= 66 LC= 2.983 LD= 3.484 FD= 2.533 PD= 27.880 : Fail@ LC= 13.535 LD= 3.149 FD= 2.391 PD= 20.081
ReplaceCuts: Dst[2197,2234] Src[2239,2276] Len= 38 LC= 3.000 LD= 3.051 FD= 2.368 PD= 21.219 : Fail@ LC=255.000 LD= 62.001 FD= 43.710 PD=241.407
ReplaceCuts: Dst[2236,2294] Src[2277,2335] Len= 59 LC= 3.033 LD= 2.677 FD= 2.106 PD= 20.018 : Fail@ LC=214.055 LD= 49.992 FD= 35.412 PD=244.243
ReplaceCuts: Dst[2305,2390] Src[2347,2432] Len= 86 LC= 5.947 LD= 3.124 FD= 2.393 PD= 31.176 : Fail@ LC= 22.429 LD= 9.406 FD= 6.997 PD= 94.320
ReplaceCuts: Dst[2435,2557] Src[2476,2598] Len= 123 LC= 3.205 LD= 4.527 FD= 3.440 PD= 48.502 : Fail@ LC= 9.425 LD= 6.296 FD= 4.524 PD= 48.187
ReplaceCuts: Out[ 0,2 ] Len=3 from Dest[ 0,2 ]
ReplaceCuts: Out[ 3,699 ] Len=697 from Sour[ 70,766 ]
ReplaceCuts: Out[ 700,700 ] Len=1 from Dest[ 700,700 ]
ReplaceCuts: Out[ 701,729 ] Len=29 from Sour[ 767,795 ]
ReplaceCuts: Out[ 730,738 ] Len=9 from Dest[ 730,738 ]
ReplaceCuts: Out[ 739,1416 ] Len=678 from Sour[ 806,1483 ]
ReplaceCuts: Out[ 1417,1500 ] Len=84 from Dest[ 1417,1500 ]
ReplaceCuts: Out[ 1501,1713 ] Len=213 from Sour[ 1567,1779 ]
ReplaceCuts: Out[ 1714,1996 ] Len=283 from Sour[ 1781,2063 ]
ReplaceCuts: Out[ 1997,2066 ] Len=70 from Dest[ 1997,2066 ]
ReplaceCuts: Out[ 2067,2132 ] Len=66 from Sour[ 2108,2173 ]
ReplaceCuts: Out[ 2133,2196 ] Len=64 from Dest[ 2133,2196 ]
ReplaceCuts: Out[ 2197,2234 ] Len=38 from Sour[ 2239,2276 ]
ReplaceCuts: Out[ 2235,2235 ] Len=1 from Dest[ 2235,2235 ]
ReplaceCuts: Out[ 2236,2294 ] Len=59 from Sour[ 2277,2335 ]
ReplaceCuts: Out[ 2295,2304 ] Len=10 from Dest[ 2295,2304 ]
ReplaceCuts: Out[ 2305,2390 ] Len=86 from Sour[ 2347,2432 ]
ReplaceCuts: Out[ 2391,2434 ] Len=44 from Dest[ 2391,2434 ]
ReplaceCuts: Out[ 2435,2557 ] Len=123 from Sour[ 2476,2598 ]
ReplaceCuts: Out[ 2558,2640 ] Len=83 from Dest[ 2558,2640 ]
ReplaceCuts:
ReplaceCuts: Ranges Replaced = 10 Frames Replaced = 2272
ReplaceCuts: Total (Including QwikScan DBase creation) Time taken = 330.36 secs (5.51 mins)
ReplaceCuts: Frames Searched = 270745 FPS=819.54 : O/P FPS=7.99
ReplaceCuts: Excluding QwikScan DBase creation, Time taken = 279.66 secs (4.66 mins)
ReplaceCuts: Frames Searched = 270745 FPS=968.11 : O/P FPS=9.44

[SilSic]

I tried it again but I got this error again:
Script error: syntax error
(.../plugins/spatialalign.avs, line 1, column 19)
(.../plugins/replacecuts.avs, line 2)

I guess the function is missing in spatialalign.avs

Oh about the fps, indeed one clip is 29.976 and the other is 29.97 FPS. I didn't noticed that, could be the very reason for too many founds.

[creaothceann]

Will not fit in single post 16KB limit.

There's also Pastebin.

[StainlessS]

Posts #41 to #44 updated, zip in post 41 (includes SpatialAlign.avs).
I tried it again but I got this error again:
Script error: syntax error
(.../plugins/spatialalign.avs, line 1, column 19)
(.../plugins/replacecuts.avs, line 2)

I guess the function is missing in spatialalign.avs

EDIT: It was complaining about the extra text you added ie ".../plugins/".
EDIT: Changed my mind, think maybe you messed up copying the text into avs script, you do not put avs scripts in
autoload plugin directory, only plugins and *.avsi files.
You need SpatialAlign.avs in same directory as client script, same with ReplaceCuts.avs
(or rename to avsi extensions and put in plugs, then remove the Import() lines from client script).



The link was in the script, ie
Import("ReplaceCuts.avs")
Import("SpatialAlign.avs") # http://forum.doom9.org/showthread.php?p=1722142#post1722142

I dont like maintaining a script in multiple places, have put zip in post 41, but you may need to get updates from that link.
The zip will likely be temporary and deleted soon.

Done a bit more tinkering.

[Motenai Yoda]

@StainlessS there are so much differences between your script and fsinapsi's tool?
http://fsinapsi.altervista.org

[StainlessS]

Thank you Motenai Yoda, looks interesting.

I found a paper after finishing RT_Qwikscan which at first I thought might be similar, but they use info from already compressed
video (without decompression) to do fast sequence matching, looks like FFmatch may do the same.

The Paper I found named "novel scheme for fast and efficient video sequence matching using compact signatures"

https://www.google.co.uk/?gws_rd=ssl#q=A+novel+scheme+for+fast+and+efficient+video+sequence+matching+using+compact+signatures

EDIT: SilSic, Take a look at link pointed to by Motenai Yoda, likely to be much faster than anything I could do in Avisynth.

[SilSic]

Got finally time to check it out. Thank you Motenai!
The tool of fsinapsi is a lot different from Stainlesss.
FFMatch supports only avi and mp4 files and need converting both clips to wav in addition for wav only output.
That author shows in youtube how much adjustment clips still need even with the best settings.
Unfortunately user guide is only in italian and no updates since 2012.

Stainless script is amazing it's absolutely flexible, and here it comes even more accurately. It's possible to use almost any kind of videos
and have both video and audio output. You can change and even add new features as you wish. There will be never any tool giving you so much freedom about how to handle your clips as Stainless Replacecuts. It's my odds-on favourite!
Hats off Stainless you are unbeatable!

I figure out the main problem of FFmatch and Replacecuts which is the very reason for misleadingly to non sync.
The "Uncut clip" is mandatory as reference and keeps always untouched.
The "cut clip" have to be filled with frames from untouched "uncut clip" never the other way round.
But sometimes the "cut clip" have more frames(eg deinterlacing, longer transits...) than "uncut clip" and that's a problem.
Which will cause certainly "misleadingly to non sync". Therefore the "cut clip" itself need cuts of any additional frames(which doesn't exist in "uncut clip").
The "cut clip" have eg just 4 frames in addition(could be 1-50frames or even more) as soon as eg 4 frames has been cut out, the "cut clip" gets
sync with the "uncut clip" and there is no need to take frames from "uncut clip" too early as it is already sync.

Could you Stainless implement this very feature(with on/off switch) in Replacecuts please?
It's a very important feature which is missing for the completion of Replacecuts.

Also important a timecode range which will be only processed or skipped for both clips eg timecode 00:03:10-00:10:01 to be skipped and
00:12:10-00:18:01 to be skipped and 00:25:10-00:39:01, this will reduce significantly processing time in cases when I know where the cuts are mostly to be.

Could you also avoid rescanning when fingerprint data is already made.
I know it can be saved on hdd but there is no option to call the fingerprint data for reuse.