raffriff42
3rd December 2015, 03:23
I had to sync two clips recently where one of the clips sometimes ran way ahead, other times way behind. To get it to work I needed to sync the two clips at many intermediate points. After much @#$%& I arrived at this solution. It works well enough for me! Hope you like it too! Thanks to Gavino for GScript (http://forum.doom9.org/showthread.php?t=147846), which made this code possible, or at least much, much easier!
> > > Not a script for beginners! < < <
Usage: Frame rates of the 2 clips must match! (use ConvertFPS (http://avisynth.nl/index.php/FPS)etc as required)
Intra-frame-only (https://en.wikipedia.org/wiki/Intra-frame) sources are a plus! Without them, watch out for possible out-of-order frames.
Set bypass=true (in blue below) to disable any speed changes.
(note - I use "1>0" for "true" and "0>0" for "false" - it saves typing)
Create two comma-delimited lists of frame numbers (sync points), one for each video;
(clips must be approximately in sync @ frame 0)
(first sync point must be > 0 for both clips)
Get sync at any useful point which is common to the 2 clips;
(how you get these sync points depends on the source material - the classic method is to use a clapperboard (https://en.wikipedia.org/wiki/Clapperboard))
Once you have found a your sync point, add the frame numbers to each list.
Add more sync points until clip synchronization is "good enough"
(note - since we are doing linear interpolation, correcting for nonlinear speed variations can never be exact)
(note - sync points must be monotonic - in increasing order)
(any video remaining after the last sync point simply plays at normal speed)
Set bypass=false to enable the speed changes and see the results.
Review, adjust and maybe add sync points as needed.
There will be some unavoidable audio artifacts in the sync-corrected clip; try to use audio from the reference clip only.
#Import("uusync.avsi")
clpRef=XXSource("...")
clpSyn=XXSource("...") ## we want to match sync to clpRef
## two lists of frame numbers, one for each video
sSyn="207,1182,2650,3251,4021,4577,5039,6416"
sRef="212,1218,2715,3312,4082,4640,5100,6451"
clpSyn_2 = UUSyncSegments(clpSyn, clpRef, sSyn, sRef, debug=1>0, bypass=0>0)
return StackHorizontal(clpRef, clpSyn_2)
Debug output is useful for finding what the nearest sync points are and what speed adjustment is being applied.
The clips in the samples below are still out by a frame or two; that's because I rushed getting my sync points.
(this was a test video with random speed changes intentionally added)
https://www.dropbox.com/s/ove5gvn1053p12g/UUSyncSegments-demo1-4060.jpg?raw=1
https://www.dropbox.com/s/kz73u95tfw0z2iw/UUSyncSegments-demo1-5784.jpg?raw=1
uusync.avsi# http://forum.doom9.org/showthread.php?t=147846
#LoadPlugin("GScript\GScript.dll")
GScript("""
#######################################
### sync 'syn' clip to 'ref' based on multiple common markers
## requires GScript http://forum.doom9.org/showthread.php?t=147846
##
## NOTE:
## - frame rates must match (use AssumeFPS/ChangeFPS etc)
## - clips must be approximately in sync @ frame 0
## - first sync point must be > 0 for both clips
##
## @ syn - clip to be synced
## @ ref - reference clip
## @ ssyn - 'syn' sync points (comma-delimited list)
## @ sref - 'ref' sync points
## @ pitchfix - if true, maintain audio pitch with speed
## (default false; allow pitch to rise or fall)
## @ noblend - if true, do not blend frames (default true)
## @ debug - if true, show debugging info (default false)
## @ bypass - if true, pass 'syn' unchanged to output
##
## @ return clip 'syn' with speed adjusted to sync to clip 'ref'
##
## @version 1.0 22-Oct-2015 raffriff42
## @version 1.1 2-Dec-2015 head & tail handling; debug output; pitchfix
## @version 1.2 3-Dec-2015 handling of first sync point
##
function UUSyncSegments(clip syn, clip ref, string sSyn, string sref,
\ bool "pitchfix", bool "noblend", bool "debug", bool "bypass")
{
Assert(ref.FrameRateNumerator==syn.FrameRateNumerator
\ && ref.FrameRateDenominator==syn.FrameRateDenominator,
\ "UUSyncSegments: clips 'syn' and 'ref' must have same framerate")
pitchfix = Default(pitchfix, false)
noblend = Default(noblend, true)
debug = Default(debug, false)
bypass = Default(bypass, false)
## chop list into "_L" (left) and "_R" (right) parts
ps = FindStr(ssyn, ",")
pr = FindStr(sref, ",")
ssyn_L = (ps==0) ? ssyn : LeftStr(ssyn, ps-1)
ssyn_R = (ps==0) ? "" : MidStr(ssyn, ps+1)
sref_L = (pr==0) ? sref : LeftStr(sref, pr-1)
sref_R = sref
## get first sync point
fsyn_L = Round(Value(ssyn_L))
fref_L = Round(Value(sref_L))
Assert(fsyn_L > 0 && fref_L > 0,
\ "UUSyncSegments: first sync point must be > 0 for both clips")
Assert(fsyn_L < syn.FrameCount,
\ "UUSyncSegments: 'syn' frame number >= FrameCount")
Assert(fref_L < ref.FrameCount,
\ "UUSyncSegments: 'ref' frame number >= FrameCount")
syn = (!debug) ? syn
\ : syn.ShowFrameNumber(x=16, y=syn.Height-16)
factor = Float(fsyn_L) / Float(fref_L)
R = syn.Trim(0, fsyn_L-1)
\ .UUChangeSpeed2(factor, pitchfix, noblend)
R = (!debug) ? R : R.Subtitle(
\ "syn in, out: 0, " + String(fsyn_L)
\ + "\nref in, out: 0, " + String(fref_L)
\ + "\nspeed mult.: " + String(factor, "%0.3f")
\ , font="courier", size=R.Height/16, lsp=0)
count = 256 ## infinite loop prevention
while (count>0 && StrLen(ssyn_R)>0) {
fsyn_L = Round(Value(ssyn_L))
fsyn_R = Round(Value(ssyn_R))
pr = FindStr(sref_R, ",")
sref_L = (pr==0) ? sref_R : LeftStr(sref_R, pr-1)
sref_R = (pr==0) ? "" : MidStr(sref_R, pr+1)
fref_L = Round(Value(sref_L))
fref_R = Round(Value(sref_R))
Assert(fsyn_L < syn.FrameCount && fsyn_R < syn.FrameCount,
\ "UUSyncSegments: 'syn' frame number >= FrameCount")
Assert(fref_L < ref.FrameCount && fref_R < ref.FrameCount,
\ "UUSyncSegments: 'ref' frame number >= FrameCount")
Assert(fsyn_R > fsyn_L,
\ "UUSyncSegments: 'syn' sync point " + String(count) + " <= previous")
Assert(fref_R > fref_L,
\ "UUSyncSegments: 'ref' sync point " + String(count) + " <= previous")
factor = Float(fsyn_R - fsyn_L) / Float(fref_R - fref_L)
S = syn.Trim(fsyn_L, fsyn_R-1)
\ .UUChangeSpeed2(factor, pitchfix, noblend)
S = (!debug) ? S : S.Subtitle(
\ "syn in, out: " + String(fsyn_L) + "," + String(fsyn_R)
\ + "\nref in, out: " + String(fref_L) + "," + String(fref_R)
\ + "\nspeed mult.: " + String(factor, "%0.3f")
\ , font="courier", size=S.Height/16, lsp=0)
R = R ++ S
ps = FindStr(ssyn_R, ",")
ssyn_L = (ps==0) ? ssyn_R : LeftStr(ssyn_R, ps-1)
ssyn_R = (ps==0) ? "" : MidStr(ssyn_R, ps+1)
count = count - 1
}
R = R ++ syn.Trim(fsyn_R, 0)
return (bypass) ? syn : R
}
""")
#######################################
### change speed over a wide range
### with frame blending in fast forward by default.
##
## @ factor - 0.33 for 1/3 speed, etc;
## min = 0.001; max = min(FrameCount, 1000)
## (if > FrameCount, duration would be 0) ;
## if argument is out of range, an error occurs.
##
## @ pitchfix - if true, maintain audio pitch with speed
## (default false; allow pitch to rise or fall)
##
## @ noblend - if true, never blend frames (default false)
##
## @ version 1.0 24-Feb-2014 raffriff42
##
function UUChangeSpeed2(
\ clip C, float factor, bool "pitchfix", bool "noblend")
{
Assert(factor>=0.001,
\ "UUChangeSpeed2 bad argument: factor<0.001")
Assert(factor<=1000.0,
\ "UUChangeSpeed2 bad argument: factor>1000")
Assert(factor<=C.FrameCount,
\ "UUChangeSpeed2 bad argument: factor>FrameCount")
pitchfix = Default(pitchfix, false)
noblend = Default(noblend, false)
AS = C.KillAudio.AssumeFPS(C.FrameRate * factor)
ASI = C.KillAudio.AssumeFPS(C.FrameRate / factor)
temprad = Round(0.5 * factor)
TS = (factor<1.0 || temprad==0 || noblend)
\ ? AS
\ : Overlay(
\ AS,
\ AS.TemporalSoften(
\ temprad, 255, 255, 48, 2),
\ opacity=0.7)
ffx = Min(Max(0.0, 25.0/factor), 100.0)
(factor<1.5 && noblend==false)
\ ? AS.ConvertFPS(C)
\ : TS.ChangeFPS(C.Framerate, linear=(factor<10.0))
A = (pitchfix)
\ ? C.KillVideo.ConvertAudioToFloat.TimeStretch(tempo=factor*100.0)
\ : C.KillVideo.ConvertAudioToFloat.TimeStretch(rate=factor*100.0)
A = (C.AudioBits==32) ? A.ConvertAudioTo32bit
\ : (C.AudioBits==24) ? A.ConvertAudioTo24bit
\ : (C.AudioBits==16) ? A.ConvertAudioTo16bit
\ : (C.AudioBits==8) ? A.ConvertAudioTo8bit
\ : A
return (Abs(factor-1.0) < (0.5/C.FrameCount)) ? C
\ : (C.HasAudio==false) ? Last
\ : Last.AudioDub(A)
}
> > > Not a script for beginners! < < <
Usage: Frame rates of the 2 clips must match! (use ConvertFPS (http://avisynth.nl/index.php/FPS)etc as required)
Intra-frame-only (https://en.wikipedia.org/wiki/Intra-frame) sources are a plus! Without them, watch out for possible out-of-order frames.
Set bypass=true (in blue below) to disable any speed changes.
(note - I use "1>0" for "true" and "0>0" for "false" - it saves typing)
Create two comma-delimited lists of frame numbers (sync points), one for each video;
(clips must be approximately in sync @ frame 0)
(first sync point must be > 0 for both clips)
Get sync at any useful point which is common to the 2 clips;
(how you get these sync points depends on the source material - the classic method is to use a clapperboard (https://en.wikipedia.org/wiki/Clapperboard))
Once you have found a your sync point, add the frame numbers to each list.
Add more sync points until clip synchronization is "good enough"
(note - since we are doing linear interpolation, correcting for nonlinear speed variations can never be exact)
(note - sync points must be monotonic - in increasing order)
(any video remaining after the last sync point simply plays at normal speed)
Set bypass=false to enable the speed changes and see the results.
Review, adjust and maybe add sync points as needed.
There will be some unavoidable audio artifacts in the sync-corrected clip; try to use audio from the reference clip only.
#Import("uusync.avsi")
clpRef=XXSource("...")
clpSyn=XXSource("...") ## we want to match sync to clpRef
## two lists of frame numbers, one for each video
sSyn="207,1182,2650,3251,4021,4577,5039,6416"
sRef="212,1218,2715,3312,4082,4640,5100,6451"
clpSyn_2 = UUSyncSegments(clpSyn, clpRef, sSyn, sRef, debug=1>0, bypass=0>0)
return StackHorizontal(clpRef, clpSyn_2)
Debug output is useful for finding what the nearest sync points are and what speed adjustment is being applied.
The clips in the samples below are still out by a frame or two; that's because I rushed getting my sync points.
(this was a test video with random speed changes intentionally added)
https://www.dropbox.com/s/ove5gvn1053p12g/UUSyncSegments-demo1-4060.jpg?raw=1
https://www.dropbox.com/s/kz73u95tfw0z2iw/UUSyncSegments-demo1-5784.jpg?raw=1
uusync.avsi# http://forum.doom9.org/showthread.php?t=147846
#LoadPlugin("GScript\GScript.dll")
GScript("""
#######################################
### sync 'syn' clip to 'ref' based on multiple common markers
## requires GScript http://forum.doom9.org/showthread.php?t=147846
##
## NOTE:
## - frame rates must match (use AssumeFPS/ChangeFPS etc)
## - clips must be approximately in sync @ frame 0
## - first sync point must be > 0 for both clips
##
## @ syn - clip to be synced
## @ ref - reference clip
## @ ssyn - 'syn' sync points (comma-delimited list)
## @ sref - 'ref' sync points
## @ pitchfix - if true, maintain audio pitch with speed
## (default false; allow pitch to rise or fall)
## @ noblend - if true, do not blend frames (default true)
## @ debug - if true, show debugging info (default false)
## @ bypass - if true, pass 'syn' unchanged to output
##
## @ return clip 'syn' with speed adjusted to sync to clip 'ref'
##
## @version 1.0 22-Oct-2015 raffriff42
## @version 1.1 2-Dec-2015 head & tail handling; debug output; pitchfix
## @version 1.2 3-Dec-2015 handling of first sync point
##
function UUSyncSegments(clip syn, clip ref, string sSyn, string sref,
\ bool "pitchfix", bool "noblend", bool "debug", bool "bypass")
{
Assert(ref.FrameRateNumerator==syn.FrameRateNumerator
\ && ref.FrameRateDenominator==syn.FrameRateDenominator,
\ "UUSyncSegments: clips 'syn' and 'ref' must have same framerate")
pitchfix = Default(pitchfix, false)
noblend = Default(noblend, true)
debug = Default(debug, false)
bypass = Default(bypass, false)
## chop list into "_L" (left) and "_R" (right) parts
ps = FindStr(ssyn, ",")
pr = FindStr(sref, ",")
ssyn_L = (ps==0) ? ssyn : LeftStr(ssyn, ps-1)
ssyn_R = (ps==0) ? "" : MidStr(ssyn, ps+1)
sref_L = (pr==0) ? sref : LeftStr(sref, pr-1)
sref_R = sref
## get first sync point
fsyn_L = Round(Value(ssyn_L))
fref_L = Round(Value(sref_L))
Assert(fsyn_L > 0 && fref_L > 0,
\ "UUSyncSegments: first sync point must be > 0 for both clips")
Assert(fsyn_L < syn.FrameCount,
\ "UUSyncSegments: 'syn' frame number >= FrameCount")
Assert(fref_L < ref.FrameCount,
\ "UUSyncSegments: 'ref' frame number >= FrameCount")
syn = (!debug) ? syn
\ : syn.ShowFrameNumber(x=16, y=syn.Height-16)
factor = Float(fsyn_L) / Float(fref_L)
R = syn.Trim(0, fsyn_L-1)
\ .UUChangeSpeed2(factor, pitchfix, noblend)
R = (!debug) ? R : R.Subtitle(
\ "syn in, out: 0, " + String(fsyn_L)
\ + "\nref in, out: 0, " + String(fref_L)
\ + "\nspeed mult.: " + String(factor, "%0.3f")
\ , font="courier", size=R.Height/16, lsp=0)
count = 256 ## infinite loop prevention
while (count>0 && StrLen(ssyn_R)>0) {
fsyn_L = Round(Value(ssyn_L))
fsyn_R = Round(Value(ssyn_R))
pr = FindStr(sref_R, ",")
sref_L = (pr==0) ? sref_R : LeftStr(sref_R, pr-1)
sref_R = (pr==0) ? "" : MidStr(sref_R, pr+1)
fref_L = Round(Value(sref_L))
fref_R = Round(Value(sref_R))
Assert(fsyn_L < syn.FrameCount && fsyn_R < syn.FrameCount,
\ "UUSyncSegments: 'syn' frame number >= FrameCount")
Assert(fref_L < ref.FrameCount && fref_R < ref.FrameCount,
\ "UUSyncSegments: 'ref' frame number >= FrameCount")
Assert(fsyn_R > fsyn_L,
\ "UUSyncSegments: 'syn' sync point " + String(count) + " <= previous")
Assert(fref_R > fref_L,
\ "UUSyncSegments: 'ref' sync point " + String(count) + " <= previous")
factor = Float(fsyn_R - fsyn_L) / Float(fref_R - fref_L)
S = syn.Trim(fsyn_L, fsyn_R-1)
\ .UUChangeSpeed2(factor, pitchfix, noblend)
S = (!debug) ? S : S.Subtitle(
\ "syn in, out: " + String(fsyn_L) + "," + String(fsyn_R)
\ + "\nref in, out: " + String(fref_L) + "," + String(fref_R)
\ + "\nspeed mult.: " + String(factor, "%0.3f")
\ , font="courier", size=S.Height/16, lsp=0)
R = R ++ S
ps = FindStr(ssyn_R, ",")
ssyn_L = (ps==0) ? ssyn_R : LeftStr(ssyn_R, ps-1)
ssyn_R = (ps==0) ? "" : MidStr(ssyn_R, ps+1)
count = count - 1
}
R = R ++ syn.Trim(fsyn_R, 0)
return (bypass) ? syn : R
}
""")
#######################################
### change speed over a wide range
### with frame blending in fast forward by default.
##
## @ factor - 0.33 for 1/3 speed, etc;
## min = 0.001; max = min(FrameCount, 1000)
## (if > FrameCount, duration would be 0) ;
## if argument is out of range, an error occurs.
##
## @ pitchfix - if true, maintain audio pitch with speed
## (default false; allow pitch to rise or fall)
##
## @ noblend - if true, never blend frames (default false)
##
## @ version 1.0 24-Feb-2014 raffriff42
##
function UUChangeSpeed2(
\ clip C, float factor, bool "pitchfix", bool "noblend")
{
Assert(factor>=0.001,
\ "UUChangeSpeed2 bad argument: factor<0.001")
Assert(factor<=1000.0,
\ "UUChangeSpeed2 bad argument: factor>1000")
Assert(factor<=C.FrameCount,
\ "UUChangeSpeed2 bad argument: factor>FrameCount")
pitchfix = Default(pitchfix, false)
noblend = Default(noblend, false)
AS = C.KillAudio.AssumeFPS(C.FrameRate * factor)
ASI = C.KillAudio.AssumeFPS(C.FrameRate / factor)
temprad = Round(0.5 * factor)
TS = (factor<1.0 || temprad==0 || noblend)
\ ? AS
\ : Overlay(
\ AS,
\ AS.TemporalSoften(
\ temprad, 255, 255, 48, 2),
\ opacity=0.7)
ffx = Min(Max(0.0, 25.0/factor), 100.0)
(factor<1.5 && noblend==false)
\ ? AS.ConvertFPS(C)
\ : TS.ChangeFPS(C.Framerate, linear=(factor<10.0))
A = (pitchfix)
\ ? C.KillVideo.ConvertAudioToFloat.TimeStretch(tempo=factor*100.0)
\ : C.KillVideo.ConvertAudioToFloat.TimeStretch(rate=factor*100.0)
A = (C.AudioBits==32) ? A.ConvertAudioTo32bit
\ : (C.AudioBits==24) ? A.ConvertAudioTo24bit
\ : (C.AudioBits==16) ? A.ConvertAudioTo16bit
\ : (C.AudioBits==8) ? A.ConvertAudioTo8bit
\ : A
return (Abs(factor-1.0) < (0.5/C.FrameCount)) ? C
\ : (C.HasAudio==false) ? Last
\ : Last.AudioDub(A)
}