Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd December 2015, 03:23   #1  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,375
[solved] syncing clips with speed variation

I had to sync two clips recently where one of the clips sometimes ran way ahead, other times way behind. To get it to work I needed to sync the two clips at many intermediate points. After much @#$%& I arrived at this solution. It works well enough for me! Hope you like it too! Thanks to Gavino for GScript, which made this code possible, or at least much, much easier!

> > > Not a script for beginners! < < <

Usage:
  • Frame rates of the 2 clips must match! (use ConvertFPS etc as required)
  • Intra-frame-only sources are a plus! Without them, watch out for possible out-of-order frames.
  1. Set bypass=true (in blue below) to disable any speed changes.
    (note - I use "1>0" for "true" and "0>0" for "false" - it saves typing)
  2. Create two comma-delimited lists of frame numbers (sync points), one for each video;
    (clips must be approximately in sync @ frame 0)
    (first sync point must be > 0 for both clips)
  3. Get sync at any useful point which is common to the 2 clips;
    (how you get these sync points depends on the source material - the classic method is to use a clapperboard)
  4. Once you have found a your sync point, add the frame numbers to each list.
  5. Add more sync points until clip synchronization is "good enough"
    (note - since we are doing linear interpolation, correcting for nonlinear speed variations can never be exact)
    (note - sync points must be monotonic - in increasing order)
    (any video remaining after the last sync point simply plays at normal speed)
  6. Set bypass=false to enable the speed changes and see the results.
  7. Review, adjust and maybe add sync points as needed.
  8. There will be some unavoidable audio artifacts in the sync-corrected clip; try to use audio from the reference clip only.
Code:
#Import("uusync.avsi")

clpRef=XXSource("...")
clpSyn=XXSource("...") ## we want to match sync to clpRef

## two lists of frame numbers, one for each video
sSyn="207,1182,2650,3251,4021,4577,5039,6416"
sRef="212,1218,2715,3312,4082,4640,5100,6451"

clpSyn_2 = UUSyncSegments(clpSyn, clpRef, sSyn, sRef, debug=1>0, bypass=0>0)
return StackHorizontal(clpRef, clpSyn_2)
Debug output is useful for finding what the nearest sync points are and what speed adjustment is being applied.

The clips in the samples below are still out by a frame or two; that's because I rushed getting my sync points.
(this was a test video with random speed changes intentionally added)





uusync.avsi
Code:
# http://forum.doom9.org/showthread.php?t=147846
#LoadPlugin("GScript\GScript.dll")

GScript("""
#######################################
### sync 'syn' clip to 'ref' based on multiple common markers
## requires GScript http://forum.doom9.org/showthread.php?t=147846
##
## NOTE: 
## - frame rates must match (use AssumeFPS/ChangeFPS etc)
## - clips must be approximately in sync @ frame 0
## - first sync point must be > 0 for both clips
##
## @ syn     - clip to be synced
## @ ref     - reference clip
## @ ssyn    - 'syn' sync points (comma-delimited list)
## @ sref    - 'ref' sync points
## @ pitchfix - if true, maintain audio pitch with speed 
##                (default false; allow pitch to rise or fall)
## @ noblend - if true, do not blend frames (default true)
## @ debug   - if true, show debugging info (default false)
## @ bypass  - if true, pass 'syn' unchanged to output
##
## @ return clip 'syn' with speed adjusted to sync to clip 'ref'
##
## @version 1.0 22-Oct-2015 raffriff42
## @version 1.1  2-Dec-2015 head & tail handling; debug output; pitchfix 
## @version 1.2  3-Dec-2015 handling of first sync point
##
function UUSyncSegments(clip syn, clip ref, string sSyn, string sref, 
\           bool "pitchfix", bool "noblend", bool "debug", bool "bypass")
{
    Assert(ref.FrameRateNumerator==syn.FrameRateNumerator
    \   && ref.FrameRateDenominator==syn.FrameRateDenominator,
    \     "UUSyncSegments: clips 'syn' and 'ref' must have same framerate")

    pitchfix = Default(pitchfix, false)
    noblend  = Default(noblend, true)
    debug    = Default(debug, false)
    bypass   = Default(bypass, false)

    ## chop list into "_L" (left) and "_R" (right) parts
    ps     = FindStr(ssyn, ",")
    pr     = FindStr(sref, ",")
    ssyn_L = (ps==0) ? ssyn : LeftStr(ssyn, ps-1)
    ssyn_R = (ps==0) ?  ""  : MidStr(ssyn, ps+1)
    sref_L = (pr==0) ? sref : LeftStr(sref, pr-1)
    sref_R = sref
    
    ## get first sync point
    fsyn_L = Round(Value(ssyn_L))
    fref_L = Round(Value(sref_L))

    Assert(fsyn_L > 0 && fref_L > 0, 
    \   "UUSyncSegments: first sync point must be > 0 for both clips")
    Assert(fsyn_L < syn.FrameCount, 
    \   "UUSyncSegments: 'syn' frame number >= FrameCount")
    Assert(fref_L < ref.FrameCount, 
    \   "UUSyncSegments: 'ref' frame number >= FrameCount")

    syn = (!debug) ? syn 
    \   : syn.ShowFrameNumber(x=16, y=syn.Height-16)

    factor = Float(fsyn_L) / Float(fref_L) 
    R = syn.Trim(0, fsyn_L-1)
    \      .UUChangeSpeed2(factor, pitchfix, noblend)

    R = (!debug) ? R : R.Subtitle(
    \        "syn in, out: 0, " + String(fsyn_L)
    \    + "\nref in, out: 0, " + String(fref_L)
    \    + "\nspeed mult.: " + String(factor, "%0.3f")
    \    , font="courier", size=R.Height/16, lsp=0)

    count = 256 ## infinite loop prevention
    while (count>0 && StrLen(ssyn_R)>0) {

        fsyn_L = Round(Value(ssyn_L))
        fsyn_R = Round(Value(ssyn_R))

        pr     = FindStr(sref_R, ",")
        sref_L = (pr==0) ? sref_R : LeftStr(sref_R, pr-1)
        sref_R = (pr==0) ?   ""   : MidStr(sref_R, pr+1)

        fref_L = Round(Value(sref_L))
        fref_R = Round(Value(sref_R))

        Assert(fsyn_L < syn.FrameCount && fsyn_R < syn.FrameCount, 
        \   "UUSyncSegments: 'syn' frame number >= FrameCount")
        Assert(fref_L < ref.FrameCount && fref_R < ref.FrameCount, 
        \   "UUSyncSegments: 'ref' frame number >= FrameCount")
        Assert(fsyn_R > fsyn_L, 
        \   "UUSyncSegments: 'syn' sync point " + String(count) + " <= previous")
        Assert(fref_R > fref_L, 
        \   "UUSyncSegments: 'ref' sync point " + String(count) + " <= previous")

        factor = Float(fsyn_R - fsyn_L) / Float(fref_R - fref_L)
        S = syn.Trim(fsyn_L, fsyn_R-1)
        \      .UUChangeSpeed2(factor, pitchfix, noblend)

        S = (!debug) ? S : S.Subtitle(
        \        "syn in, out: " + String(fsyn_L) + "," + String(fsyn_R)
        \    + "\nref in, out: " + String(fref_L) + "," + String(fref_R)
        \    + "\nspeed mult.: " + String(factor, "%0.3f")
        \    , font="courier", size=S.Height/16, lsp=0)

        R = R ++ S

        ps     = FindStr(ssyn_R, ",")
        ssyn_L = (ps==0) ? ssyn_R : LeftStr(ssyn_R, ps-1)
        ssyn_R = (ps==0) ?   ""   : MidStr(ssyn_R, ps+1)

        count = count - 1
    }
    R = R ++ syn.Trim(fsyn_R, 0)
    return (bypass) ? syn : R
}
""")

#######################################
### change speed over a wide range 
### with frame blending in fast forward by default.
##
## @ factor - 0.33 for 1/3 speed, etc; 
##    min = 0.001; max = min(FrameCount, 1000) 
##    (if > FrameCount, duration would be 0) ;
##    if argument is out of range, an error occurs.
##
## @ pitchfix - if true, maintain audio pitch with speed 
##    (default false; allow pitch to rise or fall)
##
## @ noblend - if true, never blend frames (default false)
##
## @ version 1.0 24-Feb-2014 raffriff42
##
function UUChangeSpeed2(
\           clip C, float factor, bool "pitchfix", bool "noblend")
{
    Assert(factor>=0.001, 
    \  "UUChangeSpeed2 bad argument: factor<0.001")
    Assert(factor<=1000.0, 
    \  "UUChangeSpeed2 bad argument: factor>1000")
    Assert(factor<=C.FrameCount, 
    \  "UUChangeSpeed2 bad argument: factor>FrameCount")

    pitchfix = Default(pitchfix, false)
    noblend  = Default(noblend, false)

    AS  = C.KillAudio.AssumeFPS(C.FrameRate * factor)
    ASI = C.KillAudio.AssumeFPS(C.FrameRate / factor)

    temprad = Round(0.5 * factor) 
    TS = (factor<1.0 || temprad==0 || noblend) 
    \  ? AS 
    \  : Overlay(
    \       AS, 
    \       AS.TemporalSoften(
    \           temprad, 255, 255, 48, 2), 
    \       opacity=0.7)

    ffx = Min(Max(0.0, 25.0/factor), 100.0)

    (factor<1.5 && noblend==false)  
    \  ? AS.ConvertFPS(C) 
    \  : TS.ChangeFPS(C.Framerate, linear=(factor<10.0))

    A = (pitchfix)
    \  ? C.KillVideo.ConvertAudioToFloat.TimeStretch(tempo=factor*100.0)
    \  : C.KillVideo.ConvertAudioToFloat.TimeStretch(rate=factor*100.0)

    A = (C.AudioBits==32) ? A.ConvertAudioTo32bit
    \ : (C.AudioBits==24) ? A.ConvertAudioTo24bit
    \ : (C.AudioBits==16) ? A.ConvertAudioTo16bit
    \ : (C.AudioBits==8)  ? A.ConvertAudioTo8bit
    \ : A

    return (Abs(factor-1.0) < (0.5/C.FrameCount)) ? C
    \ : (C.HasAudio==false) ? Last 
    \   : Last.AudioDub(A)
}

Last edited by raffriff42; 16th March 2017 at 23:57. Reason: (fixed image links)
raffriff42 is offline   Reply With Quote
Old 3rd December 2015, 03:44   #2  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,958
That was a great show, back in 1994. And Paula Cole was so sweet, wasn't she...
manolito is offline   Reply With Quote
Old 3rd December 2015, 11:20   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 8,850
Perfect, if you could automate creation of frame number lists (not an insurmountable problem)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 3rd December 2015, 12:57   #4  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,375
> if you could automate creation of frame number lists
Thanks StainlessS, but I had to match two visually unrelated clips*; I don't see how to really automate that process. Only a human such as myself, who knew how the 2nd clip (think alternate camera angle) related to the reference, could know when it was in sync or not. The video was only about twelve minutes anyway, so finding a dozen sync points took only a few minutes. (I'll release it to YouTube when & if I get around to finishing it) Now, if the task had been really huge and repetitive, I might have tried to work up some automation. Maybe next time...

* (and audibly unrelated - the 2nd clip was MOS)

Last edited by raffriff42; 5th December 2015 at 13:19. Reason: MOS
raffriff42 is offline   Reply With Quote
Old 3rd December 2015, 17:46   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
If you want a one-button solution to this problem, I recommend you look at:

PuralEyes

You used to be able to get it by itself, and for not much money. Now it looks like it is only available as part of an expensive suite. It's too bad that so many companies are following the Adobe Creative Suite model.
johnmeyer is offline   Reply With Quote
Reply

Tags
avisynth, gscript, synchronization

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.