[johnmeyer]

Update October 13, 2021 based on discussion in this thread: Dropped frames
------------
The updated script with the conditionals that block diff values = 0 from being included in the moving average is shown below.

Also, I tried the two methods for downloading the video. Streamtransport was straightforward, but it grabbed the low-res version of the video. I couldn't find any documentation as to how to specify the higher res version, nor was I able to find any menu or setting in the program.

The RTMPDump was a little more convoluted (reading source code, using the DOS command line, etc.), but ultimately gave me exactly what I wanted.

Finally, I figured out how to use AVISynth and VFAPIConv to serve the resulting FLV files back into Sony Vegas (which can't read FLV) without re-encoding.

So, even though the script posted below ultimately may not be used for this project, I know I'll be using it on various things that clients send to me.

Once again, many thanks for the help!

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
# Further modification on June 8, 2013
# Tdecimate modifications on October 26, 2021
#
# 1. Create interpolated frames a 2x original frame rate using MVTools2.
# 2. Detect jumps.
# 3. Create white mask at each jump point; black mask for all other frames.
# 4. Repeat each frame of original video.
# 5. For each frame in double-frame video, use a mask to "choose" between that original video, or the motion estimated video
# 6. Decimate exactly 50% to get back to original frame rate. 
#
# This decimation removes the dup frames that existed in the original video and also removes the dups created by 
# repeating each frame of original video.
# However, at each point where motion-estimated frame was inserted, no decimation occurs. 
# Thus, if dups=drops, all dups in the original video will be removed and the drop will be filled. 
# On the other hand, if no drops or dups occur, then no motion estimation appears in the final result, and the decimation 
# merely gets back to original, unchanged video.
#*********************************************

#*********************************************
#User-settable paramaters

#Puts dot on screen during debugging. True for troubleshooting; otherwise, false.
showdot = true 

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8

#Frame rate of your video 
#PAL: Vidnum=25; VidDen=1;   NTSC: Vidnum=30000; VidDen=1001;  etc.
VidNum = 60000
VidDen = 1001

#Detection window size. Try changing if you find residual dups or drops. Script gets slow if you use big numbers
DupCycle = 18

threads = 1  #TDecimate does not like multithreading
#*********************************************

loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools2 (latest)\mvtools2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll") 
loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\VideoFred's 2014 Script 06_2012\scripts\plugins\RemoveGrain.dll")

#These were used when trying to get AVISynth+ multi-threading to work
#SetFilterMTMode("TDecimate", MT_SERIALIZED)   
#SetFilterMTMode("TDecimate", MT_MULTI_INSTANCE)

global source=AVISource("E:\fs.avi").ConvertToYV12.killaudio()

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

prefiltered = RemoveGrain(source,22)
super = MSuper(source,hpad=16, vpad=16, levels=1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad=16, vpad=16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize=16,overlap=4,search=3,dct=0)
forward = MAnalyse(superfilt, isb = false, blksize=16,overlap=4,search=3,dct=0)
forward_re = MRecalculate(super, forward, blksize=8, overlap=2, thSAD=100)
backward_re = MRecalculate(super, backward, blksize=8, overlap=2, thSAD=100)

#This is alternative way to double the number of frames. 
#double = SmoothFPS2 (source,threads)

double = source.MFlowFps(super, backward_re, forward_re, num=VidNum*2, den=VidDen, blend=false)
double = showdot ? double.subtitle("***") : double

test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)

# *** NOTE *** mt_merge does NOT produce a clean merge, and the dups are slightly altered from the originals -- unable to fix

interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate half of all frames 
RequestLinear
tdecimate(display=false,debug=false,mode=1,Cycle=DupCycle*2,CycleR=DupCycle,vfrdec=1,sdlim=0)  

#Alternate decimation -- should work, but doesn't
#tdecimate(display=false,debug=false,mode=2,rate=59.94006,maxndl=19,m2PA=true)  

#Alternate ways to end the script. Must assign variable to tdecimate output instead of using implied "last."
#prefetch(threads)	
#return test
#Stacked view, for debugging
#return stackvertical(source,decimated)

#End Main Program

#----------------
#Begin Functions
#----------------

#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff~=0 (i.e., a duplicate) from moving average
#This is why there are three lines for each YDiff. Each block uses the next or preceeding
#frame, if the current YDiff is near zero, indicating a dup.
#Note: This function works just fine

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}


#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average.
#Since the first two frames of the clip have no preceding frame, don't return a mask
#for those two frames (which means there will be no fix for the first two frames)

#This function works just fine (i.e., really great detection)

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \  && current_frame > 2
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}

#This function is hard-wired for 60 fps. Not worth the time to generalize it, since SVP
#didn't seem to provide a huge benefit over MVTools2, although it should be faster.

function SmoothFPS2(clip source, threads) { 
  super_params="{pel:2,gpu:1}"
  analyse_params="""{
        block:{w:16,h:16}, 
	main:{search:{coarse:{distance:0}}},
	refine:[{thsad:200}]
        }"""
  smoothfps_params="{rate:{num:120,den:60,abs:false},scene:{mode:0,limits:{scene:8500}},algo:21,cubic:1}"

  super =   SVSuper(source,super_params)
  vectors = SVAnalyse(super, analyse_params)
  SVSmoothFps(source,super,  vectors,  smoothfps_params,  url="www.svp-team.com",  mt=threads)
}