Dropped frames

[stephen22]

I have a video with dropped frames. There is always a duplicated frame adjacent but unfortunately it is sometimes before and sometimes after.

Filldrops or a slight modification thereof can repair either type but not both. Using YDifferenceFromPrevious or ToNext from current and previous frames, it's easy to distinguish between the two types. Is it possible to get this information for use in a runtime filter?

Didee and John Meyer did some nifty scripting using SelectEvery (clip,1,x) which seemed to address the problem but the script (I think called "dropped") doesn't work here and I think I may have copied it wrong and I can't find it again in this forum.

This must be quite a common problem, so before I start to attack it, has it already been done? Any other suggestions?

[johnmeyer]

This link will take you directly to the best version of the script that I posted in that thread. You can go to the first post to read what the script does and how it works.

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?p=1510813#post1510813)

Even though the title says that the drops are "followed" by dups, the way the script works it will fix the video whether the dups are before or after the drops, so don't get hung up on that wording.

Depending on your video, you may want to keep the detection threshold pretty high, as I did in the script I posted. It has to be <1, and I had it at 0.8.

The only issue with the script is that it still has to make some attempt to figure out where to fill in the drop (finding dups is relatively easy; finding the drop location is tricky). StainlessS or someone else who has spent a lot of time comparing frames and looking for difference metrics might be able to suggest 1-2 lines of different code that would be more sensitive to a big discontinuity (a "drop") than my code.

The code that might be improved is all in the GenerateMask function. All that function does is to compare the current frame YDifference metric to the YDifference metrics from the preceeding two frames and the two frames which follow. If there is a local "spike" then a mask is generated which, given how the script works, will offer that frame as a potential for filling in with a motion estimated frame.

Fortunately, the way the code works, if you end replacing the wrong frame, it usually isn't too noticeable.

I am not aware of any other script which attempts to do this, so I think you are better off starting with this and then trying to perfect it. I included quite a few comments in the script, so hopefully you can understand what it does and how to tweak it.

If you post 5-10 seconds of video that is really bad, others may be willing to see if they can improve my code.

[StainlessS]

You might like to try out DropDeadGorgeous() [Part of (and requires) Duplicity2]
It works something like this


Duplicity2(), is a dupe Detector, Dupe frame list creator, Dupes replaced with exact dupes, Dupes replaced with interpolated frames + DBase of dupes creator.
DropDeadGorgeous2() uses Duplicity2 DBase to do the almost impossible, smoothing out jerking clips (where source has jerks and dupes, much better than simple interpolation of dupes).
DropDeadGorgeous2 can also [EDIT: help] up framerate (eg 25.0 -> 29.97) keeping all source frames verbatim, and creating smooth result
[squeezes out dupes, where are replaced by interpolated frames between frames of most movement - non dupe frames are relocated between jerk interpolations and dupes to squeeze out dupes (hard to explain it)].

EDIT: Perhaps this explains better

abcXdefgDDhi Src, Two dupe [D] frames preceded (within a set range) by a jerk frame. Jerk frame X, has greater motion to its predecessor than do d, e, f, or g.
abc12Xdefghi DropDeadGorgeous dupe removal and Jerk interpolation

Lower case non dupe non Jerk
X = Frame of greatest motion from predecessor (Jerk)
D = Dupe Frames
Digit = Interpolated frames between X and its predecessor c (Same Count as D dupes)


Duplicity2:- https://forum.doom9.org/showthread.php?t=175357

[johnmeyer]

I haven't used DropDeadGorgeous (although I've seen the movie), but the way you describe it, I don't think it will help with the OP's video. His problem -- and the problem most people have with captures which drop frames -- is NOT the duplicate that is inserted sometime after the drop in order to keep the audio in sync, but is the drop itself. Detecting the exact duplicates that are inserted is trivial, but figuring out where there is a gap in the motion is dreadfully difficult, even though it is (unfortunately) very easy for the eye to see.

To do a REALLY good job at detecting temporal gaps would require access to the internals of a motion estimation DLL, like MVTools2. That tool keeps track of small blocks of video and attempts to track their movement. I would expect that there is some set of numbers which describe the vector of each block of pixels. If those vectors could be exposed, then I think they would "blow up" on any frame where the contents have moved too much, relative to nearby frames.

That makes me think of something: maybe one of the motion estimated scene detection metrics, like MVTools2 MSCDetection, could be used instead of my comparison metric.

[StainlessS]

In above quote, the X Jerk frame is the frame following a missing dropped frame, and DD, one or more dupes.

EDIT:
During Capture, when a frame is dropped, subsequent frames in the clip are temporally advanced, up until a duplicate is produced, after that it is back in sync.
What we do [as in my prev post] is to reverse that temporal advance, by inserting [in descibed case] 2 interpolated frames before the X jerk frame, and relocate X->DD [excluding DD] later in clip, squeezing out the dupes.


abcXdefgDDhi Src, Two dupe [D] frames preceded (within a set range) by a jerk frame. Jerk frame X, has greater motion to its predecessor than do d, e, f, or g.
abc12Xdefghi DropDeadGorgeous dupe removal and Jerk interpolation

Lower case non dupe non Jerk
X = Frame of greatest motion from predecessor (Jerk)
D = Dupe Frames
Digit = Interpolated frames between X and its predecessor c (Same Count as D dupes)


# below assumes that 2 frames both dropped and duped, similar where 1 frame dropped/duped. [2 frames dropped is extreme and more difficult case, we describe the more difficult]

abc12Xdefghi # How it should have been in capture, 1,2 will both be dropped, X is jerk frame following frames that will be dropped.
abcXdefgDDhi # when 1,2 dropped by CAP device, and DD dupes inserted.
abc12Xdefghi # DropDeadGorgeous() fixed. (1,2 are both interpolated frames [c<=>X], originals GONE $4E4 [forever])


EDIT: Also, all non-dupe source frames are kept verbatim.

[MysteryX]

Take a look at this: InterpolateDoubles (https://github.com/mysteryx93/FrameRateConverter/blob/master/FrameRateConverter.avsi#L291)

"Replace double frames with interpolated frames using FrameRateConverter"

[StainlessS]

Thanks MX, I'm sure that we do something similar-ish already.
[Although we temporally relocate frames between X jerk frame [inclusive] and dupe frames [exclusive] to squeeze out the dupes,
and interpolate dropped double frames between the new relocated X jerk frame and the frame that was immediately prior to the X jerk frame].

EDIT: And see prev post edits.

[johnmeyer]

what we do [as in my prev post] is to reverse that temporal advance, by inserting [in descibed case] 2 interpolated frames.


abcXdefgDDhi Src, Two dupe [D] frames preceded (within a set range) by a jerk frame.
Jerk frame X, has greater motion to its predecessor than do d, e, f, or g.
abc12Xdefghi DropDeadGorgeous dupe removal and Jerk interpolationYup, that is what is needed.

By contrast, InterpolateDoubles seems to only be fixing the duplicate problem. MugFunky developed FillDrops() years ago, doing pretty much the same thing. I refined that a little by extending it to interlaced footage.

It would be great if the OP could post some footage so we could each try out our toys and see what the results look like.

[StainlessS]

Yep, I thought that MX doubles whotis was same as MugFuncky thingy, below is better, in-case you did not see it.

# below assumes that 2 frames both dropped and duped, similar where 1 frame dropped/duped. [2 frames dropped is extreme and more difficult case, we describe the more difficult]

abc12Xdefghi # How it should have been in capture, 1,2 will both be dropped, X is jerk frame following frames that will be dropped.
abcXdefgDDhi # when 1,2 dropped by CAP device, [EDIT: X->g are temporally advanced] and DD dupes inserted.
abc12Xdefghi # DropDeadGorgeous() fixed. [EDIT: X->g are temporally Delayed and dupes squeezed out]. 1,2 are both interpolated frames [c<=>X], originals GONE $4E4 [forever].

In bottom line, 1,2 are interpolated from c,X.

[poisondeathray]

There is always a duplicated frame adjacent but unfortunately it is sometimes before and sometimes after.


I'm assuming it's something like this, where c is the dropped frame. "adjacent" implies right next to , at least to me, not a few frames later or before

abbdef - if dup occurs before, replace 2nd dup
abddef - if dup occurs after, replace 1st dup

Here is a test clip with both cases
https://www.mediafire.com/file/cvawflhtq73okdg/drops_and_dups_testclip.mp4/file

I played with DropDeadGorgeous2(), and it caught the 2nd, but not the 1st. But I just used default settings. SS probably tweak it correctly

[StainlessS]

But I just used default settings. SS probably tweak it correctly
Probably not, I did not anticipate dupe before drop, and to try do both at once is a big ask,
is bloody difficult doing just the implemented one. :)

EDIT: Might be do-able if is KNOWN that dupes are IMMEDIATELY ADJACENT to drop.

EDIT: Got the clip thanks PDR.
EDIT: Nice test clip.

[poisondeathray]

Smoothskip caught both in that testvid . I don't know if they are immediately adjacent for the OP, but that's how I read it

I chose 65, because that's roughly the distance between the blips


orig=LSmashVideoSource("drops_and_dups_testclip.mp4")

s=orig.SmoothMotion(cycle=65, skips=1, dupes=1)

stackvertical(orig,s)


function SmoothMotion(clip last, int "cycle", int "dupes", int "skips", int "debug" ) {
cycle = default(cycle, 4)
dupes = default(dupes, 1)
skips = default(skips, 1)
debug = default(debug, 0)
TDecimate(cycle=cycle, cycleR=dupes, display=(debug == 1))
super = MSuper()
bv = MAnalyse(super, overlap=4, isb = true, search=3)
fv = MAnalyse(super, overlap=4, isb = false, search=3)
inter = MFlowInter(super, bv, fv, time=50, ml=70)
SmoothSkip(inter, cycle=(cycle-dupes), create=skips, offset=-1, debug=(debug == 2))
}



https://forum.doom9.org/showthread.php?t=172377
https://github.com/jojje/SmoothSkip

[poisondeathray]

Also, the OP says "Filldrops or a slight modification thereof can repair either type but not both. Using YDifferenceFromPrevious or ToNext from current and previous frames, it's easy to distinguish between the two types." - so that has to be directly adjacent n-1 or n+1

[Frank62]

What would you think about simply:

Filldrops()

Reverse()
Filldrops()
Reverse()

?

[StainlessS]

What would you think
Maybe, so long as it did not stuff up the other ones on first pass.
[Not sure if it would get slower and slower though - could try I suppose, but not me - I'm off to Ballet practice soon.]

[poisondeathray]

What would you think about simply:

Filldrops()

Reverse()
Filldrops()
Reverse()

?

It's what SS said - it messed up on that test clip because it should have left one alone for the 2nd pass Filldrops. But there is a duplicate there and it so it "blindly" interpolates it. Since the interpolated frame is in the "wrong" place, the jump remains

[StainlessS]

Anyways, to get you all in the mood to best solve this, see this VERY enlightening post from the big G himself, Gavino:- https://forum.doom9.org/showthread.php?p=1409378#post1409378
Its only 3 pages of posts, best read the lot, very interesting, Didée also makes an appearance.

I''m gonna see if I can get some sleep, went to bed at about 06:30, up again about 09:00 and about 10 mins kip in between,
had a couple of orange juices at ballet practice, maybe that helps a little. [Althougn I might have a large glass of apple juice too before the feathers].

[johnmeyer]

My script posted above fixed it perfectly (this is WAY too easy a test case). I did, however, have to change the double fps from 59.974 to 48, since the test case is 24 fps.

If this is going to be a "thing," then I should probably re-work the script to put the parameters that need to be changed at the front of the script. Here is the one change to fix the trivially easy test case:

Change from this:
double = source.MFlowFps(super, bvec, fvec, num=60000, den=1001, blend=false)to this:

double = source.MFlowFps(super, bvec, fvec, num=48, den=1, blend=false)

P.S. I'm not knocking poisondeathray and the test he manufactured, but only pointing out that, since it is so trivially easy to fix, it won't really provide much information on which approach is most likely to work for real-world cases.

[poisondeathray]

My script posted above fixed it perfectly (this is WAY too easy a test case). I did, however, have to change the double fps from 59.974 to 48, since the test case is 24 fps.

If this is going to be a "thing," then I should probably re-work the script to put the parameters that need to be changed at the front of the script. Here is the one change to fix the trivially easy test case:

Change from this:
double = source.MFlowFps(super, bvec, fvec, num=60000, den=1001, blend=false)to this:

double = source.MFlowFps(super, bvec, fvec, num=48, den=1, blend=false)

P.S. I'm not knocking poisondeathray and the test he manufactured, but only pointing out that, since it is so trivially easy to fix, it won't really provide much information on which approach is most likely to work for real-world cases.

Did you change anything else other than source filter and that "double" line? Your script does not work for the 2nd blip when I tried it earlier (I already made those changes when trying earlier). Check around frame 119,120, there is a jump in motion

Smoothskip works on this "trivial" test case

[poisondeathray]

here is the comparison video (top and bottom borders cropped, I labelled it "john" because that script function did not have name)
https://www.mediafire.com/file/c2qo3pyuca5e8b4/drops_and_dups_testclip_compare.mp4/file

[johnmeyer]

OK, I'll shut up about your test case being too simple, since my script failed to correctly fix this "trivial" case. :)

It is a problem with TDecimate. The problem is easily seen in this image:

https://i.imgur.com/gBrO1fz.png

The astericks indicate what frame is being decimated and, for reasons I have not yet figured out, TDecimate has decided to delete a "new" frame rather than a dup.

It did the correct thing when the jump happened after the dup.

I'll scratch my head for awhile and see if I can figure out why TDecimate decided to kill a non-dup.

[poisondeathray]

128,129 are duplicates now too.

But a larger "window" seems to work eg. cycleR=33,cycle=66

[johnmeyer]

The problem is the fixed "M in N" windows size and the "boundary" it creates between each group of frames. If interesting things happen across that boundary, it decimates the wrong things.

So, as you just found out, if you vary the window, you can make it work at a point where it previously failed, but it will then fail at another point.

The screen grab below illustrates the problem.

For the test shows in the screen grab, I changed to cycleR=7,cycle=14. This put the three non-duplicated frames that result from my fix, along with the dups in the capture and the dups I temporarily created, all in the same M in N window. TDecimate then left the "new" frames alone and didn't decimate them.

My algorithm and Didee's old code that it was based on are both sound, but I have to revise it, or else find a different mode in TDecimate that will not fail if my fixed frames happen to fall on a window boundary.

https://i.imgur.com/1QcDN4P.png

[StainlessS]

I think we really need a sample instead of guesswork, come on stephen22, post one up somewhere, probably need 1000 frames or so, maybe.

EDIT: Or maybe about 10 [or more] of each type drop-dupe pair, no more than about 1000 frames [ideally not too big].

EDIT: @ JohnMeyer, just making an observation.
I did the drop followed by dupe thing [DropDeadGorgeous] because of one of your threads, however as it turned out I had done it the wrong way around
[you pointed this out somewhere, should have been dupe followed by drop].

I've just stumbled over the thread that caused my misinterpretation, here:- https://forum.doom9.org/showthread.php?p=1510813
Thread title: "Automatically fix dups followed (eventually) by drops".

And a text a little way into first post

Problem I Tried To Solve

In a nutshell, the video randomly (i.e., not periodically) drops frames and then, to restore sync, duplicates a frame, usually within about five frames of the dup. So, here's the statement of the problem: I want to delete the dups and simultaneously synthesize a frame to insert at the point where the video jumps.

I guess that I was misled by the text above, when I did DropDeadGorgeous to fix the wrong problem in BLUE :)

[johnmeyer]

Well, I just killed a couple of hours trying to figure out how to get TDecimate to work. Maybe I'll come back to this at a different time. It should work, and it does work, but for some reason TDecimate, even when I do a two pass decimation, just won't do the job.

Perhaps I need to make sure I'm feeding it the right number of frames (i.e., exactly and precisely double the number of the original frames).

[StainlessS]

Perhaps I need to make sure I'm feeding it the right number of frames (i.e., exactly and precisely double the number of the original frames).
How are you attempting to double the number of frames ?
[just curious, not even thought about trying to do any fix yet, just scanning existing threads].
EDIT: Not even looked yet at PDR TDecimate/SmoothSkip thingy. [no idea how it works] - waiting for real sample.

[StainlessS]

John, this about right for you ?
Stole it from Didee post here:- https://forum.doom9.org/showthread.php?p=1409324#post1409324


AviSource("..\hard sub - 01 WEBdlRip 720p, 23.976.mkv.AVI")
source=Last
o = assumefps(1.0) ox=o.width() oy=o.height()

showdot = true # false # shows a "dot" in the interpolated frames of the result

super = showdot ? o.subtitle(".").MSuper(pel=2) : o.MSuper(pel=2)
bvec = MAnalyse(super, overlap=4, isb = true, search=4, dct=5)
fvec = MAnalyse(super, overlap=4, isb = false, search=4, dct=5)

nearlydouble = o.MFlowFps(super, bvec, fvec, num=2, den=1, blend=false) # 1 frame short of double frames, last frame is not repeated/interpolated
double = nearlyDouble.DuplicateFrame(nearlyDouble.FrameCount-1) # Duplcate frame at very end of nearlydouble clip.
double

assumefps(source) # Restore original frame rate
AssumeScaledFPS(2,1) # then double it

return last

Some extraneous/redundant code in that script.

EDIT: Edited, bit of a glitch.

[johnmeyer]

Updated script on October 13, 2021

StainlessS,

Just to be clear, I have copied my current version of the script below. The comments at the beginning of the script describe how it works, and answer your questions about doubling.

I've spent WAY too much time on this although I did that because this problem comes up very frequently, not only in this forum, but in the video I receive that needs fixing. It would be good to have a solid solution.

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
# Further modification on June 8, 2013
# Better Tdecimate modification on October 13, 2021
#
# 1. Create interpolated frames a 2x original frame rate using MVTools2.
# 2. Detect jumps.
# 3. Create white mask at each jump point; black mask for all other frames.
# 4. Repeat each frame of original video.
# 5. For each frame in double-frame video, use a mask to "choose" between that original video, or the motion estimated video
# 6. Decimate exactly 50% to get back to original frame rate.
#
# This decimation removes the dup frames that existed in the original video and also removes the dups created by
# repeating each frame of original video.
# However, at each point where motion-estimated frame was inserted, no decimation occurs.
# Thus, if dups=drops, all dups in the original video will be removed and the drop will be filled.
# On the other hand, if no drops or dups occur, then no motion estimation appears in the final result, and the decimation
# merely gets back to original, unchanged video.
#*********************************************

#*********************************************
#User-settable paramaters

#Puts dot on screen during debugging. True for troubleshooting; otherwise, false.
showdot = true

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8

#Frame rate of your video
VidFrameRateNum = 60000
VidFrameRateDen = 1001
#Detection window size. Try changing if you find residual dups or drops. Script gets slow if >100
DupCycle = 13
#*********************************************

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll")

global source=AVISource("E:\fs.avi").ConvertToYV12.killaudio()

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2)
bvec = MAnalyse(super, overlap=0, blksize=16, isb = true, search=4, dct=0)
fvec = MAnalyse(super, overlap=0, blksize=16, isb = false, search=4, dct=0)
double = source.MFlowFps(super, bvec, fvec, num=VidFrameRateNum*2, den=VidFrameRateDen, blend=false)

#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics
#in order to determine proper JumpThresh
test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)
interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)
RequestLinear
decimated = tdecimate(display=false,debug=false,mode=1,Cycle=DupCycle*2,CycleR=DupCycle)

#Alternate decimation
#decimated = tdecimate(display=true,debug=false,mode=2,m2PA=true,rate=VidFrameRateNum/VidFrameRateDen,maxndl=19)

return decimated

#Stacked view, for debugging
#return stackvertical(source,final)

#End Main Program

#----------------
#Begin Functions
#----------------

#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average
#This is why there are three lines for each YDiff. Use the next, or preceeding
#frame, if the current YDiff is near zero, indicating a dup.

function ShowMetrics (clip c)
{
fixed=source.ScriptClip("Subtitle(String(
\ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) )
\ )/4) /
\ (YDifferenceFromPrevious(source) + 0.01)
\ ))")
return fixed
}


#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average.
#Since the first two frames of the clip have no preceding frame, don't return a mask
#for those two frames (which means there will be no fix for the first two frames)

function GenerateMask (clip c)
{
MyMask=c.ScriptClip("""
\ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) )
\ )/4) /
\ (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh
\ && current_frame > 2
\ ? WhiteFrame : BlackFrame """)
return MyMask
}

[StainlessS]

I'll take a bit of time to digest that, I wanna try get back into some kind of normal sleep pattern.
One small thing, suggest mod.

Filldrops Mod: EDIT: script removed. See Below.

EDIT: Now I see that JKyle converted your original script in other thread to an script function, [ https://forum.doom9.org/showthread.php?p=1947624#post1947624 ]
https://github.com/JJKylee/Filter-Scripts/blob/main/AviSynth/DeJump.avsi with HBD support.
and pointed out Selur mod of filldrops [FillDrops mod 2021-07-10 by Selur]:- (https://github.com/JJKylee/Filter-Scripts/blob/main/AviSynth/FillDrops.avsi)

[johnmeyer]

II see that JKyle converted your original script in other thread to an script function, [ https://forum.doom9.org/showthread.php?p=1947624#post1947624 ] https://github.com/JJKylee/Filter-Scripts/blob/main/AviSynth/DeJump.avsi

I always thought this script was worthy of more work, given how frequently the problem happens (as I mentioned previously).

I like the fact that he parameterized the function, but I have no idea why he removed the show metrics. Heck, how can you tune any script like this if you don't know anything about the parameters your particular video generates??

I have no problems having him remove FillDrops (StanilessS, you've mentioned doing something similar as well). Including it was a "belt and suspenders" addition, meaning that I didn't think there would be any dups left, but if there were, then Filldrops would remove them. Not much downside, because if it doesn't find anything, it doesn't slow things down much (i.e., no call to MVTools2 if the logic doesn't detect a dup).

My thinking at this late hour is that since TDecimate doesn't like what I'm feeding it, and since FDecimate doesn't do any better, I have to re-think my overall approach and come up with something else, even though I think Didée's idea was brilliant.

[StainlessS]

@JohnMeyer,
Here is mod of your FillDropsI() script, takes a "thresh" arg instead of hacking function code, Is similar to Selur mod of FillDrops()
Can you check this is current version script [ignoring the modded part]


function filldropsI (clip c, float "thresh")
{
thresh = Default(thresh, 0.1)
even = c.SeparateFields().SelectEven()
super_even=MSuper(even,pel=2)
vfe=manalyse(super_even,truemotion=true,isb=false,delta=1)
vbe=manalyse(super_even,truemotion=true,isb=true,delta=1)
filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

odd = c.SeparateFields().SelectOdd()
super_odd=MSuper(odd,pel=2)
vfo=manalyse(super_odd,truemotion=true,isb=false,delta=1)
vbo=manalyse(super_odd,truemotion=true,isb=true,delta=1)
filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

evenfixed = ConditionalFilter(even, filldrops_e, even, "YDifferenceFromPrevious()", "lessthan", String(thresh))
oddfixed = ConditionalFilter(odd, filldrops_o, odd, "YDifferenceFromPrevious()", "lessthan", String(thresh))

Interleave(evenfixed,oddfixed)
Weave()
}


@Selur,
Assuming that JohnMeyer gives it the go-ahead, would you mind adding it to your FillDrops() modded avsi script.
[Leaving out all mention of me - you, and JM can take all of the blame :) ]

EDIT: And John, maybe do search on eg "function filldrops" and "function filldropsI", and delete all instances from your script bank,
so you only use the avsi in plugins, Havin' a gazillion different versions of them aint a good thing.

EDIT: We'll also coerce JKyle to add the metrics whotsit to his mod DeJump.avsi of your script.

EDIT: Source of FillDropsI() gotten from here [Thread: Filter to remove duplicate frames ]:-

I created a version of this script that uses the newer MVTools2 and is designed for interlaced video:

function filldropsI (clip c)
{
even = c.SeparateFields().SelectEven()
super_even=MSuper(even,pel=2)
vfe=manalyse(super_even,truemotion=true,isb=false,delta=1)
vbe=manalyse(super_even,truemotion=true,isb=true,delta=1)
filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

odd = c.SeparateFields().SelectOdd()
super_odd=MSuper(odd,pel=2)
vfo=manalyse(super_odd,truemotion=true,isb=false,delta=1)
vbo=manalyse(super_odd,truemotion=true,isb=true,delta=1)
filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

evenfixed = ConditionalFilter(even, filldrops_e, even, "YDifferenceFromPrevious()", "lessthan", "0.1")
oddfixed = ConditionalFilter(odd, filldrops_o, odd, "YDifferenceFromPrevious()", "lessthan", "0.1")

Interleave(evenfixed,oddfixed)
Weave()
}


EDIT: @ JM,
I note this change of thresh in post #28 FillDrops

function filldrops (clip c)
{
super=MSuper(c,pel=2)
vfe=manalyse(super,truemotion=true,isb=false,delta=1)
vbe=manalyse(super,truemotion=true,isb=true,delta=1)
filldrops = mflowinter(c,super,vbe,vfe,time=50)
fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.2")
return fixed
}

Maybe you and Selur can argue about default Thresh in the Avsi. [a good reason to have only a single version of the functions ]
EDIT: Actually, default in post #28 FillDrops is 0.2, and in Selur FillDrops is 0.1, and fillDropsI is 0.1, Which of these default values are correct and as intended,
give Selur a clue please.
I see no reason why interlaced default should be half that of non interlaced - Discuss.
(time difference between progressive frames @ 25FPS = 1/25, between even fields of 50i = 2*1/50 = 1/25 secs).

[johnmeyer]

@JohnMeyer,
Here is mod of your FillDropsI() script, takes a "thresh" arg instead of hacking function code, Is similar to Selur mod of FillDrops()
Can you check this is current version script [ignoring the modded part]Yes, it is identical to the script in my library.

Thanks for the mod. It is certainly a good idea, and I will update my script. However, since the threshold value in this particular script is not usually important, I don't think I'm going to take the time to go back and edit my posts. For dups which are created by a capture card, in order to make up for a dropped frame, the difference is often 0.0.

Also, as I tried to say in my previous post, FillDrops() is completely non-essential to the topic of this thread, and the problems we are trying to solve are not related to it.

Just to reiterate, there are two outstanding problems with my script, and the second one applies to ALL other attempts at fixing the drop/dup problem that have been posted here:

1. TDecimate doesn't perform as I want in order to remove the original dups along with the dups I intentionally added in order to match the 2x optical flow clip. Other approaches to the problem may not use TDecimate and must find some other way to ensure that the number of dups deleted exactly equals the number of motion estimated frames inserted, so as to guarantee that the number of frames stays the same so audio stays in sync. (I put that phrase in italics, because it is essential, and it is not easy to do.)

2. Detecting jumps is crucial to this fix, and my YDifference function is not sufficiently sensitive. I attempted to improve detection by comparing to the adjacent YDifference values. This is essential. For instance, you will note that in the test clip, the initial frames, which are almost all black, have extraordinarily low YDifference values, and if you attempted to use a fixed value for a threshold, it simply would not work on that initial section of the clip.

There has to be a better way to detect temporal gaps (jumps). The jump which results from the dropped frame is obvious to the eye. Therefore, there has to be an algorithm that can take advantage of this "obviousness." I have tried various scene change metrics that motion estimation filters make available, but so far have not been able to get anything that works better.

However, the reason my script works (well, sort of, given the problems poisondeathray found) is that if it replaces a few frames that it didn't need to, you may not notice, given that motion estimation works extremely well about 98% of the time.

[edit]I forgot to mention that the various "FillDropsI" thresholds you see in various posts are just the result of typing a different number. There is no correlation with interlaced/progressive, or anything else.

[johnmeyer]

I updated my script in post #28. It is my latest and it does now work for this test clip.

However, I have not been able to come up with any TDecimate approach that I think will work perfectly for all video. Even with tuning for each clip, the script might not make it all the way through the clip without failing, due to the TDecimate Cycle/CycleR boundary issues I discussed earlier. TDecimate's Mode 2 should, in theory, get around this problem, but it did not distribute the dup removal evenly, resulting in decimation of non-dup frames, thus introducing jumps. I've included a mode 2 decimation as an alternative in the script above, in case anyone wants to take it out for a spin.

I'm finished with this for now, but if anyone has some interest, perhaps they can come up with some alternative solutions, or can find some values that make TDecimate do what is needed for this "off label" application.

[zorr]

How about running the script three times with different offsets and taking the median?

For example if the DupCycle = 13 then use offsets 0, 1/3*DupCycle (=4) and 2/3*DupCycle (=8). Offsetting simply means adding black frames to the start of the video before processing and removing them afterwards.

The median then selects the most common solution for each frame, so if only one of those three variations has an error it will be corrected. Obviously this is at least thee times as slow as the current script.

[johnmeyer]

How about running the script three times with different offsets and taking the median?

For example if the DupCycle = 13 then use offsets 0, 1/3*DupCycle (=4) and 2/3*DupCycle (=8). Offsetting simply means adding black frames to the start of the video before processing and removing them afterwards.

The median then selects the most common solution for each frame, so if only one of those three variations has an error it will be corrected. Obviously this is at least thee times as slow as the current script.That is a delightfully kludgy approach to the problem, but despite its lack of elegance, it might work. I hadn't though about the Median function for a long time.

[edit]I just deleted most of my original post. Now that I thought about it some more, there might be a way to do something like this. Perhaps the following is what you were saying.

The problem is that the algorithm fails when a set of dups and drops falls along TDecimate's "Cycle" group-of-frames boundary. Therefore, all that may be required is to move that boundary, via the offset, by some intelligent amount (haven't figured that out yet what that amount should be) which will ensure that if one set of dups/drops crosses the boundary for one offset, that it would be ensured not to be on a boundary for another. Then do it a third time and put the three results to the Median "vote."

The trick -- and this may be difficult -- will be to make sure that the three passes don't create as many new boundary problems as they each eliminate. You have to guarantee that two out of three outputs will be valid for each drop/dup pair. To do this will probably involve not only an offset, but also a different Cycle size for each pass.

I am reasonably good at certain types of math, but this is an algorithm that may be more than I can figure out.

However, I am now intrigued.

[johnmeyer]

I just did some quick tests by simply feeding the script with the video shifted to earlier frames. I did not bother, at this point, with blank frame padding. So all I did was this:

selectevery(1,-4)
selectevery(1,-8)

etc.

Sure enough, using my original Cycle=20, CycleR=10 decimation, which failed on the second of the two blips as poisondeathray found, when I started using earlier frames, it worked just fine for that problem frame as the large "S" scrolls onto the screen.

I then used selectevery(1,20) which, with Cycle=20, put the failed frame right back on the boundary and, sure enough, it failed again.

So, the concept works. The problem is that the dups/drops can be anywhere, and while I can reduce the incidence of the problem doing this, I'm not yet convinced that this approach can eliminate all failures due to this problem.

I need a really good real-world test case, so I can determine if it is worth the effort to re-engineer the script.

However, if I were to incorporate this idea, I think this is what I must do:

1. Make the main part of the program into a function so it can be called mutliple times. I know how to do this and it will probably only take five minutes.
2. Figure out how to add "n" number of black frames to the beginning and end of the video before running the operation three times with different selectevery() options (probably at the 1/3 and 2/3 marks within each Cycle).
3. Figure out exactly how to use Median so that it can accurately sense differences between the three results. Having not used Median, I have no idea what median is computing. Luma? If it doesn't work, I may have to create my own voting function, based on YDifference or some similar function.
4. Remove from the stream the black padding frames added in #2.

I'll need to come back to this after I've thought about it for a few days. Numbers 1,2, and 4 above I'm pretty sure I can do, but I'm not convinced that Median will be able to determine which clip is in the "middle" and should therefore be the one that is kept.

[StainlessS]

Just incase anybody is interested, this is what I've eventually decided is the problem pattern. [Now to see what Didee code I can rob]

# Immediately adjacent, Single drop and dupe.
# Drop 1, then Dupe 1
abcdefgh # How it should have been in capture, d will be dropped, frame following drop frame is e and will become jerk frame. e will be duplicated.
abceEfgh # When d dropped by CAP device, the JERK frame e is advanced temporally and takes the place of dropped d, and E is the dupe inserted after it. [eE=Jerk Dupe]
abcDefgh # Fixed. D is interpolated frame [c<=>e], original d GONE $4E4 [forever]

# Dupe 1, then Drop 1
abcdefgh # How it should have been in capture, c will be duplicated, and d will be dropped, e will become the jerk frame.
abcCefgh # C is duplcate of c and replaces dropped d. e becomes the jerk frame. [Ce=Dupe Jerk]
abcDefgh # Fixed. D is interpolated frame [c<=>e], original d GONE $4E4 [forever]


@Stephen22, when are you gonna post a reasonable length sample ?
You seem to visit every day, why arn't you responding, have you fixed it ? [They're a nasty bunch here, but not that scary really]
MediaFire.com is a favourite filehost of many here.

[johnmeyer]

StainlessS,

Yes, that is the pattern in the simple test case poisondeathray created. My latest script fixes both cases (dup before; dup after) perfectly, but will fail if the dup/drop happens to occur at another place in the video stream (because of the TDecimate Cycle boundary issue I've discussed in previous posts). If TDecimate could be made to work in mode 2 as the documentation says it should, that part of the problem would be fully solved, and the only remaining thing to do would be to improve the detection scheme.

Didée's concept, which I then coded, is absolutely rock-solid sound. The brilliance of the idea is that it guarantees that frames inserted will precisely equal dups removed. If you try to solve the problem using other approaches, you'll find that this is very difficult to do.

I'm pretty certain that the only way to improve the detection is to use motion estimation: something using MCompensate, for instance. I wish I could get a solution to the TDecimate problem because the real focus should be on detection. Once we have a real-world example, my detection scheme will show its weakness (Didée's original idea for detection, using the FrameDiff function in TIVTC was much worse, so I did at least make some progress).

This is where we need a real-world video example. There are plenty of examples out there, so we don't need to wait for the OP to respond. Anyone reading this is free to post something.

If I can't figure out how to make TDecimate behave then, having thought about it overnight, I'm pretty sure I can make Zorr's idea work, as long as the distance between the dup and the drop is less than 1/3 the "Cycle" number of frames.

BTW, I'm 99.99% certain that all dropped frame problems are always a drop, followed by a dup. I am guilty of starting the idea that there can be a dup followed by a drop. In that old thread, I was dealing with a source that had so many drops that I wrongly attributed the dup from a previous drop to the drop that happened after that dup. I think I got this idea because the video I was given had already been cut, and therefore when I walked through it frame-by-frame, the first thing I saw was a dup, and then a drop, hence my incorrect title for that old thread.

My script works regardless of whether the chicken or the egg comes first, so I guess it doesn't really matter. Much more important is the fact that the dup very often does not happen until many frames later. You cannot assume they will be adjacent (and my script does not make that assumption).

I'm not up for putting in the work to do any more mods now, since I've fixed the test case perfectly, but if I had a client, or even a real-world example, I might be tempted. As I keep saying, this is an extremely common problem, and a good, solid fix is needed.

[johnmeyer]

OK, I think I have an idea of how to create a more reliable detection function for dropped frames.

1. Use motion estimation to create the next frame based on vectors from the previous two frames.
2. Subtract that estimated frame from the real next frame.
3. Use some comparison metric from RT_Stats to get a figure of merit for that subtraction.

I just have to figure out how to code this.

[johnmeyer]

I'm going to start a new thread for getting help on detecting jumps. I always screw up motion estimation and masking, and I need some help. I have a simple detection script and it blows out of the water anything I've done before, but it could be much better with a little help.

[StainlessS]

I'm currently concentrating on the OP's specific problem, dup/drop OR drop/dupe, single instance adjacent in either order, [and not based on any cycle nor FrameRate].

EDIT: Dont know if of any use


Function MMaskFromPrevClip(clip c,Int "MaskType",Float "Gamma",Int "thSCD1",Int "thSCD2") {
# MaskType:- 0=Motion, 1=SAD, 2=Occlusion, 3=Horizontal, 4=Vertical, 5=ColorMap.
MaskType=Default(MaskType,0) Gamma=Default(Gamma,1.0) thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MMask(fv,Gamma=1.0,kind=MaskType,thSCD1=thSCD1,thSCD2=thSCD2)
}

Function MMaskFromNextClip(clip c,Int "MaskType",Float "Gamma",Int "thSCD1",Int "thSCD2") {
# MaskType:- 0=Motion, 1=SAD, 2=Occlusion, 3=Horizontal, 4=Vertical, 5=ColorMap.
MaskType=Default(MaskType,0) Gamma=Default(Gamma,1.0) thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
bv=sup.MAnalyse(isb=True,delta=1,blksize=16)
Return c.MMask(bv,Gamma=1.0,kind=MaskType,thSCD1=thSCD1,thSCD2=thSCD2)
}



Function EndOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at EOS, else 0
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
bv=sup.MAnalyse(isb=true, delta=1,blksize=16)
Return c.MSCDetection(bv,thSCD1=thSCD1,thSCD2=thSCD2)
}

Function StartOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at SOS, else 0
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MSCDetection(fv,thSCD1=thSCD1,thSCD2=thSCD2)
}

Function SceneCutClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma pixel = 0 =Norm, 1=EOS, 2=SOS, 3=EOS & SOS
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
Sup = c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
BvEos= Sup.MAnalyse(isb=True, delta=1,blksize=16)
FvSos= Sup.MAnalyse(isb=False, delta=1,blksize=16)
Eos = c.MSCDetection(BvEos,thSCD1=thSCD1,thSCD2=thSCD2)
Sos = c.MSCDetection(FvSos,thSCD1=thSCD1,thSCD2=thSCD2)
Return MT_Lutxy(Eos,Sos,yexpr="y 0 == x 0 == 0 1 ? x 0 == 2 3 ? ?",u=-128,v=-128)
}


EDIT: Added below
It works sorta like ScSelect/ScSelect_HBD, but you must also Provide a "BOTH" [ie both EOS and SOS detect] clip (you can choose whatever solution you like for that possible outcome).
You will likely fire a BOTH if there is a single frame scene cut, ie and 'odd' frame that belongs neither with previous nor following scenes.

Function SceneCutSelectClip(clip dClip,clip Start,clip End,clip Both,clip Motion,Int "thSCD1",Int "thSCD2") { # (c) ssS: https://forum.doom9.org/showthread.php?p=1955111#post1955111
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
Sup = dClip.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
BvEos= Sup.MAnalyse(isb=True, delta=1,blksize=16)
FvSos= Sup.MAnalyse(isb=False, delta=1,blksize=16)
Eos = dClip.MSCDetection(BvEos,thSCD1=thSCD1,thSCD2=thSCD2).crop(0,0,4,4)
Sos = dClip.MSCDetection(FvSos,thSCD1=thSCD1,thSCD2=thSCD2).crop(0,0,4,4)
CondS="""
ix = (Sos.AverageLuma>0?1:0) + (Eos.AverageLuma>0?2:0)
Return ix==0?Motion:ix==1?Start:ix==2?End:Both
"""
ARGS="Motion,CondS,Motion,Start,End,Both,Eos,Sos"
Motion.GSCriptClip(CondS,Args=ARGS,After_Frame=True,Local=True) # Requires Grunt
}

# Client
AviSource("D:\hard sub - 01 WEBdlRip 720p, 23.976.mkv.AVI")
dclip = BilinearResize(320,240).Blur(1.0) # Whatever (just testing frame size can be different to other clips)
ConvertToRGB32 # Just testing works where Dclip colorspace is differenct from the other clips.
Motion = Last
Start = Subtitle("START",size=64,align=5)
End = Subtitle("END",size=64,align=5)
Both = Subtitle("BOTH",size=64,align=5)
SceneCutSelectClip(dClip,Start,End,Both,Motion,400,130)

[stephen22]

@Stephen22, when are you gonna post a reasonable length sample ?
You seem to visit every day, why arn't you responding, have you fixed it ? [They're a nasty bunch here, but not that scary really].

Sorry for delay in responding. Thanks so much for all your interest. Thought I'd chat more when I failed to solve the problem. But actually this seems to work quite well

loadplugin ("C:\program files (x86)\avisynth\plugins\mvtools2.dll")
c=AviSource("D:\Video\wares-28-09-2021-17-35-08.avi")


super=MSuper(c, pel=2)
vfe=manalyse(super,truemotion=true,isb=false,delta =1)
vbe=manalyse(super,truemotion=true,isb=true,delta= 1)
filldrops = mflowinter(c,super,vbe,vfe,time=50)
#filldrops=subtitle(filldrops,"mv")

filldrops2=duplicateframe(filldrops,0).trim(0,framecount-1) #nudge by 1 frame

#1 fix dupedrop
dupedrop = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "1")
fixed1=ConditionalFilter(c, dupedrop, c, "ydifferencefromprevious(selectevery(c,1,-1))", "lessthan", "ydifferencefromprevious(selectevery(c,1,1))")

#2 fix remaining dupes, assuming dropdupe
fixed2 = ConditionalFilter(fixed1, filldrops2, fixed1, "YDifferencetonext()", "lessthan", "1")

return fixed2

My video is from a screen copier (Flashback) and the dupes are 99% just before or just after the drop. It doesn't work at all for those that aren't adjacent, or obviously for the occasional double dupe.

Found quite a few entertaining quirks in AviSynth. Amazing how it can (quickly) reference clips generated by its own runtime routines.

John I can't get your script to work - there seems to be an undefined function "mt_merge" at line 47.

Must now study all these scripts - great to learn from the experts.

[StainlessS]

Good, you seem happy, your god must be smiling upon you.

mt_merge is masktools2 filter.

[stephen22]

Thanks

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

All of them.

[StainlessS]

I had sorta meant how many are infinitely bigger than the other infinities, [dont remember exactly how I wanted to phrase it]
but I did not have room in the limited sig text allowed, about 260 chars, so I just cut it short with about 1 char spare.

[stephen22]

Yes, I understood right.
Answer: all of them.
Thanks so much for your help and encouragement.

John your script is marvellous. I have videos from an HDMI/PAL converter that are not nearly so easy, with random dupes, sometimes 2 or 3 at a time, usually well away from the drops. Your script makes a big difference. I think I've got the general idea, starting with drops and then smoothing everything out, but I'll have to study it further to get the details.

It needed one or two tweaks for us lesser mortals, loading in masktools2 and tivtc, and generalising the framerate numerator in MFlowFps to round(framerate(source)*2) for the benefit of PAL users, but it's ingenious and very effective. Thanks very much.

[johnmeyer]

I'm trying to improve the matching (see the thread I started), and then will get around to building a workaround to the TDecimate. The script is good, but I think it can be a LOT better.

[stephen22]

It works extremely well for me - though it does weird things at scene changes. I've respectfully suggested adding a tiny modification to your brilliant Black/White algorithm

MyMask2=ConditionalFilter(source,blackframe,MyMask,
\"((YDifferenceFromPrevious(selectevery(source, 1, -1)) +
\ YDifferenceFromPrevious(selectevery(source, 1,-2)) +
\ YDifferenceFromPrevious(selectevery(source, 1,-3)) +
\ YDifferenceFromPrevious(selectevery(source, 1,-4)) +
\ YDifferenceFromPrevious(selectevery(source, 1,-5)) +
\ YDifferenceFromPrevious(selectevery(source, 1,-6)) ) / 6 ) /
\ (YDifferenceFromPrevious(source) + 0.01)","lessthan","SceneThresh")

Return MyMask2

SceneThresh = 0.25 seems to work quite well here. I do 6 measurements because in my footage there can be 3 dupes in 4 frames.

[johnmeyer]

Yeah, I never got around to detecting scene changes, so thanks for doing that. I assume you sum MyMask2 with my existing mask?

As for my work on improving detection, I did quite a bit of work on using motion estimation (http://forum.doom9.org/showthread.php?t=183313), but that effort yielded no usable results. More importantly, when I tried my existing YDifference detection on some pretty challenging (low motion) test cases that I manufactured, it worked every time.

So, the only thing left is to fix the TDecimate problem. Zorr's idea of doing the decimation three times and then comparing will make the script too slow and, as I said, in good humor when the idea was first proposed, it is a delightful kludge.

I have another idea, which is to simply do the decimation in two passes. After all, TDecimate works with 3:2 pulldown and there is never any boundary problem. Therefore, I think the TDecimate failure may be related to having such a high decimation rate, where almost every single frame is duplicated. So my idea is to decimate half of the frames to be removed and then, with a slightly different Cycle parameter, decimate the other half.

I'm on another project now, so I won't get to it right away.