Dupe Frame Script

[TruthOverFacts]

Is there any type of script I can run in VirtualDub where it detects all of the duplicate frames in the file, takes me to each one and I can view the frame before and after before deleting it to double check? I would need to delete the audio with the 1 frame at the same time.

[johnmeyer]

I don't know of any way to detect dups in VirtualDub, but I have many AVISynth scripts that do this.

[TruthOverFacts]

I don't know of any way to detect dups in VirtualDub, but I have many AVISynth scripts that do this.

What do you recommend?

[johnmeyer]

This is the AVISynth script I use to find duplicates. It outputs the duplicate frame numbers to a text file. You can then use that with whatever editing program you have to go to those duplicates and then do something, like delete the duplicate. Be aware, however, that if you delete the duplicate, the audio will now be one frame ahead. Also be aware that if your video files has duplicates, those are often added because the streaming or frame capture dropped a frame and then, to keep the audio in sync, it duplicates a frame. If you only delete the duplicates but do not fill the gaps, you will create more problems.

Read the comments to see how to use this script. As posted below, it merely displays the difference threshold on the screen. To actually get it to write the frame numbers, you'll have to comment out the ScriptClip line and then un-comment one of the two WriteFileIf lines (I suggest using the second one). You also may need to change AssumeBFF to AssumeTFF if your video is HD or PAL.

Finally, you can control how similar the frames must be by changing the BlankThreshold value. The point of the ScriptClip line is to display those threshold values so you can adjust this number. Setting it to some really small value, like 0.1, will cause it to only show perfectly identical frames.
#This script finds exact duplicate frames and outputs the frame numbers
loadPlugin("c:\Program Files\AviSynth 2.5\plugins\MPEGDecoder.dll")
loadPlugin("c:\Program Files\AviSynth 2.5\plugins\dgdecode.dll")

global blankthreshold=3

filename = "e:\output_duplicate_frames.txt"
AVISource("e:\fs.avi").killaudio()
i=AssumeBFF.ConvertToYV12

#This line below will output EVERY frame that is below threshold, which results in LOTS of frames
#Normally you don't do this, but it is included for those who want this capability.
#WriteFileIf(last, filename, "(YDifferenceFromPrevious(i)<=blankthreshold)", "current_frame+1", append = false)

#The line below writes the FIRST frame that falls below the threshold
#WriteFileIf(last, filename, "(YDifferenceFromPrevious(i)>blankthreshold)&&YDifferenceToNext(i)<=blankthreshold", "current_frame", append = false)

#Use this instead of WriteFile in order to determine blankthreshold
ScriptClip("Subtitle(String(YDifferenceFromPrevious(i)))")
You might also find some use for what I developed in this thread:

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?t=161758)

I kept trying to improve on that, and worked on it some more here:

Need help "perfecting" script to delete drops and dups (http://forum.doom9.org/showthread.php?t=183351)

but unfortunately ran into a problem I could not solve (TDecimate couldn't handle the decimation properly).

[StainlessS]

You could use something like this [requires FrameSel] to show dupe frame + Prev and Next frames, of all frames in a frames list file.

Avisynth script

Avisource("...")
Frames = ".\Frames.txt"
SHOW=True
FrameSel(cmd=Frames, show=SHOW,Extract=3)

Untested.

EDIT: FrameSel:- http://avisynth.nl/index.php/FrameSel

FrameSel(Clip, int F1, ... , int Fn, string 'scmd',string 'cmd', bool 'show', bool 'ver',bool "reject",bool "ordered",
\ bool "debug", int "Extract"=1)

# ...

Extract Int, Default 1, MUST be ODD, 1 to 11. Error if not default 1 AND neither Ordered nor Reject == True.
Let us call the UNIQUE resultant frames after Ordered and/or Reject Processing 'Target' frames.
The number of frames pulled out on either side of Target frame is Extract / 2 (integer divide), so when Extract=3, it will
extract 1 frame before target, the Target frame, and 1 frame after target, for each and every Target frame.
This arg allows you to select eg 3 frames for each Target frame, the previous frame, Target frame and next frame.
May be of use to extract bad frames together with eg 1 frame either side for saving as bitmaps and editing
in some kind of RotoScope editor to repair bad frames using image from adjacent frames. The edited frames could
then be re-loaded, middle one selected via SelectEvery(3,1), and then put back into original source clip via FrameRep().
Another simple use could be when you have a clip where each frame before a scene change is bad, with Ordered=True
and Extract=3, you pull out 3 frames, a SelectEvery(3,0) would select the frames previous to the bad frames and
then use Framerep() to replace the bad frames with those previous to them. A SelectEvery(3,2) would select the frames
after Target frames.
The total number of frames pulled out of the source clip will be (Extract * number of Target frames).
Reject is less likely to be of use with Extract, unless frame select commands specify good frames rather than bad,
where Reject in FrameRep should also be true (same as in FrameSel, as always).
See FrameRep() for example uses of Extract.