MorphDups - replace duplicate frames by interpolations

[sven_x]

USB video grabbers automatically replace disturbed frames (with bad sync) by identical copies of the last intact frame. As a result parts of the grabbed video are filled with one ore more duplicate frames.
Especially grabbings from VHS suffer from this problem.
A grabbing from a bad VHS copy that was recorded on a cheap recorder without a sync track had up to 1500 duplicate frames per hour.

..abc...........opq..stuvwxyz......... Original frames of the movie (good frames)
..a-c...........o-q..s------z......... VHS with drop outs (tape wrinkel)
..aac...........ooq..sssssssz......... Output of USB Video Grabber (duplicates instead of original frames)
Figg 1: Duplicate frames replace corrupted frames

The following script searches for duplicate frames. From the result it generates a second script, that is using Mud Funky's Morph function (http://forum.doom9.org/showpost.php?p=1499620&postcount=2) to replace duplicate frames by interpolated frames. The result is very smooth and looks fine.

Version 0.7a of the script.
# http://forum.doom9.org/showthread.php?p=1563921
# Morphdups.avs by Sven_x 2012 version 0.7a
# search duplicate frames in a clip and generate a function "morphmydups.avs"
# that interpolates between the border frames

# Guide:
# Run a first pass with this script. It searches for duplicate frames.
# From that it generates a script with a function "morphmydups.avs"
# that you can use to replace automaticly the duplicate frames by interpolations

# To do:
# perhaps YDifferenceFromPrevious(N) can be replaced by YDifferenceToNext(N-1)

# Changelog

# YDifferenceFromPrevious() can ONLY be used within conditional functions, as it changes on every frame
# Also remember that conditional variables must be assigned bottom up.
# http://avisynth.org.ru/docs/english/corefilters/conditionalfilter.htm

# source and parameters ..................................................................................
global source1=AVISource("J:\test\test79400.avi")
#DirectshowSource("raw.avi",30,true,false,true)
#global source1=SegmentedAVISource("segmented.avi")

last=source1
ConvertToYV12() # YV12 needed for MVTools and for CCE Basic

reduce=true #true = crop output to 16x16 px to speed up encoding (150%) if you run the script by encoding

# variables .................................................................................
dup_thresh=0.0001 #threshold, below threshold is considered as duplicate frame
sc_change_thresh=5.1 #a difference above this threshold is considered as scene change (excluded from list)

gap_pos=0 #Gap start
gap_l=0 #Gap length
pregap=0 #Gap YDifference before gap
postgap=0 #Gap YDifference after gap
gap_end=false # Bolean, triggers writing of gap properties into dup.txt file
gapmarker="" #Visual Gap length marker

#Test function: display parameter results in clip .................................................................................

#ScriptClip("""Subtitle("frame_number: "+String(current_frame)+", YdiffP: "+String(ydfp)+", YdiffN: "+String(ydfn)) """\
# +"""Subtitle(x=8, Y=20,text_color=$FF4422,\
# "pos: "+String(gap_pos)+", gap_l: "+String(gap_l)+", pregap: "+String(pregap)\
# +", postgap: "+String(postgap)+", gapmarker: "+LeftStr("===============",gap_l) ) """)

#Search gaps with duplicate frames and write dups_report.txt + txt-files
#................................................................................
fg1() #find begin of gap
fg2() #find middle of gap
fg3() #find end of gap

colon=" " #define colon


WriteFileStart("morphmydups.avs", """ "# Function that replaces dups by interpolated streams of frames" +chr(13) """, \
+""" "# generated by MorphDups.avs v0.7, see http://forum.doom9.org/showpost.php?p=1561928&postcount=52" +chr(13)+chr(13)""", \
+""" "# needs:" +chr(13)""", \
+""" "# - Mud Funky's Morph function, see http://forum.doom9.org/showpost.php?p=1499620&postcount=2" +chr(13)""", \
+""" "# - MvTools2" +chr(13)+chr(13) """,\
+""" "# Example: Morph (19,22) replaces frame 20 and 21 by an interpolation between 19 and 22" +chr(13)+chr(13) """,\
+""" "# sc_change_thresh=" ""","sc_change_thresh",""" " # morph statements are disabled when YDifferenceToNext is greater" +chr(13)+chr(13) """,\
+""" "function morphmydups (clip c) {" +chr(13) """,\
+""" " c #load clip" """)

WriteFileIf("morphmydups.avs", "gap_end==true && postgap<sc_change_thresh", """ " morph(" """, "gap_pos-1", """ "," """ ,"gap_pos+gap_l" ,\
+""" ") # YDiffNext=" ""","postgap" ,""" ", length = " """ , "gap_l", """LeftStr("===========",gap_l)""")

# when postgap > sc_change_thresh morph statements are disabled by a leading "'"
WriteFileIf("morphmydups.avs", "gap_end==true && postgap>=sc_change_thresh", """ " # morph(" """, "gap_pos-1", """ "," """ ,"gap_pos+gap_l" ,\
+""" ") # YDiffNext=" ""","postgap" ,""" ", length = " """ , "gap_l", """LeftStr("===========",gap_l)""")

WriteFileEnd ("morphmydups.avs", """ chr(13) +"} # End of function" +chr(10)""" )


# functions to locate gap ..............................................................
# only one boolean expression per FrameEvaluate
# begin of gap: gap_pos=current_frame, gap_l=1
# YDifferenceFromPrevious() > dup_thresh && YDifferenceToNext() < dup_thresh ?
function fg1(clip c) {
c=FrameEvaluate(c,"gap_pos= ydfp > dup_thresh && ydfn < dup_thresh ? current_frame+1 : gap_pos" )
c=FrameEvaluate(c,"gap_l= ydfp > dup_thresh && ydfn < dup_thresh ? 0 : gap_l" )
c=FrameEvaluate(c,"pregap= ydfp > dup_thresh && ydfn < dup_thresh ? ydfp : pregap" )
return(c)
}

#middle gap: gap_l=gap_l+1
#YDifferenceFromPrevious() < dup_thresh && YDifferenceToNext() < dup_thresh ?
function fg2(clip c) {
c=FrameEvaluate(c,"gap_l= ydfp < dup_thresh && ydfn < dup_thresh ? gap_l+1 : gap_l" )
return(c)
}

#end gap: gap_l=gap_l+1 , gap_end=true, postgap=yDifferenceToNext, call writefileif
#YDifferenceFromPrevious() < dup_thresh && YDifferenceToNext() > dup_thresh ?
#store ydiffp and ydiffn in variables
function fg3(clip c) {
c=FrameEvaluate(c,"gap_end= ydfp < dup_thresh && ydfn > dup_thresh && current_frame > 0 ? true : false" )
c=FrameEvaluate(c,"gap_l= ydfp < dup_thresh && ydfn > dup_thresh ? gap_l+1 : gap_l" )
c=FrameEvaluate(c,"ydfp=YDifferenceFromPrevious(source1)"\
+"ydfn=YDifferenceToNext(source1)" \
+"postgap= ydfp < dup_thresh && ydfn > dup_thresh ? ydfn : postgap" )
return(c)
}

#Cropping to increase encoding speed ......................................................................
#cropping increases speed up to 400% for first pass if you run the script by encoding
reduce ? last.crop(0, 0, 16, 16) : last


The generated output of the script, named morphmydups.avs, looks so:
function morphmydups (clip c) {
c
# morph(56,67) # YDiffNext=10.803629, length = 10==========
morph(103,105) # YDiffNext=4.838520, length = 1=
morph(106,108) # YDiffNext=4.491889, length = 1=
morph(109,111) # YDiffNext=4.452096, length = 1=

} # End of function

Please note, that this script provides you with the possibility of interactive correction. You get a list that you can edit. Before rendering the whole video, you can open the script in AvsP and test only that gaps that look critical. Either because they are marked as very long gaps (========) , or because YDifference across the gap is to high. In the latter case the script automaticly disables those morph statements that have a YDifference above a limit. You have the opportunity to delete the "#'" characters, thus re-enabling interpolation across that gap.

And this is how a calling script may look:
#0. Imports, Loadplugin ..................................................................................
import ("J:\plugins\Morph1b.avs") # overlays info text on interpolated frames
#import ("J:\plugins\Morph.avs")
loadplugin ("J:\plugins\mvtools2.dll")
import ("morphmydups.avs")

stack=true #true= debug view: small version of source and interpolated are stacked

#1. Open file ..................................................................................
AVIFileSource("J:\test\test79400.avi")
source=last

#morph function ..................................................................................
morphmydups(source)

#Debug view ..................................................................................
stack ? stackvertical(last, source.subtitle("Input (unprocessed)") ).BilinearResize(width(source)/2, height(source)) : last

Changelog:
Version 0.7a implemented correction from Gavino, please see posting below

German explanation/deutsche Erklärung hier (http://www.engon.de/audio/vhs_avisynth.htm#Interpolate).

[Gavino]

The generated output of the script, named morphmydups.avs, looks so:
function morphmydups (clip c) {

# morph(c,56,67) # YDiffNext=10.803629, length = 10==========
morph(c,103,105) # YDiffNext=4.838520, length = 1=
morph(c,106,108) # YDiffNext=4.491889, length = 1=
morph(c,109,111) # YDiffNext=4.452096, length = 1=

} # End of function
...
v 0.7 problems
Currently the output script morphmydups.avs only works for the last entry of the listed morph statements if you call it as a function from another script.
I haven't looked at how your function code is actually generated (and don't have the time right now), but the output needs to look like this:
function morphmydups (clip c) {
c
morph(56,67)
morph(103,105)
morph(106,108)
morph(109,111)
}
so that the output of each morph call becomes the input to the following one.

[johnmeyer]

You might take a look at some of the following posts from several years ago.

automated framedrop filler script (http://forum.doom9.org/showthread.php?p=753779#post753779)

FillDropsI (http://forum.doom9.org/showthread.php?p=1493082#post1493082)

I used the original code that was created many years ago ("filldrops," which is found in the first link above) which automatically finds duplicates and replaces the second instance with a motion-estimated frame, and I updated that script (the second link) to use MVTools2 instead of MVTools, and to work on interlaced video as well as progressive.

[johnmeyer]

BTW, I forgot to add something in the previous post:

Sometimes the capture card drops a frame, and this creates a "jump" in the motion. Then, to keep the audio in sync, it adds a duplicate frame, but doesn't get around to doing this until several frames after the dropped frame. In this case, you want to decimate (remove) one of the two duplicate frames, but then you have to go looking for a nearby "jump" in the motion, and insert your motion-estimated frame at that point. This is a much, much more difficult chore, but one that can still be done (in some cases) in a single pass, with conditional logic. I started a thread last spring about this exact subject, and got a tremendous amount of help that resulted in (for me) a very good solution. Here's the link to that thread:

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?p=1510780#post1510780)

[jmac698]

Keska,
You didn't understand the first post, but this is an english board, sorry. Try different translation software.

Johnmeyer,
Yes that's exactly the problem I've had before. I'm glad you found a solution, I will have to try it! However I have an even more difficult problem. The duplicate or jump finding of my sample is inaccurate, because there is so many comets/dropouts. The whole point of syncing the video is to remove them :) I don't know what to do about this problem.

[Nightwhistle]

I cannot make that script to generate duplicates list. I have tons of duplicates in my footage yet it finds nothing :(
I have fraps video i recorded with lots of duplicate frames i want to replace by interpolations. Bad thing is, i tried almost everything i found on this site and nothing worked.
Can someone give me some tips? I'm very new in this field and I'm very bad at scripting, but from what i read it is possible to do something like that somehow.

[jmac698]

Raise dup_thresh

[matfra]

Hi sven_x,
I manage to get working function morphmydups() using this version of morph.
function morph (clip c, int in, int "out", int "blksize")
{

Function fill_loop(string s, int stop, int count)
{
return (stop == 0) ? s : string("tofill.mflowinter(morph_spr,fill_vb,fill_vf,time=" + string(100*(count - stop)/float(count))) + ",thscd1=255,thscd2=255).selectevery(tofill.framecount(),0)," + fill_loop(s,stop-1,count)
}

out=default(out, 0)
blksize=default(blksize,16)

d=c.trim(in,out)# in-1?

numframes=d.framecount-2
tofill=d.selectevery(c.framecount(),0,c.framecount()-1)
global morph_spr=tofill.msuper()
fill_vf=morph_spr.manalyse(truemotion=true,blksize=blksize,isb=false,overlap=blksize/2)
fill_vb=morph_spr.manalyse(truemotion=true,blksize=blksize,isb=true,overlap=blksize/2)
filled=eval("interleave(" + fill_loop("" ,d.framecount()-1,d.framecount()-1) + "tofill.selectevery(tofill.framecount(),tofill.framecount())).assumefps(c.framerate())")
c.trim(0,in-1)++filled++c.trim(out+1,0)
}

What is your version of Morph are you using ? Morph1b.avs Can you paste ti for me please.


Is there a way to put the analysis part into a AVS file that I could call in a function like: Analisys()

[Nightwhistle]

Ok, i made it write morphmydups file with increasing the dup_tresh
Now, how i start 2nd pass with that last script, it says 'there is no function named "morph" (morphmydups.avs), line 14.

Edit: I made it to work by importing Mud's 'morph' function from other post. Oversight, sorry.
Now I'm getting Vdub error after few frames renedered: An out-of-bounds memory access (access violation) occurred in module 'VirtualDub'...
...reading address 5BEFFC38.
It is 1920x1080 full uncompressed fraps video.

[matfra]

Hey,
I would like to know if its possible to modify the script to replace the Next Frame and the Before frame only ?? Like this.

Legend: K for KeyFrame
D for DupFrame
R for ReplacedFrame

K, D, D, D, K, D, D, D, D, D, D, D, K, D, D, D, K .. This to this.


K, R, D, R, K, R, D, D, D, D, D, R, K, R, D, R, K

[Nightwhistle]

Johnmeyer's script is like only one i found that is doing exactly what i need: forum.doom9.org/archive/index.php/t-160623.html

[johnmeyer]

Johnmeyer's script is like only one i found that is doing exactly what i need: forum.doom9.org/archive/index.php/t-160623.htmlI'm glad that worked for you.

As I posted above, several months ago, if you have the more difficult -- and more usual -- problem of the duplicate and the drop not happening on adjacent frames, then you need to look at some of the ideas presented in this thread:

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?p=1510780#post1510780)

[Nightwhistle]

Here's what i've got:
[Bunch of duplicated frames recording (http://www.youtube.com/watch?v=rLzn4DvRadc)] [Edited video (http://www.youtube.com/watch?v=SXuuJOcx7f8)]

It had some problems at the end of large file i tried, but eventually, i'll just cut large files down in size before using the script so that wont be huge problem.