Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th March 2012, 12:38   #1  |  Link
sven_x
Registered User
 
Join Date: Oct 2011
Location: Germany
Posts: 39
MorphDups - replace duplicate frames by interpolations

USB video grabbers automatically replace disturbed frames (with bad sync) by identical copies of the last intact frame. As a result parts of the grabbed video are filled with one ore more duplicate frames.
Especially grabbings from VHS suffer from this problem.
A grabbing from a bad VHS copy that was recorded on a cheap recorder without a sync track had up to 1500 duplicate frames per hour.

..abc...........opq..stuvwxyz......... Original frames of the movie (good frames)
..a-c...........o-q..s------z......... VHS with drop outs (tape wrinkel)
..aac...........ooq..sssssssz......... Output of USB Video Grabber (duplicates instead of original frames)

Figg 1: Duplicate frames replace corrupted frames

The following script searches for duplicate frames. From the result it generates a second script, that is using Mud Funky's Morph function to replace duplicate frames by interpolated frames. The result is very smooth and looks fine.

Version 0.7a of the script.
PHP Code:
# http://forum.doom9.org/showthread.php?p=1563921 
# Morphdups.avs by Sven_x 2012 version 0.7a 
# search duplicate frames in a clip and generate a function "morphmydups.avs"  
# that interpolates between the border frames 

# Guide: 
# Run a first pass with this script. It searches for duplicate frames. 
# From that it generates a script with a function "morphmydups.avs"  
# that you can use to replace automaticly the duplicate frames by interpolations 

# To do: 
# perhaps YDifferenceFromPrevious(N)  can be replaced by YDifferenceToNext(N-1)  

# Changelog 

# YDifferenceFromPrevious() can ONLY be used within conditional functions, as it changes on every frame 
# Also remember that conditional variables must be assigned bottom up. 
# http://avisynth.org.ru/docs/english/corefilters/conditionalfilter.htm 

# source  and parameters .................................................................................. 
global source1=AVISource("J:\test\test79400.avi"
#DirectshowSource("raw.avi",30,true,false,true) 
#global source1=SegmentedAVISource("segmented.avi") 

last=source1 
ConvertToYV12
()  #  YV12 needed for MVTools and for CCE Basic 

reduce=true #true = crop output to 16x16 px to speed up encoding (150%) if you run the script by encoding 

# variables  ................................................................................. 
dup_thresh=0.0001    #threshold, below threshold is considered as duplicate frame 
sc_change_thresh=5.1   #a difference above this threshold is considered as scene change (excluded from list) 

gap_pos=0               #Gap start 
gap_l=0                   #Gap length 
pregap=0                 #Gap YDifference before gap 
postgap=0               #Gap YDifference after gap 
gap_end=false         # Bolean, triggers writing of gap properties into dup.txt file 
gapmarker=""         #Visual Gap length marker 

#Test function: display parameter results in clip ................................................................................. 

#ScriptClip("""Subtitle("frame_number: "+String(current_frame)+", YdiffP: "+String(ydfp)+", YdiffN: "+String(ydfn)) """\ 
#               +"""Subtitle(x=8, Y=20,text_color=$FF4422,\ 
#           "pos: "+String(gap_pos)+", gap_l: "+String(gap_l)+", pregap: "+String(pregap)\ 
#         +", postgap: "+String(postgap)+", gapmarker: "+LeftStr("===============",gap_l) ) """) 

#Search gaps with duplicate frames and write dups_report.txt + txt-files 
#................................................................................ 
fg1() #find begin of gap 
fg2() #find middle of gap 
fg3() #find end of gap 

colon=" "   #define colon 


WriteFileStart("morphmydups.avs"""" "# Function that replaces dups by interpolated streams of frames" +chr(13) """, \ 
+""" "# generated by MorphDups.avs v0.7, see http://forum.doom9.org/showpost.php?p=1561928&postcount=52" +chr(13)+chr(13)""", \ 
+""" "# needs:" +chr(13)""", \ 
+""" "# - Mud Funky's Morph function, see http://forum.doom9.org/showpost.php?p=1499620&postcount=2" +chr(13)""", \ 
+""" "# - MvTools2" +chr(13)+chr(13) """,\ 
+""" "# Example: Morph (19,22) replaces frame 20 and 21 by an interpolation between 19 and 22" +chr(13)+chr(13) """,\ 
+""" "# sc_change_thresh=" ""","sc_change_thresh",""" "   # morph statements are disabled when YDifferenceToNext is greater" +chr(13)+chr(13) """,\ 
+""" "function morphmydups (clip c) {" +chr(13) """,\
+
""" "   c     #load clip"  """) 

WriteFileIf("morphmydups.avs""gap_end==true && postgap<sc_change_thresh"""" "   morph("  """,  "gap_pos-1"""" "," """ ,"gap_pos+gap_l" ,\ 
+
""" ")           # YDiffNext=" ""","postgap" ,""" ", length = "  """ , "gap_l", """LeftStr("===========",gap_l)""") 

# when postgap > sc_change_thresh morph statements are disabled by a leading "'" 
WriteFileIf("morphmydups.avs""gap_end==true && postgap>=sc_change_thresh"""" " # morph("  """,  "gap_pos-1", """ "," """ ,"gap_pos+gap_l" ,\ 
+""" ")           # YDiffNext=" ""","postgap" ,""" ", length = "  """ , "gap_l", """LeftStr("===========",gap_l)""") 

WriteFileEnd ("morphmydups.avs"""" chr(13) +"# End of function" +chr(10)""" ) 


# functions to locate gap .............................................................. 
# only one boolean expression per FrameEvaluate 
# begin of gap: gap_pos=current_frame, gap_l=1 
# YDifferenceFromPrevious() > dup_thresh && YDifferenceToNext() < dup_thresh ?  
function fg1(clip c) { 
c=FrameEvaluate(c,"gap_pos= ydfp > dup_thresh && ydfn < dup_thresh ? current_frame+1 : gap_pos" 
c=FrameEvaluate(c,"gap_l= ydfp > dup_thresh && ydfn < dup_thresh ? 0 : gap_l" 
c=FrameEvaluate(c,"pregap= ydfp > dup_thresh && ydfn < dup_thresh ? ydfp : pregap" 
return(
c


#middle gap: gap_l=gap_l+1 
#YDifferenceFromPrevious() < dup_thresh && YDifferenceToNext() < dup_thresh ?  
function fg2(clip c) { 
c=FrameEvaluate(c,"gap_l= ydfp < dup_thresh && ydfn < dup_thresh ? gap_l+1 : gap_l" 
return(
c


#end gap: gap_l=gap_l+1 , gap_end=true, postgap=yDifferenceToNext, call writefileif 
#YDifferenceFromPrevious() < dup_thresh && YDifferenceToNext() > dup_thresh ?  
#store ydiffp and ydiffn in variables 
function fg3(clip c) { 
c=FrameEvaluate(c,"gap_end= ydfp < dup_thresh && ydfn > dup_thresh && current_frame > 0 ? true : false" 
c=FrameEvaluate(c,"gap_l= ydfp < dup_thresh && ydfn > dup_thresh ? gap_l+1 : gap_l" 
c=FrameEvaluate(c,"ydfp=YDifferenceFromPrevious(source1)"
                            +
"ydfn=YDifferenceToNext(source1)" 
                            +
"postgap= ydfp < dup_thresh && ydfn > dup_thresh ? ydfn : postgap" 
return(
c
}  

#Cropping to increase encoding speed ...................................................................... 
#cropping increases speed up to 400% for first pass if you run the script by encoding 
reduce last.crop(001616) : last 
The generated output of the script, named morphmydups.avs, looks so:
Code:
function morphmydups (clip c) {
   c
 # morph(56,67)           # YDiffNext=10.803629, length = 10==========
   morph(103,105)           # YDiffNext=4.838520, length = 1=
   morph(106,108)           # YDiffNext=4.491889, length = 1=
   morph(109,111)           # YDiffNext=4.452096, length = 1=

} # End of function
Please note, that this script provides you with the possibility of interactive correction. You get a list that you can edit. Before rendering the whole video, you can open the script in AvsP and test only that gaps that look critical. Either because they are marked as very long gaps (========) , or because YDifference across the gap is to high. In the latter case the script automaticly disables those morph statements that have a YDifference above a limit. You have the opportunity to delete the "#'" characters, thus re-enabling interpolation across that gap.

And this is how a calling script may look:
PHP Code:
#0. Imports, Loadplugin ..................................................................................
import ("J:\plugins\Morph1b.avs"# overlays info text on interpolated frames
#import ("J:\plugins\Morph.avs") 
loadplugin ("J:\plugins\mvtools2.dll")
import ("morphmydups.avs")

stack=true #true= debug view: small version of source and interpolated are stacked 

#1. Open file ..................................................................................
AVIFileSource("J:\test\test79400.avi")
source=last

#morph function ..................................................................................
morphmydups(source)

#Debug view  ..................................................................................
stack stackvertical(lastsource.subtitle("Input (unprocessed)") ).BilinearResize(width(source)/2height(source))  : last 
Changelog:
Version 0.7a implemented correction from Gavino, please see posting below

German explanation/deutsche Erklärung hier.

Last edited by sven_x; 9th March 2012 at 11:26.
sven_x is offline   Reply With Quote
Old 8th March 2012, 13:33   #2  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,385
Quote:
Originally Posted by sven_x View Post
The generated output of the script, named morphmydups.avs, looks so:
Code:
function morphmydups (clip c) {

 # morph(c,56,67)           # YDiffNext=10.803629, length = 10==========
   morph(c,103,105)           # YDiffNext=4.838520, length = 1=
   morph(c,106,108)           # YDiffNext=4.491889, length = 1=
   morph(c,109,111)           # YDiffNext=4.452096, length = 1=

} # End of function
...
v 0.7 problems
Currently the output script morphmydups.avs only works for the last entry of the listed morph statements if you call it as a function from another script.
I haven't looked at how your function code is actually generated (and don't have the time right now), but the output needs to look like this:
Code:
function morphmydups (clip c) {
   c
   morph(56,67)
   morph(103,105)
   morph(106,108)
   morph(109,111)
}
so that the output of each morph call becomes the input to the following one.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 8th March 2012, 17:37   #3  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,218
You might take a look at some of the following posts from several years ago.

automated framedrop filler script

FillDropsI

I used the original code that was created many years ago ("filldrops," which is found in the first link above) which automatically finds duplicates and replaces the second instance with a motion-estimated frame, and I updated that script (the second link) to use MVTools2 instead of MVTools, and to work on interlaced video as well as progressive.

Last edited by johnmeyer; 8th March 2012 at 17:37. Reason: Removed accidentally-created emoticon
johnmeyer is offline   Reply With Quote
Old 8th March 2012, 17:46   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,218
BTW, I forgot to add something in the previous post:

Sometimes the capture card drops a frame, and this creates a "jump" in the motion. Then, to keep the audio in sync, it adds a duplicate frame, but doesn't get around to doing this until several frames after the dropped frame. In this case, you want to decimate (remove) one of the two duplicate frames, but then you have to go looking for a nearby "jump" in the motion, and insert your motion-estimated frame at that point. This is a much, much more difficult chore, but one that can still be done (in some cases) in a single pass, with conditional logic. I started a thread last spring about this exact subject, and got a tremendous amount of help that resulted in (for me) a very good solution. Here's the link to that thread:

Automatically fix dups followed (eventually) by drops
johnmeyer is offline   Reply With Quote
Old 8th March 2012, 23:23   #5  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
Keska,
You didn't understand the first post, but this is an english board, sorry. Try different translation software.

Johnmeyer,
Yes that's exactly the problem I've had before. I'm glad you found a solution, I will have to try it! However I have an even more difficult problem. The duplicate or jump finding of my sample is inaccurate, because there is so many comets/dropouts. The whole point of syncing the video is to remove them I don't know what to do about this problem.
jmac698 is offline   Reply With Quote
Old 10th August 2012, 10:17   #6  |  Link
Nightwhistle
Registered User
 
Join Date: Aug 2012
Posts: 4
I cannot make that script to generate duplicates list. I have tons of duplicates in my footage yet it finds nothing
I have fraps video i recorded with lots of duplicate frames i want to replace by interpolations. Bad thing is, i tried almost everything i found on this site and nothing worked.
Can someone give me some tips? I'm very new in this field and I'm very bad at scripting, but from what i read it is possible to do something like that somehow.
Nightwhistle is offline   Reply With Quote
Old 10th August 2012, 10:24   #7  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
Raise dup_thresh
jmac698 is offline   Reply With Quote
Old 13th August 2012, 17:45   #8  |  Link
matfra
Registered User
 
Join Date: Jul 2009
Posts: 111
Hi sven_x,
I manage to get working function morphmydups() using this version of morph.
function morph (clip c, int in, int "out", int "blksize")
{

Function fill_loop(string s, int stop, int count)
{
return (stop == 0) ? s : string("tofill.mflowinter(morph_spr,fill_vb,fill_vf,time=" + string(100*(count - stop)/float(count))) + ",thscd1=255,thscd2=255).selectevery(tofill.framecount(),0)," + fill_loop(s,stop-1,count)
}

out=default(out, 0)
blksize=default(blksize,16)

d=c.trim(in,out)# in-1?

numframes=d.framecount-2
tofill=d.selectevery(c.framecount(),0,c.framecount()-1)
global morph_spr=tofill.msuper()
fill_vf=morph_spr.manalyse(truemotion=true,blksize=blksize,isb=false,overlap=blksize/2)
fill_vb=morph_spr.manalyse(truemotion=true,blksize=blksize,isb=true,overlap=blksize/2)
filled=eval("interleave(" + fill_loop("" ,d.framecount()-1,d.framecount()-1) + "tofill.selectevery(tofill.framecount(),tofill.framecount())).assumefps(c.framerate())")
c.trim(0,in-1)++filled++c.trim(out+1,0)
}

What is your version of Morph are you using ? Morph1b.avs Can you paste ti for me please.


Is there a way to put the analysis part into a AVS file that I could call in a function like: Analisys()

Last edited by matfra; 13th August 2012 at 19:18.
matfra is offline   Reply With Quote
Old 14th August 2012, 11:50   #9  |  Link
Nightwhistle
Registered User
 
Join Date: Aug 2012
Posts: 4
Ok, i made it write morphmydups file with increasing the dup_tresh
Now, how i start 2nd pass with that last script, it says 'there is no function named "morph" (morphmydups.avs), line 14.

Edit: I made it to work by importing Mud's 'morph' function from other post. Oversight, sorry.
Now I'm getting Vdub error after few frames renedered: An out-of-bounds memory access (access violation) occurred in module 'VirtualDub'...
...reading address 5BEFFC38.
It is 1920x1080 full uncompressed fraps video.

Last edited by Nightwhistle; 14th August 2012 at 12:10.
Nightwhistle is offline   Reply With Quote
Old 14th August 2012, 12:09   #10  |  Link
matfra
Registered User
 
Join Date: Jul 2009
Posts: 111
Hey,
I would like to know if its possible to modify the script to replace the Next Frame and the Before frame only ?? Like this.

Legend: K for KeyFrame
D for DupFrame
R for ReplacedFrame

K, D, D, D, K, D, D, D, D, D, D, D, K, D, D, D, K .. This to this.


K, R, D, R, K, R, D, D, D, D, D, R, K, R, D, R, K
matfra is offline   Reply With Quote
Old 18th August 2012, 20:01   #11  |  Link
Nightwhistle
Registered User
 
Join Date: Aug 2012
Posts: 4
Johnmeyer's script is like only one i found that is doing exactly what i need: forum.doom9.org/archive/index.php/t-160623.html
Nightwhistle is offline   Reply With Quote
Old 18th August 2012, 21:54   #12  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,218
Quote:
Originally Posted by Nightwhistle View Post
Johnmeyer's script is like only one i found that is doing exactly what i need: forum.doom9.org/archive/index.php/t-160623.html
I'm glad that worked for you.

As I posted above, several months ago, if you have the more difficult -- and more usual -- problem of the duplicate and the drop not happening on adjacent frames, then you need to look at some of the ideas presented in this thread:

Automatically fix dups followed (eventually) by drops
johnmeyer is offline   Reply With Quote
Old 19th August 2012, 10:05   #13  |  Link
Nightwhistle
Registered User
 
Join Date: Aug 2012
Posts: 4
Here's what i've got:
[Bunch of duplicated frames recording] [Edited video]

It had some problems at the end of large file i tried, but eventually, i'll just cut large files down in size before using the script so that wont be huge problem.
Nightwhistle is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.