Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th June 2011, 23:30   #1  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
Automatically fix dups followed (eventually) by drops

Overview

A friend wanted a portion of this video of his son placing third in the state 3200 M track race:

2011 CIF Track & Field Championships

Keepvid wasn’t an option; WMRecorder and similar tools didn’t work. So, I had to use Camtasia (or SnagIt) which records screen video. Unfortunately, despite reading all the hints, I didn’t get smooth video.

Here’s a link to a really bad section of the video:

10 Second Problem Clip

If you want to try the script, download this clip. If you want to suggest a better way to capture the video, click on the first link above.

The first sixty frames (of the downloadable clip) has a huge number of problems, and then things settle down.


Problem I Tried To Solve

In a nutshell, the video randomly (i.e., not periodically) drops frames and then, to restore sync, duplicates a frame, usually within about five frames of the dup. So, here's the statement of the problem: I want to delete the dups and simultaneously synthesize a frame to insert at the point where the video jumps.


Prior art

This has been discussed before, both here:

Filter to remove duplicate frames

and here:

Inverse of Decimate

Also, MugFunky, a long time ago wrote a script for detecting and replacing exact duplicates:

FillDrops

However, his problem only included duplicates: apparently the capture mechanism did not drop frames to make up for the dups.


Solution

Didée created a nifty script for interpolating missing frames. This can be found in the "Inverse of Decimate" link above. However, there were three things missing in that script that I needed for my situation:
  1. I couldn’t find any way to easily and reliably tweak the mask settings in order to change the threshold for determining the size of the "jump" between frames needed to trigger the logic for creating a motion-estimated replacement.
  2. As a corollary to #1, I couldn’t figure out a way to put meaningful metrics onto the screen or into debugview so I could see, frame-by-frame, what sort of metrics correlated to the amount of visual discontinuity between frames. It is hard to tweak a threshold setting without metrics.
  3. I needed different decimation than did the OP in that thread, both because I didn’t need to go between 30 and 24 fps, but also because my duplicate and drop distribution is quite different (much more random).
  4. Despite the power of masking, I actually found that I could achieve far more "accurate" and reliable detection of major movement by doing a simple moving average of the YDiffPrevious values from surrounding frames, and then taking a ratio of that average with the value for the current frame. This approach largely eliminates the problem with Ydiff, namely that it can be very large for some parts of the video, and very small in other parts, depending on the amount of motion, contrast, etc.
So, with that as background, I created the script you see below. It seems to work well for this particular problem. However, if anyone wants to use it, here are a few issues that I didn’t quite solve.
  1. The TDecimate logic is pretty slow. I had to use rather large Cycle parameters because sometimes the dups/drops clustered in bunches. I tried to do a 2-pass TDecimate, which is supposed to be designed for exactly this sort of thing, and I have included that code below (commented out) to show what I tried. I tried various maxndl settings for pass-2, but never got results that were better (compared to the 1-pass solution shown below) by a big enough amount to warrant the extra grief of doing a two-pass decimation.
  2. Decimation still leaves some jumps. This seems to be a decimation problem and not a problem in the logic which detects jumps.
  3. The motion estimation is pretty ugly for some frames. I didn’t spend any time on this, and simply used the vectors that Didée created in that other thread. If this video were more important, I might try to tweak the settings to get better results.
  4. SetMTMode doesn’t seem to work reliably. I’m pretty sure, based on past posts by Didée and others that this is due to TDecimate. I was able to get the script to work for enough frames before it crashed to determine that SetMTMode wouldn’t improve the performance much unless I could come up with a better approach to decimation than TDecimate (i.e., because of the large cycle size, TDecimate was causing most of the slowness).

[edit]Click on the following link to go to updated script later in this thread:

Later version of script

Code:
# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
#
# Script overview
#
# Create interpolated frames a 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video.
# However, at each point where motion-estimated frame is inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.74
showdot = true # true for troubleshooting; otherwise, false

#SetMTMode(5,4)
global source=AVISource( "E:\CIF Track\3200-Complete (edited).avi" ).ConvertToYV12
#SetMTMode(2)

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2) 
bvec  = MAnalyse(super, overlap=4, isb = true, search=4, dct=5) 
fvec  = MAnalyse(super, overlap=4, isb = false, search=4, dct=5) 
double = source.MFlowFps(super, bvec, fvec, num=60, den=1, blend=false)

#Remove comment from ShowMetrics, and change "return final" to "return test" to look at metrics in order to determine proper JumpThresh
#test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)

interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate
RequestLinear
final=tdecimate(display=false,mode=1,cycleR=10,cycle=20)  # Decimate half of all frames (set to twice the length of "normal" dup/drop cycle)

#---------------------
#Alternate two-pass approach to decimation 

#Pass 1
#RequestLinear(debug=false)
#final=tdecimate(display=false,mode=4,output="e:\metrics.txt")

#Pass 2 (remember to un-comment "requestlinear")
#RequestLinear
#final=tdecimate(display=false,mode=2,rate=30,input="e:\metrics.txt",maxndl=20)
#---------------------

return final
#return stackvertical(source,final)

#----------------
#This function displays the YDiff value that will be used for detecting big jumps

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ ((YDifferenceFromPrevious(selectevery(source, 1, 2)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1, 1)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1,-1)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1,-2)) ) / 4 ) / 
    \ (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}

#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ ((YDifferenceFromPrevious(selectevery(source, 1, 2)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1, 1)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1,-1)) + 
    \ YDifferenceFromPrevious(selectevery(source, 1,-2)) ) / 4 ) / 
    \ (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}

Last edited by johnmeyer; 30th June 2011 at 08:28. Reason: Fixed wrong explanation of dups & drops (under "Problem ..." heading); Added link to later script
johnmeyer is offline   Reply With Quote
Old 30th June 2011, 00:57   #2  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Just a quick comment on the use of ScriptClip, for future reference.

Quote:
Originally Posted by johnmeyer View Post
Code:
...
global source=AVISource( "E:\CIF Track\3200-Complete (edited).avi" ).ConvertToYV12
...
#test=ShowMetrics(source)
...
BWMask=GenerateMask(source)
...
function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ ((YDifferenceFromPrevious(selectevery(source, 1, 2)) + 
  ...
}

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ ((YDifferenceFromPrevious(selectevery(source, 1, 2)) + 
  ...
}
The input clip to ScriptClip is (initially) available as 'last' inside the run-time script, so you don't need to refer there to 'source' (which consequently need not be global).
Code:
function ShowMetrics (clip c) 
{
  fixed=c.ScriptClip("Subtitle(String(
    \ ((YDifferenceFromPrevious(selectevery(1, 2)) + 
  ... # etc
}

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ ((YDifferenceFromPrevious(selectevery(1, 2)) + 
  ... # etc
}
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 30th June 2011, 01:14   #3  |  Link
Robert Martens
Registered User
 
Join Date: Feb 2010
Location: New York
Posts: 116
You can grab the stream with some minor detective work, if you want.

At that Fox Sports West page, view the source and search for the video ID ( _SeMs5xdJTNhA5dHUbpNK_WG9UvYWUtW ). The fourth match, under "// Select a release for playback", gives you the URL the player contacts to choose the appropriate stream. Visit that address and you'll be presented with a SMIL file named "content.select". Open it in a text editor, and you'll see the RTMP addresses for the three available versions of the video.

I'm not a streaming expert, so as far as getting through that URL to simply nab the MP4 from the server, I'm lost. I was, however, able to use RTMPDump to drop the output into an FLV file. Different container, same content (as far as I know). There are Windows binaries in the download directory; I grabbed version 2.3, at the bottom.

Keep in mind that at an hour and forty-eight minutes, even the low quality version of this video is over 520MB, so be prepared to wait a bit for the dump to finish.
Robert Martens is offline   Reply With Quote
Old 30th June 2011, 01:20   #4  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Thanks for the script

If you want an easy to use GUI to download that video, streamtransport appears to work (I didn't download the entire video, but it looks to be working fine)
poisondeathray is offline   Reply With Quote
Old 30th June 2011, 03:14   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
First of all, I am going to post (tonight or tomorrow) an update to the script I posted above. I feel pretty stupid about this, but my moving average includes the frames which are duplicates. These frames yield a Ydiff of exactly zero, and this invalidates the moving average. So, I wrote some tortured code (given how conditionals work) to use an adjacent clip's values whenever zero is detected. I'm testing this now, and I think it is going to completely eliminate the residual jumps.

Quote:
Originally Posted by Gavino View Post
Just a quick comment on the use of ScriptClip, for future reference ...

The input clip to ScriptClip is (initially) available as 'last' inside the run-time script, so you don't need to refer there to 'source' (which consequently need not be global).
I'll see if that works. I am sure you are correct, and this is how I originally wrote the code. However, as I modified things, some of my attempts involved using outside variables inside the conditional which, as you know, requires them to be declared as global.

I have some vague understanding that "global = bad", but I'm not quite sure what calamity awaits me if I do that.

Quote:
Originally Posted by Robert Martens View Post
You can grab the stream with some minor detective work, if you want ... I was, however, able to use RTMPDump to drop the output into an FLV file ...
I'll definitely look into this after dinner. While I have enjoyed the challenge of creating this script, the original objective (other than draining the swamp ...) was to give this guy the video of his kid, and make it look as good as possible. A better capture is obviously the right way to go.

Quote:
Originally Posted by poisondeathray View Post
If you want an easy to use GUI to download that video, streamtransport appears to work (I didn't download the entire video, but it looks to be working fine)
Ditto my last comment. I just Googled that name and it looks like a site similar to Keepvid. I didn't know about this one. I think I'll try this first.

Thanks to everyone!
johnmeyer is offline   Reply With Quote
Old 30th June 2011, 06:36   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
Update October 13, 2021 based on discussion in this thread: Dropped frames
------------
The updated script with the conditionals that block diff values = 0 from being included in the moving average is shown below.

Also, I tried the two methods for downloading the video. Streamtransport was straightforward, but it grabbed the low-res version of the video. I couldn't find any documentation as to how to specify the higher res version, nor was I able to find any menu or setting in the program.

The RTMPDump was a little more convoluted (reading source code, using the DOS command line, etc.), but ultimately gave me exactly what I wanted.

Finally, I figured out how to use AVISynth and VFAPIConv to serve the resulting FLV files back into Sony Vegas (which can't read FLV) without re-encoding.

So, even though the script posted below ultimately may not be used for this project, I know I'll be using it on various things that clients send to me.

Once again, many thanks for the help!
Code:
# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
# Further modification on June 8, 2013
# Tdecimate modifications on October 26, 2021
#
# 1. Create interpolated frames a 2x original frame rate using MVTools2.
# 2. Detect jumps.
# 3. Create white mask at each jump point; black mask for all other frames.
# 4. Repeat each frame of original video.
# 5. For each frame in double-frame video, use a mask to "choose" between that original video, or the motion estimated video
# 6. Decimate exactly 50% to get back to original frame rate. 
#
# This decimation removes the dup frames that existed in the original video and also removes the dups created by 
# repeating each frame of original video.
# However, at each point where motion-estimated frame was inserted, no decimation occurs. 
# Thus, if dups=drops, all dups in the original video will be removed and the drop will be filled. 
# On the other hand, if no drops or dups occur, then no motion estimation appears in the final result, and the decimation 
# merely gets back to original, unchanged video.
#*********************************************

#*********************************************
#User-settable paramaters

#Puts dot on screen during debugging. True for troubleshooting; otherwise, false.
showdot = true 

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.8

#Frame rate of your video 
#PAL: Vidnum=25; VidDen=1;   NTSC: Vidnum=30000; VidDen=1001;  etc.
VidNum = 60000
VidDen = 1001

#Detection window size. Try changing if you find residual dups or drops. Script gets slow if you use big numbers
DupCycle = 18

threads = 1  #TDecimate does not like multithreading
#*********************************************

loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\MVTools2 (latest)\mvtools2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll") 
loadplugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\Film Restoration\VideoFred's 2014 Script 06_2012\scripts\plugins\RemoveGrain.dll")

#These were used when trying to get AVISynth+ multi-threading to work
#SetFilterMTMode("TDecimate", MT_SERIALIZED)   
#SetFilterMTMode("TDecimate", MT_MULTI_INSTANCE)

global source=AVISource("E:\fs.avi").ConvertToYV12.killaudio()

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

prefiltered = RemoveGrain(source,22)
super = MSuper(source,hpad=16, vpad=16, levels=1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad=16, vpad=16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize=16,overlap=4,search=3,dct=0)
forward = MAnalyse(superfilt, isb = false, blksize=16,overlap=4,search=3,dct=0)
forward_re = MRecalculate(super, forward, blksize=8, overlap=2, thSAD=100)
backward_re = MRecalculate(super, backward, blksize=8, overlap=2, thSAD=100)

#This is alternative way to double the number of frames. 
#double = SmoothFPS2 (source,threads)

double = source.MFlowFps(super, backward_re, forward_re, num=VidNum*2, den=VidDen, blend=false)
double = showdot ? double.subtitle("***") : double

test=ShowMetrics(source)

#Generate a white or black frame, depending on frame difference
BWMask=GenerateMask(source)

#Generate the 2x framerate mask needed to choose the motion-estimated frames
themask = interleave(BlackFrame,trim(BWMask,1,0))

#Merge double framerate from original with motion-esimated frames, but only where there are jumps
#(i.e., original frames are used except at jump points)

# *** NOTE *** mt_merge does NOT produce a clean merge, and the dups are slightly altered from the originals -- unable to fix

interleave(source,source).mt_merge(double,themask,luma=true,U=3,V=3)

#Decimate half of all frames 
RequestLinear
tdecimate(display=false,debug=false,mode=1,Cycle=DupCycle*2,CycleR=DupCycle,vfrdec=1,sdlim=0)  

#Alternate decimation -- should work, but doesn't
#tdecimate(display=false,debug=false,mode=2,rate=59.94006,maxndl=19,m2PA=true)  

#Alternate ways to end the script. Must assign variable to tdecimate output instead of using implied "last."
#prefetch(threads)	
#return test
#Stacked view, for debugging
#return stackvertical(source,decimated)

#End Main Program

#----------------
#Begin Functions
#----------------

#This function displays the YDiff value that will be used for detecting big jumps
#Each YDiff must eliminate Ydiff~=0 (i.e., a duplicate) from moving average
#This is why there are three lines for each YDiff. Each block uses the next or preceeding
#frame, if the current YDiff is near zero, indicating a dup.
#Note: This function works just fine

function ShowMetrics (clip c) 
{
  fixed=source.ScriptClip("Subtitle(String(
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01)
    \ ))")
  return fixed
}


#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average.
#Since the first two frames of the clip have no preceding frame, don't return a mask
#for those two frames (which means there will be no fix for the first two frames)

#This function works just fine (i.e., really great detection)

function GenerateMask (clip c)
{
  MyMask=c.ScriptClip("""
    \ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ? 
    \       YDifferenceFromPrevious(selectevery(source, 1, 3))  :
    \       YDifferenceFromPrevious(selectevery(source, 1, 2)) )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, 2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, 1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -1))  )
    \  +
    \    (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ? 
    \     YDifferenceFromPrevious(selectevery(source, 1, -3))  :
    \     YDifferenceFromPrevious(selectevery(source, 1, -2))  )
    \     )/4) / 
    \    (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh 
    \  && current_frame > 2
    \ ? WhiteFrame : BlackFrame """)
  return MyMask
}

#This function is hard-wired for 60 fps. Not worth the time to generalize it, since SVP
#didn't seem to provide a huge benefit over MVTools2, although it should be faster.

function SmoothFPS2(clip source, threads) { 
  super_params="{pel:2,gpu:1}"
  analyse_params="""{
        block:{w:16,h:16}, 
	main:{search:{coarse:{distance:0}}},
	refine:[{thsad:200}]
        }"""
  smoothfps_params="{rate:{num:120,den:60,abs:false},scene:{mode:0,limits:{scene:8500}},algo:21,cubic:1}"

  super =   SVSuper(source,super_params)
  vectors = SVAnalyse(super, analyse_params)
  SVSmoothFps(source,super,  vectors,  smoothfps_params,  url="www.svp-team.com",  mt=threads)
}

Last edited by johnmeyer; 27th October 2021 at 22:30. Reason: Updated script
johnmeyer is offline   Reply With Quote
Old 30th June 2011, 10:25   #7  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by johnmeyer View Post
However, as I modified things, some of my attempts involved using outside variables inside the conditional which, as you know, requires them to be declared as global.
Not if you use GRunT.
GRunT will also allow you to replace things like
YDifferenceFromPrevious(selectevery(source, 1, -2))
by YDifferenceFromPrevious(-2) and, as I said, 'source' is unnecessary even without GRunT.
Quote:
I have some vague understanding that "global = bad", but I'm not quite sure what calamity awaits me if I do that.
It's not really a problem here, since your globals are effectively constants and your functions are specific to this particular script. But in more general functions (especially in conjunction with ScriptClip, etc), it can lead to bugs such as the function not working if called more than once. And generally speaking, it's easier to test and understand functions that act as 'black boxes' (the modularity principle).
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 30th June 2011, 14:51   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
Quote:
Originally Posted by Gavino View Post
Not if you use GRunT.
I just quickly skimmed the GRunT and GScript threads, and downloaded both extensions. I see you did them several years ago. I wish I'd seen them sooner because they are going to make my life much easier.

Thanks!
johnmeyer is offline   Reply With Quote
Old 17th July 2011, 21:34   #9  |  Link
bizz & buzz
Banned
 
Join Date: Jun 2008
Posts: 94
I'm trying to use this script but it generates a wired "shadow" frame on top of most of the frames.This shadow frame is actually several frames apart from the base frame. There is also some crazy jumping back and forth whenever there's a scene change

source: http://www.mediafire.com/?hmjzf86w1gfd0m6

processed: http://www.mediafire.com/?dfo56cueeix83xk

Script is exactly as in post #6.

Thanks for any help

Pic:

bizz & buzz is offline   Reply With Quote
Old 18th July 2011, 04:44   #10  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
The script won't work with your video because most, if not all, of your duplicates occur exactly at the beginning of each scene change. The script replaces the first instance of the duplicated frames with a motion estimated version from the previous frame, which is from another scene. The problem isn't so much scene detection as it is the fact that the script fails when the duplicate is at the beginning of a scene change. This could be change by estimating the second instance of the dup from the following frame. Of course that approach will fail when the duplicate happens exactly at the end of a scene.

Also, you have to change the Mflowfps parameter to num=48000, den=1001 because your source is 23.976. The Tdecimate CycleR and Cycle parameters will probably have to be changed.

The script worked perfectly on my source clip and was meant as a starting point for others to solve similar problems. However, I never generalized it to all frame rates, and your duplicates exactly at the beginning of each scene is a case that the script as written will not handle.

Last edited by johnmeyer; 18th July 2011 at 04:58. Reason: Added last sentence to first paragraph
johnmeyer is offline   Reply With Quote
Old 18th July 2011, 08:56   #11  |  Link
bizz & buzz
Banned
 
Join Date: Jun 2008
Posts: 94
OK... I'll try to modify the parameters according to your suggestions and also to find a way to make the script behave differently according to whether the dup is in the end or the beggining of a scence.
Thanks a lot.

Last edited by bizz & buzz; 18th July 2011 at 08:59.
bizz & buzz is offline   Reply With Quote
Old 15th July 2021, 06:00   #12  |  Link
JKyle
App Digger
 
JKyle's Avatar
 
Join Date: Sep 2018
Posts: 411
Just for the record...

I transformed @johnmeyer's original script into a filter function named DeJump for use in general cases. This is the result of a conversation trying to fix similar artifacts in VideoHelp. Here's the thread on VideoHelp.

For reference, I made the following mods:

Quote:
- Transformed into `DeJump` filter function (July 12, 2021)
- parametrized input clip, `jumpthresh`, and `thresh` (for `FillDrops`)
- parametrized `num` and `den` options for `double` via `MFlowFps` to take input clip properties
- removed `ShowMetrics` function needed for test
- removed `FillDrops` function to make use of the externalized filter script `FillDrops.avsi` (mod by Selur)
- unglobalized and internalized some variables to be used in `GenerateMask`: `jumpthresh`, `blackframe`, `whiteframe`
- Enabled HBD support (July 13, 2021)
- removed 8-bit conversion of input clip, `ConvertToYV12`, to enable HBD support
- (removed `KillAudio`)
- put input clip pixel type warning
- (changed color names from hex values to preset names)
- Parametrized decimation cycle (July 14, 2021)
The script code is here: DeJump.avsi.
JKyle is offline   Reply With Quote
Old 15th July 2021, 07:28   #13  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,148
Thanks
kedautinh12 is offline   Reply With Quote
Old 14th October 2021, 11:53   #14  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
@JKyle,
Just a couple of observations, on your mod of Johns script.

There is no mention of Colors_RGB.avsi requirement for color_black/color_white.
Also, blackframe = BlankClip(source, color=color_black), would be TV_Levels black, and similar for white.
The Mt_Merge line "TheMask", will not be using PC_Level full Black, nor full White Mask, which has got to affect output.
Code:
Interleave(source, source).mt_merge(double, themask, luma=true, U=3, V=3)
Quote:
mt_merge : clip clip1, clip clip2, clip mask, bool "luma"(false)

It's the backbone of the framework. It merges two clips according to the mask. The bigger the mask value, the more the second clip will be taken into account ( the actual formula is y = ((256 - m) * x1 + m * x2 + 128) / 256 )
luma is a special mode, where only the luma plane of the mask is used to process all three channels.
u and v are defaulted to 2 (that way, the resulting clip contains the chroma of clip1, and looks right).
Also, you have mention [not all bad] in this thread:- https://forum.doom9.org/showthread.php?t=183300

Nice Job, I had started on the script function conversion myself when I found your mod,
cheers.
EDIT: Twould be nice if you also included Johns metrics whotsit too.

EDIT: Same in your script John [although you dont use Colors_rgb.avsi].
For Avs+, there is a new BlankClip Color_YUV settings, or can use something like
Mt_lut("0") for eg black. [just looked, there is also an mt_Invert so Mt_lut("0").mt_Invert would be HBD Ok for white (i think)].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 14th October 2021 at 12:08.
StainlessS is offline   Reply With Quote
Old 15th February 2022, 15:10   #15  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,821
A few days ago another forum member PM'd me to ask for help using the script in post #6, so I set about making a function for him without scrolling down far enough to discover it's already been done.

I'll upload my version soon anyway, but until then I thought I'd point out I don't think JKyle's DeJump function works correctly. As the output from ScriptClip is used for the B/W mask, the error isn't making itself known, but if you add the GenerateMask function to a script on it's own....

GenerateMask(0.8)



That's something I've never known. Is it possible to make an error within ScriptClip a show-stopper rather than it just displaying a text message and soldiering on?

And while I'm here... I see FillDrops is quite fashionable for this sort of thing, so can I confirm I'm using the correct version?
If you're still around StainlessS, the version you posted here seems to have been completely ignored, although it appears to be the one to use. There's FillDrops and FillDropsI? Is the latter one for people who won't de-interlace?

Cheers.

Last edited by hello_hello; 15th February 2022 at 15:22.
hello_hello is offline   Reply With Quote
Old 15th February 2022, 21:10   #16  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,821
Here's the initial version if anyone would care to test it and check my work.

FixJumps.avsi (link removed, see post #20)

Things to point out:
I was using ChangeFPS as the frame interpolation method while doing the initial "did I do something silly" error checking, and not knowing if the method of duplicating frames matters as such, I've left it as an interpolation choice. I also added FrameRateConverter as another option. If that's all wrong they can be removed.

The FillDrops function is not required. That stuff is included in the function (although only the progressive version is included at the moment).
I did wonder about ShowDots. I'm trying to wrap my head around why adding the dots doesn't change the result, but run time functions confuse me no end. If three asterisks appear near the top, they're the ones added to the double frame rate clip. If they appear near the bottom, they're added to the "FillDrops" clip.

There's a little help file at the top of the script with a list of arguments and what they're for.

Here's the link I was sent, if anyone would care to download a video for testing. It's a little over an hour long so it's about 650MB, but it's a fair crapfest of duplicate and dropped frames.
https://ufile.io/ymo81bzy

Last edited by hello_hello; 27th February 2022 at 03:26.
hello_hello is offline   Reply With Quote
Old 15th February 2022, 22:01   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by hello_hello View Post
That's something I've never known. Is it possible to make an error within ScriptClip a show-stopper rather than it just displaying a text message and soldiering on?
I think that it will only error abort before frameserving starts,
once started, it can only put error on frame [including in dll plugins].
(something like that)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th February 2022 at 22:04.
StainlessS is offline   Reply With Quote
Old 15th February 2022, 23:07   #18  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,685
I am eager to try out FixJumps.avsi. I finally gave up trying to perfect my approach because I could not find a way around the decimation problem when the decimation is a window-based algorithm. That approach fails when dups do not equal drops within a given "window" (i.e., the Cycle range used by TDecimate). If you solved that problem, I will be in hog heaven. The dectection portion of my script generally worked pretty well and was able to reliably detect duplicates (that part is relatively easy), as well as drops (much tougher problem).

Just for reference, in addition to the work I posted in this thread, I posted in three other related threads last fall:

Need help "perfecting" script to delete drops and dups

Dropped frames

Script for reliably detecting dropped frames

Anyone trying to improve the state of the art should probably skim through those as well.

Last edited by johnmeyer; 15th February 2022 at 23:49.
johnmeyer is offline   Reply With Quote
Old 16th February 2022, 02:33   #19  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,821
johnmeyer,
I'll check out those links tomorrow. At the moment the function only consists of your script from post #6 and the guts of FillDrops. Hopefully I didn't change anything and that's all it's doing at the moment.

This is actually just a quick post to make my OCD shut up. I thought two instances of ScriptClip in a function would be okay, but apparently I'm not allowed two when one will suffice, so FixJumps v2.avsi (link removed, see post #20) is the same as FixJumps.avsi from my previous post, but with less ScriptClip.
And with GRunT as a requirement, because life's too short...

Edit: A question regarding FrameRateConverter. Currently it's doing it's thing in the function like so:
FrameRateConverter(NewNum=Num*2, NewDen=Den, preset=Preset)
but would it be better to do it this way?
FrameRateConverter(FrameDouble=true, preset=Preset)

Last edited by hello_hello; 16th February 2022 at 12:28.
hello_hello is offline   Reply With Quote
Old 16th February 2022, 12:30   #20  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,821
FixJumps v3.avsi (link removed)
FixJumps v3b.avsi v3b should be exactly the same as v3. I just fiddled with the syntax a little.

Version 2 wasn't displaying the correct metrics when ShowMetrics=true. The calculation was okay, but not all of it was converted to a string for the subtitle.

When ShowMetrics=true, a subtitle also indicates whether the mask is black or white for each frame.

Farewell to ShowDots. It was a bit tacky. To compensate, Debug is new and improved. See the Debug description.

ScriptClip now uses Grunt syntax. It's pretty.

Last edited by hello_hello; 27th February 2022 at 03:12.
hello_hello is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.