Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th April 2011, 15:28   #1  |  Link
alexh110
Registered User
 
Join Date: Mar 2008
Posts: 131
Filter to remove duplicate frames

Hi,
I was wondering whether there's a Virtual Dub filter or an AviSynth script that will automatically discard duplicate frames?

I need it to fix some flv files where the timing speeds up and slows down continually. If the duplicate frames were removed, and the frame rate then adjusted, it would run at roughly the correct speed.
Hope that makes sense!

I can do this laboriously by splitting the video to an image sequence, then uising Clone Remover to delete the duplicate frames, then renumbering the remaining frames, and recompiling them into a video.
However it's such a longwinded method that it's not realistic to do all that work, given the number of files I have to fix!

Ideally I would like to find a media player that will do all this automatically, so that I don't have to create new video files at all. I can just playback the flv files I have, and the media player will fix them on-the-fly.
However I suspect that's too much to hope for!

N.B. There is audio on these files; but I don't particularly care if the audio is lost. Only really interested in the video stream.
alexh110 is offline   Reply With Quote
Old 15th April 2011, 15:44   #2  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 596
http://avisynth.org/mediawiki/Extern...rame_Detectors
http://avisynth.org/mediawiki/Extern....26_Decimation
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 15th April 2011, 15:58   #3  |  Link
Midzuki
Unavailable
 
Midzuki's Avatar
 
Join Date: Mar 2009
Location: offline
Posts: 1,478
The few VFR .FLVs I've ever met, I "fixed" them by simply using DirectShowSource() w/ *convertfps=true*

Just my opinion --- I do not see the point of using VFR in FLV files, especially when they "resolve" to 1000fps
Midzuki is offline   Reply With Quote
Old 15th April 2011, 16:18   #4  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 596
Timescale of FLV format is 1 milli second.
Thus all ntsc videos (xxxx / 1001 fps) are converted to VFR.
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 15th April 2011, 23:00   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
Quote:
Originally Posted by alexh110 View Post
Hi,
I was wondering whether there's a Virtual Dub filter or an AviSynth script that will automatically discard duplicate frames
MugFunky posted a script years ago that detects duplicate frames, and then replaces the duplicate with a sysnthesized frame using MVTools:

automated framedrop filler script

I created a version of this script that uses the newer MVTools2 and is designed for interlaced video:

Code:
function filldropsI (clip c)
{
  even = c.SeparateFields().SelectEven()
  super_even=MSuper(even,pel=2)
  vfe=manalyse(super_even,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super_even,truemotion=true,isb=true,delta=1)
  filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

  odd  = c.SeparateFields().SelectOdd()
  super_odd=MSuper(odd,pel=2)
  vfo=manalyse(super_odd,truemotion=true,isb=false,delta=1)
  vbo=manalyse(super_odd,truemotion=true,isb=true,delta=1)
  filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

  evenfixed = ConditionalFilter(even, filldrops_e, even, "YDifferenceFromPrevious()", "lessthan", "0.1")
  oddfixed  = ConditionalFilter(odd,  filldrops_o, odd,  "YDifferenceFromPrevious()", "lessthan", "0.1")

  Interleave(evenfixed,oddfixed)
  Weave()
}
You may or may not be interested in replacing the duplicate frames: it sounds like you simply want to decimate them. However, depending on why these duplicates were created, decimating them may or may not create jumps in the playback.

Replacing them is a lot easier: the script above works quite well, and most of the time, the formerly duplicated (now synthesized) frame is impossible to spot.

Regardless, both of these similar scripts show you one way to detect duplicate frames (using Ydifference). Both of these scripts (including mine) have the threshold hard-coded -- not particularly good programming practice. MugFunky's script looks for absolutely identical duplicates (Ydifference=0). Perfect duplicates are sometimes created by capture cards when they fail to keep up with the capture. My script looks for frames that are almost the same, but not quite.

If you need to determine a non-zero threshold, you should use something like (this code may not be exactly correct):

ScriptClip(last,"Subtitle(String(YDifferenceToNext() ))" )

and walk through some video to see what threshold value to use for the duplicates in your video.

It has been a long time since I've written a script that deletes an arbitrary number of frames, but I think in AVISynth that you have to do this in two passes: one pass to mark the duplicates (sending the list to a file) and a second pass where you read the file and actually delete the duplicates. Someone else can correct me on that, if there is a one-pass way of doing this.

Last edited by johnmeyer; 16th April 2011 at 01:06. Reason: Fixed code to correct problem noted in next post
johnmeyer is offline   Reply With Quote
Old 16th April 2011, 00:38   #6  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Quote:
Originally Posted by johnmeyer View Post
Code:
function filldropsI (clip c)
{
  ...
  Weave()
  AssumeFieldBased()
  AssumeBFF()
}
AssumeFieldBased() should not normally be used after Weave(), as it indicates a field-separated clip.
(I pointed this out the last two times you posted this function, and you corrected it, but it seems to have snuck in yet again. )
I don't think AssumeBFF should be in there either - what if the clip is TFF?

Quote:
It has been a long time since I've written a script that deletes an arbitrary number of frames, but I think in AVISynth that you have to do this in two passes: one pass to mark the duplicates (sending the list to a file) and a second pass where you read the file and actually delete the duplicates. Someone else can correct me on that, if there is a one-pass way of doing this.
Deleting arbitrary frames in one pass inside ScriptClip is possible (I'm sure I've posted several examples in the past), but it's a bit tricky, as you have to use a variable to keep track of how many have been deleted and the script must be rendered linearly. However, reading frames to be deleted from a file (with ConditionalReader) poses the same difficulties, I think.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 16th April 2011, 01:08   #7  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
Thanks Gavino! I have over 200 scripts and variants floating around in a folder, and obviously forgot to change them all. I edited my previous post, and have now (hopefully) made changes to the other scripts in that folder so this won't happen again. Better yet, I'll just refer back to the post above.

I don't think it actually changes the functionality when this function is used by itself, but it might cause a problem if this function was used inside a larger script, so it's worth correcting.
johnmeyer is offline   Reply With Quote
Old 16th April 2011, 01:17   #8  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Quote:
Originally Posted by johnmeyer View Post
I don't think it actually changes the functionality when this function is used by itself, but it might cause a problem if this function was used inside a larger script
That's right, for example a use of SeparateFields() later in the script would raise an error, and some other filters behave differently depending on whether their input is field-based or frame-based.

I believe it may also affect the behaviour of some encoders.
__________________
GScript and GRunT - complex Avisynth scripting made easier

Last edited by Gavino; 16th April 2011 at 01:21.
Gavino is offline   Reply With Quote
Old 16th April 2011, 06:24   #9  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Wouldn't that fail if two freezeframes in a row? Each would only be flowed to the wrong time.
jmac698 is offline   Reply With Quote
Old 16th April 2011, 15:22   #10  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
Quote:
Originally Posted by jmac698 View Post
Wouldn't that fail if two freezeframes in a row? Each would only be flowed to the wrong time.
I'm not sure what you meant by two in a row. The whole point of MugFunky's original script was to remove one of the two duplicate frames that happen when a capture card repeats a frame when it fails to capture (drops) a frame.

Now if you are saying it fails when three frames in a row are the same, you are correct.

I've used this script extensively, so I'm pretty sure it works.
johnmeyer is offline   Reply With Quote
Old 16th April 2011, 23:08   #11  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
I've tested that problem with capture cards, in one case I got a pattern like 0,0,1,3 where frame 0 was a dup and to catch up, it skipped frame 2. You're right, it works if you have a pattern like 0,1,1,3. Try my glitch analyzer sometime, it creates a text file of dup/dropped frames. (see new plugins and utilities).
jmac698 is offline   Reply With Quote
Old 17th April 2011, 21:30   #12  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
Quote:
Originally Posted by jmac698 View Post
I've tested that problem with capture cards, in one case I got a pattern like 0,0,1,3 where frame 0 was a dup and to catch up, it skipped frame 2. You're right, it works if you have a pattern like 0,1,1,3.
Good points about the drop not always happening in the frame immediately after the duplicate. I've definitely seen situations like the 0,0,1,3 pattern you describe. Somewhere in my sea of AVISynth scripts, I have a variation on the one I posted (the filldropsI script) that handles this.

However, the REALLY tough situation, which I've also seen, is where the drop doesn't always happen the same number of frames after the duplicate. As you walk through the frames on these clips you see the duplicate frame, and then one, two, three, or four frames later you see the jump from the dropped frame. I actually started to develop a script that was going to use conditional logic that was going to look at the motion vectors for the next four frames after a drop and, after decimating the duplicated frame, would insert an estimated frame at the point of the largest motion. I decided that the film in question wasn't worth that much effort to fix, but I'm sure such a script would be reasonably simple to construct.
johnmeyer is offline   Reply With Quote
Old 18th April 2011, 19:54   #13  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Thanks for mentioning that! I'm working on a drop repair script now. Using gscript and setting current_frame, I can loop over all frames in videos arbitrarily. All that remains is some good logic, some realistic estimates of how far to search (4 in this case), a good interpolator and a good dup/missing detector.
I really need to solve this as it's too big to do by hand and it's a continual problem for me. Even worse I'm trying to sync 5 videos with different types of problems.
I think you could try detecting missing frame by interpolating and comparing, and if it's not flowing as expected it may be a missing frame. Of course this isn't 100% so you'd need to look only after dups and also detect scene changes in logic. Would take some thinking.
Want to help on it?
jmac698 is offline   Reply With Quote
Old 20th April 2011, 05:02   #14  |  Link
rec
Registered User
 
Join Date: Oct 2007
Posts: 66
Hey, this is just what I'm looking for! I don't think I have the scripting chops to help you, but just want to let you know, you've got a customer!
rec is offline   Reply With Quote
Old 20th April 2011, 18:30   #15  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,394
Quote:
Originally Posted by jmac698 View Post
Thanks for mentioning that! I'm working on a drop repair script now. ... Want to help on it?
It would be fun, but I'm too busy right now.
johnmeyer is offline   Reply With Quote
Old 1st May 2011, 05:59   #16  |  Link
alexh110
Registered User
 
Join Date: Mar 2008
Posts: 131
Thanks for all the suggestions guys: that's very helpful.
alexh110 is offline   Reply With Quote
Old 4th May 2012, 15:57   #17  |  Link
OldFilmer
Registered User
 
Join Date: Dec 2011
Location: Illinois
Posts: 2
Shutterless projector capture frustrastions

We've tried various combinations of the above scripts to no avail. Would anybody be willing to take a look at our scripts and source clip to give opinion as to what we may have wrong?

Thanks in advance.
OldFilmer is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.