Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th March 2014, 19:20   #1  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Deleting every frame between equal frames in video

Hi great community, Im looking for your help to create an avisynth script that is similar to the popular avisynth "duplicated frame function script".

I have lots of videos (1920 x 1080, tsc2 compression, avi format, about 45min long each one) that I need to trim automatically.

In these videos the movie goes forward normally but then returns exactly to a previous frame (like 3 segs of rewind),
then the movie resumes and goes forward again, but then it returns to the same previous frame again, and then resumes again, until finally the movie pass this segment and goes on, but in other segment of the movie the same thing happens until it pass that segment, and so on. In the hole movie this "rewind to especific frame, forward, rewind, forward" happens almost 30 times.

For example if each letter is a different frame, the movie goes like this:

AbcdAbcAbcdeABCDefDefgDefDefgDEFGHijHijkHIJKLMnopMnopMNOPQ

Note that once in a while the movie goes back to exactly the same frame.
So, the lowercase letters are frames that got rewound, frames that I would like to discard,
and the capital letters are frames that I want to keep
So in the example above I would like to transform the video into this:

ABCDEFGHIJKLMNOPQ

I guess the average way to achieve this, would be detecting all 100% equal frames in the video and deleting everyframe between two equal frames, perhaps also use a threshold about 10 minutes, to delete only the frames that are between two equal frames that are not separated longer than 10 minutes.
Please note that these are HD videos and the difference between each frame in the video is very subtle.
Also if you recomend me some links to learn to use avisynth with several videos (kinda batch mode). I use premiere.

Thank you very much for your support doom9, regards.

Last edited by eduardobedoya; 9th March 2014 at 19:23.
eduardobedoya is offline   Reply With Quote
Old 9th March 2014, 23:39   #2  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,924
Welcome to the forum. Sorry but I have to ask, where did you get these files?
Guest is offline   Reply With Quote
Old 10th March 2014, 17:08   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
This serves as a reasonable example of how you can use RT_Stats Dbase funcs,
Used your example string to produce a test clip.

Code:
##############################
# Requires GScript(), RT_Stats(), and FrameSel() Plugins
##############################

# Choose one of below lines (comment out the other)
#TestSource(Upper=true)     # Test Source clip, We do tests on Upper case frames, but later select from upper/lower case mix frames
AviSource("...whatever...")

######
ScanAheadSeconds= 10 * 60
TEST_THRESH     = 1.0
DIF_THRESH      = 0.1
###
ScanAheadFrames = Int(ScanAheadSeconds*FrameRate)
DB="MyDbase.DB"
FN="Frames.txt"
DEBUG=True                                          # Info to DebugView window (Google)
DELETEDBASE=True                                    # Delete DBase file
##############################
GSCript("""
    RT_FileDelete(FN)                               # Delete any existing frames file
    RT_DBaseAlloc(DB,FrameCount,"f")                # Allocate a single field database of type Float
    LastFrame=FrameCount-1
    (DEBUG) ? RT_DebugF("Filling DBase with AverageLuma data") : NOP
    For(i=0,LastFrame) {                            # Set AveLuma for each and every frame
        RT_DBaseSet(DB,i,RT_AverageLuma(Last,i))
    }
    (DEBUG) ? RT_DebugF("DBase Filled, Scanning frames.") : NOP
    For(i=0,LastFrame) {
        LumaI = RT_DBaseGetField(DB,i,0)
        EndLimit= Min(i + ScanAheadFrames,LastFrame)
        OutFrame = i                                # Init frame to output
        For(j=EndLimit,i + 1,-1) {
            LumaJ = RT_DBaseGetField(DB,j,0)
            if(Abs(LumaI - LumaJ) <= TEST_THRESH) { # about same ave luma, test it
                if(RT_LumaDifference(Last,Last,n=i,n2=j) <= DIF_THRESH) {
                    OutFrame = j                    # Last frame within ScanAheadFrames that is similar to i
                    j = i                           # Early break
                }
            }
        }
        RT_TxtWriteFile(String(OutFrame),FN,Append=True)
        if(i != OutFrame) {
            (DEBUG) ? RT_DebugF("%d ] (Skipped=%d)",OutFrame,OutFrame-i) : NOP  # Shows in DebugView (Google)       
            i = OutFrame                            # Skip all frames before Outframe
        } Else {
            (DEBUG) ? RT_DebugF("%d ]",OutFrame) : NOP
        }
    }
    (DELETEDBASE) ? RT_FileDelete(DB) : NOP
""")

# If using TestSource(), comment out below line
Return FrameSel(Last,cmd=FN)  # This will return frames using FN frames file

# Here Just testing using synthesized test clip, we do NOT upper case strings this time, to see if it worked OK
TestSource(Upper=False)
Return FrameSel(Last,cmd=FN)
#####################

# Only of use in testing, (If you dont have a real problem clip)
Function TestSource(Bool "Upper") {
    Upper=Default(Upper,False)
    Test    = "AbcdAbcAbcdeABCDefDefgDefDefgDEFGHijHijkHIJKLMnopMnopMNOPQ"
    Test    = (Upper) ? Ucase(TesT) : Test
    CC = ColorBars().Trim(0,-1).KillAudio   # Single frame
    C=CC.BlankClip(length=0)                # zero length clip
    GScript("""
        For(i=1,StrLen(Test)) {
            s = MidStr(Test,i,1)
            s = s + s + s + s + s + s + s + s + s + s
            C = C + CC.Subtitle(S,size=24,align=5)
        }
    """)
    return C
}
Script to load only good frames, once main script has created FN ("Frame.txt")
Code:
FN="Frames.txt"
AviSource("...whatever...")
Return FrameSel(Last,cmd=FN)
EDIT: It will NOT be super fast !!! (main script)
EDIT: Any colorspace OK.
EDIT: Clip should be seekable, eg NOT DivX with single keyframe (way too slow).

EDIT: DebugView output (1st frame 12 is 12th letter in your example string ie 'A', 2nd is 13 ie 'B' etc)
Code:
00000010    5.15711737  [1824] RT_DebugF: Filling DBase with AverageLuma data   
00000011    5.96010494  [1824] RT_DebugF: DBase Filled, Scanning frames.    
00000012    6.72434425  [1824] RT_DebugF: 12 ] (Skipped=12) 
00000013    6.91525364  [1824] RT_DebugF: 13 ]  
00000014    7.10134315  [1824] RT_DebugF: 14 ]  
00000015    7.22855568  [1824] RT_DebugF: 29 ] (Skipped=14) 
00000016    7.34694576  [1824] RT_DebugF: 30 ]  
00000017    7.46073008  [1824] RT_DebugF: 31 ]  
00000018    7.57038546  [1824] RT_DebugF: 32 ]  
00000019    7.65050507  [1824] RT_DebugF: 40 ] (Skipped=7)  
00000020    7.72205544  [1824] RT_DebugF: 41 ]  
00000021    7.78988838  [1824] RT_DebugF: 42 ]  
00000022    7.85289049  [1824] RT_DebugF: 43 ]  
00000023    7.91187668  [1824] RT_DebugF: 44 ]  
00000024    7.93679857  [1824] RT_DebugF: 53 ] (Skipped=8)  
00000025    7.95319843  [1824] RT_DebugF: 54 ]  
00000026    7.96542358  [1824] RT_DebugF: 55 ]  
00000027    7.97333288  [1824] RT_DebugF: 56 ]  
00000028    7.97700548  [1824] RT_DebugF: 57 ]
EDIT:
Tested on a SD PAL 30 mins, 39 secs clip without any repeats, with 10 mins scan ahead and it takes about 12 seconds to output each frame,
would be faster if repeats were present and did not have to scan ahead 10 * 60 * 25 frames, for each and every output frame.
Above on Core Duo Dual Core @ 2.13GHz, Sata 2.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 10th March 2014 at 19:44.
StainlessS is offline   Reply With Quote
Old 11th March 2014, 05:27   #4  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Thanks!

Thanks you StainlessS

yes 10 min is the average time it would have to look ahead, looking for repeats, the source is very clean, meaning that are HD digitally recorded videos, good quality, no blur, no death pixels.


Please StainlessS could you give me some links, where could I study, (figure out) how to apply avisynth script inside an windows 8 laptop?
wich software do I have to install? wich plugins?
Thanks you very much man, u have save me hours of editing, THANKS!

Last edited by eduardobedoya; 11th March 2014 at 05:34.
eduardobedoya is offline   Reply With Quote
Old 11th March 2014, 05:33   #5  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
the source is...

Quote:
Originally Posted by neuron2 View Post
Welcome to the forum. Sorry but I have to ask, where did you get these files?
the source is very clean, meaning that are HD digitally recorded videos, good quality, no blur, no death pixels.


Please neuron2 could you give me some links, where could I study, (figure out) how to apply avisynth script inside an windows 8 laptop?
wich software do I have to install? wich plugins? in orther to run Stainless script??
Thank you advanced.
eduardobedoya is offline   Reply With Quote
Old 13th March 2014, 06:16   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
eduardobedoya,

Neuron2 wantz to know how you obtained the samples, ie " Where did you get these files ? ".

For a Laptop, suggest you get the FILTERSDK-HELP (compressed CHM help file version of docs installed with Avisynth v2.6,
suggest you get this even if using v2.58), available via my sig ie StainlessS@MediaFire below this post.
You can put it on Hot-Key for easy acces and it is easily searchable.

When on-line, Avisynth Wiki (has list of many plugins) here http://avisynth.nl/index.php/Main_Page

2 Lists of plugins at top of Usage forum 1st page in stickies.

Most of my plugins are not not in any of the lists (one or maybe two only EDIT: they may be in the New plugins sticky),
see Mediafire in sig for mine.

FrameSel thread here, http://forum.doom9.org/showthread.php?t=167971

RT_Stats here http://forum.doom9.org/showthread.ph...light=rt_stats

GScript here, http://forum.doom9.org/showthread.ph...hlight=gscript

There is a search option at top of all forum pages where you could have found the plugins yourself.

EDIT: And DebugView from SysInternals (Now MicroSoft) Here http://technet.microsoft.com/en-gb/s.../bb896647.aspx

and Welcome to the Forum.

EDIT: I may have a go it increasing speed of given script, perhaps speed increase of 50X or 100X possible
(but maybe not possible, I'll try to give it a go though).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 13th March 2014 at 06:56.
StainlessS is offline   Reply With Quote
Old 14th March 2014, 00:23   #7  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Post #1 of 3

QwikAveLumaScan.Avs
Code:
# QwikAveLumaScan.Avs

Function QwikAveLumaScanCreateDB(clip c, String DB, String "PrevDB", String "NextDB", Bool "Debug") {
# QwikAveLumaScanCreateDB: By StainlessS
# Requires GScript(), RT_Stats() plugins
#
# Creates a DB DataBase file of c.FrameCount records with 1 Float field per record, set to AverageLuma of each frame.
#   Usage:- eg AveLuma = RT_DBaseGetField(DB,FrameNo,0)
#
# Optional creation of PrevDB and NextDB used by QwikAveLumaScanGetNear() function.
#  PrevDB is a database of c.FrameCount records, and 256 Int fields per record, where content of fields point to the
# nearest frame BEFORE the record number (frame) which has the same Int(AverageLuma) value as the field index. If there is no
# frame prior to the record number (frame) that has an Int(AverageLuma) the same as the field index, then the field will hold Int -1.
# As the above is about as clear as mud, I shall try to explain with an example.
#  If we want to know which frame is the nearest frame prior to frame 1000 which has an AverageLuma greater or equal to 128.0
# and less than 129.0 then PrevDB(1000,128) will hold that frame number, and the RT_Stats call to get it is
#   prevfrm = RT_DBaseGetField(PrevDB,1000,128)
# NextDB is the same as PrevDB but the fields point to the nearest frame AFTER the record number (frame).
#
# c,        Clip. Planar, YUY2, RGB24, RGB32.
# DB,       String. Filename of created DB file.
# PrevDB,   String. Filename of optional PrevDB file name.
# NextDB,   String. Filename of optional NextDB file name.
# Debug,    Bool, default false. True outputs info (eg progress) to DebugView window (Google).
#
#   This function is really quite slow and took about 30 mins to create all three DataBases on a 10 Mins 13 secs PAL SD clip.
# Core Duo Duel Core 2.13Ghz and Sata 2 Drives. The AverageLuma of each frame is only gotten once, and the optional DataBases
# are created using the DBase data rather than scanning the frames again. The Optional DBases could be quite big with
# (Framecount * 256 * 4) + $4000 bytes each, but should support up to about 1.9 million frames (where it would break the
# 2 Gig +ve Int maximum, ie go -ve)
# It is recommended to create the databases only once and to re-use them where possible rather than re-creating time and again.
#   Once created, the databases make it possible to fairly quickly find similar frames and could perhaps greatly speed up
# scripts to find where cuts have been made between 2 clips or some kind of frame matching scripts.
#
    GSCript("""
    c
    myName="QwikAveLumaScanCreateDB: "
    VER = 0.0
    Debug  = Default(Debug, False)
    Start = RT_Timer()
    PrevDB=Default(PrevDB,"")
    NextDB=Default(NextDB,"")   
    (DEBUG) ? RT_DebugF("\nQwikAveLumaScanCreateDB() v%.2f by StainlessS\n",VER,name=myName) : NOP
    (DEBUG) ? RT_DebugF("Filling DBase with AverageLuma data (Will take some time)\n",name=myName) : NOP
    LastFrame=FrameCount-1
    RT_DBaseAlloc(DB,FrameCount,"f")                                # FrameCount records with 1 field each, Float
    if(PrevDB=="") {
        (DEBUG) ? RT_DebugF("Creating DB",name=myName) : NOP
        For(frame=0,LastFrame) {
            L = RT_AverageLuma(Last,frame)                          # Ave luma of frame
            RT_DBaseSet(DB,frame,L)                                 # Set DBase Aveluma
            (DEBUG && frame % 1000 == 0) ? RT_DebugF("Record(%d) %.1f%%",frame,(frame+1)*100.0/(LastFrame+1),name=myName) : NOP
        }   
    }
    if(PrevDB!="" || NextDB!="") {
        DBT=DB+"_"+RT_LocalTimeString(File=True)                    # Temporary DBase filename
        RT_DBaseAlloc(DBT,256,"i")                                  # Create temp DBT
        DBS = RT_StrPad("",256,"i")                                 # Type String of 256 'i' characters, ie 256 int fields
        if(PrevDB!="") {
            (DEBUG) ? RT_DebugF("Initializing TEMP DB",name=myName) : NOP
            For(i=0,255) {                                          # Init DBT field 0 to not valid
                RT_DBaseSet(DBT,i, -1)
            }
            (DEBUG) ? RT_DebugF("Creating PrevDB",name=myName) : NOP
            RT_DBaseAlloc(PrevDB,FrameCount,DBS)
            For(frame=0,LastFrame) {
                L   = RT_AverageLuma(Last,frame)                    # Ave luma of frame
                RT_DBaseSet(DB,frame,L)                             # Set Aveluma
                for(i=0,255) {
                    prev = RT_DBaseGetField(DBT,i,0)                # Get latest frame number with approx luma = i
                    RT_DBaseSetField(PrevDB,frame,i,prev)
                }
                Li  = Int(L)                                        # AveLuma of frame as index into temp DBase DBT
                RT_DBaseSet(DBT,Li,frame)                           # Update temp DB with current frame as previous for following frames
                (DEBUG && frame % 1000 == 0) ? RT_DebugF("Record(%d) %.1f%%",frame,(frame+1)*100.0/(LastFrame+1),name=myName) : NOP
            }       
        }
        if(NextDB!="") {
            (DEBUG) ? RT_DebugF("Initializing TEMP DB",name=myName) : NOP
            For(i=0,255) {                                          # Init DBT field 0 to not valid
                RT_DBaseSet(DBT,i, -1)
            }
            (DEBUG) ? RT_DebugF("Creating NextDB",name=myName) : NOP
            RT_DBaseAlloc(NextDB,FrameCount,DBS)
            For(frame=LastFrame, 0, -1) {
                L = RT_DBaseGetField(DB,frame,0)                    # Aveluma
                for(i=0,255) {
                    next = RT_DBaseGetField(DBT,i,0)
                    RT_DBaseSetField(NextDB,frame,i,next)
                }
                Li  = Int(L)                                        # AveLuma of frame as index into temp DBase DBT
                RT_DBaseSet(DBT,Li,frame)                           # Update temp DB with current frame as next for prev frames
                Ftmp=LastFrame - Frame
                (DEBUG && Ftmp % 1000 == 0) ? RT_DebugF("Record(%d) %.1f%%",Ftmp,(Ftmp+1)*100.0/(LastFrame+1),name=myName) : NOP
            }       
        }       
        RT_FileDelete(DBT)                                          # Delete Temp Dbase
    }       
    End = RT_Timer()
    (DEBUG) ? RT_DebugF("\nTotal Time = %.2f Seconds (%.2f Mins)\n",End-Start,(End-Start)/60.0,name=myName) : NOP
    """)
}

Function QwikAveLumaScanGetNear(String DB, String PNDB,int Frame,Float "AveLuma", Float "AveLumaDiff",int "MaxDistance") {
#   Function to find the frame nearest (in frame number distance) to the target Frame, whose AverageLuma is within
#   +- AveLumaDiff (inclusive) of AveLuma. 
#
#   DB, String. Filename of DB file as created by QwikAveLumaScanCreateDB()
#   PNDB, String, Filename of PrevDB OR NextDB file as created by QwikAveLumaScanCreateDB()
#   Frame, Int. Frame number nearest to which, you want to find a NEAR frame. 
#   AveLuma, Float. Optional value of AverageLuma, defaults to the AverageLuma of arg Frame.
#   AveLumaDiff, Float. Optional (default 1.0). Difference Threshold acceptable as similar to AveLuma. AveLumaDiff is inclusive.
#   MaxDistance, Int. Optinal maximum number of frames to search relative to Frame arg. Default number of records in DB-1, ie FrameCount - 1.
#
#   The PNDB arg controls whether we are looking for frames BEFORE or AFTER the target Frame, use the PrevDB to find BEFORE
#   frames or NextDB to find AFTER frames (as described for QwikAveLumaScanCreateDB().
#     To find the nearest frame BEFORE frame 1000 with a Aveluma difference within 1.5 of AverageLuma of frame 1000, use eg
#       PrvFrm = QwikAveLumaScanGetNear(DB,PrevDB,1000,AveLumaDiff=1.5)
#     To find the nearest frame AFTER frame 1000 with an aveluma difference within 1.5 of AverageLuma = 128.0, use eg
#       NxtFrm = QwikAveLumaScanGetNear(DB,NextDB,1000,AveLuma=128.0,AveLumaDiff=1.5)
#         In this case the AverageLuma of the returned frame number would be greater or equal to (128.0 - 1.5) and less or
#         equal to (128.0 + 1.5). This allows you to use an AveLumaDiff of 0.0 to find an EXACT MATCH.
#     To Find exact match eg (needs clip and prior DB setup),
#       q = RT_AverageLuma(n=1000)                                             # Get AverageLuma of frame 1000
#       z = QwikAveLumaScanGetNear(DB, NextDB, 0, AveLuma=q, AveLumaDiff=0.0)  # Get next frame after frame 0 exactly same as frame 1000 AverageLuma.
#       Subtitle(String(q)+"  " + String(z))
#     Above will show AverageLuma of frame 1000 and likely show '1000' as next nearest frame to 0 with that exact same AverageLuma.
#
#   The Function returns -1 if there is no frame that satisfies conditions.
#
    AveLumaDiff=Float(Default(AveLumaDiff,1.0))
    L = Defined(AveLuma) ? Float(AveLuma) : RT_DBaseGetField(DB,Frame,0)    # Use AveLuma if given, else get from DB
    MaxDistance = Default(MaxDistance,RT_DBaseRecords(DB) - 1)      # Defaults to FrameCount - 1
    Li = Int(L)                                                     # Approx ave luma as Int, index into PNDB
    tmn = L - AveLumaDiff                                           ## Valid luma range +- AveLumaDiff
    tmx = L + AveLumaDiff                                           ##
    mni = Max(0,    Int(tmn))                                       # Start PNDB Index to search
    mxi = Min(255,  Int(tmx))                                       # End PNDB Index to search
    NearLumaFrm = -1                                                # Not Found
    Distance = MaxDistance + 1                                      # Has to be closer than this
    GSCript("""
        # Search center interval 1st as likely nearest to target frame (assuming any given AveLuma is that of target Frame)
        # Intent to cut down on search distance in mni->mxi loop
        frm=RT_DBaseGetField(PNDB,Frame,Li)                         # Get nearest before/after frame @ approx Li ave luma
        if(frm != -1) {                                             # Are there any frames in that luma interval ?
            Break=False                                             # Wish there was a Break (sigh)
            While(!Break && Abs(Frame-frm) < Distance) {
                v = RT_DBaseGetField(DB,frm,0)                      # AveLuma for roving frm
                if(v >= tmn && v <= tmx) {                          # Is it within FLOAT Thresh range of L ?
                    NearLumaFrm = frm                               # We got a nearer one, HeHe
                    Distance = Abs(Frame-frm)
                    Break=True                                      # We got less distant, break out of While Loop
                } Else {
                    frm = RT_DBaseGetField(PNDB,frm,Li)             # Scan more distant
                    if(frm== -1) {
                        Break = True                                # No More
                    }
                }
            }
        }
        if(Distance > 1) {                                          # If Distance == 1, then we cannot find closer frame
            for(i=mni,mxi) {                                        # Int intervals where AveLuma +- AveLumaDiff is valid
                if(i != Li) {                                       # We already did Li
                    frm=RT_DBaseGetField(PNDB,Frame,i)              # Get nearest before/after frame @ approx i ave luma
                    if(frm != -1) {                                 # Are there any frames in that luma interval ?
                        Break=False                                 # Wish there was a Break (sigh)
                        While(!Break && Abs(Frame-frm) < Distance) {
                            v = RT_DBaseGetField(DB,frm,0)          # AveLuma for roving frm
                            if(v >= tmn && v <= tmx) {              # Is it within FLOAT Thresh range of L ?
                                NearLumaFrm = frm                   # We got a nearer one, HeHe
                                Distance = Abs(Frame-frm)
                                Break=True                          # We got less distant, break out of While Loop
                            } Else {
                                frm = RT_DBaseGetField(PNDB,frm,i)  # Scan more distant
                                if(frm== -1) {
                                    Break = True                    # No More
                                }
                            }
                        }
                        if(Distance==1) {
                            i = mxi                                 # Found adjacent frame, No more search needed, Early Break out of For Loop
                        }
                    }
                }
            }
        }
    """)
    return NearLumaFrm
}
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 14th March 2014 at 00:38.
StainlessS is offline   Reply With Quote
Old 14th March 2014, 00:38   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Post #2 of 3.

MakePrevDB.avs, Creates Database's
Code:
# MakePrevDB.avs

Import("QwikAveLumaScan.avs")

AviSource("Test.avi")           # Or Whatever

DB      ="MyDB.DB"
PrevDB  ="MyDB_Prev.DB"

QwikAveLumaScanCreateDB(Last, DB, PrevDB, debug=True)

return MessageClip("DB PREV set Created")
Eduardobedoya.avs, needs pre-created database's
Code:
# Eduardobedoya.avs

# MUST create DataBases before using this script using MakePrevDB.avs
# This script creates a Frames.txt file, can then use SelectFrames.avs to get frames instantly.

Import("QwikAveLumaScan.avs")

AviSource("Test.avi")           # Or Whatever

# Alter below to suit
ScanAheadSeconds    = 10 * 60   # Search range in seconds
AVELUMA_DIFF        = 0.1       # Max diff of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
MAX_LDIFF           = 0.001     # LumaDifference between i frame and candidate duplicate (average pixel diff rather than frame diff, 0.0 Exact match)
###
ScanAheadFrames = Int(ScanAheadSeconds*FrameRate)
DB      ="MyDB.DB"
PrevDB  ="MyDB_Prev.DB"
FN="Frames.txt"                                                 # Output frame command file for FrameSel()
DEBUG=True                                                     # Info to DebugView window (Google)
####
GSCript("""
    Start = RT_Timer()
    RT_FileDelete(FN)                                           # Delete any existing frames file
    LastFrame=FrameCount-1
    OCnt = 0                                                    # Frames output count
    For(i=0,LastFrame) {
        OutFrame = i                                            # Init frame to output
        LumaI = RT_DBaseGetField(DB,i,0)                        # AverageLuma of i frame from DB
        EndLimit = Min(i + ScanAheadFrames,LastFrame)           # Searching Endlimit - 1 to i+1 (downwards)
        Fails = 0                                               # Count false candidates, too many may indicate AVELUMA_DIFF too big
        For(j=EndLimit,i + 1,-1) {
            # Search for frame nearest to j but higher than i that has AverageLuma difference with i, less or equal to AVELUMA_DIFF
            j = QwikAveLumaScanGetNear(DB,PrevDB,j,AveLuma=LumaI,AveLumaDiff=AVELUMA_DIFF,maxdistance=j-i-1)
            if(j > i) {                                         # We found a candidate frame
                dif=RT_LumaDifference(Last,Last,n=i,n2=j)       # Ave pixel diff between i frame and candidate
                if(dif <= MAX_LDIFF) {
                    OutFrame = j                                # Last frame within ScanAheadFrames that is similar to i
                    j = i                                       # Early break
                } Else {
                    Fails = Fails + 1                           # Count False candidate frames for Debug
                }
            }
        }
        RT_TxtWriteFile(String(OutFrame),FN,Append=True)        # For Prune instead of FrameSel (supports audio), use "0,"+String(OutFrame)
        OCnt = OCnt + 1
        if(i != OutFrame) {
            If(DEBUG) {
                E=RT_Timer
                RT_DebugF("%d ] Failed_LDifs= %-4d LDif=%f ALDif=%f  Skipped=%d InCnt=%d OutCnt=%d InFPS=%.3f OutFPS=%.3f",
                    \ OutFrame,Fails,dif,Abs(LumaI-RT_DBaseGetField(DB,OutFrame,0)),OutFrame-i,i+1,OCnt,OCnt/(E-Start),(OutFrame+1)/(E-Start))
            }
            i = OutFrame                                        # Skip all frames before Outframe
        } Else {
            If(DEBUG) {
                E=RT_Timer
                RT_DebugF("%d ] Failed_LDifs= %-4d  InCnt=%d OutCnt=%d InFPS=%.3f OutFPS=%.3f",
                    \ OutFrame,Fails,i+1,OCnt,OCnt/(E-Start),(OutFrame+1)/(E-Start))
            }
        }
    }
    E = RT_Timer()
    S=RT_String("\nTotal Time = %.2f Seconds (%.2f Mins) InCnt=%d OutCnt=%d InFPS=%.3f OutFPS=%.3f\n",
        \ E-Start,(E-Start)/60.0,LastFrame+1,OCnt,OCnt/(E-Start),(LastFrame+1)/(E-Start))
    RT_DebugF(S)
    RT_TxtWriteFile(S,"Eduardobedoya.Log",Append=False)
""")

Return FrameSel(Last,cmd=FN)                                    # Select frames via FrameSel plug, or for audio support see a few lines above.
SelectFrames.avs, instantly get frames afterdatabases created and frames file created via Eduardobedoya.avs
Code:
# SelectFrames.avs

AviSource("Test.avi")                         # Or Whatever
FN="Frames.txt"
Return FrameSel(Last,cmd=FN)            # Select frames via FrameSel plug.
Somewhat faster than previous script, but still not startling. The QwikAveLumaScan.Avs funcs may well benefit by converting to plugins.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 14th March 2014 at 00:43.
StainlessS is offline   Reply With Quote
Old 14th March 2014, 00:39   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Post #3 of 3.

DummyTest.avs, if you dont have suitable test clip
Code:
# DummyTest.avs

Import("QwikAveLumaScan.avs")

TestClip(True)                  # Get Testclip all UpperCase
#Return Last

# Below are just for show for TestClip, both difs can be 0.0 as we will find exact matches
ScanAheadSeconds    = 10 * 60   # Search range in seconds
AVELUMA_DIFF        = 0.1       # Max diff of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
MAX_LDIFF           = 0.001     # LumaDifference between i frame and candidate duplicate (average pixel diff rather than frame diff, 0.0 Exact match)
###
ScanAheadFrames = Int(ScanAheadSeconds*FrameRate)
DB      ="Dummy.DB"
PrevDB  ="Dummy_Prev.DB"
FN      ="DummyFrames.txt"                                      # Output frame command file for FrameSel()
DEBUG=True                                                      # Info to DebugView window (Google)
DELETEDBASE=TRUE                                                # Delete DBase files
QwikAveLumaScanCreateDB(Last, DB, PrevDB,debug=true)            # Create DataBases (would normally do as separate job, SLOW)
####
GSCript("""
    Start = RT_Timer()
    RT_FileDelete(FN)                                           # Delete any existing frames file
    LastFrame=FrameCount-1
    For(i=0,LastFrame) {
        OutFrame = i                                            # Init frame to output
        LumaI = RT_DBaseGetField(DB,i,0)                        # AverageLuma of i frame from DB
        EndLimit = Min(i + ScanAheadFrames,LastFrame)           # Searching Endlimit - 1 to i+1 (downwards)
        Fails = 0                                               # Count false candidates, too many may indicate AVELUMA_DIFF too big
        For(j=EndLimit,i + 1,-1) {
            # Search for frame nearest to j but higher than i that has AverageLuma difference with i, less or equal to AVELUMA_DIFF
            j = QwikAveLumaScanGetNear(DB,PrevDB,j,AveLuma=LumaI,AveLumaDiff=AVELUMA_DIFF,maxdistance=j-i-1)
            if(j > i) {                                         # We found a candidate frame
                dif=RT_LumaDifference(Last,Last,n=i,n2=j)       # Ave pixel diff between i frame and candidate
                if(dif <= MAX_LDIFF) {
                    OutFrame = j                                # Last frame within ScanAheadFrames that is similar to i
                    j = i                                       # Early break
                } Else {
                    Fails = Fails + 1                           # Count False candidate frames for Debug
                }
            }
        }
        RT_TxtWriteFile(String(OutFrame),FN,Append=True)
        if(i != OutFrame) {
            (DEBUG) 
                \ ? RT_DebugF("%d ] Failed_LDifs= %-4d LDif=%f ALDif=%f  Skipped=%d",
                    \ OutFrame,Fails,dif,Abs(LumaI-RT_DBaseGetField(DB,OutFrame,0)),OutFrame-i)
                \ : NOP
            i = OutFrame                                        # Skip all frames before Outframe
        } Else {
            (DEBUG) ? RT_DebugF("%d ] Failed_LDifs= %-4d",OutFrame,Fails) : NOP
        }
    }
    if(DELETEDBASE) {
        RT_FileDelete(DB)
        RT_FileDelete(PrevDB)       
    }
    End = RT_Timer()
    (DEBUG) ? RT_DebugF("\nTotal Time = %.2f Seconds (%.2f Mins)\n",End-Start,(End-Start)/60.0) : NOP
""")

Testclip(False)                                                 # Now change to upper/lower case mixed to see if it worked OK
Return FrameSel(Last,cmd=FN)                                    # Select frames via FrameSel plug.

# Only of use in testing, (If you dont have a real problem clip)
Function TestClip(Bool "Upper") {
    Upper=Default(Upper,False)
    Test    = "AbcdAbcAbcdeABCDefDefgDefDefgDEFGHijHijkHIJKLMnopMnopMNOPQ"
    Test    = (Upper) ? Ucase(Test) : Test
    CC = ColorBars().Trim(0,-1).KillAudio   # Single frame
    C=CC.BlankClip(length=0)                # zero length clip
    GScript("""
        For(i=1,StrLen(Test)) {
            s = MidStr(Test,i,1)
            s = RT_StrPad("",20,s)
            s=RT_String("%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n%s\n",
                \ s,s,s,s,s,s,s,s,s,s,s,Esc=0)
            C = C + CC.Subtitle(s,lsp=0,size=40,align=8,y=20)
                
        }
    """)
    return C
}
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 14th March 2014, 09:33   #10  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Thanks Stainless

Thanks Stainless, man, thank you very much for your support, you just freak me out with all that huge amount of scripts, I will read and try everything tomorrow. Please man, just give me the minimal ammount of links necesary to achieve this specific task, the easier way possible, sorry I dident post my laptop specs, it is a laptop with 16gbram, nvidia GTX 680M, multicore 2.9ghz. Please Stainless since I need to apply this script in many videos (batch operation) I would like you suggest me the fastest application possible, if it is possible the easier app also, sorry I am not a programmer. Thanks Advanced.

PD: I get the videos from camtasia recording the laptop screen.
Regards.
eduardobedoya is offline   Reply With Quote
Old 14th March 2014, 19:52   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Have already given a few links in previous post,
Would also suggest VirtualDub, see main forum page.

Have only just now converted the QwikAveLumaScanCreateDB func to a plugin (in coming version of RT_Stats),
previous script created all 3 databases (for a test avi) in about 45 mins, currently got it running
as plugin doing same in 41 seconds, so there is some improvement.

I shall now have a go at the QwikAveLumaScanGetNear function, hoping for a reasonable improvement there also,
I shall not though be touching the Eduardobedoya.avs script, its too much a one-off to bother with,
but it should be pretty nippy I think when other plugs done.

I'll see what I can do to make a simple idiot proof script to auto create Databases and FrameSelect command files,
and also create scripts to load the resultant fixed files, all you will have to do is run VirtualDub, select compression and
then Queue them for Batch Saving (file menu).

maybe a day or two.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 15th March 2014, 03:19   #12  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Man from 45 min to 41 seconds, thats optimization!
thanks you stainless
I have read your scripts so I kinda understand that...

First I run the QwikAveLumaScan.avs (the plugin version) inside Virtualdub in orther to create a database (but I dont understand, which application plugin? avisynth? Virtualdub? do I need to install this plugin??)
Then I run the MakePrevDB.avs also inside Virtualdub in orther to create a database
Then I run the Eduardobedoya.avs also inside Virtualdub in orther to create a frames.txt file
and finally use the SelectFrames.avs to get the frames inside Virtualdub, right? inside Virtualdub timeline? or I will get avi files directly?

I have 20 videos (45 minutes footage each one), Do I need to proceed with these steps above for each one of these videos??? or, Can I put in the avisource 4 videos, in orther to create the database of 4 videos in one shot????

I have read this
http://neuron2.net/LVG/avisynth.html
So I beleive that I will have to install avisynth and virtualdub, then create the avs scriptfiles that you provided changing the sorcefiles and paths, then open the avs scripts inside virtualdub in the order posted above??? That would be all right? Do I missing something?? those are all the steps right?

I will wait for you last instruction, to perform a test. thanks again StainlessS.

Last edited by eduardobedoya; 15th March 2014 at 04:11.
eduardobedoya is offline   Reply With Quote
Old 15th March 2014, 10:16   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Dont bother with those scripts just now, There are a few little problems, especially around the
end of clip (last frame), so I'm still tinkering a little bit. Shall leave those scripts (when updated)
in place for example usage.

I'll make a script, that allows avi group selection via a file selector (GUI),
does ALL of the dbase creation, processes the avi's using the DB's and output
frames files, and also auto create avs scripts, which should be loaded later into vdub and queued for
compressed output. You can execute the creation script by simply loading into VDub or
playing in Windows Media Player (but will seem to hang until all done).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 24th March 2014, 03:52   #14  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
pls man, let me know when you release it, I kinda look around everyday xD, Thanks

pls man, let me know when you will release it, I kinda look around this page everyday xD, Thanks, thanks advanced.
eduardobedoya is offline   Reply With Quote
Old 24th March 2014, 18:29   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Sorry, eduardobedoya, had to go away for a few days but have been working on it.
Have had to increase maximum number of fields in RT_Stats DBase's from 256 to 1024
(same for Attributes) and made a few other small changes. Have nearly implemented
plugin functions for fast finding of frames using a 'fingerprint' of the frame to find.
Fingerprint uses (RT_Stats equivalents to) YPlaneMedian, AveraLuma, YPlaneStdev,
and YInRange (last two RT_Stats only). With a tolerance of about +- 1.0 on fingerprint
primitives (eg difference in AvergeLuma between find and found frames) I'm getting
something like 20 FPS output (to frames text file) on a clip that has no duplicates (ie there is no
frame to find so does a full search) where test clip has 10,000 SD PAL frames searching from
current frame to last frame for every output frame. If tolerances are 0.0, ie exact matches
only, then am getting about 170 to 180 FPS in same case as above. (Core 2 Duo 2.13 Ghz, Sata 2).

Have been reluctant to release RT_Stats update until the new funcs are implemented as flexibly as possible,
will be developer ONLY functions, and not for the casual scriptor. Will I think be released as experimental, as
could still change args, but mostly already there.

Question on your particular problem, are there any unique sequences within duplicate sequence range (10 mins you say) ?,
as currently implemented it searches for LAST frame that matches, so it can skip large numbers of frames all in a single
jump. However, if there are isolated unique sequences beteen Find and Found frames, you would lose the unique sequence.
I could implement modified script to drop the current frame instead of all frames between current Find frame and Found frame,
but it would be significantly slower dropping 1 frame at a time instead of vast swathes of frames.

Please be patient, I am still working on it.

EDIT: Also, do we need to cope with Audio ?

Code:
    # Below XYX isolated unique sequence, Keep uppercase.
    "abcdabcdXYZdefdefgABCdefdefgDEFghijGHIJKLmnopmnopMNOPQRSTUVW"
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 25th March 2014 at 09:29.
StainlessS is offline   Reply With Quote
Old 25th March 2014, 09:07   #16  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Thanks Stainless - audio is not important, in fact, the files are avi. without audio.

Thanks Stainless - audio is not important, in fact, the files are avi. without audio.

Thanks advanced, will wait till you release it.
eduardobedoya is offline   Reply With Quote
Old 25th March 2014, 09:20   #17  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Thanks Stainless- No, there are not any unique sequence between two duplicated frames

Thanks Stainless - No, there are not any unique sequence between two duplicated frames, it would be very strange to have something usefull between two duplicated frames, basically I need to delete ALL frames between to equal frames. Not only delete the duplicated frames, but delete everything between two duplicated frames. (inside a range of 10 minutes).

The footage is 30fps Techsmith codec avi - no audio.

Thanks Advanced, I will wait till your release
eduardobedoya is offline   Reply With Quote
Old 25th March 2014, 09:39   #18  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Post #1 of 2

Not releasing RT_Stats v1.31 with docs just yet, but here it is without new docs or source.
Also includes Prune for Audio. (copy plugs to Plugins dir).

Have implemented for Audio anyways, and two scripts, one to keep unique frames, and one that dont.

See beginning of avs for config.

Here are dll's (still need Gscript and FrameSel v1.12) : LINK REMOVED

Keep Unique script:
Code:
# ############################
# Fsel_EduardobedoyaKeepUnique_Batch.avs, by StainlessS
# ############################

# Alter below to Config
########################################################################
########################################################################
########################################################################

ScanAheadSecs       = 10 * 60   # Search ahead range in seconds

####
# Below settings, as close to zero as possible (faster but might miss duplicates), used by QWIK scan routines
LODIFFMED           = 1         # INT, Max diff (-ve tolerance) of YMedian between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFMED           = 1         # INT, Max diff of YMedian between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFAL            = 0.01      # Float, Max diff (-ve tolerance) of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFAL            = 0.01      # Float, Max diff of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFSTD           = 0.01      # Float, Max diff (-ve tolerance) of YPlaneStdev between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFSTD           = 0.01      # Float, Max diff of YPlaneStdev between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFINR           = 0.01      # Float, Max diff (-ve tolerance) of YInRange between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFINR           = 0.01      # Float, Max diff of YInRange between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
####

####
# Below used to identify if candidate frames found by QWIK SCAN routines are good.
MAX_LDIFF           = 0.01      # Float, LumaDifference between candidate frame and duplicate (average pixel diff rather than frame diff, 0.0 Exact match)
####

####
AUDIO               = True      # False, no audio. True Supports audio, Audio requires Prune Plugin
####

########################################################################
########################################################################
########################################################################

FSEL_TITLE="Select AVI, files Will Batch create FrameSel command files"
FSEL_DIR="."
FSEL_FILT="Avi files|*.avi"
FSEL_MULTI=True
AVIFILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(AVIFILE_LIST.IsString,"RT_FSelOpen: Error="+String(AVIFILE_LIST))

NFILES=RT_TxtQueryLines(AVIFILE_LIST)                                           # Query Number of lines in String ie number of files.
myName="Fsel_EduardobedoyaKeepUnique_Batch: "
LOG="Fsel_EduardobedoyaKeepUnique_Batch.Log"
RT_TxtWriteFile(LOG,LOG,Append=False)
GSCript("""
    TOTSTART = RT_Timer
    For(i=0,NFILES-1) {
        START = RT_Timer
        FN=RT_TxtGetLine(AVIFILE_LIST,i)                                        # Filename of avi file i
        S=RT_String("\n\n%d/%d ] Processing %s",i+1,NFILES, FN)
        RT_DebugF(S,name=myName)
        RT_TxtWriteFile(S,LOG,Append=True)
        DropDupSequencesKeepUnique(FN,ScanAheadSecs,
            \ ldm=LODIFFMED,    hdm=HIDIFFMED,
            \ ldL=LODIFFAL,     hdL=HIDIFFAL,
            \ lds=LODIFFSTD,    hds=HIDIFFSTD,
            \ ldr=LODIFFINR,    hdr=HIDIFFINR,
            \ LDThresh=MAX_LDIFF,audio=AUDIO,log=LOG)
        T = RT_Timer - START
        S=RT_String("  %s  Tot File Time = %.2f Seconds (%.2f Mins)",FN, T,T/60.0)
        RT_DebugF(S,name=myName)
        RT_TxtWriteFile(S,LOG,Append=True)
    }
    T = RT_Timer - TOTSTART
    S=RT_String("\n\nTOTAL Time = %.2f Seconds (%.2f Mins)\n",T,T/60.0)
    RT_DebugF(S,name=myName)
    S=RT_String("\n\nDONE\n\n%s\n",S)
    RT_TxtWriteFile(S,LOG,Append=True)
    S=RT_StrReplace(S,Chr(10),"\n")
""")
Return blankclip(length=24*60*60*24).Subtitle(S,Align=5,Y=100,lsp=0,Size=30)


Function DropDupSequencesKeepUnique(String "AviName",int "ScanAheadSecs",
    \ int "ldM",int "hdM",Float "ldL",Float "hdL",Float "ldS",Float "hdS",Float "ldR",Float "hdR",Float "LDThresh",Bool "Audio",String "Log") { 
    myName="DropDupSequencesKeepUnique: "
    ScanAheadSecs=Default(ScanAheadSecs,10*60)
    ldM = Default(ldM,1)                hdM = Default(hdM,1)
    ldL = Float(Default(ldL,0.01))      hdL = Float(Default(hdL,0.01))
    ldS = Float(Default(ldS,0.01))      hdS = Float(Default(hdS,0.01))
    ldR = Float(Default(ldR,0.01))      hdR = Float(Default(hdR,0.01))
    LDThresh = Float(Default(LDThresh,0.01))
    LOG = Default(LOG,"DropDupSequencesKeepUnique.LOG")
    Assert(Exist(AviName),myName+AviName+" Does Not Exist")
    Avisource(AviName)
    Audio=Default(Audio,False)                                                      # Needs Prune plugin if Audio   
    Audio=(!HasAudio) ? False : Audio
    Assert(ScanAheadSecs>=0,myName+"ScanAheadSecs Must be greater than zero")
    Assert(ldM>=0 && hdM >0,myName+"ldM and hdM Must be greater than zero")
    Assert(ldL>=0.0 && hdL >=0.0,myName+"ldL and hdL Must be greater or equal to zero")
    Assert(ldS>=0.0 && hdS >=0.0,myName+"ldS and hdS Must be greater or equal to zero")
    Assert(ldR>=0.0 && hdR >=0.0,myName+"ldR and hdR Must be greater or equal to zero") 
    
    PathAndNode = RT_FilenameSplit(AviName,7)       # Drive + Dir + Name    
    CMDFrames=PathAndNode+"_KUnique_Frames.txt"
    ScriptFile=PathAndNode+"_KUnique_SelectFrames.AVS"
    ScanAheadFrames = Int(ScanAheadSecs*FrameRate)
    DB=PathAndNode+".DB"
    NextDB=PathAndNode+"_Next.DB"
    RT_FileDelete(CMDFrames)                                                       # Delete any existing frames file
    FrameSel_Select="""
        Avisource("%s")
        CmdFrames="%s"
        (Exist(CmdFrames)) ? FrameSel(cmd=CmdFrames,reject=True) : NOP
        Return Last
    """
    Prune_Select="""
        Avisource("%s")
        CmdFrames="%s"
        PruneFrames=CmdFrames+"_Prune.txt"
        Ex=(Exist(CmdFrames))
        (Ex) ? FrameSel_CmdReWrite(PruneFrames,Cmd=CmdFrames,reject=True,Range=True,Prune=True) : NOP
        (Ex) ? Prune(cmd=PruneFrames,Fade=10,FadeIn=True,FadeSplice=True,FadeOut=True) : NOP
        Return Last
    """

    Select_S = (Audio) ? Prune_Select : FrameSel_Select                             # Select FrameSel or Prune extraction
    Select_S = RT_StrReplaceDeep(RT_StrReplace(Select_S,Chr(9)," "),"  "," ")       # TAB and SPACE compact
    Select_S = RT_String(Select_S,AviName,CmdFrames)                                # Insert filenames
    
    START = RT_Timer
    RT_QwikAveLumaScanCreateDB(DB,prevdb="",nextdb=NextDB,debug=true)               # Forward scanning
    T= RT_Timer - START
    S = RT_String("  QWIK Scan DBase creation = %.2f Secs (%.2f Mins)",T,T/60.0)
    RT_TxtWriteFile(S,LOG,Append=True)
    GSCript("""
        START = RT_Timer
        Dropped = 0
        LastFrame=FrameCount-1
        RT_DebugF("  QWIK Scanning file ... Please Wait",name=myName)
        For(i=0,LastFrame) {
            EndLimit = Min(i + ScanAheadFrames,LastFrame)           # Searching i+1 to Endlimit inclusive
            # QWIK Find a series of likely matching frames
            For(j = i + 1,EndLimit) {
                j = RT_QwikAveLumaScanGetNear(DB,NextDB,j,
                        \ findframe=i,
                        \ LoDiffMed =ldM,           HiDiffMEd   =hdM,
                        \ LoDiffAl  =ldL,           HiDiffAL    =hdL,
                        \ LoDiffStd =ldS,           HiDiffStd   =hdS,
                        \ LoDiffInr =ldR,           HiDiffInr   =hdR,
                        \ maxdistance=EndLimit-j,   Inclusive=True)
                if(j < 0) {                                         # j Will be -1, ie Not Found
                    j = EndLimit                                    # Early break from j, DONT delete frame
                } Else {                                            # We found a candidate frame, j will be greater than i
                    dif=RT_LumaDifference(Last,Last,n=i,n2=j)       # Average pixel diff between i frame and candidate j
                    if(dif <= LDThresh) {                           # is candidate frame a good match ?
                        Dropped = Dropped + 1
                        RT_TxtWriteFile(String(i),
                            \ CMDFrames,Append=True)                # Frame to DELETE, Keep later frame, delete earlier
                        j = EndLimit                                # Early break, move on to next i frame
                    } # Otherwise continue QWIK scan for next candidate in series
                }
            }
        }
    RT_TxtWriteFile(Select_S,ScriptFile,Append=False)
    RT_FileDelete(DB)
    RT_FileDelete(NextDB)
    T = RT_Timer - START
    FT=FrameCount / FrameRate
    S=RT_String("  Dropping %d of %d frames [%dx%d %.2f secs (%.2f Mins) @ %.2f FPS]\n  QWIK SCAN %.2f Secs (%.2f Mins) InFPS=%.2f OutFPS=%.2f", \
            Dropped, FrameCount,Width,Height,FT,FT/60.0,FrameRate,T,T/60.0,FrameCount/T,(FrameCount-Dropped)/T)
    RT_DebugF(S,name=myName)
    RT_TxtWriteFile(S,LOG,Append=True)
    """)
    Return 0
}
EDITED:

and log

Code:
Fsel_EduardobedoyaKeepUnique_Batch.Log
  

1/4 ] Processing D:\AVS\AVI\IN\FLASHTEST.avi
  QWIK Scan DBase creation = 15.75 Secs (0.26 Mins)
  Dropping 0 of 7342 frames [640x400 293.68 secs (4.89 Mins) @ 25.00 FPS]
  QWIK SCAN 5.42 Secs (0.09 Mins) InFPS=1354.36 OutFPS=1354.36
  D:\AVS\AVI\IN\FLASHTEST.avi  Tot File Time = 21.25 Seconds (0.35 Mins)
  

2/4 ] Processing D:\AVS\AVI\IN\Scrambled.avi
  QWIK Scan DBase creation = 244.28 Secs (4.07 Mins)
  Dropping 25500 of 45000 frames [720x576 1800.00 secs (30.00 Mins) @ 25.00 FPS]
  QWIK SCAN 524.80 Secs (8.75 Mins) InFPS=85.75 OutFPS=37.16
  D:\AVS\AVI\IN\Scrambled.avi  Tot File Time = 769.27 Seconds (12.82 Mins)
  

3/4 ] Processing D:\AVS\AVI\IN\TEST2.avi
  QWIK Scan DBase creation = 46.72 Secs (0.78 Mins)
  Dropping 0 of 15319 frames [720x576 612.76 secs (10.21 Mins) @ 25.00 FPS]
  QWIK SCAN 20.47 Secs (0.34 Mins) InFPS=748.40 OutFPS=748.40
  D:\AVS\AVI\IN\TEST2.avi  Tot File Time = 67.47 Seconds (1.12 Mins)
  

4/4 ] Processing D:\AVS\AVI\IN\TEST.avi
  QWIK Scan DBase creation = 57.47 Secs (0.96 Mins)
  Dropping 0 of 13861 frames [720x576 554.44 secs (9.24 Mins) @ 25.00 FPS]
  QWIK SCAN 13.17 Secs (0.22 Mins) InFPS=1052.31 OutFPS=1052.31
  D:\AVS\AVI\IN\TEST.avi  Tot File Time = 70.84 Seconds (1.18 Mins)


DONE

TOTAL Time = 928.83 Seconds (15.48 Mins)
Only Scrambled.avi is scrambled, others just to test that it dont delete frames where not required.

Here MakeScrambled.avs, test clip
Code:
###################
# MakeScrambled.avs
###################
# Only of use in testing, (If you dont have a real problem clip)

Function ScrambleClip(clip c) { # intended for PAL source [numbers may not work for NTSC, have not tried, use AssumeFPS(25.0) beforehand]
    c
    fc=FrameCount
    Need = 27 * 750
    Assert(fc >= Need,"ScrambleClip:, need at least "+String(Need)+" Frame clip as source(got "+String(fc)+")")

    # Below XYZ isolated sequence. Will keep all upper case letters (including XYZ in Keep Unique script).
    # Keep unique script will keep LAST instance of ALL frames within Scanahead.
    # Non Keep Unique avs will jump past XYZ frames and they will not be kept, but faster. (jumps from 1st lowercase 'a' to
    # last 'A', and omits everything between.
    
    Test    = Ucase("abcdabcdXYZdefdefgABCdefdefgDEFghijGHIJKLmnopmnopMNOPQRSTUVW")
    CC=C.BlankClip(length=0)                # zero length clip
    GScript("""
        For(i=1,StrLen(Test)) {
            s = MidStr(Test,i,1)
        si = RT_ORD(s) - RT_Ord("A")        # Subtract 65, make 'A'=0, 'Z' = 25 
        CT=C.Trim(si*750,-750).Subtitle(S,Size=20)
        CT = (s>="X" && s<="Z") ? CT.Subtitle("UNIQUE",Align=5,size=60) : CT
        CC = CC ++ CT
        }
    """)
    return CC
}

avisource("JurassicPark.avi")
Trim(16854,182049)              # Chop off intro
ScrambleClip()

return last
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 17th December 2014 at 21:40.
StainlessS is offline   Reply With Quote
Old 25th March 2014, 09:43   #19  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,161
Post #2 of 2

No Keep unique frames
Code:
# ############################
# Fsel_Eduardobedoya_Batch.avs, by StainlessS
# ############################

# Alter below to Config
########################################################################
########################################################################
########################################################################

ScanAheadSecs       = 10 * 60   # Search ahead range in seconds

####
# Below settings, as close to zero as possible (faster but might miss duplicates), used by QWIK scan routines
LODIFFMED           = 1         # INT, Max diff (-ve tolerance) of YMedian between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFMED           = 1         # INT, Max diff of YMedian between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFAL            = 0.01      # Float, Max (-ve tolerance) diff of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFAL            = 0.01      # Float, Max diff of AverageLuma between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFSTD           = 0.01      # Float, Max (-ve tolerance) diff of YPlaneStdev between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFSTD           = 0.01      # Float, Max diff of YPlaneStdev between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
#
LODIFFINR           = 0.01      # Float, Max (-ve tolerance) diff of YInRange between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
HIDIFFINR           = 0.01      # Float, Max diff of YInRange between i frame and possible duplicate (may vary with clip, 0.0 for EXACT match)
####

####
# Below used to identify if candidate frames found by QWIK SCAN routines are good.
MAX_LDIFF           = 0.01      # Float, LumaDifference between candidate frame and duplicate (average pixel diff rather than frame diff, 0.0 Exact match)
####

####
AUDIO               = True      # False, no audio. True Supports audio, Audio requires Prune Plugin
####

########################################################################
########################################################################
########################################################################

FSEL_TITLE="Select AVI, files Will Batch create FrameSel command files"
FSEL_DIR="."
FSEL_FILT="Avi files|*.avi"
FSEL_MULTI=True
AVIFILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(AVIFILE_LIST.IsString,"RT_FSelOpen: Error="+String(AVIFILE_LIST))

NFILES=RT_TxtQueryLines(AVIFILE_LIST)                                           # Query Number of lines in String ie number of files.
myName="Fsel_Eduardobedoya_Batch: "
LOG="Fsel_Eduardobedoya_Batch.Log"
RT_TxtWriteFile(LOG,LOG,Append=False)
GSCript("""
    TOTSTART = RT_Timer
    For(i=0,NFILES-1) {
        START = RT_Timer
        FN=RT_TxtGetLine(AVIFILE_LIST,i)                                        # Filename of avi file i
        S=RT_String("\n\n%d/%d ] Processing %s",i+1,NFILES, FN)
        RT_DebugF(S,name=myName)
        RT_TxtWriteFile(S,LOG,Append=True)
        DropDupSequences(FN,ScanAheadSecs,
            \ ldm=LODIFFMED,    hdm=HIDIFFMED,
            \ ldL=LODIFFAL,     hdL=HIDIFFAL,
            \ lds=LODIFFSTD,    hds=HIDIFFSTD,
            \ ldr=LODIFFINR,    hdr=HIDIFFINR,
            \ LDThresh=MAX_LDIFF,audio=AUDIO,log=LOG)
        T = RT_Timer - START
        S=RT_String("  %s  Tot File Time = %.2f Seconds (%.2f Mins)",FN, T,T/60.0)
        RT_DebugF(S,name=myName)
        RT_TxtWriteFile(S,LOG,Append=True)
    }
    T = RT_Timer - TOTSTART
    S=RT_String("\n\nTOTAL Time = %.2f Seconds (%.2f Mins)\n",T,T/60.0)
    RT_DebugF(S,name=myName)
    S=RT_String("\n\nDONE\n\n%s\n",S)
    RT_TxtWriteFile(S,LOG,Append=True)
    S=RT_StrReplace(S,Chr(10),"\n")
""")
Return blankclip(length=24*60*60*24).Subtitle(S,Align=5,Y=100,lsp=0,Size=30)


Function DropDupSequences(String "AviName",int "ScanAheadSecs",
    \ int "ldM",int "hdM",Float "ldL",Float "hdL",Float "ldS",Float "hdS",Float "ldR",Float "hdR",Float "LDThresh",Bool "Audio",String "Log") { 
    myName="DropDupSequences: "
    ScanAheadSecs=Default(ScanAheadSecs,10*60)
    ldM = Default(ldM,1)                hdM = Default(hdM,1)
    ldL = Float(Default(ldL,0.01))      hdL = Float(Default(hdL,0.01))
    ldS = Float(Default(ldS,0.01))      hdS = Float(Default(hdS,0.01))
    ldR = Float(Default(ldR,0.01))      hdR = Float(Default(hdR,0.01))
    LDThresh = Float(Default(LDThresh,0.01))
    LOG = Default(LOG,"DropDupSequences.LOG")
    Assert(Exist(AviName),myName+AviName+" Does Not Exist")
    Avisource(AviName)
    Audio=Default(Audio,False)                                                      # Needs Prune plugin if Audio   
    Audio=(!HasAudio) ? False : Audio
    Assert(ScanAheadSecs>0,myName+"ScanAheadSecs Must be greater than zero")
    Assert(ldM>=0 && hdM >=0,myName+"ldM and hdM Must be greater or equal to zero")
    Assert(ldL>=0.0 && hdL >=0.0,myName+"ldL and hdL Must be greater or equal to zero")
    Assert(ldS>=0.0 && hdS >=0.0,myName+"ldS and hdS Must be greater or equal to zero")
    Assert(ldR>=0.0 && hdR >=0.0,myName+"ldR and hdR Must be greater or equal to zero") 
    
    PathAndNode = RT_FilenameSplit(AviName,7)       # Drive + Dir + Name    
    CMDFrames=PathAndNode+"_Frames.txt"
    ScriptFile=PathAndNode+"_SelectFrames.AVS"
    ScanAheadFrames = Int(ScanAheadSecs*FrameRate)
    DB=PathAndNode+".DB"
    PrevDB=PathAndNode+"_Prev.DB"
    RT_FileDelete(CMDFrames)                                                       # Delete any existing frames file
    FrameSel_Select="""
        Avisource("%s")
        CmdFrames="%s"
        (Exist(CmdFrames)) ? FrameSel(cmd=CmdFrames,reject=False) : NOP
        Return Last
    """
    Prune_Select="""
        Avisource("%s")
        CmdFrames="%s"
        PruneFrames=CmdFrames+"_Prune.txt"
        Ex=(Exist(CmdFrames))
        (Ex) ? FrameSel_CmdReWrite(PruneFrames,Cmd=CmdFrames,reject=False,Range=True,Prune=True) : NOP
        (Ex) ? Prune(cmd=PruneFrames,Fade=10,FadeIn=True,FadeSplice=True,FadeOut=True) : NOP
        Return Last
    """
    
    Select_S = (Audio) ? Prune_Select : FrameSel_Select                             # Select FrameSel or Prune extraction
    Select_S = RT_StrReplaceDeep(RT_StrReplace(Select_S,Chr(9)," "),"  "," ")       # TAB and SPACE compact
    Select_S = RT_String(Select_S,AviName,CmdFrames)                                # Insert filenames
    
    START = RT_Timer
    RT_QwikAveLumaScanCreateDB(DB,prevdb=PrevDB,nextdb="",debug=true)               # Backwards scanning
    T= RT_Timer - START
    S = RT_String("  QWIK Scan DBase creation = %.2f Secs (%.2f Mins)",T,T/60.0)
    RT_TxtWriteFile(S,LOG,Append=True)
    GSCript("""
        START = RT_Timer
        Dropped = 0
        LastFrame=FrameCount-1
        RT_DebugF("  QWIK Scanning file ... Please Wait",name=myName)
        For(i=0,LastFrame) {
            EndLimit = Min(i + ScanAheadFrames,LastFrame)           # Searching Endlimit to i+1 inclusive (downwards)
            # QWIK Find a series of likely matching frames
            For(j=EndLimit,i + 1,-1) {
                # Search for frame nearest to EndLimit but higher than i
                j = RT_QwikAveLumaScanGetNear(DB,PrevDB,j,
                        \ findframe=i,
                        \ LoDiffMed =ldM,           HiDiffMEd   =hdM,
                        \ LoDiffAl  =ldL,           HiDiffAL    =hdL,
                        \ LoDiffStd =ldS,           HiDiffStd   =hdS,
                        \ LoDiffInr =ldR,           HiDiffInr   =hdR,
                        \ maxdistance=j-(i+1),      Inclusive=True)
                if(j > i) {                                         # We found a candidate frame
                    dif=RT_LumaDifference(Last,Last,n=i,n2=j)       # Ave pixel diff between i frame and candidate
                    if(dif <= LDThresh) {
                        Dropped = Dropped + (j-i)
                        i = j                                       # Last frame within ScanAheadFrames that is similar to i
                        j = 0                                       # Early break
                    }
                } # Otherwise continue QWIK scan for next candidate in series
            }
            RT_TxtWriteFile(String(i),CMDFrames,Append=True) 
        }
    RT_TxtWriteFile(Select_S,ScriptFile,Append=False)
    RT_FileDelete(DB)
    RT_FileDelete(PrevDB)
    T = RT_Timer - START
    FT=FrameCount / FrameRate
    S=RT_String("  Dropping %d of %d frames [%dx%d %.2f secs (%.2f Mins) @ %.2f FPS]\n  QWIK SCAN %.2f Secs (%.2f Mins) InFPS=%.2f OutFPS=%.2f", \
            Dropped, FrameCount,Width,Height,FT,FT/60.0,FrameRate,T,T/60.0,FrameCount/T,(FrameCount-Dropped)/T)
    RT_DebugF(S,name=myName)
    RT_TxtWriteFile(S,LOG,Append=True)
    """)
    Return 0
}
EDITED:

and log
Code:
Fsel_Eduardobedoya_Batch.Log  

1/4 ] Processing D:\AVS\AVI\IN\FLASHTEST.avi
  QWIK Scan DBase creation = 16.30 Secs (0.27 Mins)
  Dropping 0 of 7342 frames [640x400 293.68 secs (4.89 Mins) @ 25.00 FPS]
  QWIK SCAN 8.44 Secs (0.14 Mins) InFPS=870.11 OutFPS=870.11
  D:\AVS\AVI\IN\FLASHTEST.avi  Tot File Time = 24.83 Seconds (0.41 Mins)
  

2/4 ] Processing D:\AVS\AVI\IN\Scrambled.avi
  QWIK Scan DBase creation = 236.22 Secs (3.94 Mins)
  Dropping 27750 of 45000 frames [720x576 1800.00 secs (30.00 Mins) @ 25.00 FPS]
  QWIK SCAN 23.17 Secs (0.39 Mins) InFPS=1942.00 OutFPS=744.43
  D:\AVS\AVI\IN\Scrambled.avi  Tot File Time = 259.59 Seconds (4.33 Mins)
  

3/4 ] Processing D:\AVS\AVI\IN\TEST2.avi
  QWIK Scan DBase creation = 53.75 Secs (0.90 Mins)
  Dropping 0 of 15319 frames [720x576 612.76 secs (10.21 Mins) @ 25.00 FPS]
  QWIK SCAN 27.56 Secs (0.46 Mins) InFPS=555.80 OutFPS=555.80
  D:\AVS\AVI\IN\TEST2.avi  Tot File Time = 81.61 Seconds (1.36 Mins)
  

4/4 ] Processing D:\AVS\AVI\IN\TEST.avi
  QWIK Scan DBase creation = 69.00 Secs (1.15 Mins)
  Dropping 0 of 13861 frames [720x576 554.44 secs (9.24 Mins) @ 25.00 FPS]
  QWIK SCAN 18.75 Secs (0.31 Mins) InFPS=739.25 OutFPS=739.25
  D:\AVS\AVI\IN\TEST.avi  Tot File Time = 87.91 Seconds (1.47 Mins)


DONE



TOTAL Time = 453.94 Seconds (7.57 Mins)
Is somewhat better than 1st script in thread where we were getting about 1 frame every 12 seconds.

EDIT: Has GUI File Selector to group select AVI files, auto creates DataBases, Frames command files and
avs files to instantly select edited clips, and deletes Dbases files.

EDIT: Just spotted potential (but unlikely) problem, if mutiple consecutive SPACES in AVI filenames, will
convert to single space and NOT find them. (RT_StrReplace line).

EDIT: Scripts edited for minor problems or typo's.
The Prune extractor removes possible audio glitches ('cracks') at splices when audio present.

PS, were the scripts idiot proof ?
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but are any of them infinitely bigger ???

Last edited by StainlessS; 25th March 2014 at 19:02.
StainlessS is offline   Reply With Quote
Old 27th March 2014, 16:28   #20  |  Link
eduardobedoya
Registered User
 
eduardobedoya's Avatar
 
Join Date: Mar 2014
Posts: 39
Thanks Stainlesss, I read your notes and tried your script

Thanks Stainlesss, I read your notes and tried your script

First I installed the 32bit version of Virtualdub (I dont know why windows media player fail to open Fsel_Eduardobedoya_Batch.avs)
Is it the same if I have the 32 or 64 bits version of Virtualdub?

Then I installed Avisynth_258 and the RT Stats plugin that you provided
I couldn't find FrameSel v1.12 so I guessed v2.12 could do the trick, so I installed from here
http://forum.doom9.org/showthread.php?p=1566581#post1566581
Then I installed GScript_11 from here
http://forum.doom9.org/showthread.php?t=147846
(I copy all the plugins' dlls inside the Avisynth's plugin directory)
Finally I installed debug view from here
http://technet.microsoft.com/en-gb/sysinternals/bb896647

I run VirtualDub and open Fsel_Eduardobedoya_Batch.avs
the open windows showed, then I pick up some 1920x1080 20min video
here is the LOG:

00000001 0.00000000 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000002 15.38796234 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000003 15.40174389 [7920] Fsel_Eduardobedoya_Batch:
00000004 15.40177345 [7920] Fsel_Eduardobedoya_Batch:
00000005 15.40181732 [7920] Fsel_Eduardobedoya_Batch: 1/1 ] Processing C:\Users\Eduardo\Desktop\FOR AVISYNTH\cap19 -4.avi
00000006 15.41219711 [7920] TSC2: Instantiating codec with Q=2/6, force KFs=800, force KFbytes=5000000
00000007 15.43127728 [7920]
00000008 15.43127728 [7920] RT_QwikAveLumaScanCreateDB: RT_QwikAveLumaScanCreateDB() by StainlessS
00000009 15.43131065 [7920] RT_QwikAveLumaScanCreateDB: Filling DBase with YStats data (Will take some time)
00000010 15.43134689 [7920] RT_QwikAveLumaScanCreateDB: Creating DB
00000011 23.23528481 [7920] RT_QwikAveLumaScanCreateDB: record(1024) 5.5%
00000012 31.00768471 [7920] RT_QwikAveLumaScanCreateDB: record(2048) 10.9%
00000013 35.70775223 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000014 35.70792007 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000015 38.72195816 [7920] RT_QwikAveLumaScanCreateDB: record(3072) 16.4%
00000016 46.43200684 [7920] RT_QwikAveLumaScanCreateDB: record(4096) 21.8%
00000017 54.15245438 [7920] RT_QwikAveLumaScanCreateDB: record(5120) 27.3%
00000018 61.86000061 [7920] RT_QwikAveLumaScanCreateDB: record(6144) 32.7%
00000019 69.56040955 [7920] RT_QwikAveLumaScanCreateDB: record(7168) 38.2%
00000020 77.31515503 [7920] RT_QwikAveLumaScanCreateDB: record(8192) 43.6%
00000021 78.57512665 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000022 80.63726044 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000023 82.39134979 [7920] DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt = 2
00000024 85.02700043 [7920] RT_QwikAveLumaScanCreateDB: record(9216) 49.1%
00000025 88.59996033 [7920] DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt = 2
00000026 92.72863007 [7920] RT_QwikAveLumaScanCreateDB: record(10240) 54.5%
00000027 100.43165588 [7920] RT_QwikAveLumaScanCreateDB: record(11264) 60.0%
00000028 108.12608337 [7920] RT_QwikAveLumaScanCreateDB: record(12288) 65.4%
00000029 115.83764648 [7920] RT_QwikAveLumaScanCreateDB: record(13312) 70.9%
00000030 123.53649902 [7920] RT_QwikAveLumaScanCreateDB: record(14336) 76.3%
00000031 131.22393799 [7920] RT_QwikAveLumaScanCreateDB: record(15360) 81.8%
00000032 138.88714600 [7920] RT_QwikAveLumaScanCreateDB: record(16384) 87.2%
00000033 146.58955383 [7920] RT_QwikAveLumaScanCreateDB: record(17408) 92.7%
00000034 154.30934143 [7920] RT_QwikAveLumaScanCreateDB: record(18432) 98.1%
00000035 156.98666382 [7920] RT_QwikAveLumaScanCreateDB: record(18785) 100.0%
00000036 156.98908997 [7920] RT_QwikAveLumaScanCreateDB: Creating PrevDB
00000037 157.05241394 [7920] RT_QwikAveLumaScanCreateDB: record(1024) 5.5%
00000038 157.05616760 [7920] RT_QwikAveLumaScanCreateDB: record(2048) 10.9%
00000039 157.05995178 [7920] RT_QwikAveLumaScanCreateDB: record(3072) 16.4%
00000040 157.06365967 [7920] RT_QwikAveLumaScanCreateDB: record(4096) 21.8%
00000041 157.06732178 [7920] RT_QwikAveLumaScanCreateDB: record(5120) 27.3%
00000042 157.07098389 [7920] RT_QwikAveLumaScanCreateDB: record(6144) 32.7%
00000043 157.07464600 [7920] RT_QwikAveLumaScanCreateDB: record(7168) 38.2%
00000044 157.07839966 [7920] RT_QwikAveLumaScanCreateDB: record(8192) 43.6%
00000045 157.08206177 [7920] RT_QwikAveLumaScanCreateDB: record(9216) 49.1%
00000046 157.08575439 [7920] RT_QwikAveLumaScanCreateDB: record(10240) 54.5%
00000047 157.08943176 [7920] RT_QwikAveLumaScanCreateDB: record(11264) 60.0%
00000048 157.09320068 [7920] RT_QwikAveLumaScanCreateDB: record(12288) 65.4%
00000049 157.09689331 [7920] RT_QwikAveLumaScanCreateDB: record(13312) 70.9%
00000050 157.10060120 [7920] RT_QwikAveLumaScanCreateDB: record(14336) 76.3%
00000051 157.10430908 [7920] RT_QwikAveLumaScanCreateDB: record(15360) 81.8%
00000052 157.10803223 [7920] RT_QwikAveLumaScanCreateDB: record(16384) 87.2%
00000053 157.11178589 [7920] RT_QwikAveLumaScanCreateDB: record(17408) 92.7%
00000054 157.11546326 [7920] RT_QwikAveLumaScanCreateDB: record(18432) 98.1%
00000055 157.11676025 [7920] RT_QwikAveLumaScanCreateDB: record(18785) 100.0%
00000056 157.11859131 [7920] RT_QwikAveLumaScanCreateDB: total time = 141.69 seconds (2.36 mins)
00000057 157.12307739 [7920] DropDupSequences: QWIK Scanning file ... Please Wait
00000058 178.10548401 [7920] DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt = 2
00000059 217.35881042 [7920] DropDupSequences: Dropping 18526 of 18785 frames [1920x1080 1252.33 secs (20.87 Mins) @ 15.00 FPS]
00000060 217.35885620 [7920] DropDupSequences: QWIK SCAN 60.23 Secs (1.00 Mins) InFPS=311.88 OutFPS=4.30
00000061 217.36831665 [7920] Fsel_Eduardobedoya_Batch: C:\Users\Eduardo\Desktop\FOR AVISYNTH\cap19 -4.avi Tot File Time = 201.97 Seconds (3.37 Mins)
00000062 217.37132263 [7920] Fsel_Eduardobedoya_Batch:
00000063 217.37135315 [7920] Fsel_Eduardobedoya_Batch:
00000064 217.37139893 [7920] Fsel_Eduardobedoya_Batch: TOTAL Time = 201.97 Seconds (3.37 Mins)
00000065 217.37141418 [7920] Fsel_Eduardobedoya_Batch:
00000066 217.37489319 [7920] 003C8740->CAVIFileSynth::GetStream(*, 73647561(auds), 0)
00000067 217.37492371 [7920] 05533D48->CAVIStreamSynth(audio)
00000068 217.37496948 [7920] 05533D48->CAVIStreamSynth::AddRef() (audio) gRefCnt=3, m_refs=1
00000069 217.37498474 [7920] 003C8740->CAVIFileSynth::AddRef() gRefCnt=4, m_refs=3
00000070 217.37503052 [7920] 05533D48->CAVIStreamSynth::Info(0018F948, 204) (audio)
00000071 217.37504578 [7920] 05533D48->CAVIStreamSynth::ReadFormat() (audio)
00000072 217.37536621 [7920] 05533D48->CAVIStreamSynth::ReadFormat() (audio)
00000073 217.37565613 [7920] 05533D48->CAVIStreamSynth::Info(0018F9E0, 204) (audio)
00000074 217.37568665 [7920] 05533D48->CAVIStreamSynth::Info(0018F9D8, 204) (audio)
00000075 217.37573242 [7920] 05533D48->CAVIStreamSynth::Info(0018F9D8, 204) (audio)
00000076 217.37577820 [7920] 003C8740->CAVIFileSynth::GetStream(*, 73647561(auds), 1)
00000077 217.37579346 [7920] 003C8740->CAVIFileSynth::GetStream(*, 73766169(iavs), 1)
00000078 217.37583923 [7920] 003C8740->CAVIFileSynth::GetStream(*, 73646976(vids), 0)
00000079 217.37586975 [7920] 05533C28->CAVIStreamSynth(video)
00000080 217.37591553 [7920] 05533C28->CAVIStreamSynth::AddRef() (video) gRefCnt=5, m_refs=1
00000081 217.37593079 [7920] 003C8740->CAVIFileSynth::AddRef() gRefCnt=6, m_refs=4
00000082 217.37596130 [7920] 05533C28->CAVIStreamSynth::Info(0018F978, 204) (video)
00000083 217.37600708 [7920] 05533C28->CAVIStreamSynth::ReadFormat() (video)
00000084 217.37629700 [7920] 05533C28->CAVIStreamSynth::ReadFormat() (video)
00000085 217.37661743 [7920] 05533C28->CAVIStreamSynth::Info(0018FA0C, 204) (video)
00000086 217.37663269 [7920] 05533C28->CAVIStreamSynth::Info(0018FA04, 204) (video)
00000087 217.37666321 [7920] 05533C28->CAVIStreamSynth::Info(0018FA04, 204) (video)
00000088 217.37803650 [7920] 05533C28->CAVIStreamSynth::QueryInterface() (video) {00020022-0000-0000-c000-000000000046} (IAVIStreaming)
00000089 217.37808228 [7920] 05533C28->CAVIStreamSynth::AddRef() (video) gRefCnt=7, m_refs=2
00000090 217.37809753 [7920] 05533C28->CAVIStreamSynth::Begin(0, 2073600, 2000) (video)
00000091 217.37814331 [7920] 05533C28->CAVIStreamSynth::Release() (video) gRefCnt=6, m_refs=1
00000092 217.43962097 [7920] DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt = 6

the process finished and an black screen appeared with yellow letters indicating some time
also created the "cap19 -4_Frames.txt" and "cap19 -4_SelectFrames.AVS" files inside the source directory
but I dont know how to export the video, If I put File>save as avi, it export a black screen video
If I try to open the created "cap19 -4_SelectFrames.AVS" it will pop a dialog box saying...
Avisynth open failure:
Script error: there is no function named "FrameSel"

Do I need to install some plugin in VirtualDub?
Is there something I am doing wrong?
Is there a way to import the result frames directly inside premiere? or another editor that could render faster than VirtualDub?
I found that VirtualDub render to slow, I need to render almost 20 hours with x264 compression, pls Stainlesss what do you suggest?

Last edited by eduardobedoya; 27th March 2014 at 21:35.
eduardobedoya is offline   Reply With Quote
Reply

Tags
avisynth, detection, duplicated, frames, trim

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.