Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th May 2014, 12:03   #1  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Memory problem opening multiple clips

Hi All,

I'm running into what seems to be a memory problem when cutting and splicing together AVCHD camera files. The behaviour is the same as when previous versions of L-SMASH WORKS leaked out of memory. I'm using AvsPmod 2.5.1 with Avisynth 2.6.0 Alpha 4 and L-SMASH WORKS r725's LWLibavAudio/VideoSource() on Windows 7 Pro SP1 32-bit.

Because the camera creates one file per scene, I want to first open all files assigning each to a unique variable, and then use the variable names to more conveniently reorder and Trim() them. As it stands the script therefore looks like this (just for now, later I plan to use more meaningful clip variable names ):
Code:
# ===================
# Files
# ===================
c20140510154014 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154014.mts")
c20140510154111 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154111.mts")
c20140510154135 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154135.mts")
c20140510154151 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154151.mts")
c20140510154217 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154217.mts")
c20140510154243 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154243.mts")
c20140510154322 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154322.mts")
c20140510154429 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154429.mts")
c20140510154445 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154445.mts")
c20140510154458 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154458.mts")
c20140510154521 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154521.mts")
c20140510154558 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154558.mts")
c20140510154744 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154744.mts")
c20140510154854 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154854.mts")
c20140510155012 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155012.mts")
c20140510155050 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155050.mts")
c20140510155133 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155133.mts")
c20140510155144 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155144.mts")
c20140510155152 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155152.mts")
c20140510155207 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155207.mts")
c20140510155218 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155218.mts")
c20140510155231 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155231.mts")
c20140510155252 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155252.mts")
c20140510155338 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155338.mts")
c20140510155412 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155412.mts")
c20140510155427 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155427.mts")
c20140510155451 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155451.mts")
c20140510155550 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155550.mts")
c20140510155634 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155634.mts")
c20140510155703 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155703.mts")
c20140510155732 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155732.mts")
c20140510155848 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155848.mts")
c20140510155931 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510155931.mts")
c20140510160017 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160017.mts")
c20140510160100 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160100.mts")
c20140510160129 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160129.mts")
c20140510160226 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160226.mts")
c20140510160437 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160437.mts")
c20140510160606 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160606.mts")
c20140510160721 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160721.mts")
c20140510160729 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160729.mts")
c20140510160916 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510160916.mts")
c20140525171805 = LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140525171805.mts")

# ===================
# Timeline editing
# ===================
# ...

# ===================
# Final output
# ===================

MessageClip("No edited output available yet")
LWOpen() is defined in the auto-include utility script below as:
Code:
function LWOpen(string source) {
    return(AudioDub(LWLibavVideoSource(source), LWLibavAudioSource(source)))
}
When I run my script, I get:
Quote:
LWLibavAudioSource: failed to get the audio track.
(FV_utils-1.9.0.avsi, line 116)
(C:\Users\fvisagie\Videos\Home Videos\20140510 Collene & Kallie Pre-shoot\1. Footage.avs, line 38)
This error would occur anywhere from line 38 to line 41, whenever AvsPmod's memory usage reaches around 1666 MB RAM. At that point total physical memory utilisation across all processes is only ~75%. The machine has 4 GB.

Setting higher-than-default values with SetMemoryMax() makes no difference I could notice. When I use FFVideo/AudioSource() instead of LWLibavAudio/Source(), I get similar behaviour:
Quote:
FFVideoSource: Insanity detected: decoder returned an empty frame
(FV_utils-1.9.0.avsi, line 184)
(C:\Users\fvisagie\Videos\Home Videos\20140510 Collene & Kallie Pre-shoot\1. Footage.avs, line 45)
This happens at about 1620 MB memory usage.

When I do NOT assign variables to the opened clips e.g.
Code:
LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154014.mts")
LWOpen("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154111.mts")
...
I do not get this problem. I know the filtergraph etc. probably look different in this case, but even so I confirmed that each file is still opened and indexed.

This suggests I would be able to complete this task by opening the files and editing the timeline in-line, e.g. LWOpen("file1").Trim(a, b) ++ LWOpen("file2").Trim(c, d). However I find this way of working without variables much less convenient.

Am I doing something wrong, or is this a bug? Is there a way to get around this memory problem so I can use my preferred approach, i.e. assigning a variable to each opened clip?

Many thanks,
Francois

Last edited by fvisagie; 26th May 2014 at 12:08. Reason: Typos
fvisagie is offline   Reply With Quote
Old 26th May 2014, 16:28   #2  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
If I understand what's going on here, it's not the use of variables per se which is causing the problem. In your script without variables, each source filter instance is no longer referenced once the line following has been evaluated (overwriting the variable last). They will therefore be immediately freed by Avisynth (the filters' destructors will be called). In the 'variables' script, because distinct variables are used, each one holds a reference so nothing is freed.

Using the source filters (or wrapper function) directly (instead of variables) won't gain anything because the filter graph itself will hold a reference to each one used.

The only thing I can suggest is to render your script in sections, saving intermediate results to a lossless format.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 26th May 2014, 17:02   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
In my experience, all video (not image) source filters screw up at a certain number of open clips,
I think I decided that 28 (maybe it was 23) opened files was usually as many as could be reliably opened at once, so
if I had more than this I would do them in several multiclip sections, saving to lossles as Gavino suggests.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 26th May 2014, 17:41   #4  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Thanks for your feedback, guys.

Quote:
Originally Posted by StainlessS View Post
if I had more than this I would do them in several multiclip sections, saving to lossles as Gavino suggests.
That's a daunting prospect - with one of my recent projects the camera generated 449 clips/files!

Some more results.

As you suspect, even when I open and splice files in-line with no intermediate functions or variables, e.g.
Code:
...
AudioDub(LWLibavVideoSource("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154111.mts"), LWLibavAudioSource("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154111.mts")) ++ \
AudioDub(LWLibavVideoSource("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154135.mts"), LWLibavAudioSource("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140510 Collene & Kallie Pre-shoot\20140510154135.mts")) ++ \
...
I get the same problem.

When I set the LWLibavVideoSource() threads parameter to values in the range 1 - 3 inclusive, the problem does not occur. The lower the threads value, the less memory used and the slower the processing. My machine has a quad-core i5-2540M.

So, conversely, the higher the number of threads used the more memory used until ultimately the plugin fails. This begs the question of why the plugin errors out when not even the physical memory has been exhausted yet at that point - ~75%?

Last edited by fvisagie; 27th May 2014 at 07:42.
fvisagie is offline   Reply With Quote
Old 26th May 2014, 19:20   #5  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Quote:
Originally Posted by StainlessS View Post
In my experience, all video (not image) source filters screw up at a certain number of open clips,
That's been my experience as well. However you can use FFmpeg to join your videos together into a few (1 to 20) longer clips, then work with those in Avisynth, cutting them up as much as you like*

*edit - I don't know what the limit is, but I have just tested 200 clip variables (trimmed from a single Source) with no trouble.

Last edited by raffriff42; 26th May 2014 at 20:33.
raffriff42 is offline   Reply With Quote
Old 26th May 2014, 20:08   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
You could do worse than take a look here to see a work in progress for joining multiple clips (althoug you would not need most
of the functionality of the scripts given)
http://forum.doom9.org/showthread.ph...ynthesizer_mod
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 27th May 2014, 08:33   #7  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
I assume there is no existing or easy solution to the issue of too many open clips then.

Quote:
Originally Posted by StainlessS View Post
You could do worse than take a look here to see a work in progress for joining multiple clips (althoug you would not need most
of the functionality of the scripts given)
http://forum.doom9.org/showthread.ph...ynthesizer_mod
Thanks for the suggestion. Apart from the purpose of RoboSplice which I managed to figure out, what does the rest of that project do? The original project and documentation seem to have been removed from the Tangentsoft site.

Quote:
Originally Posted by raffriff42 View Post
However you can use FFmpeg to join your videos together
...
I have just tested 200 clip variables (trimmed from a single Source) with no trouble.
FFmpeg's been my preferred back-up plan for joining, thanks, primarily because that approach preserves stream metadata. Thanks also for the reassuring test.

The downside of joining source files (in addition to more time and disk space) is that metadata is sacrificed. In this case, once joined the start and end points of clips are lost. Typically in the projects I work with this adds a whole new dimension of effort.

What existing solutions are there to this issue other than the non-precise approach of image-based scene change detection? What I would find very elegant is concatenating in Avisynth to lossless output with a runtime script that records timecode discontinuities in a scene file/AvsPmod bookmark file etc. The concept could conceivably be extended to continuously record any metadata such as date and timestamp to e.g. subtitle format. As opposed to extracting metadata on-the-fly (which I haven't yet found a solution for) I currently foresee parsing pre-generated metadata text files. Therefore this approach could even be used to apply timeline edits to existing subtitle text files without needing to parse potentially complex script timeline edits. I've found a way of providing AVCHD metadata such as timecode and timestamp, and current_frame very nicely identifies source file frame number at runtime. However, I've found no way of identifying the current input file(s) at runtime. Any suggestions for that (bearing in mind I'm in no position to write my own plugins )?

Last edited by fvisagie; 27th May 2014 at 08:59. Reason: Clarification
fvisagie is offline   Reply With Quote
Old 27th May 2014, 14:43   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
The original TangentSoft doc is included in the DOC folder inside the zip, "Videographica AVISynthesizer Manual.htm".
Looks like he might have been getting unwanted communications about what was for him a dead project.

The original Avisynthesizer project allows group selection of files in Windows Explorer, right click and 'Send To" Avisynthesizer
will output an avs file which concatenates selected files using a template script.

If you chose to use Avisynthesizer_Mod, it would be no trouble at all to capture filenames, ie in equivalent of my DoFile()
function, could update a global variable representing File number, writing filename and extracted data to a file, or maybe even
an RT_Stats database (you can have up to 256 fields in current RT_Stats Dbase, 1024 in the Beta version recently posted in that thread).
You can also use eg RT_Call() to call an external function (eg MediaInfo cmd line version) that extracts metadata to a text file, parse and extract from that,
and maybe add it to the Dbase file.
The Dbase can hold all Avisynth variable types (excluding clip), so eg first field of a record (clip) might be filename, 2nd frame count, etc.
Reading writing and parsing extracted text files can reasonably easily done using RT_Stats functions.
One user (Filker I think) used RT_Stats and RT_Call to extract EXIF data from jpegs using a command line utility that extracts EXIF data from those
jpegs, might give you a few ideas if you look up the concerning posts.
You could even eg have a master DBase, holding eg names of a sub DBase, where sub Dbase holds individual clip data, and perhaps eg name of text file
holding subtitles text. Master Dbase holding name of Sub Dbase file, and data relating to the complete assembled final clip.
You dont need to be able to write your own plugins, RT_Stats probably provides pretty much everything you need, apart from external
commands to extract the various types of data you are interested in keeping.

EDIT: So this Avisynthesizer_mod template
Code:
#ASYNTHER RoboSplice

Global G_FILECOUNT=___FILECNT___

Global G_DEBUG=True
Global G_QBC=True 		  	Global G_SAMPLES=32 		Global G_QBC_THRESH=-32.0  Global G_INTERLACED=False
Global G_AL_STRENGTH=0.95  	Global G_GAMMA=1.0			Global G_AUTOGAIN=False

Global G_DEBLOCK=20
Global G_RPOW2=false
Global G_DISPWID=0 		Global G_DISPHIT=0

Global G_FRNUM=0 		Global G_FRDEN=0
Global G_USE_INTERFRAME=False

Global G_ASPECT_DARX=0       Global G_ASPECT_DARY=0

Global G_MCD=0      	Global G_SIGMA=1.65 		Global G_SHARPEN=0.0
Global G_CHANNELS=2 	Global G_SAMPLERATE=44100	Global G_BITS=16 		Global G_NORMALIZE=(G_FILECOUNT>1)

[GetTrim("___FILE___")]
would produce this script (if "Send To" the 4 filenames in blue):
Code:
#ASYNTHER RoboSplice

Global G_FILECOUNT=4

Global G_DEBUG=True
Global G_QBC=True 		  	Global G_SAMPLES=32 		Global G_QBC_THRESH=-32.0  Global G_INTERLACED=False
Global G_AL_STRENGTH=0.95  	Global G_GAMMA=1.0			Global G_AUTOGAIN=False

Global G_DEBLOCK=20
Global G_RPOW2=false
Global G_DISPWID=0 		Global G_DISPHIT=0

Global G_FRNUM=0 		Global G_FRDEN=0
Global G_USE_INTERFRAME=False

Global G_ASPECT_DARX=0       Global G_ASPECT_DARY=0

Global G_MCD=0      	Global G_SIGMA=1.65 		Global G_SHARPEN=0.0
Global G_CHANNELS=2 	Global G_SAMPLERATE=44100	Global G_BITS=16 		Global G_NORMALIZE=(G_FILECOUNT>1)

GetTrim("D:\AVS\AVI\TEST2YV12.avi") ++ \
GetTrim("D:\AVS\AVI\TEST2RGB.avi") ++ \
GetTrim("D:\AVS\AVI\TESTYV12.avi") ++ \
GetTrim("D:\AVS\AVI\TESTRGB.avi")
Where GetTrim() can do anything it wants with the filename, resulting clips being joined together.
EDIT: There is facility to order the given filenames within Avisynthesizer_mod.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th May 2014 at 16:00.
StainlessS is offline   Reply With Quote
Old 27th May 2014, 17:41   #9  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
@StainlessS,

Thanks!
Quote:
RT_Stats probably provides pretty much everything you need, apart from external commands
Are you also implying that it can identify the current input clip at runtime? Just wanted to check before diving into it .
fvisagie is offline   Reply With Quote
Old 27th May 2014, 21:38   #10  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
Quote:
Where GetTrim() can do anything it wants with the filename
As it says above. Read my previous post again.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th May 2014 at 21:40.
StainlessS is offline   Reply With Quote
Old 28th May 2014, 05:59   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
fvisagie,

OK, spent a few hours on the stuff below, it's not very pretty but works OK, still would need a lot of
work to perfect such usage, and some kind of function to do a binary search on the Dbase when
user jumps about on the timeline (to find which trim it is in and current trim frame and filename).
You cannot jump about, MUST only play straight though.
It is in the form of an Avisynthesizer Template that has already been initialized by Avisynthesizer,
ie has had "Global G_FILECOUNT" and the input filenames aready set. You just need to delete those lines
and uncomment the original template lines to restore it as a template.
If you want to try it out, suggest cut out 5 clips from an avi, named "0.avi" to "4.avi" with either
1000 or 100 frames (I used 1000).
It will create a single Dbase and enter in a little data, just frame numbers really and filename (you can add whatever else you like
and change the source filter to whatever).
During ScriptClip processing it subtitles:- whole movie frame number, Trim number, Trim frame number, and trim Filename.
Probably not all globals were necessary, but I gotta get some sleep sometime soon.
Here tis:
Code:
#ASYNTHER Test
############################################
#Global G_FILECOUNT=___FILECNT___               # Total clips being processed, filled in by Avisynthesizer_Mod
Global G_FILECOUNT=5                            # Total clips being processed
############################################
Global G_CURRENT_TRIM=0                         # Count of clips done so far : MUST BE 0
############################################
Global G_PROJECTNAME="Test"                     # Change to whatever you want
Global G_DB=G_PROJECTNAME+".DB"                 # DBase
Global G_TYPESTRING="s512s512iii"               # Fields:-
                                                # 0) String(512) ClipFileName
                                                # 1) String(512) ClipSubTitles (not used)
                                                # 2) Int ClipFrameCount
                                                # 3) Int MovieStartFrame
                                                # 4) Int MovieEndFrame,
#############################################
RT_DBaseAlloc(G_DB,G_FILECOUNT,G_TYPESTRING)    # Create Movie DB
###
#[ProcessTrim("___FILE___")]                    # filled in by Avisynthesizer_Mod
ProcessTrim("0.avi") ++ \
ProcessTrim("1.avi") ++ \
ProcessTrim("2.avi") ++ \
ProcessTrim("3.avi") ++ \
ProcessTrim("4.avi")

Global  G_CURRENT_TRIM = -1                     # Init for play
Global  G_MOVIETRIM_START= -1                   # Init for play
Global  G_MOVIETRIM_END = -1                    # Init for play
Global  G_FN=""                                 # Init for play

ScriptClip("""
    DoInit = (current_Frame > G_MOVIETRIM_END)
    Global G_CURRENT_TRIM   = (DoInit) ? G_CURRENT_TRIM + 1 : G_CURRENT_TRIM
    Global G_MOVIETRIM_START= (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,3) : G_MOVIETRIM_START
    Global G_MOVIETRIM_END  = (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,4) : G_MOVIETRIM_END
    Global G_FN= (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,0) : G_FN
    RT_SubTitle("%d] {%d:%d} %s",current_frame,G_CURRENT_TRIM,current_frame-G_MOVIETRIM_START,G_FN)
    return Last
""")
Return Last

#############################################
### Contents of Process.avsi
Function ProcessTrim(string fn) {
    RT_DebugF("Processing %d ] %s",G_CURRENT_TRIM,fn)
    c = Avisource(fn)
    FC = c.FrameCount
    SubTitleFN=""                                       # We dont have subtitle file, maybe we could get it from some external program.
    S = (G_CURRENT_TRIM == 0) ? 0 : RT_DBaseGetField(G_DB,G_CURRENT_TRIM-1,4) + 1     # Previous MovieEndFrame + 1
    E = S + FC - 1
    RT_DBaseSet(G_DB,G_CURRENT_TRIM,fn,SubTitleFN,FC,S,E)
    Global G_CURRENT_TRIM = G_CURRENT_TRIM + 1          # Increment current trim number. NOTE Assign Global
    return c
}
And this is a little script to show contents of DBase, might come in handy to check out if working OK.
Results sent to DebugView.
Code:
DB="TEST.DB"
###
TYPENAMES=RT_String("Bool\nInt\nFloat\nString\nBin\n")
GSCript("""
    Records=RT_DBaseRecords(DB)
    Fields=RT_DBaseFields(DB)
    RT_DebugF("\n%s\n",DB)
    RT_DebugF("Records      = %d",RT_DBaseRecords(DB))
    RT_DebugF("RecordSize   = %d",RT_DBaseRecordSize(DB))
    RT_DebugF("RecordsMax   = %d ($%X)",RT_DBaseRecordsMax(DB),RT_DBaseRecordsMax(DB))
    RT_DebugF("Fields       = %d",FIELDS)
    for(i=0,Fields - 1) {
        TYPE_S=RT_TxtGetLine(TYPENAMES,RT_DBaseFieldType(DB,i))
        RT_DebugF("  %2d ) Type = %d(%6s) Size = %d",i,RT_DBaseFieldType(DB,i),TYPE_S,RT_DBaseFieldSize(DB,i))
    }

    RT_debugF("\nShowing Contents\n")
    QUOT=Chr(34)
    for(i=0,Records-1) {
        RT_DebugF("%d] ----------------",i)
        for(j=0,fields-1) {
            Typ=RT_DBaseFieldType(DB,j)
            RT_DebugF("        %d] %s%s%s",j,Typ==3?QUOT:"",String(RT_DBaseGetField(DB,i,j)),Typ==3?QUOT:"")
        }
    }
""")
colorbars.killaudio
return last
EDIT: You will have to edit path to filenames, I used "D:\AVS\DD\".
EDIT: Forget above, I've edited to use current directory eg just "0.avi".
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2014 at 08:17.
StainlessS is offline   Reply With Quote
Old 28th May 2014, 07:38   #12  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by StainlessS View Post
OK, spent a few hours on the stuff below
This is valuable, thanks for the effort. Hope you got some sleep .

Cheers,
Francois
fvisagie is offline   Reply With Quote
Old 28th May 2014, 08:13   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
Quote:
Hope you got some sleep
Not yet, I'll have to hurry because I've got to get up half an hour ago

OK, done the findTrim() thing, you can now jump about on the timeline.
Code:
#ASYNTHER Test
############################################
#Global G_FILECOUNT=___FILECNT___               # Total clips being processed, filled in by Avisynthesizer_Mod
Global G_FILECOUNT=5                            # Total clips being processed
############################################
Global G_CURRENT_TRIM=0                         # Count of clips done so far : MUST BE 0
############################################
Global G_PROJECTNAME="Test"                     # Change to whatever you want
Global G_DB=G_PROJECTNAME+".DB"                 # DBase
Global G_TYPESTRING="s512s512iii"               # Fields:-
                                                # 0) String(512) ClipFileName
                                                # 1) String(512) ClipSubTitles (not used)
                                                # 2) Int ClipFrameCount
                                                # 3) Int MovieStartFrame (of trim)
                                                # 4) Int MovieEndFrame  (of trim)
#############################################
RT_DBaseAlloc(G_DB,G_FILECOUNT,G_TYPESTRING)    # Create Movie DB
###
#[ProcessTrim("___FILE___")]                    # filled in by Avisynthesizer_Mod
ProcessTrim("0.avi") ++ \
ProcessTrim("1.avi") ++ \
ProcessTrim("2.avi") ++ \
ProcessTrim("3.avi") ++ \
ProcessTrim("4.avi")

Global  G_PREV_FRAME        = -2                # Force Init
Global  G_CURRENT_TRIM      = -1                # Init for play
Global  G_MOVIETRIM_START   = -1                # Init for play
Global  G_MOVIETRIM_END     = -1                # Init for play
Global  G_FN=""                                 # Init for play

ScriptClip("""
    DoInit = (G_PREV_FRAME + 1 != current_frame || current_frame > G_MOVIETRIM_END)
    Global G_CURRENT_TRIM   = (DoInit) ? FindTrim(G_DB,3,4,current_frame) : G_CURRENT_TRIM
    Global G_MOVIETRIM_START= (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,3) : G_MOVIETRIM_START
    Global G_MOVIETRIM_END  = (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,4) : G_MOVIETRIM_END
    Global G_FN             = (DoInit) ? RT_DBaseGetField(G_DB,G_CURRENT_TRIM,0) : G_FN
    RT_SubTitle("%d] {%d:%d} %s",current_frame,G_CURRENT_TRIM,current_frame-G_MOVIETRIM_START,G_FN)
    G_PREV_FRAME = current_frame                # Remember previous frame visited for next iteration
    return Last
""")
Return Last

#############################################
### Contents of Process.avsi
Function ProcessTrim(string fn) {
    RT_DebugF("Processing %d ] %s",G_CURRENT_TRIM,fn)
    c = Avisource(fn)
    FC = c.FrameCount
    SubTitleFN=""                                                   # We dont have subtitle file, maybe we could get it from some external program.
    S = (G_CURRENT_TRIM == 0) ? 0 : RT_DBaseGetField(G_DB,G_CURRENT_TRIM-1,4) + 1     # Previous MovieEndFrame + 1
    E = S + FC - 1
    RT_DBaseSet(G_DB,G_CURRENT_TRIM,fn,SubTitleFN,FC,S,E)
    Global G_CURRENT_TRIM = G_CURRENT_TRIM + 1                      # Increment current trim number. NOTE Assign Global
    return c
}

Function FindTrim(String DB,Int S_Field,Int E_Field,Int Frame) {
# DB Find Trim (record) containing movie frame Frame, S_Field = MovieStartFrame, E_Field = MovieEndFrame
    result = -1                                                     # Init NOT FOUND
    low     = 0
    high    = RT_DBaseRecords(DB) - 1
    GSCript("""
        while(low <= high) {
            mid = (low + high) / 2
            if(RT_DBaseGetField(DB,mid,E_Field) < Frame) {
                low = mid + 1
            } Else If (RT_DBaseGetField(DB,mid,S_Field) > Frame) {
                high = mid - 1
            } Else {
                low = high + 1                                      # Force exit
                Result = mid
            }
        }
    """)
    return result
}
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 28th May 2014, 14:21   #14  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Thanks muchly.

Quote:
OK, done the findTrim() thing
So having digested what RT-Stats, Avisynthesizer_Mod and Gscript are all about, let alone the logic, lemme see if I get this straight, big breath... (my only real coding work was M68K assembler), this works by:
  • storing clip data in a DB,
  • having knowledge of the script's timeline editing (in this case UnalignedSplices and the order in which they occur)
  • frame-counting at runtime,
  • and using that to infer from the DB the current input clip and frame (the latter also being the same as current_frame?)
Feel free to correct .

Last edited by fvisagie; 28th May 2014 at 15:27. Reason: UnalignedSplices, not Trims
fvisagie is offline   Reply With Quote
Old 28th May 2014, 17:18   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
1) Yep.

2) Nope, Aligned Splice. (I refer to them as trims probably because RoboSplice also originally trimmed black frames from start/end of each input clip).
(Avisynthesizer used UnAligned Splice, I modded because I did not like that).

3) Yep.
4) Yep.

Having Global data available in ScriptClip makes in unnecessary to interogate the DBase for everything at every frame, eg getting
Filename of clip would swallow a chunk of memory at every frame (not released until avisynth closure). In the given instance we subtitle splice
info on every frame, and so we need to know it at every frame. We only need to interogate DBase when user jumps about, or when we enter a new splice region.
If you only needed to do something when entering a new splice, then it would probably suffice to track Global G_PREV_FRAME, and Global G_MOVIETRIM_END,
the 1st to see if user jumped about, 2nd to detect entering of new splice region.
EDIT And Global G_MOVIETRIM_START if you needed to know Splice frame number.

EDIT: Note, The Dbase also has 256 (currently or 1024 in beta) Int or Float attributes that can be read/written,
during DBase creation phase, could use one of them to track current splice/trim, but not much different to using a Global.

EDIT: It might be better to move fields MovieStartFrame and MovieEndFrame to fields 0 and 1, ie movie data at start of DBase
and after that everything belongs to trim/splice.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2014 at 17:45.
StainlessS is offline   Reply With Quote
Old 28th May 2014, 21:43   #16  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 5,871
Here modified FselShowDB.avs
Code:
# FselShowDB.avs
FSEL_TITLE="Select DBase file"
FSEL_DIR="."
FSEL_FILT="DB files|*.DB"
FSEL_MULTI=False
DBFILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(DBFILE_LIST.IsString,"FselShowDB.avs: RT_FSelOpen Error="+String(DBFILE_LIST))
DB=RT_TxtGetLine(DBFILE_LIST,0)
###
TYPENAMES=RT_String("Bool\nInt\nFloat\nString\nBin\n")
GSCript("""
    Records=RT_DBaseRecords(DB)
    Fields=RT_DBaseFields(DB)
    RT_DebugF("\n%s\n",DB)
    RT_DebugF("Records      = %d",RT_DBaseRecords(DB))
    RT_DebugF("RecordSize   = %d",RT_DBaseRecordSize(DB))
    RT_DebugF("RecordsMax   = %d ($%X)",RT_DBaseRecordsMax(DB),RT_DBaseRecordsMax(DB))
    RT_DebugF("Fields       = %d",FIELDS)
    for(i=0,Fields - 1) {
        TYPE_S=RT_TxtGetLine(TYPENAMES,RT_DBaseFieldType(DB,i))
        RT_DebugF("  %2d ) Type = %d(%6s) Size = %d",i,RT_DBaseFieldType(DB,i),TYPE_S,RT_DBaseFieldSize(DB,i))
    }
    RT_debugF("\nShowing Contents\n")
    QUOT=Chr(34)
    for(i=0,Records-1) {
        RT_DebugF("%d] ----------------",i)
        for(j=0,fields-1) {
            Typ=RT_DBaseFieldType(DB,j)
            RT_DebugF("        %d] %s%s%s",j,Typ==3?QUOT:"",String(RT_DBaseGetField(DB,i,j)),Typ==3?QUOT:"")
        }
    }
""")
colorbars.killaudio.Subtitle("See DebugView Output")
return last
Uses file selector to pick DB file.

AvsPMod (just installed) is a bit wierd with this script, sometimes swallows DebugView output, sometimes interferes with the file selector
mainly the FSEL_FILT string (changes it to ""), but sometimes OK, External Player (F6) is OK though (windows Mplayer2 for me).
EDIT: Also inserts a DirectShowSource("Test.DB") into the script when it plays up, whats with that then.

Here output
Code:
00000001    0.00000000  RT_DebugF:
00000002    0.00002762  RT_DebugF: D:\AVS\DD\Test.DB
00000003    0.00005642  RT_DebugF:
00000004    0.00047348  RT_DebugF: Records      = 5
00000005    0.00088868  RT_DebugF: RecordSize   = 1036
00000006    0.00162306  RT_DebugF: RecordsMax   = 2072844 ($1FA10C)
00000007    0.00172965  RT_DebugF: Fields       = 5
00000008    0.00277997  RT_DebugF:    0 ) Type = 3(String) Size = 512
00000009    0.00384017  RT_DebugF:    1 ) Type = 3(String) Size = 512
00000010    0.00488063  RT_DebugF:    2 ) Type = 1(   Int) Size = 4
00000011    0.00592725  RT_DebugF:    3 ) Type = 1(   Int) Size = 4
00000012    0.00698488  RT_DebugF:    4 ) Type = 1(   Int) Size = 4
00000013    0.00709123  RT_DebugF:
00000014    0.00712013  RT_DebugF: Showing Contents
00000015    0.00714840  RT_DebugF:
00000016    0.00743387  RT_DebugF: 0] ----------------
00000017    0.00846935  RT_DebugF:         0] "0.avi"
00000018    0.00944840  RT_DebugF:         1] ""
00000019    0.01044262  RT_DebugF:         2] 1000
00000020    0.01141452  RT_DebugF:         3] 0
00000021    0.01237941  RT_DebugF:         4] 999
00000022    0.01248989  RT_DebugF: 1] ----------------
00000023    0.01347254  RT_DebugF:         0] "1.avi"
00000024    0.01463834  RT_DebugF:         1] ""
00000025    0.01563924  RT_DebugF:         2] 1000
00000026    0.01661687  RT_DebugF:         3] 1000
00000027    0.01759837  RT_DebugF:         4] 1999
00000028    0.01770981  RT_DebugF: 2] ----------------
00000029    0.01875268  RT_DebugF:         0] "2.avi"
00000030    0.01972829  RT_DebugF:         1] ""
00000031    0.02070023  RT_DebugF:         2] 1000
00000032    0.02167538  RT_DebugF:         3] 2000
00000033    0.02266695  RT_DebugF:         4] 2999
00000034    0.02277710  RT_DebugF: 3] ----------------
00000035    0.02374526  RT_DebugF:         0] "3.avi"
00000036    0.02472806  RT_DebugF:         1] ""
00000037    0.02570513  RT_DebugF:         2] 1000
00000038    0.02668370  RT_DebugF:         3] 3000
00000039    0.02765870  RT_DebugF:         4] 3999
00000040    0.02776932  RT_DebugF: 4] ----------------
00000041    0.02874646  RT_DebugF:         0] "4.avi"
00000042    0.02990422  RT_DebugF:         1] ""
00000043    0.03091367  RT_DebugF:         2] 1000
00000044    0.03188236  RT_DebugF:         3] 4000
00000045    0.03285628  RT_DebugF:         4] 4999
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th May 2014 at 21:47.
StainlessS is offline   Reply With Quote
Old 29th May 2014, 07:02   #17  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
Quote:
Originally Posted by StainlessS View Post
2) Nope, Aligned Splice.
Hehe, exhaustion takes its toll .

Quote:
Originally Posted by StainlessS View Post
Here modified FselShowDB.avs
Great stuff.

Quote:
mainly the FSEL_FILT string (changes it to "")
When I had something similar it was due to AvsPmod parsing incorrectly. It will probably not work but just in case, see if turning off Options.Program settings.Save/Load.Save *.avs scripts etc. turns off AvsPmod's editing of scripts also.

Quote:
EDIT: Also inserts a DirectShowSource("Test.DB") into the script when it plays up, whats with that then.
Sounds like an AvsPmod template for the ".db" extension that somehow got triggered.

Anyhow, many thanks for all this effort, sincerely appreciated.
fvisagie is offline   Reply With Quote
Old 29th May 2014, 16:04   #18  |  Link
fvisagie
Registered User
 
Join Date: Aug 2008
Location: Isle of Man
Posts: 586
This works quite nicely too, attaching the necessary knowledge to the clip's filtergraph.
Code:
function Open(string source, bool "trace") {
    trace = Default(trace, True)
    LWOpen(source)
    trace ? ScriptClip("""source = """" + source + """"
                          Subtitle(source + "\n" + String(current_frame), x=0, y=0, size=24, lsp=1)""", after_frame=true) : NOP()
}

c1 = Open("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140221 Kam'Bati\2014-02-22_11-20-24.mp4")
c2 = Open("C:\Users\fvisagie\Videos\Home Videos\Camera Tapes\20140221 Kam'Bati\2014-02-22_12-09-10.mp4")

c1.Trim(10, 20) ++ c2.Trim(30, 40)
ShowFrameNumber(x=Width()-60, y=20)
Whenever a clip's filtergraph is accessed, its ScriptClipt() code now knows the corresponding source file name and frame number .

For completeness, LWOpen() is
Code:
function LWOpen(string source, int "threads") {
    threads = Default(threads, 0)
    return(AudioDub(LWLibavVideoSource(source, threads=threads), LWLibavAudioSource(source)))
}

Last edited by fvisagie; 29th May 2014 at 17:27.
fvisagie is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:23.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.